The rear wheel on the same side as the gear shift wont come loose after pulling cotter pin and nut?

Can I damage anything by using a 4×4 and a hammer?

Or is a special tool needed?

Thanks guys