1) 500 men (as in crew members - so I suggest a big ship..)
2)You need to attack the fort of such city and destroy it (you will know you are damaging it when you are shown a one liner that says "Fort cannon destroyed")
3) After destroying it you need to board it and kill all the soldiers that are inside
4) Finally you will be teleported to the city and there you need to kill all the soldiers of the city and go to the recidential place of the governor and talk to him.