Light objects have much less resistance to change of movement and so need much less time to start moving. The flick of a finger is enough to make a balloon accelerate quickly away. When moving, it has little momentum and even the friction of the air quickly slows it up.