You don't need a drill press to make straight holes Sight the bit against two squares to ensure that the hole is drilled
perpendicular to the workpiece. Or you can use an accessory, like a drill guide, which turns the handheld drill into a mini drill press.
When you want to be more precise than is possible by sighting a bit, there are several commercial jigs that can be used to keep a bit square to a workpiece. Many of these jigs convert a handheld drill into a miniature drill press.