Visit the Temple of Literature. This is located not far from the old quarter and not far from the Hoan Kiem lake.
This temple is about 1000 years old and it is also Vietnam's first university. It cost 2-3 dollars (in local money) to get in, it surely you can afford and it worth it.