Fishing need not be an expensive sport. Fishing rods and reels make great birthday and Christmas presents, but a hand line or fishing line strung on a bottle works just as well.
Visit a bait and tackle shop and ask for some advice. They will be happy to help and let you know about local conditions and the best bait to use. (It is in their best interest to encourage children to take up fishing, isn't it!)