Several bungalow resorts on the Rai Leh West side of the peninsula have sea kayaks available for rental for around 600 baht/half-day, 1,000 baht/full-day (including life-vests). The kayaks are simple two-seat plastic models, but perform fine on the millpond-smooth water of the bay. A half-day is probably plenty long enough to explore the immediate environs of Rai Leh. A bottle of water, a hat, and plenty of sun protection are essential!
If you are a strong kayaker, and feeling quite adventurous, you can get to Poda island by Kayak. The 5 mile trip takes 1-1.5 hours each way however, so bring water and plenty of sunscreen.