The bus from Bangkok to Koh Samet leaves every hour from the Eastern Bus terminal (Ekkamai). You need to take the bus to Ban Phe which will drop you directly at the Nuanthip Pier from where you can take the ferry to Koh Samet. The buses are operated by Cherdchai tours and are comfortable and cheap.