The poem opens with the speaker addressing a woman (presumably). Because of the title, we're guessing she's named Celia.
He tells her to "drink" to him "only" with her "eyes." In other words, he's telling her that she doesn't have to hold up a beer and say cheers, but only has to use her eyes. It's kind of like when you say, "I'll drink to the Bears winning the Super Bowl."
The speaker says that he, too, will "pledge" – i.e., "drink" or say "cheers" or something to that effect – with his eyes.