Also, before the scene at the waterfall Loomis asks, "What about Anson?" To which Caleb responds "Like you said, there's probably nothing there" indicating that he has no plans on leaving. After Caleb disappears, Loomis tells Anne that he in fact left for Anson. I doubt Caleb would change his mind just moments afterward, especially without saying goodbye to Anne.