There were two instances when Caleb was sneaking around before Ann officially met him. If you remember, there was a scene at night time when Ann's dog was barking at the door and another when there was someone listening to Ann play the organ at the church. She thought it was John, but it was actually Caleb, and John was just out chopping wood. After Caleb comes to the farm, he admits to stealing their eggs in a conversation with Ann and John.