In a lot of cases of “summoning to a different world” in novels or manga, you could be summoned, but not returned, so the summoned person was forced to live in the different world against his will, but seemingly the reality Zenjirou confronted here wasn’t that unreasonable.