Look through the window" definitely implies there is glass to look through. "Look out the window" could be either with or without glass. But I would say the main difference is that "look out the window" is American English; in the UK, we would tend to say either "look through the window" or "look out of the window", although the two are subtly different.