The E-M1 has a burlier appearance than the E-M5, gaining a more substantial hand grip and a profusion of buttons and direct controls. There's a button to access just about every imaginable feature, and of course, since it's an Olympus, it's also possible to re-configure many of these. The body is dust, splash, and freezeproof (to -10 °C/ 14 °F), although you'll need to use a similarly-sealed lens to benefit from this.
The E-M1's more pronounced hand grip allows the front and rear control dials to be better spaced-out (an occasional criticism of the smaller model), and the buttons are all larger and more positive than the E-M5's. The result is a camera that sits even more comfortably in the hand, especially if you have larger hands, and is easier to operate. The magnesium alloy construction means it's still pretty light, though.
Ultimately, despite its authentically semi-pro appearance, the E-M1 is still a pretty small camera compared to DSLRs, especially in terms of body thickness. The disadvantage the previous top-end E-series cameras had was that they ended up being as big as any other DSLR (the E-3 and E-5 were essentially the same size as full-frame DSLRs, at which point it became hard not to measure them up against those cameras). The E-M1 may be large for a Micro Four Thirds camera, but it's not big in the grand scheme of things. Its lenses tend to be smaller, too.