We have presented GyroPen, a method that reconstructs a
user’s writing path from the inertial sensors of a phone
and have shown that this is a promosing approach toward
handwriting with the phone. We describe the approaches used
for high-accuracy writing-path reconstruction and show that
the reconstructed writing paths are accurate enough to be
recognized by an off-the-shelf handwriting recognition engine
without the need for special tuning. In a first proof-of-concept
experiment the majority of the participants reacted positively
to the approach and could use it after a learning period of
just a few minutes. In a second experiment we observed that
the method can be used to write all letters with acceptable
accuracy after some practice. We have shown that the system
is an interesting prototype aiming to enable handwriting with
the phone rather than on the phone for an intuitive experience
for text entry into mobile phones which will open unique
applications in the future.
GyroPen could be improved by explicitly handling penup
strokes, for instance using a sensor-fusion approach of
gyroscopes and accelerometers or by training a handwriting
recognition system that is fully invariant to pen-up strokes.
The proposed method has the advantage that it works on
smart phones without any modification to the hardware. If
it was possible to add additional sensors to the phones, a
similar user experience could be obtained e.g. by building a
small trackball or an optical mouse sensor into the phones
writing corner. Another alternative to using the inertial sensors
would be to use computer vision techniques with the builtin
camera to reconstruct the phone movements similar to
TinyMotion [31]. This would also work without additional
sensors but might be problematic if the writing surface is very
homogeneous and thus there would be no features to track