III. PROPOSED APPROACH
The proposed approach aims to develop a strategy to share
the autonomous perception, locomotion and cognition capabilities
of robots and human operators towards high–fidelity
3D inspection of complex structures in their environment.
To address this challenge, we employ recent breakthroughs
in the field of Augmented Reality and combine them with
cutting–edge technologies in the field of aerial robotics. In
particular, we consider aerial robotic systems capable of real–
time self localization and mapping of their environment that
are further empowered by path–planning algorithms which
compute optimized inspection paths for any prior model of
the environment that is available. The derived inspection
solution corresponds to an initial path for the robot, the prior
of how the inspection mission should take place if the real
environment were identical to its geometrical model and each
surface subset of it held equal importantance. However, as the
real environment to be inspected may differ from the available
model, as important information (e.g. human–readable
indications or semantically–important texture) might not be
encoded into its a priori known geometric model, and since
it is possible that only the human operator may be able to
know where to focus the inspection mission, the Augmented
Reality interface aims to allow the operator to adjust the
viewpoints followed by the robot. To achieve this goal effi-
ciently, the AR–interface shares part of the robot cognition to
the human by projecting synthesized stereo images that fuse
the camera frames with the real–time derived 3D map of the
environment. This on one hand enables the human operator
to know both what the robot perceives as well as how well
the environment is mapped, while on the other also allowing
to derive conclusions about which part of the environment is
not inspected at all or has to be further mapped and inspected
in more detail. The operator can then employ head motion
to intuitively guide the robot through a head motion–based
teleoperation controller that adjusts the reference waypoint of
the robot. Within the next sections, the employed structural
inspection planner is summarized, followed by a description
of the Augmented Reality interface, the cognition sharing
scheme, and the control structure.