We propose new representation based on skeleton joints,namely a local skeletal descriptor and an associated representation that. This skeletal descriptor, which is referred to as skeletal quad, was inspired by a geometry hashing method that describes the positions of nearby stars in night sky images [6]. It was also used to describe keypoint constellations in video frames [4]. Our action descriptor is a Fisher kernel representation which encodes the Jacobian of the probability function that generates the skeletal quads contained in a depthimage sequence.