The ability to track skeletal movements is important in a variety of applications including animation, biological studies and animal experiments. To detect even small movements, such a method should provide highly accurate estimates. Besides that it should not impede the mammal in its motion. This motivates the usage of a passive optical motion capture system. Thereby the main challenges are the initialization, the association of the unlabeled markers to their corresponding segment also across the frames, and the estimation of the skeleton configuration. While many existing approaches can deal with the latter two problems, they typically need a specific pose for initialization. This is rather unpractical in the context of animal tracking and often requires a manual initialization process. In this paper, we present an approach to reliably track animals and humans in marker-based optical motion capture systems with freely attached markers. Our method is also able to perform an automatic initialization without any pre-or post-processing of the data. To achieve this, our approach utilizes a large database of previously observed poses. We present our algorithm and its evaluation on real-world data sets with an animal and humans. The results demonstrate that our initialization method performs accurately for the most kind of initial poses and our tracking approach outperforms a popular fully automatic skeleton tracking method especially with respect to the smoothness of the motion.