I2: 25% of social tags contained additional to authors’ information, 26% unnecessary information
and 49% no new information. Moreover, social tags were found to be more useful than formal
vocabulary terms and most of the end-users wanted to change the original metadata description to
adopt some of the social tags as indexing terms.
Another important study about social tagging has been conducted in the framework of the US-funded
project “STEVE: The Museum Social Tagging Project” (http://www.steve.museum/) (Trant, 2009a). More
specifically, the project developed a repository including around 97.000 digital resources of cultural heritage.
These resources were characterized with metadata by professional museum experts as well as with tags added
by the end-users of the repository (that is students and teachers). During the evaluation study of the project,
the STEVE Repository was including 36981 tags which had been added to 1792 digital cultural heritage
resources. This means that on average 20 social tags were added per cultural heritage resource. The main
issues that were investigated during the STEVE project evaluation study were the following:
I1: social tags correlation with museum metadata.
I2: usefulness of social tags as museum metadata descriptors of the digital cultural heritage
resources.
The evaluation results in respect to the aforementioned issues showed that:
I1: 86% of social tags didn’t match with museum metadata added by professional museum experts.
I2: 88% of social tags were considered useful by the professional museum experts
As we can notice from the aforementioned studies, there is a strong interest for investigating the added
value of social tagging on enlarging the metadata descriptions of digital educational resources, as well as the
formal vocabularies used in expert-based metadata. However, both issues have been investigated without
considering the possible implications of users’ tagging motivation to the enlargement of resulting tags and
folksonomies. Next, we aim to address this issue and we present our proposed evaluation methodology.
3. PROPOSED EVALUATION METHODOLOGY
In this section, we present our proposed methodology for identifying different types of users’ tagging
motivation and evaluating their possible influence to the metadata descriptions of digital educational
resources, as well as to the resulted folksonomy when it is compared to formal structured vocabularies used
for metadata authoring by metadata experts or content providers. More specifically, our methodology
includes the following steps:
Step 1 – Identify different underlying behaviours for users’ tagging: This step includes the
discrimination of the users based on their tagging motivation. For this purpose, we adopt two types
of user motivations proposed by Korner (2009):
o Categorizers, who are motivated by categorization and use tags to construct and maintain a
navigational aid to the resources they annotate. For this purpose, categorizers aim to
establish a stable and bounded vocabulary based on their personal preferences and
motivation.
o Describers, who are motivated by description aim to describe the resources they annotate
accurately and precisely. As a result, their tag vocabulary typically contains an open set of
tags.
In order to discriminate between categorizers and describers we adopt a set of measures proposed by
Korner et al. (2010):
o Tag/Resource Ratio: relates the vocabulary size of a user to the total number of digital
educational Resources tagged by this user. Describers, who use a variety of different tags
for their resources, score higher values for this measure than categorizers, who use fewer
tags.
o Orphaned Tag Ratio: characterizes the degree to which users produce orphaned tags (that is
tags assigned to few resources only, and therefore are used infrequently). The orphaned tag
ratio captures the percentage of tags in a user's vocabulary that represent such orphaned
tags. Categorizers vocabulary scores values closer to 0 because orphaned tags would
introduce noise to their personal vocabulary, whereas describers vocabulary scores values