The Estimation of Ground Truth from Multiple Information Sources
Ladislav Beránek, Václav Nýdl, Radim Remeš

Language: cs
Last modified: 2013-11-11


At the present time, it is possible to use different applications on Internet infrastructure (for example social networks) to obtain specific data from users. Users can also perform evaluations of certain products or services, classification of phenomena or objects, and more. These pieces of information obtained from users may be further processed and integrated. Results are then usable in marketing, science and so on. On the other side, it is difficult to assess the reliability of individual users engaged in these activities. In this paper, we present a model based on belief function theory which reduces the impact of unreliable or roguish users on the overall result. This model is based on the integration of pieces of information from individual users and their eventual discounting, whereby contributions of users very different from the others are reduced. This approach allows to determine a ground truth and to predict answers of users when the new data. Experimental results show that this approach has advantages over ”taking the average” baseline and some state-of-art models.


crowdsourcing, multiple observers, adversarial users, belief function discounting

Full Text: PDF