The Context Aggregation and Analysis tool was developed by the MeVer team within the InVID project and continues to evolve within the WeVerify project. The tool aims to facilitate the verification of user-generated videos posted by three well-known platforms – YouTube, Facebook and Twitter. It collects information surrounding the video, analyses and filters it and creates a verification report that is then presented to the end user (journalist) who is responsible to inspect the verification cues and decide about the video veracity.

To evaluate the value of the tool for journalistic verification tasks, a user study was designed and organized in collaboration with Professor Andreas Veglis and Dr. Dimitrios Giomelakis of the Media and Journalism department of the Aristotle University of Thessaloniki. On December 12, the user study was launched during the Media Informatics Lab attended by 28 students. The students, 9 male and 19 female, are at the 3rd or 4th year of their studies. MeVer member Olga Papadopoulou and also lead developer of the CAA tool, made a presentation during the course, where she described the features and utility of the tool. Then, the students were asked to verify a set of fake and real videos and report the results using the CAA tool.

PRESENTATION

The user study has been split in three stages:

-20 videos (fake and real) were provided to the students before the course and they were asked to verify them following best practices that they have already studied. -During the course 10 videos (fake and real) were given to the students and they were asked to verify them using the CAA tool after the tool was presented to them. -After the course the students were asked to verify 30 more videos (fake and real) using the CAA tool.

The students were asked to report the results: -A label of fake or real indicating the credibility of the video. -The degree of certainty which quantifies how certain the student was about his/her label. -The list of verification cues that contributed the verification (During the first stage, where the CAA tool was not used, the verification cues were not reported). -A free text description of the analysis. -The time that the student needed to verify the video.

The study will run until January 13th, and then the collected data will be analyzed and reported.

The tool received good reviews and positive feedback from the students during its use and most of them showed interest and willingness to use it again. Olga Papadopoulou will visit the course on January 16, after the user study is completed, and she will have a discussion with the students in order to understand how useful the CAA tool has been to them and what needs to be improved to make it even more usable. The feedback will be taken into account in order to enhance the tool and release an improved version.