Archived information for the tutorial which took place in ECDL2010, Glasgow, UK


During the last twenty years, the digital library domain has exhibited a significant growth aiming to fulfill the diverse information needs of heterogeneous user communities. Digital libraries, either existing as research prototype in a research center or laboratory, or operating in an intense environment enjoying actual usage from end users, have explicitly the need to measure and evaluate their operation. Digital library evaluation is a multifaceted domain aiming to compose the views and perspectives of various agents, such as digital library developers, librarians, curators, information and computer scientists. Several research fields, like information retrieval, human computer interaction, information seeking, user behavior analysis, organization and management of information systems, are contributing to capture, analyze and interpret data into useful suggestions of beneficial value for the information provider and its users.

This half-day tutorial will attempt to conclude the current state of the art on digital libraries evaluation focusing to the following critical questions that project managers, digital library developers and librarians face: the motivations forcing to evaluate, how these motivations are connected to methodologies, techniques and criteria, how effective is one methodology compared to another in relation to the context of operation, what are the appropriate personnel and resources, as well as the organizational and legal requirements for conducting an evaluation experiment and what are the expected derivatives.

The tutorial is divided in two sections, each of them counting ninety minutes duration. The first section is dedicated to outlining the digital library evaluation dimensions and approaches, the methodologies, criteria, metrics and measurement instruments employed in evaluation activities. Trough the enumeration of important projects and initiatives, the pros and cons of each approach will be sketched. The second part of the tutorial will start with a formal model presenting the dominant concepts of the evaluation research field, providing a conceptual synopsis of the issues discussed in the first part. Furthermore this section includes a practical session in which several indicative real-life tasks will be given to the participants and their response to the evaluation challenges will be discussed.

There are no specific requirements about the participants’ skills and knowledge. The attendees will be engaged in discussion groups, contributing to the overall quality of the tutorial.


Handouts (a few pdfs in a zip file)

Readings as a group in Mendeley

Tutors’ profile

Giannis Tsakonas holds a BSc in Librarianship and a PhD in Information Science from the Department of Archives and Library Sciences, Ionian University, Corfu, Greece. He is member of the User Support Department of the Library & Information Center, University of Patras, Greece. He has been actively involved in many national and European digital library projects, while he is national editor in Greece for E-LIS, the international subject repository on librarianship and information science. His research interests include digital library development in information contexts, such as the fields of academia and museums; user-centered digital library evaluation; information behavior; aspects of information services integration; and visual communication.

Christos Papatheodorou is an Associate Professor at the Department of Archives and Library Sciences, Ionian University, Corfu, Greece and a fellow researcher in the Digital Curation Unit, Institute for the Management of Information Systems, “Athena” Research Centre, Athens, Greece.  He holds a BSc and a PhD in Computer Science from the Department of Informatics, Athens University of Economics and Business, Greece. His research interests include digital library evaluation, metadata interoperability, Web 2.0 technologies, user modeling and personalization. He has participated in the program committee of various international conferences and workshops and was general co-chair of the 13th European Conference on Digital Libraries.

They are both editors of the book “Evaluation of Digital Libraries: An Insight to Useful Applications and Methods”, published by Woodhead/Chandos Publishing House.