Assessment and Evaluation (WP6)
The goal of this work package is to assess the technical success of the project in terms of:
- User satisfaction
- Technical adequacy
Of course assessment of the economic and dissemination impact is equally important. However, given the specific nature of these measures we thought they would better fit in WP7 (Business Activities).
User satisfaction:
This part of the work is mainly devoted to the evaluation of the integration between the web GUI and the underlying system. The baseline is represented by the current measure of 40% of queries duplicated in two languages for the partner libraries. Such a percentage should decrease to 20% after the first prototype will be deployed and it should further decrease to 10% on the fully deployed system.
Technical adequacy:
This measure will describe both the capability of the system to perform intelligent multilingual expansion and the quality of the translation lexicons (thus, indirectly their adaptivity and their capability to self maintain). Measures will be performed by using a set of queries and predefined human-selected answers, according to standard international methodologies such as TREC/CLEF. Concerning expansion, we expect an increase of 20% of the F-measure (precision+recall) for the prototype and of 40 % for the deployed system. As for multilingual coverage it should pass from a baseline of 50% provided by standard translation dictionaries to a top of 80% thanks to expansion and user based translation strategies.
Work package leader:
Kornik Library - Polish Academy of Sciences
Deliverables:
- D 6.1. First validation report (technical adequacy): M15 [public report]
- D 6.2. Second validation report (technical adequacy + user satisfaction): M21 [public report]
- D 6.3. Final validation report (technical adequacy + user satisfaction): M24 [public report]
• 
