2024-03-19T07:53:45Z
https://nagoya.repo.nii.ac.jp/oai
oai:nagoya.repo.nii.ac.jp:00007110
2023-01-16T04:50:48Z
309:318:319
An Application of the NTCIR-WEB Raw-data Archive Dataset for User Experiments
TAKAKU, Masao
EGUSA, Yuka
SAITO, Hitomi
TERAI, Hitoshi
寺井, 仁
open access
Evaluation
User experiments
User interface
Rawdata archive
Web information retrieval
This paper presents a simple approach to utilize past test collections as a material for user experiments. We have built a Web-based user interface for NTCIR-5 WEB run results, and conducted a user experiment with 29 subjects to investigate whether performance evaluation metrics of information retrieval systems used in test collections such as TREC and NTCIR comparable to user performance. In this experiment, we selected three types of systems from among systems that participated in NTCIR-5 WEB, and then selected three topics with roughly the same values from among several search topics. The results of the experiment showed no significant differences among these systems and topics in the time for search. While, in general, the user experiment itself have been successfully conducted and shown similar trends with prior study, the approach seems to have some limitations mainly on interactivity and cached page display.
National Institute of Informatics (NII)
2007
eng
journal article
VoR
http://hdl.handle.net/2237/8815
https://nagoya.repo.nii.ac.jp/records/7110
http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings6/EVIA/14.pdf
Proceedings of the First International Workshop on Evaluating Information Access (EVIA 2007) (NTCIR-6 Pre-Meeting Workshop)
78
81
https://nagoya.repo.nii.ac.jp/record/7110/files/EVIA.pdf
application/pdf
164.7 kB
2018-02-19