The NTCIR Fair Web Task


The NTCIR-18 FairWeb-2 Task


Updates

FairWeb-2 task slide deck (June 13, 2024)


Subtasks

web search (as in FairWeb-1, but featuring a reproducibility challenge)
conversational search (NEW!)


Important Dates (mostly tentative)

Nov 1, 2024: test topics released; task registrations due
Dec 15, 2024: run submissions due
Dec 2024-Jan 2025: Entity annotation + evaluation
Feb 1, 2025: Evaluation results + draft overview released
June 10-13, 2025: NTCIR-18@NII, Tokyo


Organisers

fairweb2org
at list.waseda.jp

Maria Maistro (University of Copenhagen, Denmark)
Tetsuya Sakai (Waseda University, Japan)
Sijie Tao (Waseda University, Japan)
Junjie Wang (Waseda University, Japan)
Hanpei Fang (Waseda University, Japan)
Yuxiang Zhang (Waseda University, Japan)
Nuo Chen (The Hong Kong Polytechnic University, China)
Haitao Li (Tsinghua University, China)
Yiteng Tu (Tsinghua University, China)


Papers

Tetsuya Sakai: Fairness-based Evaluation of Conversational Search: A Pilot Study, Proceedings of EVIA 2023, pp.5-13, 2023. paper slides YouTube video (from around 27:20)

Sijie Tao, Nuo Chen, Tetsuya Sakai, Zhumin Chu, Hiromi Arai, Ian Soboroff, Nicola Ferro, Maria Maistro: Overview of the NTCIR-17 FairWeb-1 Task, Proceedings of NTCIR-17, pp.284-305, 2023. pdf

Yiteng Tu, Haitao Li, Zhumin Chu, Qingyao Ai, and Yiqun Liu: THUIR at the NTCIR-17 FairWeb-1 Task: An Initial Exploration of the Relationship Between Relevance and Fairness, Proceedings of NTCIR-17, pp.319-324, 2023. pdf

Zhumin Chu, Tetsuya Sakai, Qingyao Ai, and Yiqun Liu: Chuweb21D: A Deduped English Document Collection for Web Search Tasks, Proceedings of ACM SIGIR-AP 2023, pp.63-72, 2023. pdf

Tetsuya Sakai, Jin Young Kim, and Inho Kang: A Versatile Framework for Evaluating Ranked Lists in terms of Group Fairness and Relevance, ACM TOIS, 2023. open access pdf


Links

NTCIR-17 FairWeb-1
NTCIR