The NTCIR Fair Web Task

Last updated: 21st March 2024.
Twitter: @ntcirfairweb


Task Specifications

Introduction to FairWeb-1 (slides; updated Feb 2023)

Timeline (GMT+9 except where indicated)

blue: what participants will do
red: what organisers will do

October 3, 2022 Release of 1st CFP with sample topics and evaluation protocol [DONE]
February 15, 2023 Pilot relevance assessments for the sample topics and a few pilot runs released [DONE]
February 1-March 10, 2023 Topic development [DONE]
March 15, 2023 Topics released [DONE]
April 15, 2023 Task registrations due (Anywhere on Earth). Sign up from this NTCIR-17 page! [DONE]
May 22, 2023 Run submissions due [DONE]
May 23-July 31, 2023 Entity annotations; runs evaluated [DONE]
August 1, 2023 Evaluation results and draft overview released [DONE]
September 1, 2023 Draft participant papers due [DONE]
November 1, 2023 Camera ready papers due
December 12-15 2023 NTCIR-17@NII, Tokyo, Japan


Sijie Tao, Nuo Chen, Tetsuya Sakai, Zhumin Chu, Hiromi Arai, Ian Soboroff, Nicola Ferro, Maria Maistro: Overview of the NTCIR-17 FairWeb-1 Task, Proceedings of NTCIR-17, pp.284-305, 2023. pdf

Zhumin Chu, Tetsuya Sakai, Qingyao Ai, and Yiqun Liu: Chuweb21D: A Deduped English Document Collection for Web Search Tasks, Proceedings of ACM SIGIR-AP 2023, pp.63-72, 2023. pdf

Tetsuya Sakai, Jin Young Kim, and Inho Kang: A Versatile Framework for Evaluating Ranked Lists in terms of Group Fairness and Relevance, ACM TOIS, 2023. pdf

(In Japanese) 酒井哲也: グループフェアネスを考慮したウェブ検索タスク, 情報処理学会研究報告 2022-DBS-175/2022-IFAT-148, No.6, 2022. paper slides


TREC Fair Ranking Track


Email: fairweb1org at

Sijie Tao, Nuo Chen, Tetsuya Sakai (Waseda University, Japan)
Zhumin Chu (Tsinghua University, P.R.C.)
Nicola Ferro (University of Padua, Italy)
Maria Maistro (University of Copenhagen, Denmark)
Ian Soboroff (NIST, USA)
Hiromi Arai (RIKEN AIP, Japan)