The NTCIR Fair Web Task

Last updated: 3rd October, 2022.
Twitter: @ntcirfairweb


Here are the four sample topics! (October 3, 2022)

Task Specifications

Introduction to FairWeb-1 (slides)


blue: what participants will do
red: what organisers will do

October 3, 2022 Release of 1st CFP with sample topics and evaluation protocol
December 16, 2022 Pilot relevance assessments for the sample topics and a few pilot runs released; topic set size determined
Dec 19-Feb 28, 2023 Topic development
March 1, 2023 Topics released; task registrations due
May 12, 2023 Run submissions due
May 15-July 31, 2023 Entity annotations; runs evaluated
August 1, 2023 Evaluation results and draft overview released
September 1, 2023 Draft participant papers due
November 1, 2023 Camera ready papers due
December 2023 NTCIR-17@NII, Tokyo, Japan


(In Japanese) 酒井哲也: グループフェアネスを考慮したウェブ検索タスク, 情報処理学会研究報告 2022-DBS-175/2022-IFAT-148, No.6, 2022. paper slides

Tetsuya Sakai, Jin Young Kim, and Inho Kang: A Versatile Framework for Evaluating Ranked Lists in terms of Group Fairness and Relevance, arXiv:2204.00280, 2022.


TREC Fair Ranking Track


Email: fairweb1org at

Sijie Tao, Nuo Chen, Tetsuya Sakai (Waseda University, Japan)
Zhumin Chu (Tsinghua University, P.R.C.)
Nicola Ferro (University of Padua, Italy)
Maria Maistro (University of Copenhagen, Denmark)
Ian Soboroff (NIST, USA)
Hiromi Arai (RIKEN AIP, Japan)