Last updated: December 2, 2021.
Twitter: @ntcirdialeval

Updates

We have released the test data to the participants!
The DialEval-2 task details page has been released!
The NTCIR-16 DialEval-2 task definition (unchanged from previous rounds) is here !
The Dialogue Evaluation (DialEval-2) Task has been accepted by the NTCIR-16 Programme Committee! (January 16, 2021)

Task Details

Details of the DialEval-2 task, including the information about training data, can be found here.

April 2021	Task registrations open: training data given to participants
April-May 2021	Annotating the test dialogues
December 1 2021	Test dialogues released/Task registrations due
January 15 2022	Run submissions due
February 1, 2022	Evaluation results released
February 1, 2022	Draft task overview paper released
March 1, 2022	Draft participant paper submissions due
May 1, 2022	All camera-ready paper submissions due
June 14-17, 2022	NTCIR-16 Conference in NII, Tokyo, Japan

dialeval2org@list.waseda.jp

Tetsuya Sakai (Waseda University, Japan)
Sijie Tao (Waseda University)
Inho Kang (Naver Corporation)

Zhaohao Zeng, Sosuke Kato, Tetsuya Sakai, and Inho Kang: Overview of the NTCIR-15 Dialogue Evaluation (DialEval-1) Task, Proceedings of NTCIR-15, pp.13-34, 2020. pdf
Zhaohao Zeng, Sosuke Kato, and Tetsuya Sakai: Overview of the NTCIR-14 Short Text Conversation Task: Dialogue Quality and Nugget Detection Subtasks, Proceedings of NTCIR-14, pp.289-315, 2019. pdf
Tetsuya Sakai: Comparing Two Binned Probability Distributions for Information Access Evaluation, Proceedings of ACM SIGIR 2018, pp.1073-1076. preprint pdf