Sakai Lab: Tools

last updated: 18th September 2017

Back to The Sakai Lab       sakailab.com (in Japanese)  Dr. Tetsuya Sakai


samplesizeANOVA2.xlsx (Tetsuya Sakai) for computing the topic set size required based on one-way ANOVA (for m>=2 systems)

samplesizeTTEST2.xlsx (Tetsuya Sakai) for computing the topic set size required based on the paired t-test (for m=2 systems)

samplesizeCI2.xlsx (Tetsuya Sakai) for computing the topic set size required for achieving a tight confidence interval

These tools are easier to use than the earlier versions (no need to scroll down the sheets).

(Sakai, T.: Laboratory Experiments in Information Retrieval: Sample Sizes, Effect Sizes, and Statistical Power, to appear, Springer)


SIGIR2017PACK (Tetsuya Sakai) for paired and unpaired Bayesian tests

(Sakai, T.: The Probability That Your Hypothesis Is Correct, Credible Intervals, and Effect Sizes for IR Evaluation, ACM SIGIR 2017)


samplesizeTTEST.xlsx (Tetsuya Sakai): for computing the topic set size required based on the paired t-test (for m=2 systems)

samplesizeANOVA.xlsx (Tetsuya Sakai): for computing the topic set size required based on one-way ANOVA (for m>=2 systems)

(Sakai, T.: Designing Test Collections for Comparing Many Systems, ACM CIKM 2014)


samplesizeCI.xlsx (Tetsuya Sakai): for computing the topic set size required for achieving a tight confidence interval

(Sakai, T.: Designing Test Collections That Provide Tight Confidence Intervals, FIT 2014)


BOOTS (Tetsuya Sakai): for conducting paired bootstrap hypothesis tests

Discpower (Tetsuya Sakai): for computing discriminative power using the randomised Tukey HSD test

NTCIREVAL (Tetsuya Sakai): for computing various IR measures

NTCIRPOOL (Tetsuya Sakai): for making pool files for relevance assessments


Back to The Sakai Lab       sakailab.com (in Japanese)  Dr. Tetsuya Sakai