University of Tsukuba | Grad. Scho. Syst. and Info. Eng. | Dept. Comp. Sci. | List of Courses
Experiment Design in Computer Science_E
Claus Aranha, Tetsuya Sakurai
Office hours Office Hours by appointment through MANABA
Cource# 01CH740
Area Common Subject
Basic/Advanced 基礎科目
Course style Lectures
Term Spring AB
Period Friday 5,6
Room# 3B301
Keywords Experiment Design, Statistical Testing, Statistics, Philosophy of Science, Ethics, R Language

No particular requirements for registration.

This course expect students to be familiar with basic statistical concepts (Random Variables, Distributions, etc). The experiments in the course will use the R language, so familiarity with this language is helpful.

This course expect students to be familiar with basic concepts of statistics (Mean, Median, Standard deviation, Population, Random Variables, etc). We also encourage the students to study the R programming language to prepare for this course.

relation degree program competence Knowledge Utilization Skills,Management Skills,Teamwork Skills,International Skills,Research Skills,Ethics

The collection and analysis of data through experiments is one of the cornerstones of the scientific method. In this course, we study the general philosophy and methods behind experimentalism: Why do we perform experiments, what is a good/rigorous experiment, how to plan and design a rigorous experiment, and how to perform statistical analysis on experimental data.

This course is centered around lectures with plenty of examples and study cases. The students will be invited to apply the techniques studied in this lecture to experiment of their own design.

Course plan
  • Class 1: Introduction - What is science, and what is a scientific experiment? What is rigorous research?
  • Class 2: Statistics review - Point Indicators and Interval Indicators (Random Variables, Means, Variances, Distributions, Confidence Intervals)
  • Class 3: Introduction to Inference Testing -- Hypothesis, Type Errors, Z testing.
  • Class 4: Introduction to Inference Testing -- Comparison Testing, Paired Testing
  • Class 5: Introduction to Inference Testing -- Equality Testing, Non-Parametric Testing
  • Class 6: Lecture Review and Case Study
  • Class 7: Power Analysis, Sample Size and Sample Choosing
  • Class 8: Multiple Comparison: Anova and post-hoc testing
  • Class 9: Blocking and Parameter Choice
  • Class 10: Lecture Review and Case Study
Textbook No designated textbook: The course will be based on lecture notes to be distributed electronically on Manaba
  • D.C. Montgomery, "Design and Analysis of Experiments", Wiley, 2005,
  • Felipe Campelo "Lecture Notes on Design and Analysis of Experiments", Online, 2018,
  • Other recommended readings on MANABA

The grade is based on three individual reports.

Each report is a case study, where the student will have to: Design an experiment to answer a scientific question, obtain experimental data (from the instructor, or by performing an experiment), analyze the experimental data using the tools studied in the lecture, and prepare a report discussing the experimental conclusions.

The report is evaluated on the quality of the experiment design, the correctness of the statistical analysis, and the quality of the discussion of the results (note that positive or negative results do not factor in the grade of the report).

  • The first report is due before lecture 3, and worth 30% of the grade.
  • The second report is due before lecture 7, and worth 30% of the grade.
  • The third report is due by the date of the final exam and worth 40% of the grade.
TF / TA 1 TA is assigned for office hours.

Lecture in English

I expect the students to contribute with questions and discussions during the class. Additionally, I will recommend that the students use data from their own research to prepare the reports. Students who cannot provide their own data will be provided with sample data, however you will learn much more if you can use your own data.