Knowledge Bank

Pre-Post Test Design

January 11, 2022, Evaluation Observatory

Suppose a teacher wants to measure the impact of their teaching techniques on students' learning outcomes. A simple way of measuring the impact would be as follows:

The teacher will conduct a test for students before teaching the class. After which, they will teach the class using a particular technique, and take another test for the same class with a similar level of difficulty. Finally, the teacher analyses the difference in score of these two tests to measure the impact of their teaching.

The evaluation design used here is referred to as Pre-Post Test Design, and it is an experiment in which outcome or impact indicator measurements are taken before and after the treatment is administered (Zach, 2020) using a standard survey pre and post the intervention. (EMpower, n.d.).

• 

Pre-Post Test Designs are flexible; and can be used across non-experimental, experimental and quasi-experimental research settings. While it has a treatment group, it may or may not include a control group (Zach, 2020). Although, it is especially used in non-experimental settings within the same treatment group when either the interventions are universally applicable, or too complex to generate a matched comparison group (Kultar et al., 2017).

In the presence of a control group, Pre-Post test designs have a strong internal validity but face challenges in achieving external validity (Shuttleworth, 2009). However, when done in a non-experimental setting, since there is no baseline measurement against groups that remained completely untreated, it becomes challenging to find whether the pre-testing process influenced the results (Shuttleworth, 2009), thus weakening internal validity. For example, children giving a pre-test may try a harder in their lectures and try to outperform children who were not given a pre-test; thus making the test results an outcome of the experimental process, rather than administering the treatment alone (Shuttleworth, 2009).

Advantages of Pre-Post Test Design (meera, n.d.):

• Pre-post-test design are relatively simple to implement
• It controls for variables that may influence outcomes prior to administering the intervention
• Compared to retrospective pre and post-tests, or post-tests, they deliver better evidence for effectiveness of the intervention

Disadvantages of Pre-Post Test Design (meera, n.d.):

• In the absence of a control group, it cannot account for confounding variables or non-programmatic influences on outcome indicators.
• When self-reporting measures are used, in the absence of a control group, it may lead to over-estimation of a particular indicator at baseline (due to lack of knowledge) and accurate estimation at endline (after undergoing treatment); thus, creating erroneous conclusions.

For a broad overview

• 
• Blog Post: Pretest-Posttest Designs

  Written by Martin Shuttleworth, this article gives a simple overview of the concept and discusses the pros and cons of implementing pre-post test designs.

• 
• Working paper: Capturing Change: Comparing Pretest-Posttest and Retrospective Evaluation Methods

  Written by Jessica L. O'Leary and Glenn D. Israel and published by the University of Florida, this provides the introduction of the Prestest-Posttest Model and discusses advantages and limitations of the model.

• 
• Guidance Document: Guidance For Pre- and Post-Test Design

  Developed by EMpower, this document aims to help programs avoid some of the most common errors in pre-and post-evaluation and provide tips for analysis and reporting results.

• 
• Blog Post: Pretest-Posttest Design: Definition & Examples

  Published by Statology, this web article defines Pre-Post Test Design and illustrates the process of using this design with the help of examples.

• 
• Working paper: Pretest-Posttest Designs and Measurement Of Change

  Written by Dimiter, Dimitrov and Phillip D. Rumrill Jr., this article examines issues in comparing groups and measuring change with pre-test and post-test data. Furthermore, it presents designs that can help rehabilitation professionals better understand and determine the effects of select interventions.

For in depth understanding

• 
• Book: Analysis of Pretest-Posttest Designs

  Written by Peter L. Bonate, this book has an in-depth discussion on various aspects of Pre-Post Test Designs, including data analysis, measurement concepts, difference scores, analysis of covariance, statistical tests, etc.

Case Study

• 
• Health: Analysing The Data with Pre-Test and Post-Test Measurement of One Group

  Published by SAGE Publications, this case study demonstrates the evaluation of recreational therapy program for people with dementia using paired t-test, one-sample t-test and binomial test.

• 
• Education: Evaluating the Effectiveness of Pre- and Post-Test Model of Learning in a Medical School

  Published by National Journal of Physiology, Pharmacy and Pharmacology, this article evaluates the knowledge of didactic lecture among students by giving pre and post-test questionnaire-based evaluation techniques.

• 
• Education: Comparative Analysis Between Pre-test/Post-test Model and Post-test-only Model in Achieving the Learning Outcomes

  Written by Tayyaba Gul Malik and Rabail Alam, this case study discusses the effectiveness of the Pre-Post Test in comparison with a post-test-only model in determining the learning outcomes of a 45 minute lecture.

Toolkits

• 
• Blog Post: Pretest-Posttest Design

  Published by Real Statistics, this post guides practitioners on using excel for Pre-Post Test design. It explains on taking a T-test, ANOVA and ANCOVA approaches to Pre Test-Post Test design.