January 11, 2022, Evaluation Observatory
Suppose a teacher wants to measure the impact of their teaching techniques on students' learning outcomes. A simple way of measuring the impact would be as follows:
The teacher will conduct a test for students before teaching the class. After which, they will teach the class using a particular technique, and take another test for the same class with a similar level of difficulty. Finally, the teacher analyses the difference in score of these two tests to measure the impact of their teaching.
The evaluation design used here is referred to as Pre-Post Test Design, and it is an experiment in which outcome or impact indicator measurements are taken before and after the treatment is administered (Zach, 2020) using a standard survey pre and post the intervention. (EMpower, n.d.).
Pre-Post Test Designs are flexible; and can be used across non-experimental, experimental and quasi-experimental research settings. While it has a treatment group, it may or may not include a control group (Zach, 2020). Although, it is especially used in non-experimental settings within the same treatment group when either the interventions are universally applicable, or too complex to generate a matched comparison group (Kultar et al., 2017).
In the presence of a control group, Pre-Post test designs have a strong internal validity but face challenges in achieving external validity (Shuttleworth, 2009). However, when done in a non-experimental setting, since there is no baseline measurement against groups that remained completely untreated, it becomes challenging to find whether the pre-testing process influenced the results (Shuttleworth, 2009), thus weakening internal validity. For example, children giving a pre-test may try a harder in their lectures and try to outperform children who were not given a pre-test; thus making the test results an outcome of the experimental process, rather than administering the treatment alone (Shuttleworth, 2009).
Advantages of Pre-Post Test Design (meera, n.d.):
Disadvantages of Pre-Post Test Design (meera, n.d.):
Written by Martin Shuttleworth, this article gives a simple overview of the concept and discusses the pros and cons of implementing pre-post test designs.
Written by Jessica L. O'Leary and Glenn D. Israel and published by the University of Florida, this provides the introduction of the Prestest-Posttest Model and discusses advantages and limitations of the model.
Developed by EMpower, this document aims to help programs avoid some of the most common errors in pre-and post-evaluation and provide tips for analysis and reporting results.
Published by Statology, this web article defines Pre-Post Test Design and illustrates the process of using this design with the help of examples.
Written by Dimiter, Dimitrov and Phillip D. Rumrill Jr., this article examines issues in comparing groups and measuring change with pre-test and post-test data. Furthermore, it presents designs that can help rehabilitation professionals better understand and determine the effects of select interventions.
Written by Peter L. Bonate, this book has an in-depth discussion on various aspects of Pre-Post Test Designs, including data analysis, measurement concepts, difference scores, analysis of covariance, statistical tests, etc.
Published by SAGE Publications, this case study demonstrates the evaluation of recreational therapy program for people with dementia using paired t-test, one-sample t-test and binomial test.
Published by National Journal of Physiology, Pharmacy and Pharmacology, this article evaluates the knowledge of didactic lecture among students by giving pre and post-test questionnaire-based evaluation techniques.
Written by Tayyaba Gul Malik and Rabail Alam, this case study discusses the effectiveness of the Pre-Post Test in comparison with a post-test-only model in determining the learning outcomes of a 45 minute lecture.
Published by Real Statistics, this post guides practitioners on using excel for Pre-Post Test design. It explains on taking a T-test, ANOVA and ANCOVA approaches to Pre Test-Post Test design.
Guidance For Pre- and Post-Test Design. (n.d.). EMpowerweb. Retrieved December 26, 2021, from https://empowerweb.org/assets/uploads/tools-resources/461/5._guidance_for_pre-_and_post-test_design.pdf
Martyn Shuttleworth (Nov 3, 2009). Pretest-Posttest Designs. Retrieved Jan 05, 2022 from Explorable.com: https://explorable.com/pretest-posttest-designs Singh K., Chandurkar D., & Dutt V. (2017). A Practitioners’ Manual on Monitoring and Evaluation of Development Projects83. Cambridge Scholar Publishing.
Types of Evaluation Designs. (n.d.). meera. Retrieved December 26, 2021, from https://meera.snre.u-mich.edu/types-evaluation-designs
Zach. (2020). Pretest-Posttest Design: Definition & Examples. Statology. https://www.statology.org/-pretest-posttest-design/
Big Data: An Introduction and Application in the Social Sector
August 30, 2021
Building an Evaluation Ecosystem: Perspectives from Evaluation Associations - Key Takeaways from the Webinars
August 30, 2021
GENSA Celebrates One Year, and Keynote by Katherine Hay
August 30, 2021