Knowledge Bank

Structured Surveys

March 25, 2022, Evaluation Observatory


The kind of tools a researcher uses to conduct their surveys, is determined largely by the evaluative questions to be answered, and whether the mode of answering them is qualitative or quantitative. When it comes to Quantitative Research, Structured Surveys are generally preferred over other tool formats (Singh et al., 2017). A Structured Survey in essence is a questionnaire which has pre-planned and predefined questions (IGNOU, n.d., p. 11), and where interviewer collects data, usually face to face, using technologies like a computer or over telephone, rather than allowing the respondent to self-administer the interview (Singh et al., 2017). In resource or technically constrained settings, these interviews may also sometimes be administered manually, with pen and a printed form over paper. The interviewer generally reads the questions in exactly the same manner as they appear in the survey questionnaire.

Survey can have following types of questions (Singh et al., 2017),

  • Open-ended questions: These questions do not have pre-coded options.
  • Dichotomous questions: These questions have two possible answers like yes/no, true/false or agree/disagree.
  • Multiple-response questions: These questions have many probable answers.

In the case of Structured Surveys, questions are predominantly close-ended, which means that respondents have to choose answers from a fixed set of choices; however, may also include certain open-ended questions.

Once a decision to use a structured survey has been made, the following aspects need to be integrated into its design (Singh et al., 2017):

Mutual Exclusivity

Response items should be mutually exclusive for multiple response questions to ensure that there is no bias.

FIlter Items/ Skip Questions

A feature which allows non-applicable questions to be skipped.

Non Exhaustive Response Set

Including responses like 'not applicable' or 'don't know' for uninformed or neutral respondents, rather than trying to force responses.

Order of Questions

Questions should proceed from general and less sensitive items to more specific and sensitive items.

Reliability

It signifies the issues of consistency in measures - the ability of a measurement instrument to measure the same thing each time it is used.

Validity

It assesses the extent to which the concept measures the thing it was designed to measure

Cross-Check Items

This helps researchers to track data consistency in research questionnaires.

While making a decision on whether Structured Surveys are to be used or not; it is useful to be cognizant of the various limitations and advantages that go along with the use of such surveys.

Advantages include (Trueman, 2015):

  • Easy to interview large number of people in efficient manner.
  • Easy to design, code and interpret.
  • Reliable research method.
  • Sensitive topics can be researched effectively compared to other research instruments.

Disadvantages are:

  • It can be time consuming and expensive.
  • It is difficult to effectively examine complex topics using close ended questions.

List of recommended resources:

For in depth understanding

  • Written by Kultar Singh, Chapter 3 of this book guide researchers on research instruments with special attention to Structured Surveys – instruments, techniques and norms which need to be followed.

Case Study

Toolkits

  • Survey CTO is most commonly used questionnaire programming software in development research. It based on Open Data Kit based software. This link will guide you for the steps and best practices in Survey CTO programming. Also, this survey CTO guide and video tutorial will be gives comprehensive overview of the software.

  • KoBoToolbox is a free and open source software which allows to collect data in the field using mobile phones, tablets or computers. This toolbox is a how-to guide on using KoBoToolbox for one’s structured surveys. Also, use this video tutorial for a comprehensive overview of the software.

  • The Census and Survey Processing System (CSPro) is a public domain software package used by hundreds of organisations for entering, editing, tabulating, and disseminating survey data. It supports data collection on android devices (phones and tablets). This guide will help practitioners use this software, while this video tutorial gives a comprehensive overview of the same.

  • SurveyMonkey is a well-known online survey software that helps you create and run professional online surveys. It makes it easy to create a survey from scratch, or from a template, add logic, customise it to your liking, and then preview and test it, before sending it to your respondents. This guide will help practitioners on using this software; while this video tutorial is useful for a comprehensive overview.

  • Google forms are one of the easiest ways to create an online survey and it also provides options to create a variety of questions as per the needs of the survey. It is a free app which anyone can use to create a short survey, and even provides for data validation checks needed for simple quesitonnaires.

  • Developed by 60 decibels, this toolkit – which was prepared in response to the COVID 19 situation, provides tips and Cheat Sheets to conduct remote phone surveys. It also provides sample questions for the same based on the Lean Data methodology.

Further Reading

  • Developed by Acumen, This document is designed to serve as a field guide to help practitioners conduct Lean Data projects. It is a practical and action-oriented document to guide researchers on using lean data methods for survey research.

  • Published by UNDP Singapore Global Centre, this video provides brief overview of lean data approach with the help of examples.

For a broad overview

  • Developed by United States Government Accountability Office, this document guides practitioners on developing structured survey questions to collect data. It discusses aspects like writing appropriate questions for high-quality responses and deciding what type of questions to ask.

You may also like

Opinions

Big Data: An Introduction and Application in the Social Sector

August 30, 2021

Events

Building an Evaluation Ecosystem: Perspectives from Evaluation Associations - Key Takeaways from the Webinars

August 30, 2021

Themes

GENSA Celebrates One Year, and Keynote by Katherine Hay

August 30, 2021