3 Web-based questionnaires

3.1 Web-based questionnaire design

• Web-based survey questionnaire design is NOT web design.

• Web-based survey questionnaire design does not solve all problems related to the web surveys; Web-based questionnaire design can decrease or increase parts of measurement, nonresponse and coverage error.

• Web-based surveys assume that respondents can read. Such an assumption may exclude populations where literacy is a problem: according to the National Adult Literacy Survey, nearly 25% of US adult population has extremely limited literacy in English. About 20% of that category have visual difficulties that interfere with their ability to read printed material; almost 25% are immigrants.

• Web-based surveys assume that respondents have access to a computer, which may exclude economically disadvantaged populations.

• Web-based surveys require that responses be entered electronically, which may exclude physically disabled populations.

• Although few substantive differences between computer-based and paper-based surveys have been found, those differences that have been found often reflect changes in the design of the instrument. Most of these changes involve apparently innocuous modification of the layout; still, these changes may make it harder or easier to complete the questions, and they may convey different expectations about the kinds of information required.

3.2 Question formatting

3.2.1 Use of color

• Take advantage of graphic design theory: A proper use of color could provide visual cues that may simplify the web-based survey process.

3.2.2 Use of Multimedia Features

• Take advantage of the media's presentation capabilities: Web-based surveys could include hypertext links to help and additional resources as part of the survey instrument without increasing its apparent length.

• Be aware of how respondents may interpret questions in light of accompanying graphics: When graphics or pictures are used, bear in mind that respondents tend to interpret questions in the context of those graphics and pictures. [Example]

• Always ask whether the multimedia material is an enhancement that solves an information problem or whether it is a distraction ("eye candy").

3.2.3 Choice of Response Format

• Use matrix questions sparingly: Matrix questions place an extra burden on the respondent because they require a lot of work to be done within a single screen. In addition, it is impossible to predict exactly how a matrix question will look on a respondent's web browser. Therefore, they should be used with caution.
  • Example: Simple Matrix
  • Example: Complex Matrix

  3.2.3.1 Radio button

  • The size does not change even when font size changes.

  • Require precision in clicking.

  • Appropriate for "select only one" from mutually exclusive items.

  • Avoid default-filled radio buttons, since they may be misunderstood as an answer when respondents do not choose any answer.

  3.2.3.2 Check boxes

  • When there are too many options, use a matrix format.
    • Example: Simple Matrix

  • Use "check all that apply" only when necessary, since respondents tend to merely satisfy their task rather than optimizing.

  • When "none of above" is needed, provide it with a radio button. Avoid erroneous check on this choice with the chosen responses.

  3.2.3.3 Drop-down boxes

  • Use sparingly: they are best for a very long list. (eg. State/country of residence, prescription drugs, etc)

  • Drop-down boxes require 3 mouse actions whereas other response formats require only one. Thus, use drop-down format only when the increased mouse action is worthwhile.

  • Not appropriate for items where typing is quicker (eg. Year of birth)

  • Permit type-ahead look up, since it prevents tedious scrolling.

  • Do not make the first option visible in drop-down boxes: it may mislead the respondents and it may be misunderstood as an answer when respondents do not choose any answer.

  • Give visual cues for the task in drop-down boxes. (eg. "select one")

  • Avoid multiple selections on drop-down boxes: use check boxes.

  • Depending on the place on the screen, use drop-up boxes.

  3.2.3.4 Text input

  • The Size of the box should meet the amount of the required information.

  • Provide sufficient space for text input.

  • Provide concise and clear input guidance. (eg. MM/DD/YYYY for the birth date)

  • Avoid scrolling around the input box. If necessary, keep it vertical.

3.2.4 Response Categories

• Preprogram to check the error in input (eg. the input do not sum to 100%; check some response options and "none of above" option) and provide feedback if erroneous input is caught.

• Implement category randomization when needed.

• Provide an open-ended field with the "others" option.

• When order matters (5-point scale), pay attention to the response labels and spacing.

• Reduce response errors by restricting response choices: A predefined response format is helpful to achieve uniformity of data, which will reduce the workload in data cleaning and processing. However, a flexible format may be more respondent-friendly. It is suggested to use radio buttons when the number of choices is relatively small, and drop boxes when the number of potential choices is large. [Example]

• Force answers only on rare occasions: Forcing respondents to answer questions should be used only on rare occasions because the respondent may become annoyed and give an arbitrary or deliberately false answer in order to proceed to the next screen or stop taking the survey altogether.

3.2.5 Automation

• Automatically validate input, if possible: Input validation improves data quality and saves time in data preparation. However, such validation should be user friendly and simply identify the mistake of the user. [Example]

3.3 Questionnaire layout

3.3.1 Use of color

• Take advantage of graphic design theory: A proper use of color could provide visual cues that may simplify the web-based survey process.

3.3.2 Use of Multimedia Features

• Take advantage of the media's presentation capabilities: Web-based surveys could include hypertext links to help and additional resources as part of the survey instrument without increasing its apparent length.

• Use graphics sparingly: In a web-based survey, graphics can significantly slow the downloading of a web page, especially if users are likely to connect to the Internet using a modem as opposed to a high-speed connection.

• Be aware of how respondents may interpret questions in light of accompanying graphics: When graphics or pictures are used, bear in mind that respondents tend to interpret questions in the context of those graphics and pictures.
  • Example: Graphics

• Always ask whether the multimedia material is an enhancement that solves an information problem or whether it is a distraction ("eye candy").

3.3.3 Overall structure

• Place logo on the top left and menus on the left vertically.

• Left alignment of the questions and answers.

• Place the response format on the left of each response categories.

• Vertical alignment has more white space thus it is visually appealing and results in less problems with alignment and formatting

• Be cautious about the sense of scale (eg. column space for Likert scale) in horizontal alignment.

• Keep the answer labels visible for the grid design.

• Predict the visual flow and align the questions, answers and proceeding buttons accordingly.

• Decrease the movement time by decreasing the distance between the selected response check box (or radio button, etc) and the 'Next/Submit' button.

• Present questions in hierarchically ordered structure.

• Provide cognitive comforts; respondents do not need to go back and forth in their memory to retrieve.

• Preprogram the skip pattern (branching).

• Decrease item nonresponse error

• Decisions about whether to display questions on the same page together or on separate screens can impact the data collected.

• A group of questions considered together may convey information about the purpose of the questions and may elicit different responses than if the questions were presented one at a time across multiple pages.

• Designers can help guide the user's mental model by providing appropriate question context. Such context may be conveyed in may ways, including question wording, response wording, introductory explanation, or as above, may be implied by surrounding questions.

• Consider several design options: all questions in one page, related questions grouped on the same page, or one question per page.

• The grouping of response options may impact the responses recorded.

• If it is significant that responses or questions fall into distinct groups, call attention to the fact in some way, for example grouping or labeling.

• List a few questions per screen: Excessive scrolling can become a burden to respondents and lengthy web pages can give the impression that the survey is too long to complete. There is some evidence that using a single screen or a few screens for short surveys minimizes respondents abandonment, whereas using a single screen and forcing the respondent to scroll down in long surveys increases abandonment.

• Eliminate unnecessary questions: Avoid questions that have answers that the computer can determine, such as the date the questionnaire is filled out.

• Begin the web questionnaire with a question that is fully visible on the first screen of the questionnaire, and will be easily comprehended and answered by all respondents.

• Present each question in a consistent format, including question numbers, instructions, space between lines, and space between question and answers.

• Limit the length to decrease the likelihood of a long line of prose being allowed to extend across the screen of the respondent's browser.

• Do not require respondents to provide an answer to each question before being allowed to answer any subsequent ones. It is suggested to include a "prefer not to answer" and/or "donŐt know" category for every item than to force respondents to answer every question.

• Organize and partition an on-line survey to correspond with the organization of semantic knowledge in memory. Then how should the survey be partitioned on-line into pages? Should it be Item-based, Form-based, or somewhere in between?

• Paged surveys that are not congruent with sections are to be avoided.

• Sectioned surveys that require scrolling should clearly indicate that additional items must be accessed by scrolling to them.

• Indexes to sections and pages are of marginal benefit and may sometimes lead to confusion.

• Hide inappropriate and irrelevant questions to shorten the apparent length of the questionnaire and make such questions available only if the respondent specifically needs or wishes to view them.

3.3.4 Instructions

• Provide instructions. Unlike the traditional face-to-face or telephone surveys, web-based surveys are not administered by an interviewer who motivates, clarifies and probes the respondents. [Example]

• Embedded instructions distract respondents from their main task.

• Provide a help button to access the instructions.

• Instructions on the pop-up windows may disturb respondents

• Enable respondents to report problems: Almost certainly, some respondents will experience some type of problem with the survey. Thus, a "help desk" should be established that respondents can contact easily by e-mail and/or toll-free phone number.

• Provide specific instructions on how to take each necessary computer action for responding to the questionnaire. Answering techniques may be obvious for experienced users, but need to be explained to less experienced respondents.

• Provide computer operation instructions as part of each question where the action is to be taken, not in a separate section prior to the beginning of the questionnaire.

• Reduce the branching instructions to a minimum to reduce reading time, confusion, and perceived difficulty of the questionnaire.

3.3.5 First Page

• Introduce the web questionnaire with a welcome screen that is motivational, emphasizes the ease of responding and instructs respondents about how to proceed to the next page. [Example]

• Choose for the first question an item that is likely to be interesting to most respondents, easily answered and fully visible on the welcome screen of the questionnaire.

• The first impression determines the web-based survey response rate.

3.3.6 Access control

• Provide a PIN number for limiting access only to authorized users in order to:
  
  
  • Prevent duplicates and foreign elements (coverage error).

  • Secure Confidentiality.

  • Let the researcher take control over the access.

• Always password protect web-based surveys: User passwords are necessary to restrict access and uniquely identify web-based survey respondents. Of course, restricting access for a convenience sample does not make sense, so passwords would not be issued in that particular case.

• Ensure that respondents' privacy and their perception of privacy are protected: Because there is the risk that transmissions sent over the Internet may be observed by unauthorized users, all survey data should be encrypted. A message explaining such procedures in a clear way, and displayed just before respondents leave a secure area can alleviate any privacy or security concerns they may have.

• Carefully handle respondents who fail a screening test: Depending on the nature of a survey and the respondent population, access to a web-based questionnaire can be restricted until a respondent has passed the screening questions. Two possible approaches involve excluding respondents from a survey as soon as they fail a screening question, or allowing all respondents to complete the entire survey and eliminate ineligible respondents later on.

3.3.7 Automation

• Automating a survey instrument provides many important benefits, including conditional branching, error checking and correction, questions search, question sort, calculations, and others.

• Be aware of the burden transferred from the user to the computer in the migration of paper-based surveys to web-based surveys to predict impacts of data collection.

• It may be an issue that users rely too much on the automation and are less likely to detect certain types of errors.
  [Top]