Seriousness check
The seriousness check is a technique that can be used in online research (also called Internet-based research, Web-based research, Web-based experiments) to significantly improve data quality. The technique has been shown to predict the dropout rate and control for motivational confounding.[1][2][3][4]. Nowadays, online studies are accessible to a large diversity of participants. However, many people just click through a questionnaire out of curiosity, rather than providing well thought out answers[4]. This poses a serious threat to the validity of online research[5][2][4]. The seriousness check addresses this problem: In this approach the respondents are asked about the seriousness of their participation or for a probability estimate that they will complete the entire study or experiment[1]. Thus, by using the seriousness check irrelevant data entries can be easily identified and be excluded from the data analysis.
Method
Seriousness checks can be implemented both before[2][3][4] and after[6] participation in the study. Reips[2][3][4] has argued to conduct seriousness checks in the first page of the experiment because this has been shown to be the best predictor of dropout rates and thus a measure of motivation. Also, the answer to the check question serves as self-commitment by the participant. Additionally, conducting a seriousness check prior to the completion of the study can reduce the dropout rates[7]. Figure 1 depicts a possible seriousness check proposed by Reips[4]. The participants are asked if whether or not they want to participate seriously in the experiment (“I would like to seriously participate now.” / ”I would like to look at the pages only.”). There are Web-based tools, e.g. WEXTOR[8] (http://wextor.eu), that implement the seriousness check by default.
Figure 1 illustrates the seriousness check[9]
Impact
Several studies have shown that a seriousness check at the beginning of a study is the best predictor of dropout rates and thus a measure of motivation.[2][3][4]. It was observed that of those answering “I would like to look at the pages only” around 75% will drop, while of those answering “I would like to seriously participate now” only ca. 10-15% will drop during the study. Overall, about 30-50% of visitors will fail the seriousness check, i.e. answer “I would like to look at the pages only”[4]
References
This article "Seriousness check" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Seriousness check. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
|
This page exists already on Wikipedia. |
- ↑ 1.0 1.1 Musch, J., & Klauer, K. C. (2002). Psychological experimenting on the World Wide Web: Investigating content effects in syllogistic reasoning. In B. Batinic, U.-D. Reips, & M. Bosnjak (Eds.), Online social sciences (pp. 181–212). Hogrefe & Huber Publishers.
- ↑ 2.0 2.1 2.2 2.3 2.4 Reips, U.-D. (2002a). Standards for internet-based experimenting. Experimental Psychology, 49(4), 243–256. https://doi.org/10.1026/1618-3169.49.4.243
- ↑ 3.0 3.1 3.2 3.3 Reips, U.-D. (2008). How Internet-mediated research changes science. In A. Barak (Ed.), Psychological aspects of cyberspace: Theory, research, applications (p. 268–294). Cambridge University Press. https://doi.org/10.1017/CBO9780511813740.013
- ↑ 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7 Reips, U.-D. (2009). Internet experiments: Methods, guidelines, metadata. Human Vision and Electronic Imaging XIV, Proc. SPIE, 7240, 724008. https://doi.org/10.1117/12.823416
- ↑ Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), 867–872. https://doi.org/10.1016/j.jesp.2009.03.009
- ↑ Aust, F., Diedenhofen, B., Ullrich, S., & Musch, J. (2013). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45(2), 527–535. https://doi.org/10.3758/s13428-012-0265-2
- ↑ Reips, U.-D. (2002b). Internet-based psychological experimenting: Five dos and five don’ts. Social Science Computer Review, 20(3), 241–249. https://doi.org/10.1177/089443930202000302
- ↑ Reips, U.-D.; Neuhaus, C. (2002). "WEXTOR: A web-based tool for generating and visualizing experimental designs and procedures" (PDF). Behavior Research Methods, Instruments, and Computers. 34(2): 234–240. doi:10.3758/BF03195449.
- ↑ Reips, U.-D., & Krantz, J. H. (2010). Conducting true experiments on the Web. In S. D. Gosling & J. A. Johnson (Eds.), Advanced methods for conducting online behavioral research (p. 193–216). American Psychological Association. https://doi.org/10.1037/12076-01
