Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Apr;29(4):731-735.
doi: 10.1158/1055-9965.EPI-19-0846. Epub 2020 Feb 17.

A Tale of Two Diverse Qualtrics Samples: Information for Online Survey Researchers

Affiliations

A Tale of Two Diverse Qualtrics Samples: Information for Online Survey Researchers

Carrie A Miller et al. Cancer Epidemiol Biomarkers Prev. 2020 Apr.

Abstract

Background: There is often a lack of transparency in research using online panels related to recruitment methods and sample derivation. The purpose of this study was to describe the recruitment and participation of respondents from two disparate surveys derived from the same online research panel using quota sampling.

Methods: A commercial survey sampling and administration company, Qualtrics, was contracted to recruit participants and implement two Internet-based surveys. The first survey targeted adults aged 50 to 75 years and used sampling quotas to obtain diversity with respect to household income and race/ethnicity. The second focused on women aged 18 to 49 years and utilized quota sampling to achieve a geographically balanced sample.

Results: A racially and economically diverse sample of older adults (n = 419) and a geographically diverse sample of younger women (n = 530) were acquired relatively quickly (within 12 and 4 days, respectively). With exception of the highest income level, quotas were implemented as requested. Recruitment of older adults took longer (vs. younger female adults). Although survey completion rates were reasonable in both studies, there were inconsistencies in the proportion of incomplete survey responses and quality fails.

Conclusions: Cancer researchers, and researchers in general, should consider ways to leverage the use of online panels for future studies. To optimize novel and innovative strategies, researchers should proactively ask questions about panels and carefully consider the strengths and drawbacks of online survey features including quota sampling and forced response.

Impact: Results provide practical insights for cancer researchers developing future online surveys and recruitment protocols.See all articles in this CEBP Focus section, "Modernizing Population Science."

PubMed Disclaimer

Conflict of interest statement

Conflict of interest statement:

The authors declare no potential conflicts of interest.

Figures

Figure 1
Figure 1
contains a depiction of recruitment and participation within each sample.

References

    1. Dillman DA, Smyth JD, and Christian LM, Internet, phone, mail, and mixed-mode surveys: the tailored design method. 2014: John Wiley & Sons.
    1. Van Selm M. and Jankowski NW, Conducting online surveys. Quality & Quantity, 2006. 40(3): p. 435–456.
    1. Wright KB, Researching Internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of computer-mediated communication, 2005. 10(3): p. JCMC1034.
    1. Andrews D, Nonnecke B, and Preece J, Electronic survey methodology: A case study in reaching hard-to-involve Internet users. International journal of human-computer interaction, 2003. 16(2): p. 185–210.
    1. Link MW, et al. , Mobile technologies for conducting, augmenting and potentially replacing surveys: Executive summary of the AAPOR task force on emerging technologies in public opinion research. 2014. 78(4): p. 779–787.

Publication types

MeSH terms