Published online Dec 20, 2024. doi: 10.5662/wjm.v14.i4.93559
Revised: June 15, 2024
Accepted: June 25, 2024
Published online: December 20, 2024
Processing time: 147 Days and 16.1 Hours
Research surveys are believed to have originated in antiquity with evidence of them being performed in ancient Egypt and Greece. In the past century, their use has grown significantly and they are now one of the most frequently employed research methods including in the field of healthcare. Modern validation techniques and processes have allowed researchers to broaden the scope of qualitative data they can gather through these surveys such as an individual’s views on service quality to nationwide surveys that are undertaken regularly to follow healthcare trends. This article focuses on the evolution and current utility of research surveys, different methodologies employed in their creation, the advantages and disadvantages of different forms and their future use in healthcare research. We also review the role artificial intelligence and the importance of increased patient participation in the development of these surveys in order to obtain more accurate and clinically relevant data.
Core Tip: Research surveys have been utilised for centuries and have grown in scope and use with regards to healthcare in the past century. Whilst undoubted strengths there are also weaknesses associated with this methodology. This article looks at past and current use, their strengths and weaknesses and likely use in the future.
- Citation: Colwill M, Pollok R, Poullis A. Research surveys and their evolution: Past, current and future uses in healthcare. World J Methodol 2024; 14(4): 93559
- URL: https://www.wjgnet.com/2222-0682/full/v14/i4/93559.htm
- DOI: https://dx.doi.org/10.5662/wjm.v14.i4.93559
Research surveys are believed to have originated in antiquity with evidence of census surveys being performed by governments in ancient Egypt and Greece in order to better understand their respective populations. The modern incarnation of the national population census started in the United States (USA) in 1790[1] and since then the type and scope of surveys has increased exponentially. In the 19th century surveys utilised for social research became a key part of guiding public policy, such as that performed by Joseph Rowntree to investigate poverty in the 1890s in the United Kingdom (UK)[2]. It was in the early 20th century that the use of surveys in healthcare research became a common methodology with one of the first being performed in the USA in the form of the National Health Interview Survey (NHIS) in 1957 which provided an insight into the healthcare problems faced by the nation[3].
Today, research surveys remain a widely practiced and important methodology in healthcare. This article will examine their methodology, current uses and how these have evolved, their strengths and weaknesses and their future utility in healthcare research.
The methodology employed in producing healthcare surveys can vary significantly but choosing the appropriate protocol is crucial to produce accurate research.
Clearly, getting information from every single person affected by a condition is impractical and therefore sampling, where a proportion of the target population provides the survey responses, is utilised. The choice of sampling method is decided by the researching team and is broadly divided into probability sampling or non-probability sampling. Probability sampling includes simple random, stratified random, systematic (or interval) and clustered random compared to non-probability which incorporates convenience, judgmental and snowball sampling. Non-probability sampling methods are in general quicker, easier and more cost-effective however they can introduce bias as they do not guarantee equal chances of each subject in the target population being involved. An example is snowball sampling where one interviewee essentially provides the researcher with the name of at least one more potential participant and, whilst this can be useful for rare pathologies or very small populations, it introduces sample bias and can reduce the diversity of the sample population so that the results are indicative of a sub-population rather than the true target population[4].
Together with an appropriate sampling method, accurate calculation of the required sample size is also required to generate accurate data and various statistical techniques and software programmes are available to perform this. The sample size calculation is also dependent upon the methodology chosen by the researching team and with regards to non-probability sampling, the commonest method used for surveys in healthcare research, sample size calculations are largely irrelevant as it is likely to produce non-generalisable results[5]. In this case, researchers should instead include as many subjects as possible from different demographics and sub-groups or consider using sample matching to minimise selection bias.
As well as accurately sampling populations as discussed, the type of research design is also important in producing scientifically valid data and testing the hypothesis. Cross-sectional surveys, which provide a snapshot of data at one specific time, can assess several variables at once but cannot demonstrate cause and effect. However, for example comparing obesity levels to physical activity levels, these surveys can demonstrate potential correlation signals which may require further research. Longitudinal surveys employ continuous or repeated measures to provide data over different time periods. This allows researchers to identify changes or trends in real time and provide insight into cause and effect. However, these studies take longer to complete, are more costly financially and temporally and results can be negatively impacted by incomplete or interrupted follow-up.
Further challenges that researchers face when utilising any form of survey is the need to ensure that the data they are collecting will provide accurate answers to the questions being addressed which is referred to as validity. There are various methods to assess validity, either through content validity, construct validity or criterion validity and these have previously been discussed in detail[6]. The process of validation, using whichever of these three metrics is felt to be most appropriate, can be challenging and cumbersome and often requires the involvement of experienced researchers to be done correctly.
As discussed, the choice of methodology employed in research surveys is crucial but the technique used to collect data is equally important to produce accurate results. This data can be collected using multiple different formats and each of these have their own strengths and weaknesses. The most common method is through online surveys where participants are sent a link to a pre-configured survey which they can complete without the need for a separate interview. Multiple companies have developed software to facilitate this to optimise participant recruitment and completion. This method is relatively cheap and can be performed quickly particularly if the survey in question is already validated. The anonymous nature of the process may also mean that patients are more willing to provide truthful answers and not be biased, consciously or unconsciously, by an interviewer. However, online research surveys also have problems with low response rates, inability to be performed by low literacy audiences and there is no scope for answers to be probed further by an interviewer.
The advantages and disadvantages of other, less frequently used, research survey methodologies are summarised in Table 1; this is adapted from the Food and Agriculture Organisation of the United Nations[7].
Survey methodology | Advantages | Disadvantages |
Low cost; Lack of interviewer means participants may be more likely to give honest, unbiased answers | Low response rates; Unable to probe answers because of lack of interviewer | |
Interview | Good response rates; Allows probing of answers; Allows observation of attitudes | Expensive; Time consuming; Requires interviewer training; Risk of interviewer introducing bias |
Phone | Easy accessibility in most countries; Anonymity; Flexible for participants; Computer aided techniques allow for rapid data collection | Lack of visual materials; Limitations in nature of questions that can be asked and answered (non-open ended); Call screening can limit response rate |
Another disadvantage, particularly with regards to patient satisfaction surveys, is that they can have a negative impact upon the very practice they are being used to assess and improve. Patient satisfaction is poorly defined and developing surveys that can be utilised to improve services based upon this is extremely difficult[8]. Consequently, by lacking a clearly defined objective, incorrect data can be corrected leading to false conclusions being drawn. There is also evidence that patient satisfaction surveys often focus on outdated metrics which do not accurately reflect the quality of care provided and can cause practices to focus solely on elements that will score highly on feedback surveys[9]. For effective use as a quality improvement tool, patient feedback surveys need to have accurately defined criteria and be regularly updated to ensure they are assessing the correct criteria and these surveys are further discussed below.
National healthcare surveys are used across the world by thousands of agencies and researchers each year. The relatively low cost of performing them means that they can be performed on a regular basis and collect different tranches of data which allow trends to be studied which can then influence healthcare policy and resource allocation. One of the longest running examples of this is the NHIS in the USA[3]. Since 1957 it has, through interviews performed by trained personnel, conducted surveys to get an accurate idea of illness in the non-institutionalised population of the USA. The survey collects data on a wide range of areas such as demographics, insurance coverage, smoking rates, mental health burden and access to, and utilisation of, preventative services. This data is used to monitor health objectives set by the department of health and human services as well as by epidemiological researchers. Crucially, the survey design is reviewed after every decennial census to ensure accurate and pertinent data is collected. Similar programmes exist in multiple other countries such as India in the form of the National Family Health Survey which started to collect data in 1992[10].
As well as interviews, national surveys can also be performed through direct mail campaigns, such as the general practitioner (GP) patient survey performed in the UK by IPSOS on behalf of the National Health Service (NHS)[11]. This long-running survey is sent through the mail to approximately 2 million patients each year to provide representative feedback on the performance of GPs across the country, highlight any concerns patients may have and, through feedback to the practices and the NHS management, can help to improve services.
Along with untargeted surveys, such as the GP patient survey, national surveys can also be targeted at specific groups. An example from the UK is the national inflammatory bowel disease (IBD) survey, performed periodically by a charity called IBD UK. Through surveys sent out to IBD care teams across the country they collect data on the services that each trust provides and compares it to the agreed national standards to help trusts identify deficiencies in their service such as understaffing.
Whilst some surveys are distributed nationally, charities such as IBD UK also send surveys out to individual patients to provide personalised feedback in order to improve their local services. These surveys are usually conducted using direct mail or e-mail recruitment and data is collected via a web portal or by returning a completed paper form. As discussed earlier, these feedback surveys have been shown in a previously published review[12] to be a validated quality improvement tool and do provide data specific to local services which can be used to improve care. However, they suffer from a high non-response rate, response bias and sample bias[13]. Correcting for this bias can be challenging and labour intensive but is necessary to generate accurate data which will then guide changes to actually improve services. There are also concerns about the overinterpretation of these surveys and the performing institutions subsequently focusing too much on measurable outcomes, rather than actual patient concerns.
Individualised research surveys are also used to collect patient reported outcomes (PROs) which have become a key part of clinical trials[14] where patients are provided with surveys to complete on a regular, sometimes daily, basis documenting their symptoms and well-being. This data is then fed back anonymously to the investigating team and conglomerated for central analysis helping to demonstrate the efficacy of the treatment being investigated. Whilst this offers researchers the opportunity to identify trends over time, it can be skewed by day-to-day complications, for example a patient developing temporary symptoms due to gastroenteritis whilst being part of a study examining a new IBD treatment, and researchers must allow and correct for this on a case-by-case basis.
Together with the growing use of PROs and patient feedback surveys, there has also been greater patient involvement in the developing and authoring of these surveys. Many organisations have developed patient involvement initiatives such as The Royal Marsden NHS Foundation Trust[15], Crohn’s and Colitis UK[16] and the Health Regulation Authority[17]. These are designed to ensure that patients are involved in the process of developing research surveys so that the data collected better represents the concerns of those impacted. With regards to clinical trials, this patient involvement also helps to recruit and retain participants. A review published in 2020 by the National Institute for Health and Care research also found that there is an active desire amongst patients to provide constructive feedback but that the NHS often does not use this feedback, collected from surveys, effectively to generate long-term improvements in services[18]. This suggests that whilst there may have been improvement in the methodology of collecting patient feedback, implementation of change has not yet caught up in the NHS.
The use of artificial intelligence (AI) is already changing the research survey landscape and will be a powerful ally in the field of research surveys including in healthcare. Tools such as Amazon’s Mechanical Turk already exist to help generate populations that produce more reliable and accurate data[19] and applying this and similar tools to healthcare surveys offers significant potential. As discussed previously, one of the commonest ways for surveys to be performed is through an online portal. AI can change the functionality of these portals so that the survey data is collected in a conversational manner, rather than simple question and answer, which will improve the depth and quality of data collected particularly with regards to qualitative data. Along with improved data collection, AI also has potential to improve analysis and interpretation. AI is already used for this purpose in healthcare settings[20] and with regards to research surveys it offers the capability to improve the efficiency and speed of survey response interpretation and analysis[21].
Whilst there has been some concern regarding the use of AI in healthcare research, particularly with regards to ethical considerations[20], data has shown that patients are broadly in favour of increasing the role of AI in healthcare[22] and provided that there is sufficient regulation and transparent practices in place then these concerns can be adequately addressed. Further questions remain over ownership, for example if an AI programme develops, distributes and collects data who has the intellectual property rights to this data and who bears responsibility for this data? It should also be noted that AI is still very much in it’s infancy and that whilst it has utility to augment human performance as described above, it does not replace experienced and competent researchers and this is unlikely to change in the foreseeable future.
Along with AI, the level of patient and public involvement in the development of research surveys is increasing[23] and, given the benefits discussed earlier, this trend will likely continue in the future in order to ensure the most accurate and appropriate data is collected.
Research surveys have been an important part of healthcare research for close to a century. Given their various advantages, particularly their inexpensive nature and speed at which they can be performed, their widespread use will continue in the future and with the careful and appropriate implementation of modern technologies and evolving methodology, their value will likely increase in the future.
1. | US Census Bureau CHS. Agency History - History - U.S. Census Bureau n.d. (accessed January 23, 2024). Available from: https://www.census.gov/history/www/census_then_now/. [Cited in This Article: ] |
2. | One hundred years of poverty and policy | Joseph Rowntree Foundation 2004. (accessed January 23, 2024). Available from: https://www.jrf.org.uk/one-hundred-years-of-poverty-and-policy. [Cited in This Article: ] |
3. | NHIS - National Health Interview Survey 2024. (accessed January 23, 2024). Available from: https://www.cdc.gov/nchs/nhis/index.htm. [Cited in This Article: ] |
4. | Enhancing the sample diversity of snowball samples: Recommendations from a research project on anti-dam movements in Southeast Asia - PMC n.d. (accessed May 27, 2024). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6104950/. [Cited in This Article: ] |
5. | Althubaiti A. Sample size determination: A practical guide for health researchers. J Gen Fam Med. 2023;24:72-78. [PubMed] [DOI] [Cited in This Article: ] [Cited by in F6Publishing: 21] [Reference Citation Analysis (70)] |
6. | Rattray J, Jones MC. Essential elements of questionnaire design and development. J Clin Nurs. 2007;16:234-243. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 603] [Cited by in F6Publishing: 588] [Article Influence: 34.6] [Reference Citation Analysis (71)] |
7. | Participatory survey methods for gathering information n.d. (accessed January 24, 2024). Available from: https://www.fao.org/3/w8016e/w8016e01.htm. [Cited in This Article: ] |
8. | Wong E, Mavondo F, Fisher J. Patient feedback to improve quality of patient-centred care in public hospitals: a systematic review of the evidence. BMC Health Serv Res. 2020;20:530. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 13] [Cited by in F6Publishing: 33] [Article Influence: 8.3] [Reference Citation Analysis (0)] |
9. | Richman BD, Schulman KA. Are Patient Satisfaction Instruments Harming Both Patients and Physicians? JAMA. 2022;328:2209-2210. [PubMed] [DOI] [Cited in This Article: ] [Cited by in F6Publishing: 7] [Reference Citation Analysis (0)] |
10. | National Family Health Survey n.d. (accessed January 24, 2024). Available from: https://rchiips.org/nfhs/. [Cited in This Article: ] |
11. | GP Patient Survey n.d. (accessed January 24, 2024). Available from: https://gp-patient.co.uk/. [Cited in This Article: ] |
12. | Al-Abri R, Al-Balushi A. Patient satisfaction survey as a tool towards quality improvement. Oman Med J. 2014;29:3-7. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 333] [Cited by in F6Publishing: 300] [Article Influence: 30.0] [Reference Citation Analysis (0)] |
13. | Fincham JE. Response rates and responsiveness for surveys, standards, and the Journal. Am J Pharm Educ. 2008;72:43. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 593] [Cited by in F6Publishing: 491] [Article Influence: 30.7] [Reference Citation Analysis (0)] |
14. | Walters SJ, Jacques RM, Dos Anjos Henriques-Cadby IB, Candlish J, Totton N, Xian MTS. Sample size estimation for randomised controlled trials with repeated assessment of patient-reported outcomes: what correlation between baseline and follow-up outcomes should we assume? Trials. 2019;20:566. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 11] [Cited by in F6Publishing: 19] [Article Influence: 3.8] [Reference Citation Analysis (0)] |
15. | Patient and Public Involvement and Engagement. The Royal Marsden n.d. (accessed January 24, 2024). Available from: https://www.royalmarsden.nhs.uk/our-research/patient-involvement. [Cited in This Article: ] |
16. | Patient involvement in Research Day n.d. (accessed January 24, 2024). Available from: https://crohnsandcolitis.org.uk/get-involved/want-to-get-involved-in-research/patient-involvement-in-research-day. [Cited in This Article: ] |
17. | Public involvement. Health Res Auth n.d. (accessed January 24, 2024). Available from: https://www.hra.nhs.uk/planning-and-improving-research/best-practice/public-involvement/. [Cited in This Article: ] |
19. | Boyer O’leary M, Georgetown University, Wilson JM, Metiu A, College of William & Mary, ESSEC Business School. Beyond Being There: The Symbolic Role of Communication and Identification in Perceptions of Proximity to Geographically Dispersed Colleagues. MISQ. 2014;38:1219-1243. [DOI] [Cited in This Article: ] [Cited by in Crossref: 85] [Cited by in F6Publishing: 88] [Article Influence: 8.8] [Reference Citation Analysis (0)] |
20. | Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc J. 2019;6:94-98. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 1432] [Cited by in F6Publishing: 815] [Article Influence: 163.0] [Reference Citation Analysis (0)] |
21. | Alowais SA, Alghamdi SS, Alsuhebany N, Alqahtani T, Alshaya AI, Almohareb SN, Aldairem A, Alrashed M, Bin Saleh K, Badreldin HA, Al Yami MS, Al Harbi S, Albekairy AM. Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Med Educ. 2023;23:689. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 6] [Cited by in F6Publishing: 132] [Article Influence: 132.0] [Reference Citation Analysis (0)] |
22. | Fritsch SJ, Blankenheim A, Wahl A, Hetfeld P, Maassen O, Deffge S, Kunze J, Rossaint R, Riedel M, Marx G, Bickenbach J. Attitudes and perception of artificial intelligence in healthcare: A cross-sectional survey among patients. Digit Health. 2022;8:20552076221116772. [PubMed] [DOI] [Cited in This Article: ] [Cited by in F6Publishing: 25] [Reference Citation Analysis (0)] |
23. | Arumugam A, Phillips LR, Moore A, Kumaran SD, Sampath KK, Migliorini F, Maffulli N, Ranganadhababu BN, Hegazy F, Botto-van Bemden A. Patient and public involvement in research: a review of practical resources for young investigators. BMC Rheumatol. 2023;7:2. [PubMed] [DOI] [Cited in This Article: ] [Reference Citation Analysis (0)] |