Research surveys and their evolution: Past, current and future uses in healthcare

Abstract

Research surveys are believed to have originated in antiquity with evidence of them being performed in ancient Egypt and Greece. In the past century, their use has grown significantly and they are now one of the most frequently employed research methods including in the field of healthcare. Modern validation techniques and processes have allowed researchers to broaden the scope of qualitative data they can gather through these surveys such as an individual’s views on service quality to nationwide surveys that are undertaken regularly to follow healthcare trends. This article focuses on the evolution and current utility of research surveys, different methodologies employed in their creation, the advantages and disadvantages of different forms and their future use in healthcare research. We also review the role artificial intelligence and the importance of increased patient participation in the development of these surveys in order to obtain more accurate and clinically relevant data.

Key Words: Research surveys; Methodology; Sampling; Artificial intelligence

Core Tip: Research surveys have been utilised for centuries and have grown in scope and use with regards to healthcare in the past century. Whilst undoubted strengths there are also weaknesses associated with this methodology. This article looks at past and current use, their strengths and weaknesses and likely use in the future.

INTRODUCTION

Research surveys are believed to have originated in antiquity with evidence of census surveys being performed by governments in ancient Egypt and Greece in order to better understand their respective populations. The modern incarnation of the national population census started in the United States (USA) in 1790[1] and since then the type and scope of surveys has increased exponentially. In the 19^th century surveys utilised for social research became a key part of guiding public policy, such as that performed by Joseph Rowntree to investigate poverty in the 1890s in the United Kingdom (UK)[2]. It was in the early 20^th century that the use of surveys in healthcare research became a common methodology with one of the first being performed in the USA in the form of the National Health Interview Survey (NHIS) in 1957 which provided an insight into the healthcare problems faced by the nation[3].

Today, research surveys remain a widely practiced and important methodology in healthcare. This article will examine their methodology, current uses and how these have evolved, their strengths and weaknesses and their future utility in healthcare research.

SURVEY METHODOLOGY

The methodology employed in producing healthcare surveys can vary significantly but choosing the appropriate protocol is crucial to produce accurate research.

Sampling

Clearly, getting information from every single person affected by a condition is impractical and therefore sampling, where a proportion of the target population provides the survey responses, is utilised. The choice of sampling method is decided by the researching team and is broadly divided into probability sampling or non-probability sampling. Probability sampling includes simple random, stratified random, systematic (or interval) and clustered random compared to non-probability which incorporates convenience, judgmental and snowball sampling. Non-probability sampling methods are in general quicker, easier and more cost-effective however they can introduce bias as they do not guarantee equal chances of each subject in the target population being involved. An example is snowball sampling where one interviewee essentially provides the researcher with the name of at least one more potential participant and, whilst this can be useful for rare pathologies or very small populations, it introduces sample bias and can reduce the diversity of the sample population so that the results are indicative of a sub-population rather than the true target population[4].

Sample size

Together with an appropriate sampling method, accurate calculation of the required sample size is also required to generate accurate data and various statistical techniques and software programmes are available to perform this. The sample size calculation is also dependent upon the methodology chosen by the researching team and with regards to non-probability sampling, the commonest method used for surveys in healthcare research, sample size calculations are largely irrelevant as it is likely to produce non-generalisable results[5]. In this case, researchers should instead include as many subjects as possible from different demographics and sub-groups or consider using sample matching to minimise selection bias.

Research design

As well as accurately sampling populations as discussed, the type of research design is also important in producing scientifically valid data and testing the hypothesis. Cross-sectional surveys, which provide a snapshot of data at one specific time, can assess several variables at once but cannot demonstrate cause and effect. However, for example comparing obesity levels to physical activity levels, these surveys can demonstrate potential correlation signals which may require further research. Longitudinal surveys employ continuous or repeated measures to provide data over different time periods. This allows researchers to identify changes or trends in real time and provide insight into cause and effect. However, these studies take longer to complete, are more costly financially and temporally and results can be negatively impacted by incomplete or interrupted follow-up.

Validity

Further challenges that researchers face when utilising any form of survey is the need to ensure that the data they are collecting will provide accurate answers to the questions being addressed which is referred to as validity. There are various methods to assess validity, either through content validity, construct validity or criterion validity and these have previously been discussed in detail[6]. The process of validation, using whichever of these three metrics is felt to be most appropriate, can be challenging and cumbersome and often requires the involvement of experienced researchers to be done correctly.

Advantages and disadvantages of research survey forms

As discussed, the choice of methodology employed in research surveys is crucial but the technique used to collect data is equally important to produce accurate results. This data can be collected using multiple different formats and each of these have their own strengths and weaknesses. The most common method is through online surveys where participants are sent a link to a pre-configured survey which they can complete without the need for a separate interview. Multiple companies have developed software to facilitate this to optimise participant recruitment and completion. This method is relatively cheap and can be performed quickly particularly if the survey in question is already validated. The anonymous nature of the process may also mean that patients are more willing to provide truthful answers and not be biased, consciously or unconsciously, by an interviewer. However, online research surveys also have problems with low response rates, inability to be performed by low literacy audiences and there is no scope for answers to be probed further by an interviewer.

The advantages and disadvantages of other, less frequently used, research survey methodologies are summarised in Table 1; this is adapted from the Food and Agriculture Organisation of the United Nations[7].

Table 1 Advantages and disadvantages of research survey methodologies.

Survey methodology	Advantages	Disadvantages
Mail	Low cost; Lack of interviewer means participants may be more likely to give honest, unbiased answers	Low response rates; Unable to probe answers because of lack of interviewer
Interview	Good response rates; Allows probing of answers; Allows observation of attitudes	Expensive; Time consuming; Requires interviewer training; Risk of interviewer introducing bias
Phone	Easy accessibility in most countries; Anonymity; Flexible for participants; Computer aided techniques allow for rapid data collection	Lack of visual materials; Limitations in nature of questions that can be asked and answered (non-open ended); Call screening can limit response rate

Another disadvantage, particularly with regards to patient satisfaction surveys, is that they can have a negative impact upon the very practice they are being used to assess and improve. Patient satisfaction is poorly defined and developing surveys that can be utilised to improve services based upon this is extremely difficult[8]. Consequently, by lacking a clearly defined objective, incorrect data can be corrected leading to false conclusions being drawn. There is also evidence that patient satisfaction surveys often focus on outdated metrics which do not accurately reflect the quality of care provided and can cause practices to focus solely on elements that will score highly on feedback surveys[9]. For effective use as a quality improvement tool, patient feedback surveys need to have accurately defined criteria and be regularly updated to ensure they are assessing the correct criteria and these surveys are further discussed below.

CURRENT UTILITY OF SURVEYS IN HEALTHCARE RESEARCH

National surveys

National healthcare surveys are used across the world by thousands of agencies and researchers each year. The relatively low cost of performing them means that they can be performed on a regular basis and collect different tranches of data which allow trends to be studied which can then influence healthcare policy and resource allocation. One of the longest running examples of this is the NHIS in the USA[3]. Since 1957 it has, through interviews performed by trained personnel, conducted surveys to get an accurate idea of illness in the non-institutionalised population of the USA. The survey collects data on a wide range of areas such as demographics, insurance coverage, smoking rates, mental health burden and access to, and utilisation of, preventative services. This data is used to monitor health objectives set by the department of health and human services as well as by epidemiological researchers. Crucially, the survey design is reviewed after every decennial census to ensure accurate and pertinent data is collected. Similar programmes exist in multiple other countries such as India in the form of the National Family Health Survey which started to collect data in 1992[10].

As well as interviews, national surveys can also be performed through direct mail campaigns, such as the general practitioner (GP) patient survey performed in the UK by IPSOS on behalf of the National Health Service (NHS)[11]. This long-running survey is sent through the mail to approximately 2 million patients each year to provide representative feedback on the performance of GPs across the country, highlight any concerns patients may have and, through feedback to the practices and the NHS management, can help to improve services.

Along with untargeted surveys, such as the GP patient survey, national surveys can also be targeted at specific groups. An example from the UK is the national inflammatory bowel disease (IBD) survey, performed periodically by a charity called IBD UK. Through surveys sent out to IBD care teams across the country they collect data on the services that each trust provides and compares it to the agreed national standards to help trusts identify deficiencies in their service such as understaffing.

Individual surveys

Whilst some surveys are distributed nationally, charities such as IBD UK also send surveys out to individual patients to provide personalised feedback in order to improve their local services. These surveys are usually conducted using direct mail or e-mail recruitment and data is collected via a web portal or by returning a completed paper form. As discussed earlier, these feedback surveys have been shown in a previously published review[12] to be a validated quality improvement tool and do provide data specific to local services which can be used to improve care. However, they suffer from a high non-response rate, response bias and sample bias[13]. Correcting for this bias can be challenging and labour intensive but is necessary to generate accurate data which will then guide changes to actually improve services. There are also concerns about the overinterpretation of these surveys and the performing institutions subsequently focusing too much on measurable outcomes, rather than actual patient concerns.

Individualised research surveys are also used to collect patient reported outcomes (PROs) which have become a key part of clinical trials[14] where patients are provided with surveys to complete on a regular, sometimes daily, basis documenting their symptoms and well-being. This data is then fed back anonymously to the investigating team and conglomerated for central analysis helping to demonstrate the efficacy of the treatment being investigated. Whilst this offers researchers the opportunity to identify trends over time, it can be skewed by day-to-day complications, for example a patient developing temporary symptoms due to gastroenteritis whilst being part of a study examining a new IBD treatment, and researchers must allow and correct for this on a case-by-case basis.

Together with the growing use of PROs and patient feedback surveys, there has also been greater patient involvement in the developing and authoring of these surveys. Many organisations have developed patient involvement initiatives such as The Royal Marsden NHS Foundation Trust[15], Crohn’s and Colitis UK[16] and the Health Regulation Authority[17]. These are designed to ensure that patients are involved in the process of developing research surveys so that the data collected better represents the concerns of those impacted. With regards to clinical trials, this patient involvement also helps to recruit and retain participants. A review published in 2020 by the National Institute for Health and Care research also found that there is an active desire amongst patients to provide constructive feedback but that the NHS often does not use this feedback, collected from surveys, effectively to generate long-term improvements in services[18]. This suggests that whilst there may have been improvement in the methodology of collecting patient feedback, implementation of change has not yet caught up in the NHS.

Future

The use of artificial intelligence (AI) is already changing the research survey landscape and will be a powerful ally in the field of research surveys including in healthcare. Tools such as Amazon’s Mechanical Turk already exist to help generate populations that produce more reliable and accurate data[19] and applying this and similar tools to healthcare surveys offers significant potential. As discussed previously, one of the commonest ways for surveys to be performed is through an online portal. AI can change the functionality of these portals so that the survey data is collected in a conversational manner, rather than simple question and answer, which will improve the depth and quality of data collected particularly with regards to qualitative data. Along with improved data collection, AI also has potential to improve analysis and interpretation. AI is already used for this purpose in healthcare settings[20] and with regards to research surveys it offers the capability to improve the efficiency and speed of survey response interpretation and analysis[21].

Whilst there has been some concern regarding the use of AI in healthcare research, particularly with regards to ethical considerations[20], data has shown that patients are broadly in favour of increasing the role of AI in healthcare[22] and provided that there is sufficient regulation and transparent practices in place then these concerns can be adequately addressed. Further questions remain over ownership, for example if an AI programme develops, distributes and collects data who has the intellectual property rights to this data and who bears responsibility for this data? It should also be noted that AI is still very much in it’s infancy and that whilst it has utility to augment human performance as described above, it does not replace experienced and competent researchers and this is unlikely to change in the foreseeable future.

Along with AI, the level of patient and public involvement in the development of research surveys is increasing[23] and, given the benefits discussed earlier, this trend will likely continue in the future in order to ensure the most accurate and appropriate data is collected.

CONCLUSION

Research surveys have been an important part of healthcare research for close to a century. Given their various advantages, particularly their inexpensive nature and speed at which they can be performed, their widespread use will continue in the future and with the careful and appropriate implementation of modern technologies and evolving methodology, their value will likely increase in the future.