Delphi methodology in healthcare research: How to decide its appropriateness

Table 1 Evaluation of Delphi studies on coronavirus disease 2019 that were published in 2020 on nine qualitative evaluation points

No.	Ref.	Medicine field	Geographical location (Country/Continent)	Aim or purpose	Guidance format
1	Vitacca et al[24]	Rehabilitation	Italy, Europe	Consensus on pulmonary rehabilitation in patients with COVID-19 after discharge from acute care.	Recommendations from experts’ panel.
2	Mikuls et al[25]	Rheumatology	USA, North America	Guidance to rheumatology providers on the management of adult rheumatic diseases during COVID-19 pandemic.	77 initial guidance statements converted to 25 final guidance statements.
3	Greenhalgh et al[26]	Primary health	UK, Europe	To develop early warning score for patients with suspected COVID-19 who need escalation to next level of care.	Development of software for early warning score in COVID-19 patients.
4	Lamb et al[27]	Respiratory medicine and critical care medicine	USA, NA	Guidance to physicians on the preparation, timing, and technique of tracheostomy in COVID-19 patients.	Eight recommendations.
5	Welsh Surgical Research Initiative (WSRI) Collaborative[28]	General surgery	Global	Identify the needs of the global OR workforce during COVID-19.	Statements, predominantly standardization of OR pathways, OR staffing, and preoperative screening or diagnosis.
6	Eibensteiner et al[29]	Nephrology	Europe	To gather expert knowledge and experience to guide the care of children with chronic kidney disease during the COVID-19 pandemic.	Qualitative expert statements and answers.
7	Bhandari et al[30]	Gastroenterology	Global	Guidance on how to resume endoscopy services during COVID-19.	Best practice recommendations to aid the safe resumption of endoscopy services globally in the era of COVID-19.
8	Guckenberger et al[31]	Radiotherapy	NA and Europe	To develop practice recommendations pertaining to safe radiotherapy for lung cancer patients during COVID-19 pandemic.	Consensus recommendations in common clinical scenarios of radiotherapy for lung cancer.
9	Aj et al[32]	General surgery	NA, Europe and Australia	Validation of international COVID-19 surgical guidance during COVID-19 pandemic.	Area of consensus and contentious areas from previous guidelines.
10	Gelfand et al[33]	Dermatology	NA	Guidance on the management of psoriatic disease during the COVID-19 pandemic.	22 guidance statements.
11	Allan et al[34]	Surgery	Global	Guidance on surgery and OR practices during COVID-19 pandemic.	Development of research priorities in discipline of surgery related to COVID-19.
12	Shanbehzadeh et al[35]	Medical informatics and public health	Iran, Middle east	Development of minimum data set for COVID-19 surveillance system.	Conceptual COVID-19 surveillance model.
13	Bergman et al[36]	Long-term nursing care	NA	Consensus guidance statements focusing on essential family caregivers and visitors in nursing homes during COVID-19 pandemic.	Recommendations for visitors in long term nursing homes.
14	Daigle et al[37]	Ophthalmology	Canada	Risk stratifying for oculofacial plastic and orbital surgeries in context of transmission of SARS-CoV-2.	Risk based algorithm for oculoplastic surgeries and recommendations for appropriate PPE.
15	Sorbello et al[38]	Anaesthesia	Europe	Review of available evidence and scientific publications about barrier-enclosure systems for airway management in suspected/confirmed COVID-19 patients.	Recommendation on enclosure barrier systems.
16	Jheon et al[39]	Cardiovascular and thoracic surgery	Asia	Thoracic cancer surgery during COVID-19 pandemic.	Recommendations on timing, approach, type of surgery, and postoperative requirements.
17	Olmos-Gómez et al[40]	Behavioural sciences	Spain, Europe	To know the impact of learning environments and psychological factors.	Future research priorities.
18	Sawhney et al[41]	Gastroenterology	Global	Study to emphasize patient-important outcomes while considering procedural timing.	Recommendations on procedural timing for common indications for advanced endoscopy during COVID-19.
19	Sciubba et al[42]	Neurosurgery	USA	Study to device scoring system to help with triaging surgical patients during the COVID-19 pandemic.	Scoring system to triage spinal surgery cases during COVID-19 pandemic.
20	Errett et al[43]	Environmental health science	USA	Study to develop an Environmental Health Sciences COVID-19 research agenda.	To validate, find limitations, and identify future research priorities.
21	Arezzo et al[44]	Minimal access surgery	Global	To study and provide recommendations for recovery plan in minimally invasive surgery amid COVID-19 pandemic.	Framework for resumption of surgery with focus on minimally invasive surgeries following COVID-19 pandemic.
22	Dashash et al[45]	Healthcare education	Syria	To identify essential competencies required for approaching patients with COVID-19.	Core competency points for health care professionals to prepare them for COVID-19 pandemic.
23	Ramalho et al[46]	Psychiatry	Global	To create a practical and clinically useful protocol for mental health care to be applied in the pandemic.	Consensus protocol for use of telemedicine in psychiatry consults during COVID-19 pandemic.
24	Saldarriaga Rivera et al[47]	Rheumatology	Columbia, SA	To produce recommendations for patients with rheumatological diseases receiving immunomodulatory and immunosuppressive therapies.	Recommendations for pharmacological management of patients with rheumatic diseases during COVID-19 pandemic.
25	Tchouaket Nguemeleu et al[48]	Public health	Canada, NA	Study for development and validation of a time and motion guide to assess the costs of prevention and control interventions for nosocomial infections.	Development and validation of a new instrument for systematic assessment of costs relating to the human and material resources used in nosocomial infection prevention and control.
26	Santana et al[49]	Nursing	Brazil, SA	To develop an adaptable acceptable nursing protocol during the pandemic.	Protocol for nurse managers to cope with pandemic.
27	Tang et al[50]	Oncology	China	To develop a risk model based on the experience of recently resumed activities in many cancer hospitals in China to reduce nosocomial transmission of SARS-CoV-2.	Risk model development on the basis of experience from recently resumed cancer hospital.
28	Jiménez-Rodríguez et al[51]	Public health	Spain	Develop recommendations for telemedicine in video consultations during COVID-19.	Consensus recommendations for healthcare professionals for proper management of video consultation.
29	Reina Ortiz et al[52]	Public health	Ecuador	Development of bio-safety measures to reduce cross-transmission of SARS-CoV-2.	Biosafety-at-home flyer for high-risk group and health care workers to reduce the risk of cross-transmission.
30	Douillet et al[53]	Internal medicine	France and Belgium	Identify reliable criteria for hospitalization or outpatient management in mild cases of COVID-19.	Development of toolkit “HOME-CoV rule”, a decision-making support mechanism for clinicians to target patients with suspected or confirmed COVID-19 requiring hospitalization.
31	Richez et al[54]	Rheumatology	France	Management of anti-inflammatory agents and disease-modifying-anti-rheumatic-drugs for rheumatological patients during COVID-19.	Recommendations to rheumatologists on management.
32	Yalçınkaya et al[55]	Physiotherapy and rehabilitation medicine	Turkey	Recommendations for the management of spasticity in Cerebral palsy children during COVID-19 pandemic.	Consensus recommendations for spasticity management in cerebral palsy children.
33	Tanasijevic et al[56]	Haemato-oncology	USA	To identify minimum hemoglobin for safe transfusion in myelodysplastic syndrome during COVID-19 pandemic.	Recommendations for lowest value of hemoglobin for which transfusions can safely forgo.
34	Alarcón et al[57]	Dentistry	Latin America	Education and practice in implant Dentistry during COVID-19 pandemic.	Consensus recommendations.

COVID-19: Coronavirus disease 2019; SARS-CoV-2: Severe acute respiratory syndrome coronavirus 2; OR: Operating room.

Table 2 Basic information of the Delphi studies included for evaluation

No.	Ref.	Identification of problem area	Selection of panel members	Anonymity of panellist	Controlled feedback	Iterative rounds	Consensus Criteria	Analysis of consensus	Closing criteria	Group stability	Number of rounds	Number of experts
1	Vitacca et al[24]	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	2	20
2	Mikuls et al [25]	Yes	Yes	Not clear	Yes	Yes	Yes	Yes	Yes	No	2	14
3	Greenhalgh et al[26]	Yes	Not clear	Yes	Yes	Yes	Not clear	Yes	Yes	No	4	72
4	Lamb et al[27]	Yes	Yes	No	No	No	Yes	Yes	Yes	No	1	13
5	Welsh Surgical Research Initiative (WSRI) Collaborative[28]	Yes	No	Yes	No	No	Yes	Yes	Yes	No	1	339
6	Eibensteiner et al[29]	Yes	Yes	Not clear	Yes	Yes	No	No	No	No	4	13
z	Bhandari et al[30]	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	2	34
8	Guckenberger et al[31]	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	3	32
9	Aj et al[32]	Yes	No	Yes	Yes	No	Yes	Yes	Yes	No	1	339
10	Gelfand et al[33]	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	No	2	18
11	Allan et al[34]	No	No	Yes	No	Yes	Yes	Yes	No	No	3	213
12	Shanbehzadeh et al[35]	Yes	Yes	No	No	Yes	Yes	Yes	Yes	No	2	40
13	Bergman et al[36]	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	No	2	21
14	Daigle et al[37]	Yes	Yes	Not clear	Yes	Yes	Yes	Yes	Yes	No	2	18
15	Sorbello et al[38]	Yes	Not clear	Not clear	Not clear	Not clear	Not clear	Not clear	Not clear	Not clear	-	0
16	Jheon et al[39]	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	2	26
17	Olmos-Gómez et al[40]	Yes	Yes	Not clear	Not clear	Yes	Yes	Yes	Yes	Yes	3	441
18	Sawhney et al[41]	Not clear	Not clear	Not clear	Yes	Yes	Yes	Yes	No	Not clear	3	14
19	Sciubba et al[42]	Not clear	Not clear	No	Not clear	Yes	No	Yes	No	Not clear	3	16
20	Errett et al[43]	Not clear	Not clear	Yes	Not clear	Yes	Yes	Yes	Yes	No	3	28
21	Arezzo et al[44]	Yes	Not clear	Yes	Yes	No	Yes	Yes	Yes	No	2	55
22	Dashash et al[45]	Yes	Not clear	Not clear	Yes	No	Yes	Yes	No	No	3	20
23	Ramalho et al[46]	Yes	Not clear	Not clear	Yes	Yes	Yes	Yes	Not clear	No	2	16
24	Saldarriaga Rivera et al[47]	Yes	No	Not clear	Yes	Yes	Yes	Yes	Not clear	No	3	11
25	Tchouaket Nguemeleu et al[48]	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	2	18
26	Santana et al[49]	No	Not clear	No	Yes	Yes	Yes	Yes	No	Not clear	4	6
27	Tang et al[50]	Yes	Yes	Not clear	No	Not clear	No	No	No	No	1	83
28	Jiménez-Rodríguez et al[51]	No	Yes	Not clear	Yes	Yes	Yes	Yes	Yes	No	3	16
29	Reina Ortiz et al[52]	No	Yes	Not clear	Yes	Yes	No	No	Not clear	No	2	12
30	Douillet et al[53]	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Not clear	4	51
31	Richez et al[54]	Yes	Yes	No	Yes	No	No	No	Not clear	No	2	10
32	Yılmaz Yalçınkaya et al[55]	Yes	Yes	Not clear	No	No	Yes	Yes	No	No	1	60
33	Tanasijevic et al[56]	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	No	3	13
34	Alarcón et al[57]	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	No	3	197