Peritraumatic Behavior Questionnaire - Observer Rated: Validation of the objective version of a measure for combat-related peritraumatic stress

Abstract

AIM: To validate the first third-person-rated measure assessing combat-related peritraumatic stress symptoms and evaluate its psychometric properties and war-zone applicability.

METHODS: The valid assessment of peritraumatic symptoms in the theater of military operations represents a significant challenge in combat-related, mental health research, which mainly relies on retrospective, subjective self-report ratings. This longitudinal observational study used data from actively deployed troops to correlate third-person observer ratings of deployment peritraumatic behaviors [Peritraumatic Behavior Questionnaire - Observer Rated (PBQ-OR)] collected on a bi-monthly basis with post-deployment (1-wk follow-up) ratings of the previously validated PBQ self-rate version (PBQ-SR), and (3-mo follow-up) clinician assessed and self-report posttraumatic stress disorder (PTSD) symptoms (Clinician Administered PTSD Scale, PTSD Checklist). Cronbach’s alpha (α) and correlation coefficients were calculated to assess internal reliability and concurrent validity respectively.

RESULTS: Eight hundred and sixty male Marines were included in this study after signing informed consents at pre-deployment (mean age 23.2 ± 2.6 years). Although our findings were limited by an overall sparse return rate of PBQ-OR ratings, the main results indicate satisfactory psychometric properties with good internal consistency for the PBQ-OR (α = 0.88) and high convergent and concurrent validity with 1-wk post-deployment PBQ-SR ratings and 3-mo posttraumatic stress symptoms. Overall, later PBQ-OR report date was associated with higher correlation between PBQ-OR and post-deployment measures. Kappa analysis between PBQ-OR and PBQ-SR single items, showed best agreement in questions relating of mortal peril, desire for revenge, and experience of intense physical reactions. Logistic regression demonstrated satisfactory predictive validity of PBQ-OR total score with respect to PTSD caseness (OR = 1.0513; 95%CI: 1.011-1.093; P = 0.02).

CONCLUSION: Since no comparable tools have been developed, PBQ-OR could be valuable as real-time screening tool for earlier detection of Service Members at risk.

Key Words: Peritraumatic reaction; Posttraumatic stress disorder; Trauma; Military service; Combat; Assessment; Dissociation; Stress

Core tip: The assessment of combat-related peritraumatic symptoms mainly relies on retrospective, subjective self-report ratings. We have therefore developed the Peritraumatic Behavior Questionnaire - Observer Rated (PBQ-OR), a third-person-rated scale for unit-embedded medical personnel to objectively assess symptoms of combat-related peritraumatic stress in deployed troops. In this study, we validated the PBQ-OR during active deployment and longitudinally evaluated its psychometric properties and war-zone applicability. Our findings show that the PBQ-OR could be used as a screening and monitoring tool in real time and may permit earlier detection of Service Members at risk for posttraumatic stress symptoms to target prevention and early intervention efforts.

INTRODUCTION

Peritraumatic stress reactions include various behavioral, emotional, cognitive, and physiological symptoms associated with sympathetic activation during and immediately following a traumatic event[1]. Prolonged continuation of these biological and psychological responses can lead to long-term adverse biological alterations[2,3], strongly associated with the subsequent development of posttraumatic stress disorder (PTSD)[4,5]. Peritraumatic stress is, hence, a very sensitive pre-clinical risk marker and its accurate assessment could improve individual risk evaluation and provision of appropriate mental health interventions in traumatized populations[6,7].

Peritraumatic reactions are especially representative of extraordinary stress challenges and could affect the maintenance of military operational resilience in Service Members[8,9]. However, the valid assessment of peritraumatic symptoms in the theater of military operations is a significant challenge in combat-related, mental health research. To address this issue, we developed a new 15-item 5-point-Likert scale measure of combat-related peritraumatic distress symptoms: the Peritraumatic Behavior Questionnaire (PBQ). The detailed description of the development procedure of the PBQ is available in Agorastos et al[10]. The PBQ was designed as a military-specific, observer rated scale for unit-embedded medical personnel (UMP) for the reliable assessment of combat-related peritraumatic stress symptoms of Service Members in the theatre of operations (PBQ-OR, Table 1). The initial validation of the self-rated PBQ version [PBQ - Self Report (PBQ-SR)] confirmed the ability and reliability of PBQ-SR to assess peritraumatic reactions as a general construct unifying the major underlying peritraumatic symptom dimensions[10]. PBQ-SR demonstrated good internal consistency and convergent and discriminant validity, and showed a high correlation to various PTSD-specific/-related symptoms and PTSD caseness.

Table 1 The Peritraumatic Behavior Questionnaire - Observer Rated.

PBQ-OR questions
For a period of time, the individual did not act like their normal self
For a period of time, the individual seemed to feel fearless and invulnerable, as if nothing could harm them
For a period of time, the individual seemed not to care about their own or others' welfare or safety
For a period of time, the individual seemed to feel no remorse for doing things that would have bothered them in the past
1For a period of time, the individual seemed to be determined to get revenge
For a period of time, the individual seemed unable to stop laughing, crying, or screaming
For a period of time, the individual seemed helpless and unable to look out for their own welfare
For a period of time, the individual appeared to be confused, as if having difficulty making sense of what was happening
For a period of time, the individual appeared to be disoriented, as if uncertain about where they were or what day or what time it was
2For a period of time, the individual appeared not to be able to move parts of their body
2For a period of time, the individual froze or seemed to be moving very slowly, such that they could not do everything they wanted to do
For a period of time, the individual’s speech changed (such as stuttering, repeating words or phrases, or having a shaky or squeaky voice)
2For a period of time, the individual was not able to fully carry out their duties (during or immediately after the event)
1For a period of time, the individual expressed the belief that they were going to die
1For a period of time, the individual had an intense physical reaction such as sweating, shaking, or heart pounding

Text from PBQ-SR.

¹Questions 5, 14 and 15 have higher overall item-to-item significance of correlation to PBQ-SR;

²Questions 10, 11 and 13 (report order 3) received no non-zero responses. PBQ-OR: Peritraumatic Behavior Questionnaire - Observer Rated; PBQ-SR: Peritraumatic Behavior Questionnaire - Self Rated.

However, PBQ-SR, as well as prior research on peritraumatic stress have relied upon retrospective, subjective self-report questionnaires. Retrospective subjective assessment of peritraumatic symptoms introduces several potential biases and distortions related to cognitive barriers, adaptive denial coping, ethical concerns or current symptoms and obviates real-time case identification and intervention[11,12]. Therefore, objective third-person ratings of behavioral changes suggestive of acute peritraumatic stress for UMP, if viable, would represent a desirable approach. However, no valid instruments currently exist for objective ratings of combat-related peritraumatic symptoms, so it is yet unknown whether behavioral manifestations of acute traumatic stress are sufficiently observable or specific enough to be evaluated by third-party-UMP observers.

Thus, the primary objective of this study was the in-theater validation and psychometric evaluation of the PBQ-OR along with the assessment of its war-zone applicability through information collected in actively deployed troops. Specifically we aimed to: (1) validate and demonstrate the psychometric properties of the PBQ-OR; (2) investigate the relationship between objective, in-theater PBQ-OR ratings and self-reported peritraumatic symptoms retrospectively assessed by the PBQ-SR in Marines after deployment; (3) explore the relation of PBQ-OR ratings to post-deployment PTSD symptoms and PTSD caseness; and (4) investigate the PBQ-OR applicability as an operational clinical tool for accurate and consistent in-theater, objective assessment of peritraumatic symptoms in Marine ground combatants by especially trained UMP.

MATERIALS AND METHODS

Study design

The PBQ-OR in-theater validation study was designed as a longitudinal observational study to correlate third-person observer ratings of deployment peritraumatic behaviors (PBQ-OR) with post-deployment (1-wk follow-up) self-report measures of peritraumatic symptoms (PBQ-SR), and 3-mo post-deployment data collection of PTSD symptoms. In-theater data collection was linked to a larger IRB and VA research committee approved study entitled, “Prospective Study of the Psychological, Social and Biological Markers of Risk and Resilience for Operational Stress in Marines”. Post-deployment information was assessed as part of the parent Marine Resiliency Study[13].

Collection of data

PBQ-OR data were collected by UMP on a bi-monthly basis for all consenting Marines in the enrolled deployment cohorts, beginning approximately 30 d after war zone deployment and until return to the United States approximately 7 mo later. The instructions for PBQ-OR required each symptom to be rated as present only to the extent it was a clear change from baseline behaviour for the rated individual, persisting for “a period of time” after exposure to an identifiable stressor. Throughout the confidential assessment, military operations and healthcare decision making in-theater were not directly affected and there was no direct contact between study personnel and unit members. PBQ-OR ratings were then confidentially forwarded to study investigators.

Rater training

All embedded UMP were trained at pre-deployment by mental health professionals in the administration and scoring of the PBQ-OR. UMP attended training comprised of a presentation on peritraumatic symptoms, an introduction to the PBQ-OR, presentation of videos and rating of symptoms upon completion, as well as participation in a question and answer period. UMP ratings were assessed for inter-rater reliability. UMP with a correlation of greater than 80 were certified as PBQ-OR raters or else repeated training until qualification.

Measures

The Clinician Administered PTSD Scale (CAPS)[14], the gold standard diagnostic interview tool for measuring PTSD in clinical research was administered by specially trained physicians or psychologists, and the PTSD Checklist (PCL)[15], a 17-item, self-report questionnaire, a validated assessment of PTSD symptom severity was filled out by each study participant at 3 mo post-deployment. A PTSD diagnosis according to DSM-IV criteria was made according to the well-established F1/I2 scoring rule[16]. All Marine participants additionally filled out the self-rate version of the PBQ (PBQ-SR)[10] 1 wk after deployment. All trauma-specific items were completed in reference to the same, indexed traumatic event.

Statistical analysis

Since PBQ-OR ratings were not completed at specific dates, Corpsmen reports were assembled into three groupings: Report order one (R1), two (R2), and three (R3) according to their submission dates with respect to post-deployment ratings. Cronbach’s alpha (α) coefficients were calculated to assess internal reliability. Convergent and concurrent validity was measured by correlation coefficients between the corpsmen rated PBQ-OR and the post-deployment PBQ-SR, CAPS and PCL individual reports. To show by-symptom correlations between the corpsman’s PBQ-OR report and Marine’s post-deployment PBQ-SR scores, convergent validity was measured question by question using Kappa (κ) analysis. Predictive validity of PBQ-OR with respect to PTSD diagnosis at 3 mo post-deployment was calculated using logistic regression. Because PBQ-OR ratings were non-normally distributed, Spearman’s rho (ρ) was used to calculate the correlations throughout the analysis. Correlations were plotted with respect to mean days from PBQ-OR to PBQ-SR assessment. All statistical analyses of this study have been conducted and reviewed by biomedical statisticians (CJH, CVC).

RESULTS

Eight hundred and sixty male Marines were included in this study after signing informed consents at pre-deployment (mean age 23.2 ± 2.6 years). Demographic information is presented in Table 2. Twenty-nine certified male Corpsmen signed informed consent to be included as raters in the PBQ-OR study. Of those, 7 actually returned PBQ-OR ratings with dropouts occurring for different reasons (e.g., non-embedment with a unit, operational schedule, serious injury, loss of data in battle). Overall, 458 PBQ-OR ratings were returned (R1: n = 248, R2: n = 128, R3: n = 62; Table 3). The mean number of days from PBQ-OR to PBQ-SR rating for R1, R2 and R3 was 198 ± 51.5, 170 ± 38.8 and 136 ± 24.9, respectively.

Table 2 Demographic information of the included study sample.

	%
Education
Some high school	1.3
General education diploma	2.4
High school diploma	63.4
Some college	30.9
4-yr college degree	1.3
Ethnicity
Not Hipsanic or Latino	74.0
Mexican	15.2
South/Central American	5.5
Other Spanish culture or origin	4.6
Race
Black/African American	6.6
American Indian or Alaskan Native	9.1
Asian	2.6
Native Hawaiian or Pacific Islander	1.2
White	83.7
More than one	3.8

Demographics are given in percentage of the total available data (percentages under 1.0% are not reported).

Table 3 Total scores of psychometric assessments by different instruments.

Report order	n	PBQ				CAPS		PCL
		PBQ-OR (Corpsman)		PBQ-SR (Marine)
		Mean	SD	Mean	SD	Mean	SD	Mean	SD
Overall	458	2.29	4.95	6.88	8.09	21.71	20.07	28.21	13.14
1	248	1.45	3.20	6.84	8.18	21.39	20.04	28.52	13.12
2	128	3.63	6.53	7.58	8.64	22.45	20.67	28.57	13.99
3	62	3.02	6.37	6.19	7.16	22.25	20.16	26.78	12.24

Means and SD of questionnaires administered post-deployment (PBQ-SR, CAPS, PCL) by report order include only Marine reports for those with returned PBQ-OR ratings. PBQ-OR: Peritraumatic Behavior Questionnaire - Observer Rated; PBQ-SR: Peritraumatic Behavior Questionnaire - Self Rated; PCL: Posttraumatic stress disorder Checklist; CAPS: Clinician Administered posttraumatic stress disorder Scale.

The means and standard deviations of all the rating instruments are presented in Table 3. Individual element response rates were item specific, and varied across item in both the PBQ-SR and the PBQ-OR. Marine participants who filled the PBQ-SR showed low response rates for questions relating to feelings of helplessness (#7), inability to move (#10), and inability to perform duties (#13). In regard to the PBQ-OR, Corpsmen response rates to these same items was comparably low. Corpsmen responses were low for the following additional items: Lack of remorse (#4), unstoppable laughing or crying (#6), disorientation (#9), and change in time perception (#11). Some questions (numbers 10, 11, 13), in addition to showing low ratings, received no non-zero responses from corpsmen at any of the reports (data not shown).

Psychometric properties

PBQ-OR Cronbach’s α analysis showed good internal consistency (α = 0.88). R1, R2, and R3 PBQ-OR reports had Cronbach’s α of 0.83, 0.90, and 0.92 respectively. PBQ-OR showed an overall significant or highly significant correlation to post-deployment PBQ-SR, PCL and CAPS total score in all three report orders (Table 4), confirming satisfactory convergent validity. A question-by-question κ analysis indicated different rates of correlations between PBQ-OR and PBQ-SR, showing best agreement between the observer and subjective ratings in questions relating to perception of mortal peril (#14) (report order 1: ρ = 0.41, P < 0.001; report order 2: ρ = 0.58, P < 0.001; report order 3: ρ = 0.50, P < 0.001) desire for revenge (#5) (report order 1: ρ = 0.38, P < 0.001; report order 2: ρ = 0.44, P < 0.001; report order 3: ρ = 0.61, P < 0.001), and experience of intense physical reactions to combat (#15) (report order 1: ρ = 0.34, P < 0.001; report order 2: ρ = 0.41, P < 0.001; report order 3: ρ = 0.53, P < 0.001) (cf. Table 1, Legend). No question showed significant or consistent negative correlation between PBQ-SR and PBQ-OR, suggesting overall agreement and effectiveness of the instrument (data not shown). Overall, increasing report order was associated with an improvement in correlation between PBQ-OR and post-deployment measures (Table 4). Finally, using logistic regression, PBQ-OR total score showed significant association with post-deployment PTSD caseness (OR = 1.0513; 95%CI: 1.011-1.093; P = 0.02), suggesting satisfactory predictive validity.

Table 4 Correlations between Peritraumatic Behavior Questionnaire - Observer Rated questionnaires and Marine Resiliency Study subject measures by report order.

Report order	CAPS total score		PCL total score		PBQ-SR total score
	P	n	P	n	P	n
1	0.21b	235	0.25b	230	0.28b	249
2	0.28a	120	0.38b	113	0.41b	126
3	0.26a	63	0.46b	55	0.43b	63

Correlations plotted at mean days from report filing to PBQ-SR assessment.

^aP < 0.05,

^bP < 0.001. PCL: Posttraumatic stress disorder Checklist; CAPS: Clinician Administered Posttraumatic stress disorder scale; PBQ-SR: Peritraumatic Behavior Questionnaire - Self Rated.

DISCUSSION

The assessment of the immediate individual response to trauma represents one of the most important challenges in traumatized populations[17]. This study contributes to the validation of the PBQ-OR, the first third-person rated, objective instrument for the assessment of peritraumatic symptoms in combat-related settings. The main findings from this study include: (1) satisfactory psychometric properties with good internal consistency for the PBQ-OR; (2) high convergent validity with respect to post-deployment PBQ-SR total score ratings; (3) high concurrent validity with respect to post-deployment PTSD symptoms as well as significant predictive validity with respect to PTSD caseness; and (4) increases in correlations between PBQ-OR and all three post-deployment measures’ total scores with increasing report order. However, an overall sparse return rate of PBQ-OR ratings and a drop off in return rate of PBQ-OR ratings with increasing report order.

The sparse return rate of PBQ-OR reports is the most major limitation of this study and mirrors the well-documented key practical limitations of data assessment and documentation in military field operational research[18]. According to prior literature, the reliability and validity of in-theatre assessed psychometric measures is mostly threatened by non-response and deployment duration[19], as also seen in our study. War-zone-related research often goes hand in hand with unpredictable parameters, inconsistency in sampling practice, unit mobility, data storage, access and tracing issues, ineptness of structured interviews, time constraints, etc.[20], and, thus, introduces a broad spectrum of potentially quality-affecting specific features leading to sampling (e.g., non-response, assessing frame bias, data access) and non-sampling error types (e.g., interviewee-, rater- or scale-related errors)[21].

However, when taken together, our psychometric results suggest that PBQ-OR is a reliable and valid observer-rated measure for the global and objective assessment of combat-related peritraumatic symptoms and their underlying dimensions by UMP in currently deployed military personnel. We, thus, suggest that third-person, objectively recognized peritraumatic symptoms as measured by the PBQ-OR may constitute a valid and reliable screening for the assessment of combat-related peritraumatic reactions. There is a trend of better psychometric properties, when the PBQ-OR is administered towards the end of deployment. However, since exact time or frequency of combat-trauma exposure was not known, we cannot positively elucidate the reasons for this trend.

Because the content development of the PBQ focuses on behavioural indicators of peri-traumatic stress in the field of operations[10], the PBQ represents a uniquely appropriate peri-traumatic measure for military members.

The administration and scoring of the PBQ-OR is easily feasible due to clear and simple rating instructions and clearly specified assessment areas. In addition, its comparability to already established self-rated peritraumatic dissociation scales (e.g., Peritraumatic Dissociative Experience Questionnaire; Peritraumatic Distress Inventory)[22,23] is promoted through the 5-point-Likert scale structure applied. Such a screening tool could be used immediately after a traumatic event, but also periodically and longitudinally for monitoring. PBQ-OR represents an instrument to be used in real time, without interfering with concurrent military operations and relying on self-perception, recollection or self-report. No comparable measures have been developed so far for the assessment of acute peritraumatic-stress-related observable reactions in military Service Members, thus the PBQ may provide a template for future training of UMP. The use of PBQ for UMP training for recognition of and response to peritraumatic stress in the battlefield setting could be one of the main values of this measure.

In conclusion, PBQ-OR utilization could add up to more accurate and timely identification of peritraumatic reactions advancing the individual risk of service members for the imminent development of combat-related acute and posttraumatic stress symptoms. Regular PBQ-OR assessment could represent a focused prevention strategy through effective regular monitoring, facilitating earlier support and evidence-based treatment. The PBQ-OR ability to embody a regularly used measure with practical applicability and incremental validity in combat-related settings should, however, be prospectively validated through additional, larger-scale studies.

ACKNOWLEDGMENTS

We acknowledge CAPT, Brian Schumacher serving medical monitor and the Camp Pendleton Navy Corpsmen who participated in the PBQ development focus group. We would also like to thank the MRS team for their contribution and acknowledge staff of the Veteran Medical Research Foundation for their support.

COMMENTS

Background

Peritraumatic stress reactions include various behavioral, emotional, cognitive, and physiological symptoms associated with sympathetic activation during and immediately following a traumatic event. Prolonged continuation of these biological and psychological responses can lead to long-term adverse biological alterations, strongly associated with the subsequent development of posttraumatic stress disorder (PTSD).

Research frontiers

The assessment of the immediate individual response to trauma represents one of the most important challenges in traumatized populations.

Innovations and breakthroughs

This study contributes to the validation of the Peritraumatic Behavior Questionnaire - Observer Rated (PBQ-OR), the first third-person rated, objective instrument for the assessment of peritraumatic symptoms in combat-related settings.

Applications

The main findings from this study include: (1) satisfactory psychometric properties with good internal consistency for the PBQ-OR; (2) high convergent validity with respect to post-deployment PBQ - Self Rated (PBQ-SR) total score ratings; (3) high concurrent validity with respect to post-deployment PTSD symptoms as well as significant predictive validity with respect to PTSD caseness; and (4) increases in correlations between PBQ-OR and all three post-deployment measures’ total scores with increasing report order.

Terminology

PTSD: Posttraumatic stress disorder; PBQ-OR: Peritraumatic Behavior Questionnaire - Observer Rated; PBQ-SR: Peritraumatic Behavior Questionnaire - Self Rated.

Peer-review

This is a nice article presenting a useful instrument.