If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Correspondence: Melissa L.D. Christopher, PharmD, National Director, Pharmacy Benefits Management, Veterans Affairs Academic Detailing Services, 3350 La Jolla Village Drive, Mission Valley Anex Suite 130, San Diego, CA 92161.
To evaluate the effects of the U.S. Veterans Health Administration (VA) National Academic Detailing Service alongside the Opioid Overdose Education and Naloxone Distribution (OEND) program on naloxone prescriptions prescribed from October 2014 to September 2016.
A retrospective, repeated measures cohort study was conducted to evaluate the effectiveness of a real-world application of academic detailing (AD) alongside OEND on providers’ outpatient naloxone prescribing from October 2014 to September 2016. Outcome was the number of naloxone prescriptions prescribed per month per provider. During the study period, VA providers were aware of OEND, but may not have been exposed to academic detailing. Therefore, providers were categorized as exposed when the first OEND-specific academic detailing session was provided during the study period. Generalized estimating equations were used to estimate the association between exposure to academic detailing and monthly naloxone prescriptions prescribed while taking into account the correlation within each provider. Incident rate ratios with 95% CIs were reported.
Seven hundred fifty (22.6%) of 3313 providers received at least 1 OEND-specific academic detailing visit. At 1 year, the average number of naloxone prescriptions per month was 3-times greater in AD-exposed providers compared with AD-unexposed providers (95% CI 2.0-5.3); and at 2 years, the average number of naloxone prescriptions was 7-times greater (95% CI 3.0-17.9). Moreover, the average difference in naloxone prescribing from baseline to 2 years was 7.1% greater in AD-exposed providers compared with AD-unexposed providers (95% CI 2.0%-12.5%).
This preliminary analysis provides the first evidence that academic detailing influenced naloxone prescribing rates in a large, integrated health care system at 1 and 2 years. In addition, AD-exposed providers had a higher average difference in naloxone prescribing rate compared with AD-unexposed providers after 2 years of follow-up.
Opioid overdose mortality is a rising epidemic in the United States, increasing by 400% from 1999
The increased use of opioids and its association with opioid overdose mortality place veterans at increased risk. Consequently in 2014, the U.S. Veterans Health Administration (VA) implemented the first nationwide Opioid Overdose Education and Naloxone Distribution (OEND) program to reduce opioid overdose mortality in U.S. Veterans.
to reduce and prevent opioid overdose mortality among veterans. The OEND program developed national tools and resources to educate providers and patients, which includes the development of an online SharePoint site, quick start guides, naloxone kit brochures, patient educational materials, and posters. To achieve its goals, OEND partnered with the VA National Academic Detailing Service to educate providers and increase the distribution of naloxone.
Academic detailing (AD) is a service for clinicians by clinicians; it provides individualized, face-to-face outreach to encourage evidence-based decision making to improve veterans’ health.
Academic detailers, who are trained clinical pharmacists, use these interactions to deliver evidence-based research, data tools, and educational materials to change prescribing behavior and to improve adherence to best practices for a variety of clinical areas (e.g., alcohol use disorder, mental health, opioid safety). AD was initially piloted at VA facilities in California, Nevada, and the Pacific Islands, and it was completed in 2014.
After demonstrating effectiveness, a memorandum was issued by the Interim Under Secretary of Health to implement AD at all Veterans Integrated Service Networks by 2015.
Academic detailers delivered OEND-specific education about opioid safety, opioid overdose prevention, risk identification, and response with naloxone to VA providers through face-to-face interactions also known as educational outreach visits. Key elements of AD include educational outreach programming, clinical dashboard development, and application, and barrier resolutions.
Overdose rescues by trained and untrained participants and change in opioid use among substance-using participants in overdose education and naloxone distribution programs: a retrospective cohort study.
but the incremental treatment effect of using AD along with OEND to affect adoption of this practice into a health care system has not been studied. Therefore, on the basis of our current knowledge of AD and OEND, we hypothesize that providers exposed to AD will have a higher rate of naloxone prescribing compared with providers who were not exposed to AD alongside OEND in the VA.
The objective of this study was to evaluate the impact of AD alongside OEND on naloxone prescriptions prescribed from October 2014 to September 2016.
This was a retrospective, repeated measures cohort study to evaluate the effectiveness of a real-world application of AD with OEND on providers’ outpatient naloxone prescribing.
We evaluated the number of naloxone prescriptions prescribed per month in providers exposed to AD and providers not exposed to AD from October 2014 to September 2016. During the study period, it was assumed that all providers were aware of OEND, regardless of AD exposure. Providers were categorized as exposed when an academic detailer completed a documented OEND-specific AD session as part of the OEND program. A phased rollout of AD occurred nationally; therefore, providers who were not exposed to AD at the beginning of the study period may be considered exposed at a later time (time-dependent exposure). The outcome was the estimated number of naloxone prescriptions prescribed per month per provider. This study was reviewed and approved by the Edward J. Hines VA Institutional Review Board in accordance with VHA Handbook 1058.05.
VA provides coverage across all 50 U.S. states and its territories, including Puerto Rico, the Virgin Islands, American Samoa, Guam, and the Philippines Islands.
Providers were included if they were actively treating patients in primary care or substance-use disorder clinics during the study period (October 2014 to September 2016). Providers who wrote for acute opioid prescriptions, such as orthopedic surgeons, were excluded to focus on providers who had constant contact with their patients’ routine care. This was a closed cohort of VA providers; hence, providers were excluded if they started in the VA after the beginning of the study period or left the VA before the end of the study period. Providers were categorized as AD-exposed when they had an OEND-specific AD session during the study period; otherwise, they were categorized as AD-unexposed.
Naloxone prescription claims and provider-level data (age, sex, classification, length of employment) were derived from the VA Corporate Data Warehouse. Data on when providers received their first OEND-specific AD session were derived from Salesforce.com, a cloud-based platform that academic detailers used to record their educational outreach with providers.
Baseline comparisons between providers exposed and unexposed to AD were performed using Student t test with unequal variances for continuous data and χ2 test for discrete data. Longitudinal analysis was performed using generalized estimating equations (GEE) with a negative binomial distribution and auto-regressive correlation structure to account for repeated measure within each provider to estimate the association between provider exposure to AD and monthly naloxone prescriptions prescribed adjusting for potential confounders. Generalized estimating equations yield population average effects, which are interpreted as the average response in providers exposed and not exposed to AD.
Incidence rate ratio and 95% CI were reported at 1 and 2 years into the study. We included an interaction term between AD exposure and time (month) to capture the difference-in-differences estimate in naloxone prescribing across the study period within and between the AD-exposed and AD-unexposed groups. Difference-in-differences estimators represent the average differences in the change (rate) of naloxone prescriptions prescribed from baseline (first difference) between providers exposed and unexposed to AD (second difference).
We clustered on provider and controlled for baseline amount of naloxone prescribed, providers’ age, sex, classification (primary care or substance use disorder), and length of AD exposure. Statistical significance was defined as a two-tailed alpha of less than 5%. All analyses were performed using STATA/SE version 13 (StataCorp, College Station, TX).
Seven hundred fifty (22.6%) of 3313 providers received at least 1 OEND-specific AD visit (Table 1). Altogether, the majority of providers worked in primary care (90.9%), were female (52.1%), and were, on average, 54.7 years old. There was no statistically significant difference in baseline age (P = 0.672), length of employment (P = 0.316), and sex (P = 0.068) between providers exposed and not exposed to AD. Conversely, AD-exposed providers had a slightly larger proportion of primary care providers (94% versus 90%) and a slightly smaller proportion of substance-use disorder providers (6% versus 10%) compared to AD-unexposed providers.
Table 1Baseline demographics comparison between the groups
AD exposed (n = 750)
AD unexposed (n = 2563)
Total (n = 3313)
Age (y), mean (SD)
Male, n (%)
Primary care provider, n (%)
Substance user disorder provider, n (%)
Both (substance use disorder and primary care provider), n (%)
In the longitudinal analysis, AD-exposed providers were, on average, associated with an increased incidence rate for naloxone prescriptions prescribed by a factor of 3.2 (95% CI 2.0-5.3; P <0.001) compared with AD-unexposed providers at 1 year, while controlling for potential confounders (Figure 1). Similarly, AD-exposed providers had a higher incident rate of naloxone prescriptions prescribed that was 7-times greater than AD-unexposed providers controlling for potential confounders (95% CI 3.0-17.9; P <0.001). In the difference-in-differences estimation, AD-exposed providers had a 7.1% higher average change in naloxone rate compared with AD-unexposed providers (95% CI 2.0%-12.5%).
This was the first real-world evaluation of the impact of AD on naloxone in a large, integrated health care network alongside OEND. AD-exposed providers had higher rates of naloxone prescribed at years 1 and 2 compared with AD-unexposed providers. In addition, AD-exposed providers had a higher average difference in naloxone prescribing rate from baseline to 2 years compared with AD-unexposed providers. Taken together, these findings provide support that OEND with AD had the intended effect of increasing naloxone prescriptions during the study period.
We expected the number of naloxone prescriptions prescribed to increase over time because of the awareness of opioid overdose in the VA, largely because of the efforts by the OEND. The addition of naloxone to the VA National Formulary and the changing clinical climate also contributed to increases in naloxone prescriptions during the study period. However, before this study, the incremental value of AD alongside OEND was unclear. The nature of AD is rooted in its active approach to providing educational outreach. Similar to the methods of a drug representative, academic detailers provide direct, face-to-face educational outreach to providers to align prescribing behavior with best practices.
This face-to-face interaction is an active intervention that requires the academic detailer to identify and develop a relationship with the provider. Although active interventions are effective at promoting behavioral change,
Collaboration between the VA National Academic Detailing Service and OEND optimizes the effectiveness of these 2 programs at improving adherence to naloxone prescribing guidelines with the ultimate goal of reducing opioid overdose mortality. This was especially important in cases where provider reluctance prevented patients from receiving naloxone.
Providers have been reluctant to prescribe naloxone for a variety of factors. Green et al.
and colleagues performed a qualitative analysis to identify themes associated with reluctance by providers to prescribe naloxone for patients at risk for opioid overdose. Some of the themes identified included concerns that providers were providing patients with a “safety net” that would encourage continued abuse of opioids. However, respondents were also supportive of training in the form of OEND that would provide them with the knowledge and skills to prescribe naloxone effectively and safely. Furthermore, respondents stated that naloxone should not be a standalone solution. Rather, they indicated that a multifaceted approach is necessary to address the growing opioid overdose issue. AD combined with OEND offers such an approach, and the preliminary analyses from the current study suggest that the impact is significant.
This analysis provides decision makers and policy analysts with information on the incremental treatment effect of AD in combination with OEND. The OEND program nevertheless still influenced providers that were not exposed to AD as indicated by the rising number of naloxone prescriptions prescribed. However, combination of AD with OEND markedly increased naloxone prescriptions prescribed at years 1 and 2.
We are aware that certain limitations threaten the interval and external validity of this preliminary analysis. First, we were unable to control for selection bias in the study. Academic detailers targeted providers based on set criteria that might not be the same with another academic detailer. Regional differences can also influence how the academic detailers identify providers for an educational outreach. Future studies will need to incorporate more robust research methods such as regression discontinuity,
Second, we were unable to validate our assumption that all providers were aware of OEND at VA. Although the OEND program was implemented nationwide, there was the possibility that some providers may not have been aware of its existence. We justified our assumption based on the reach and interest of the OEND program.
Providers who sought more information regarding naloxone would likely encounter the OEND program’s resources, which included educational materials, recorded presentations, and naloxone instruction guides.
Third, measuring providers' adherence to guidelines is a challenging endeavor.
Oftentimes, evaluating a provider’s prescribing pattern functions as a surrogate for adherence. Despite an apparent association between the two, there is no strong justification to use prescribing pattern as a surrogate for guideline adherence. Surrogate measures of adherence to guidelines need to account for other elements, such as counseling patient on naloxone use, opioid overdose education, and emergency procedures for overdose. Checklists offer a mechanism to capture these elements associated with adherence to guidelines
; however, they are not always available. We chose to use naloxone prescriptions prescribed as a proxy for adherence to naloxone prescribing guidelines because it was retrievable from the VA Corporate Data Warehouse; however, this was not a complete measure of adherence. Future studies should consider adding a checklist or survey to capture important elements of adherence to confirm findings from the quantitative piece.
Finally, VA is an integrated health care system with a unique patient demographic that might not be reflective of the general U.S. population. Despite this limitation, other large, integrated health care systems or accountable care organizations interested in addressing the rising opioid overdose epidemic in their patient population can use the findings from this study as a benchmark for naloxone prescribing to at-risk patients. At the time of this analysis, we were unaware of another large, integrated system similar to VA that implemented AD to address the opioid overdose epidemic. Therefore, the results from this preliminary analysis can benefit decisions makers by providing empirical evidence to develop comprehensive programs to address the rising opioid overdose epidemic.
This preliminary analysis addressed some of the uncertainty regarding the impact of AD on naloxone prescribed alongside OEND. Providers exposed to AD had higher rates of naloxone prescriptions prescribed at years 1 and 2 compared with providers not exposed to AD from October 2014 to September 2016. Although these results are preliminary, they provide valuable information for policy makers in making future decisions to address opioid overdose mortality.
The authors thank Dr. Virginia Torrise, Deputy Chief Consultant, Professional Practice, and Mr. Michael Valentino, Chief Consultant, PBM, for providing the leadership and support to implement academic detailing at the U.S. Veterans Health Administration (VA); the Pharmacy Benefits Management Academic Detailing Service Staff and all the academic detailers throughout the VA; the VA Academic Detailing Advisory Board; and Veterans Integrated Service Network pharmacist executives leading academic detailing programs throughout the Veteran Healthcare system; the OEND National Workgroup Members who supported programming efforts and provided additional guidance and direction on academic detailing resource development; and the providers who care for our veterans every day with dedication and honor.
Centers for Disease Control and Prevention
Wide-ranging online data for epidemiologic research (WONDER).
Overdose rescues by trained and untrained participants and change in opioid use among substance-using participants in overdose education and naloxone distribution programs: a retrospective cohort study.
Mark Bounthavong, PharmD, MPH, Data Pharmacist Program Manager, VA Academic Detailing Program Office, and PhD Student, Pharmaceutical Outcomes Research and Policy Program, University of Washington, Seattle, WA
Michael Harvey, PharmD, BCPS, Regional Data Lead Pharmacist Manager, VA Academic Detailing Program Office, St. Paul, MN
Daina Wells, PharmD, BCPS, BCPP, Education and Training Pharmacist Program Manager, VA Academic Detailing Program Office, San Francisco, CA
Sarah Popish, PharmD, BCPP, Education and Training Pharmacist Program Manager, VA Academic Detailing Program Office, Sarasota, FL
Julianne Himstreet, PharmD, BCPS, Education and Training Pharmacist Program Manager, VA Academic Detailing Program Office, Eugene, OR
Elizabeth Oliva, PhD, VA National Opioid Overdose Education and Naloxone Distribution Coordinator, VA Program Evaluation and Resource Center, Menlo Park, CA
Chad Kay, PharmD, BCPS, Regional Data Lead Pharmacist Manager, VA Academic Detailing Program Office, St. Louis, MO
Marcos Lau, PharmD, MS, BCPS, Data Lead Pharmacist Manager, VA Academic Detailing Program Office, Los Angeles, CA
Priya Randeria, MHA, MPH, Management and Program Analyst, VA Academic Detailing Program Office, Los Angeles, CA
Andrea Phillips, JD, MPH, Management and Program Analyst, VA Academic Detailing Program Office, Long Beach, CA
Melissa L.D. Christopher, PharmD, National Director, VA Academic Detailing Program Office, San Diego, CA
Published online: January 12, 2017
Disclosure: The authors declare no conflicts of interest or financial interests in any product or service mentioned in this article. During the development, analysis, and manuscript preparation of this work, the authors were employees of the U.S. Veterans Health Administration, Department of Veterans Affairs.
Disclaimer: The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any agency of the U.S. Government. Assumptions made within the analysis are not reflective of the position of any U.S. Government entity.