Advertisement

Trends in naloxone prescriptions prescribed after implementation of a National Academic Detailing Service in the Veterans Health Administration: A preliminary analysis

Published:January 12, 2017DOI:https://doi.org/10.1016/j.japh.2016.11.003

      Abstract

      Objectives

      To evaluate the effects of the U.S. Veterans Health Administration (VA) National Academic Detailing Service alongside the Opioid Overdose Education and Naloxone Distribution (OEND) program on naloxone prescriptions prescribed from October 2014 to September 2016.

      Methods

      A retrospective, repeated measures cohort study was conducted to evaluate the effectiveness of a real-world application of academic detailing (AD) alongside OEND on providers’ outpatient naloxone prescribing from October 2014 to September 2016. Outcome was the number of naloxone prescriptions prescribed per month per provider. During the study period, VA providers were aware of OEND, but may not have been exposed to academic detailing. Therefore, providers were categorized as exposed when the first OEND-specific academic detailing session was provided during the study period. Generalized estimating equations were used to estimate the association between exposure to academic detailing and monthly naloxone prescriptions prescribed while taking into account the correlation within each provider. Incident rate ratios with 95% CIs were reported.

      Results

      Seven hundred fifty (22.6%) of 3313 providers received at least 1 OEND-specific academic detailing visit. At 1 year, the average number of naloxone prescriptions per month was 3-times greater in AD-exposed providers compared with AD-unexposed providers (95% CI 2.0-5.3); and at 2 years, the average number of naloxone prescriptions was 7-times greater (95% CI 3.0-17.9). Moreover, the average difference in naloxone prescribing from baseline to 2 years was 7.1% greater in AD-exposed providers compared with AD-unexposed providers (95% CI 2.0%-12.5%).

      Conclusions

      This preliminary analysis provides the first evidence that academic detailing influenced naloxone prescribing rates in a large, integrated health care system at 1 and 2 years. In addition, AD-exposed providers had a higher average difference in naloxone prescribing rate compared with AD-unexposed providers after 2 years of follow-up.

      Introduction

      Opioid overdose mortality is a rising epidemic in the United States, increasing by 400% from 1999
      • Centers for Disease Control and Prevention
      Wide-ranging online data for epidemiologic research (WONDER).
      and surpassing motor vehicle mortality by 150% in 2014.
      • Rudd R.A.
      • Aleshire N.
      • Zibbell J.E.
      • Gladden R.M.
      Increases in drug and opioid overdose deaths–United States, 2000-2014.
      In particular, Veterans are twice as likely to die from opioid overdose compared to the general U.S. population (mortality rate = 1.96 [95% CI 1.83-2.08]).
      • Bohnert A.S.B.
      • Ilgen M.A.
      • Galea S.
      • McCarthy J.F.
      • Blow F.C.
      Accidental poisoning mortality among patients in the Department of Veterans Affairs Health System.
      Between 2001 and 2007, opioid prescribing in veterans increased by 184% for methadone, 60% for synthetic and semisynthetic opioids, and 35% for nonsynthetic opioids.
      • Bohnert A.S.B.
      • Ilgen M.A.
      • Trafton J.A.
      • et al.
      Trends and regional variation in opioid overdose mortality among Veterans Health Administration patients, fiscal year 2001 to 2009.
      The increased use of opioids and its association with opioid overdose mortality place veterans at increased risk. Consequently in 2014, the U.S. Veterans Health Administration (VA) implemented the first nationwide Opioid Overdose Education and Naloxone Distribution (OEND) program to reduce opioid overdose mortality in U.S. Veterans.
      • Oliva E.M.
      • Nevedal A.
      • Lewis E.T.
      • et al.
      Patient perspectives on an opioid overdose education and naloxone distribution program in the US Department of Veterans Affairs.
      The goals of the OEND are to educate veterans, their friends and family, and providers to improve access to naloxone, an effective opioid reversal agent,
      • Robinson A.
      • Wermeling D.P.
      Intranasal naloxone administration for treatment of opioid overdose.
      • Sumner S.A.
      • Mercado-Crespo M.C.
      • Spelke M.B.
      • et al.
      Use of naloxone by emergency medical services during opioid drug overdose resuscitation efforts.
      to reduce and prevent opioid overdose mortality among veterans. The OEND program developed national tools and resources to educate providers and patients, which includes the development of an online SharePoint site, quick start guides, naloxone kit brochures, patient educational materials, and posters. To achieve its goals, OEND partnered with the VA National Academic Detailing Service to educate providers and increase the distribution of naloxone.
      Academic detailing (AD) is a service for clinicians by clinicians; it provides individualized, face-to-face outreach to encourage evidence-based decision making to improve veterans’ health.
      • Soumerai S.B.
      • Avorn J.
      Principles of educational outreach (‘academic detailing’) to improve clinical decision making.
      • Avorn J.
      • Soumerai S.B.
      Improving drug-therapy decisions through educational outreach.
      • Wells D.L.
      • Popish S.
      • Kay C.
      • Torrise V.
      • Christopher M.L.
      VA academic detailing service: implementation and lessons learned.
      Academic detailers, who are trained clinical pharmacists, use these interactions to deliver evidence-based research, data tools, and educational materials to change prescribing behavior and to improve adherence to best practices for a variety of clinical areas (e.g., alcohol use disorder, mental health, opioid safety). AD was initially piloted at VA facilities in California, Nevada, and the Pacific Islands, and it was completed in 2014.
      • Wells D.L.
      • Popish S.
      • Kay C.
      • Torrise V.
      • Christopher M.L.
      VA academic detailing service: implementation and lessons learned.
      After demonstrating effectiveness, a memorandum was issued by the Interim Under Secretary of Health to implement AD at all Veterans Integrated Service Networks by 2015.
      Academic detailers delivered OEND-specific education about opioid safety, opioid overdose prevention, risk identification, and response with naloxone to VA providers through face-to-face interactions also known as educational outreach visits. Key elements of AD include educational outreach programming, clinical dashboard development, and application, and barrier resolutions.
      • Wells D.L.
      • Popish S.
      • Kay C.
      • Torrise V.
      • Christopher M.L.
      VA academic detailing service: implementation and lessons learned.
      Although prior studies have reported that AD has been successful in aligning provider prescribing behavior with antihypertensive guidelines,
      • Siegel D.
      • Lopez J.
      • Meier J.
      • et al.
      Academic detailing to improve antihypertensive prescribing patterns.
      judicious use of antimicrobial agents,
      • Kisuule F.
      • Wright S.
      • Barreto J.
      • Zenilman J.
      Improving antibiotic utilization among hospitalists: a pilot academic detailing project with a public health approach.
      adherence to alcohol use disorder treatment,
      • Harris A.H.S.
      • Bowe T.
      • Hagedorn H.
      • et al.
      Multifaceted academic detailing program to increase pharmacotherapy for alcohol use disorder: interrupted time series evaluation of effectiveness.
      and reduction in medication errors,
      • Shaw J.
      • Harris P.
      • Keogh G.
      • Graudins L.
      • Perks E.
      • Thomas P.S.
      Error reduction: academic detailing as a method to reduce incorrect prescriptions.
      it is unknown whether AD can affect a provider’s adherence to naloxone treatment guidelines (measured as the number of naloxone prescriptions prescribed).
      Previous studies have reported on the effectiveness of OEND on reducing opioid overdose mortality,
      • Clark A.K.
      • Wilder C.M.
      • Winstanley E.L.
      A systematic review of community opioid overdose prevention and naloxone distribution programs.
      • Walley A.Y.
      • Xuan Z.
      • Hackman H.H.
      • et al.
      Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis.
      • Doe-Simkins M.
      • Quinn E.
      • Xuan Z.
      • et al.
      Overdose rescues by trained and untrained participants and change in opioid use among substance-using participants in overdose education and naloxone distribution programs: a retrospective cohort study.
      • Binswanger I.A.
      • Koester S.
      • Mueller S.R.
      • Gardner E.M.
      • Goddard K.
      • Glanz J.M.
      Overdose education and naloxone for patients prescribed opioids in primary care: a qualitative study of primary care staff.
      but the incremental treatment effect of using AD along with OEND to affect adoption of this practice into a health care system has not been studied. Therefore, on the basis of our current knowledge of AD and OEND, we hypothesize that providers exposed to AD will have a higher rate of naloxone prescribing compared with providers who were not exposed to AD alongside OEND in the VA.

      Objectives

      The objective of this study was to evaluate the impact of AD alongside OEND on naloxone prescriptions prescribed from October 2014 to September 2016.

      Methods

       Design

      This was a retrospective, repeated measures cohort study to evaluate the effectiveness of a real-world application of AD with OEND on providers’ outpatient naloxone prescribing.
      We evaluated the number of naloxone prescriptions prescribed per month in providers exposed to AD and providers not exposed to AD from October 2014 to September 2016. During the study period, it was assumed that all providers were aware of OEND, regardless of AD exposure. Providers were categorized as exposed when an academic detailer completed a documented OEND-specific AD session as part of the OEND program. A phased rollout of AD occurred nationally; therefore, providers who were not exposed to AD at the beginning of the study period may be considered exposed at a later time (time-dependent exposure). The outcome was the estimated number of naloxone prescriptions prescribed per month per provider. This study was reviewed and approved by the Edward J. Hines VA Institutional Review Board in accordance with VHA Handbook 1058.05.
      • Office of Research Oversight
      VHA Operations Activities That May Constitute Research.

       Sample

      VA is the largest integrated health care system in the United States, with 168 VA Medical Centers and 1053 VA Outpatient Clinics serving approximately 8.9 million veterans annually.
      • Veterans Health Administration
      Reports–National Center for Veterans Analysis and Statistics.
      VA provides coverage across all 50 U.S. states and its territories, including Puerto Rico, the Virgin Islands, American Samoa, Guam, and the Philippines Islands.
      Providers were included if they were actively treating patients in primary care or substance-use disorder clinics during the study period (October 2014 to September 2016). Providers who wrote for acute opioid prescriptions, such as orthopedic surgeons, were excluded to focus on providers who had constant contact with their patients’ routine care. This was a closed cohort of VA providers; hence, providers were excluded if they started in the VA after the beginning of the study period or left the VA before the end of the study period. Providers were categorized as AD-exposed when they had an OEND-specific AD session during the study period; otherwise, they were categorized as AD-unexposed.

       Data source

      Naloxone prescription claims and provider-level data (age, sex, classification, length of employment) were derived from the VA Corporate Data Warehouse. Data on when providers received their first OEND-specific AD session were derived from Salesforce.com, a cloud-based platform that academic detailers used to record their educational outreach with providers.

       Analysis

      Baseline comparisons between providers exposed and unexposed to AD were performed using Student t test with unequal variances for continuous data and χ2 test for discrete data. Longitudinal analysis was performed using generalized estimating equations (GEE) with a negative binomial distribution and auto-regressive correlation structure to account for repeated measure within each provider to estimate the association between provider exposure to AD and monthly naloxone prescriptions prescribed adjusting for potential confounders. Generalized estimating equations yield population average effects, which are interpreted as the average response in providers exposed and not exposed to AD.
      • Ballinger G.A.
      Using generalized estimating equations for longitudinal data analysis.
      • French B.
      • Heagerty P.J.
      Analysis of longitudinal data to evaluate a policy change.
      • Liang K.-Y.
      • Zeger S.L.
      Longitudinal data analysis using generalized linear models.
      Incidence rate ratio and 95% CI were reported at 1 and 2 years into the study. We included an interaction term between AD exposure and time (month) to capture the difference-in-differences estimate in naloxone prescribing across the study period within and between the AD-exposed and AD-unexposed groups. Difference-in-differences estimators represent the average differences in the change (rate) of naloxone prescriptions prescribed from baseline (first difference) between providers exposed and unexposed to AD (second difference).
      • Villa J.M.
      Simplifying the estimation of difference in differences treatment effects with Stata.
      We clustered on provider and controlled for baseline amount of naloxone prescribed, providers’ age, sex, classification (primary care or substance use disorder), and length of AD exposure. Statistical significance was defined as a two-tailed alpha of less than 5%. All analyses were performed using STATA/SE version 13 (StataCorp, College Station, TX).

      Results

      Seven hundred fifty (22.6%) of 3313 providers received at least 1 OEND-specific AD visit (Table 1). Altogether, the majority of providers worked in primary care (90.9%), were female (52.1%), and were, on average, 54.7 years old. There was no statistically significant difference in baseline age (P = 0.672), length of employment (P = 0.316), and sex (P = 0.068) between providers exposed and not exposed to AD. Conversely, AD-exposed providers had a slightly larger proportion of primary care providers (94% versus 90%) and a slightly smaller proportion of substance-use disorder providers (6% versus 10%) compared to AD-unexposed providers.
      Table 1Baseline demographics comparison between the groups
      VariablesAD exposed (n = 750)AD unexposed (n = 2563)Total (n = 3313)P value
      Age (y), mean (SD)54.5 (8.7)54.7 (8.8)54.7 (8.8)0.672
      Male, n (%)381 (50.8)1205 (47.0)1586 (47.9)0.068
      Primary care provider, n (%)704 (93.9)2308 (90.1)3012 (90.9)0.002
      Substance user disorder provider, n (%)42 (5.6)246 (9.6)288 (8.7)0.002
      Both (substance use disorder and primary care provider), n (%)4 (0.5)9 (0.4)12 (0.4)0.002
      Months worked at the VA, mean (SD)143.2 (99.1)147.3 (104.9)146.4 (103.6)0.316
      Abbreviation used: AD, academic detailing.
      In the longitudinal analysis, AD-exposed providers were, on average, associated with an increased incidence rate for naloxone prescriptions prescribed by a factor of 3.2 (95% CI 2.0-5.3; P <0.001) compared with AD-unexposed providers at 1 year, while controlling for potential confounders (Figure 1). Similarly, AD-exposed providers had a higher incident rate of naloxone prescriptions prescribed that was 7-times greater than AD-unexposed providers controlling for potential confounders (95% CI 3.0-17.9; P <0.001). In the difference-in-differences estimation, AD-exposed providers had a 7.1% higher average change in naloxone rate compared with AD-unexposed providers (95% CI 2.0%-12.5%).
      Figure thumbnail gr1
      Figure 1Naloxone kits monthly prescribing rates from October 2014 to September 2016.

      Discussion

      This was the first real-world evaluation of the impact of AD on naloxone in a large, integrated health care network alongside OEND. AD-exposed providers had higher rates of naloxone prescribed at years 1 and 2 compared with AD-unexposed providers. In addition, AD-exposed providers had a higher average difference in naloxone prescribing rate from baseline to 2 years compared with AD-unexposed providers. Taken together, these findings provide support that OEND with AD had the intended effect of increasing naloxone prescriptions during the study period.
      We expected the number of naloxone prescriptions prescribed to increase over time because of the awareness of opioid overdose in the VA, largely because of the efforts by the OEND. The addition of naloxone to the VA National Formulary and the changing clinical climate also contributed to increases in naloxone prescriptions during the study period. However, before this study, the incremental value of AD alongside OEND was unclear. The nature of AD is rooted in its active approach to providing educational outreach. Similar to the methods of a drug representative, academic detailers provide direct, face-to-face educational outreach to providers to align prescribing behavior with best practices.
      • Soumerai S.B.
      • Avorn J.
      Principles of educational outreach (‘academic detailing’) to improve clinical decision making.
      • Avorn J.
      • Soumerai S.B.
      Improving drug-therapy decisions through educational outreach.
      This face-to-face interaction is an active intervention that requires the academic detailer to identify and develop a relationship with the provider. Although active interventions are effective at promoting behavioral change,
      • Albarracín D.
      • Gillette J.C.
      • Earl A.N.
      • Glasman L.R.
      • Durantini M.R.
      • Ho M.-H.
      A test of major assumptions about behavior change: a comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic.
      a multifaceted approach is often required to promote provider prescribing changes.
      • Ajzen I.
      The theory of planned behavior.
      • Hameed M.A.
      • Counsell S.
      • Swift S.
      A conceptual model for the process of IT innovation adoption in organizations.
      Collaboration between the VA National Academic Detailing Service and OEND optimizes the effectiveness of these 2 programs at improving adherence to naloxone prescribing guidelines with the ultimate goal of reducing opioid overdose mortality. This was especially important in cases where provider reluctance prevented patients from receiving naloxone.
      Providers have been reluctant to prescribe naloxone for a variety of factors. Green et al.
      • Green T.C.
      • Bowman S.E.
      • Zaller N.D.
      • Ray M.
      • Case P.
      • Heimer R.
      Barriers to medical provider support for prescription naloxone as overdose antidote for lay responders.
      and colleagues performed a qualitative analysis to identify themes associated with reluctance by providers to prescribe naloxone for patients at risk for opioid overdose. Some of the themes identified included concerns that providers were providing patients with a “safety net” that would encourage continued abuse of opioids. However, respondents were also supportive of training in the form of OEND that would provide them with the knowledge and skills to prescribe naloxone effectively and safely. Furthermore, respondents stated that naloxone should not be a standalone solution. Rather, they indicated that a multifaceted approach is necessary to address the growing opioid overdose issue. AD combined with OEND offers such an approach, and the preliminary analyses from the current study suggest that the impact is significant.
      This analysis provides decision makers and policy analysts with information on the incremental treatment effect of AD in combination with OEND. The OEND program nevertheless still influenced providers that were not exposed to AD as indicated by the rising number of naloxone prescriptions prescribed. However, combination of AD with OEND markedly increased naloxone prescriptions prescribed at years 1 and 2.

      Limitations

      We are aware that certain limitations threaten the interval and external validity of this preliminary analysis. First, we were unable to control for selection bias in the study. Academic detailers targeted providers based on set criteria that might not be the same with another academic detailer. Regional differences can also influence how the academic detailers identify providers for an educational outreach. Future studies will need to incorporate more robust research methods such as regression discontinuity,
      • Thistlethwaite D.L.
      • Campbell D.T.
      Regression-discontinuity analysis: an alternative to the ex post facto experiment.
      • Imbens G.W.
      • Lemieux T.
      Regression discontinuity designs: a guide to practice.
      • Lee D.S.
      • Lemieux T.
      Regression discontinuity designs in economics.
      instrumental variables,
      • Greenland S.
      An introduction to instrumental variables for epidemiologists.
      • Staiger D.
      • Stock J.H.
      Instrumental variables regression with weak instruments.
      and propensity score matching
      • Austin P.C.
      An introduction to propensity score methods for reducing the effects of confounding in observational studies.
      • Rosenbaum P.R.
      • Rubin D.B.
      The central role of the propensity score in observational studies for causal effects.
      to reduce selection bias.
      Second, we were unable to validate our assumption that all providers were aware of OEND at VA. Although the OEND program was implemented nationwide, there was the possibility that some providers may not have been aware of its existence. We justified our assumption based on the reach and interest of the OEND program.
      • Oliva E.M.
      • Nevedal A.
      • Lewis E.T.
      • et al.
      Patient perspectives on an opioid overdose education and naloxone distribution program in the US Department of Veterans Affairs.
      • Tiffany E.
      • Wilder C.M.
      • Miller S.C.
      • Winhusen T.
      Knowledge of and interest in opioid overdose education and naloxone distribution among US veterans on chronic opioids for addiction or pain.
      After the OEND program was implemented, naloxone was added to the VA National Formulary, which may have prompted interest in naloxone prescribing.
      • U.S. Department of Veterans Affairs
      Pharmacy Benefits and Management Services. National Formulary.
      Providers who sought more information regarding naloxone would likely encounter the OEND program’s resources, which included educational materials, recorded presentations, and naloxone instruction guides.
      Third, measuring providers' adherence to guidelines is a challenging endeavor.
      • Sugarman J.R.
      Challenges in measuring adherence to clinical practice guidelines.
      Oftentimes, evaluating a provider’s prescribing pattern functions as a surrogate for adherence. Despite an apparent association between the two, there is no strong justification to use prescribing pattern as a surrogate for guideline adherence. Surrogate measures of adherence to guidelines need to account for other elements, such as counseling patient on naloxone use, opioid overdose education, and emergency procedures for overdose. Checklists offer a mechanism to capture these elements associated with adherence to guidelines
      • James P.A.
      • Cowan T.M.
      • Graham R.P.
      • Majeroni B.A.
      • Fox C.H.
      • Jaén C.R.
      Using a clinical practice guideline to measure physician practice: translating a guideline for the management of heart failure.
      ; however, they are not always available. We chose to use naloxone prescriptions prescribed as a proxy for adherence to naloxone prescribing guidelines because it was retrievable from the VA Corporate Data Warehouse; however, this was not a complete measure of adherence. Future studies should consider adding a checklist or survey to capture important elements of adherence to confirm findings from the quantitative piece.
      Finally, VA is an integrated health care system with a unique patient demographic that might not be reflective of the general U.S. population. Despite this limitation, other large, integrated health care systems or accountable care organizations interested in addressing the rising opioid overdose epidemic in their patient population can use the findings from this study as a benchmark for naloxone prescribing to at-risk patients. At the time of this analysis, we were unaware of another large, integrated system similar to VA that implemented AD to address the opioid overdose epidemic. Therefore, the results from this preliminary analysis can benefit decisions makers by providing empirical evidence to develop comprehensive programs to address the rising opioid overdose epidemic.

      Conclusions

      This preliminary analysis addressed some of the uncertainty regarding the impact of AD on naloxone prescribed alongside OEND. Providers exposed to AD had higher rates of naloxone prescriptions prescribed at years 1 and 2 compared with providers not exposed to AD from October 2014 to September 2016. Although these results are preliminary, they provide valuable information for policy makers in making future decisions to address opioid overdose mortality.

      Acknowledgment

      The authors thank Dr. Virginia Torrise, Deputy Chief Consultant, Professional Practice, and Mr. Michael Valentino, Chief Consultant, PBM, for providing the leadership and support to implement academic detailing at the U.S. Veterans Health Administration (VA); the Pharmacy Benefits Management Academic Detailing Service Staff and all the academic detailers throughout the VA; the VA Academic Detailing Advisory Board; and Veterans Integrated Service Network pharmacist executives leading academic detailing programs throughout the Veteran Healthcare system; the OEND National Workgroup Members who supported programming efforts and provided additional guidance and direction on academic detailing resource development; and the providers who care for our veterans every day with dedication and honor.

      References

        • Centers for Disease Control and Prevention
        Wide-ranging online data for epidemiologic research (WONDER).
        (Available at:) (Accessed September 7, 2016)
        • Rudd R.A.
        • Aleshire N.
        • Zibbell J.E.
        • Gladden R.M.
        Increases in drug and opioid overdose deaths–United States, 2000-2014.
        MMWR Morb Mortal Wkly Rep. 2016; 64: 1378-1382
        • Bohnert A.S.B.
        • Ilgen M.A.
        • Galea S.
        • McCarthy J.F.
        • Blow F.C.
        Accidental poisoning mortality among patients in the Department of Veterans Affairs Health System.
        Med Care. 2011; 49: 393-396
        • Bohnert A.S.B.
        • Ilgen M.A.
        • Trafton J.A.
        • et al.
        Trends and regional variation in opioid overdose mortality among Veterans Health Administration patients, fiscal year 2001 to 2009.
        Clin J Pain. 2014; 30: 605-612
        • Oliva E.M.
        • Nevedal A.
        • Lewis E.T.
        • et al.
        Patient perspectives on an opioid overdose education and naloxone distribution program in the US Department of Veterans Affairs.
        Subst Abuse. 2016; 37: 118-126
        • Robinson A.
        • Wermeling D.P.
        Intranasal naloxone administration for treatment of opioid overdose.
        Am J Health Syst Pharm. 2014; 71: 2129-2135
        • Sumner S.A.
        • Mercado-Crespo M.C.
        • Spelke M.B.
        • et al.
        Use of naloxone by emergency medical services during opioid drug overdose resuscitation efforts.
        Prehospital Emerg Care. 2016; 20: 220-225
        • Soumerai S.B.
        • Avorn J.
        Principles of educational outreach (‘academic detailing’) to improve clinical decision making.
        JAMA. 1990; 263: 549-556
        • Avorn J.
        • Soumerai S.B.
        Improving drug-therapy decisions through educational outreach.
        N Engl J Med. 1983; 308: 1457-1463
        • Wells D.L.
        • Popish S.
        • Kay C.
        • Torrise V.
        • Christopher M.L.
        VA academic detailing service: implementation and lessons learned.
        Fed Pract. 2016; 33: 66-69
        • Siegel D.
        • Lopez J.
        • Meier J.
        • et al.
        Academic detailing to improve antihypertensive prescribing patterns.
        Am J Hypertens. 2003; 16: 508-511
        • Kisuule F.
        • Wright S.
        • Barreto J.
        • Zenilman J.
        Improving antibiotic utilization among hospitalists: a pilot academic detailing project with a public health approach.
        J Hosp Med. 2008; 3: 64-70
        • Harris A.H.S.
        • Bowe T.
        • Hagedorn H.
        • et al.
        Multifaceted academic detailing program to increase pharmacotherapy for alcohol use disorder: interrupted time series evaluation of effectiveness.
        Addict Sci Clin Pract. 2016; 11: 15
        • Shaw J.
        • Harris P.
        • Keogh G.
        • Graudins L.
        • Perks E.
        • Thomas P.S.
        Error reduction: academic detailing as a method to reduce incorrect prescriptions.
        Eur J Clin Pharmacol. 2003; 59: 697-699
        • Clark A.K.
        • Wilder C.M.
        • Winstanley E.L.
        A systematic review of community opioid overdose prevention and naloxone distribution programs.
        J Addict Med. 2014; 8: 153-163
        • Walley A.Y.
        • Xuan Z.
        • Hackman H.H.
        • et al.
        Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis.
        BMJ. 2013; 346: f174
        • Doe-Simkins M.
        • Quinn E.
        • Xuan Z.
        • et al.
        Overdose rescues by trained and untrained participants and change in opioid use among substance-using participants in overdose education and naloxone distribution programs: a retrospective cohort study.
        BMC Public Health. 2014; 14: 297
        • Binswanger I.A.
        • Koester S.
        • Mueller S.R.
        • Gardner E.M.
        • Goddard K.
        • Glanz J.M.
        Overdose education and naloxone for patients prescribed opioids in primary care: a qualitative study of primary care staff.
        J Gen Intern Med. 2015; 30: 1837-1844
        • Office of Research Oversight
        VHA Operations Activities That May Constitute Research.
        (VHA Handbook 1085.05. Published October 28, 2011; signed by Robert A. Petzel, MD. Available at:) (Accessed September 14, 2016)
        • Veterans Health Administration
        Reports–National Center for Veterans Analysis and Statistics.
        (Available at:) (Accessed August 26, 2016)
        • Ballinger G.A.
        Using generalized estimating equations for longitudinal data analysis.
        Organ Res Methods. 2004; 7: 127-150
        • French B.
        • Heagerty P.J.
        Analysis of longitudinal data to evaluate a policy change.
        Stat Med. 2008; 27: 5005-5025
        • Liang K.-Y.
        • Zeger S.L.
        Longitudinal data analysis using generalized linear models.
        Biometrika. 1986; 73: 13-22
        • Villa J.M.
        Simplifying the estimation of difference in differences treatment effects with Stata.
        University Library of Munich, Germany2012 (Available at:) (Accessed May 2, 2016)
        • Albarracín D.
        • Gillette J.C.
        • Earl A.N.
        • Glasman L.R.
        • Durantini M.R.
        • Ho M.-H.
        A test of major assumptions about behavior change: a comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic.
        Psychol Bull. 2005; 131: 856-897
        • Ajzen I.
        The theory of planned behavior.
        Organ Behav Hum Decis Process. 1991; 50: 179-211
        • Hameed M.A.
        • Counsell S.
        • Swift S.
        A conceptual model for the process of IT innovation adoption in organizations.
        J Eng Technol Manag. 2012; 29: 358-390
        • Green T.C.
        • Bowman S.E.
        • Zaller N.D.
        • Ray M.
        • Case P.
        • Heimer R.
        Barriers to medical provider support for prescription naloxone as overdose antidote for lay responders.
        Subst Use Misuse. 2013; 48: 558-567
        • Thistlethwaite D.L.
        • Campbell D.T.
        Regression-discontinuity analysis: an alternative to the ex post facto experiment.
        J Educ Psychol. 1960; 51: 309-317
        • Imbens G.W.
        • Lemieux T.
        Regression discontinuity designs: a guide to practice.
        J Econom. 2008; 142: 615-635
        • Lee D.S.
        • Lemieux T.
        Regression discontinuity designs in economics.
        J Econ Lit. 2010; 48: 281-355
        • Greenland S.
        An introduction to instrumental variables for epidemiologists.
        Int J Epidemiol. 2000; 29: 722-729
        • Staiger D.
        • Stock J.H.
        Instrumental variables regression with weak instruments.
        Econometrica. 1997; 65: 557-586
        • Austin P.C.
        An introduction to propensity score methods for reducing the effects of confounding in observational studies.
        Multivar Behav Res. 2011; 46: 399-424
        • Rosenbaum P.R.
        • Rubin D.B.
        The central role of the propensity score in observational studies for causal effects.
        Biometrika. 1983; 70: 41-55
        • Tiffany E.
        • Wilder C.M.
        • Miller S.C.
        • Winhusen T.
        Knowledge of and interest in opioid overdose education and naloxone distribution among US veterans on chronic opioids for addiction or pain.
        Drugs Educ Prev Policy. 2016; 23: 322-327
        • U.S. Department of Veterans Affairs
        Pharmacy Benefits and Management Services. National Formulary.
        (Available at:) (Accessed June 23, 2013)
        • Sugarman J.R.
        Challenges in measuring adherence to clinical practice guidelines.
        J Am Board Fam Pract Am Board Fam Pract. 1997; 10: 237-239
        • James P.A.
        • Cowan T.M.
        • Graham R.P.
        • Majeroni B.A.
        • Fox C.H.
        • Jaén C.R.
        Using a clinical practice guideline to measure physician practice: translating a guideline for the management of heart failure.
        J Am Board Fam Pract. 1997; 10: 206-212

      Biography

      Mark Bounthavong, PharmD, MPH, Data Pharmacist Program Manager, VA Academic Detailing Program Office, and PhD Student, Pharmaceutical Outcomes Research and Policy Program, University of Washington, Seattle, WA
      Michael Harvey, PharmD, BCPS, Regional Data Lead Pharmacist Manager, VA Academic Detailing Program Office, St. Paul, MN
      Daina Wells, PharmD, BCPS, BCPP, Education and Training Pharmacist Program Manager, VA Academic Detailing Program Office, San Francisco, CA
      Sarah Popish, PharmD, BCPP, Education and Training Pharmacist Program Manager, VA Academic Detailing Program Office, Sarasota, FL
      Julianne Himstreet, PharmD, BCPS, Education and Training Pharmacist Program Manager, VA Academic Detailing Program Office, Eugene, OR
      Elizabeth Oliva, PhD, VA National Opioid Overdose Education and Naloxone Distribution Coordinator, VA Program Evaluation and Resource Center, Menlo Park, CA
      Chad Kay, PharmD, BCPS, Regional Data Lead Pharmacist Manager, VA Academic Detailing Program Office, St. Louis, MO
      Marcos Lau, PharmD, MS, BCPS, Data Lead Pharmacist Manager, VA Academic Detailing Program Office, Los Angeles, CA
      Priya Randeria, MHA, MPH, Management and Program Analyst, VA Academic Detailing Program Office, Los Angeles, CA
      Andrea Phillips, JD, MPH, Management and Program Analyst, VA Academic Detailing Program Office, Long Beach, CA
      Melissa L.D. Christopher, PharmD, National Director, VA Academic Detailing Program Office, San Diego, CA