Applying the Intention-Action Framework to Specialty Referrals

Low-value care-services that provide little or no clinical benefit-is a significant problem in the United States. For example, low-value specialty referrals often lead to unnecessary tests, follow-ups, and procedures that burden patients, weaken trust, and reduce access to specialty care for those with greater clinical need. This study is a six-month, single-site, randomized controlled trial conducted at UCLA Health to evaluate an electronic health record (EHR)-embedded behavioral intervention designed to reduce low-value specialty referrals in cardiology, pulmonology, and gastroenterology. The study focuses on physician decision-making at the point of referral and tests whether structured, real-time decision support can reduce low-value care while maintaining patient safety and access to appropriate specialty services. The study employs a parallel-arm, randomized design. Physicians are randomized to one of three study arms and will remain in their assigned condition throughout the six-month intervention period. The intervention is delivered automatically within routine clinical workflow. * Arm 1 - Control: The order composer allows physicians to place referrals with minimal information displayed. * Arm 2 - Information + Review: The order composer displays criteria for high-value referrals (developed by UCLA Health using professional society guidelines and expert consensus from clinical leadership) and informs physicians that their referral decisions may be reviewed. * Arm 3 - Information + Review + Checklist: The order composer displays referral criteria and informs physicians that their referral decisions may be reviewed. The order composer also includes cascading checkboxes that prompt physicians to confirm that referral criteria are met before submitting a referral. The target study population consists of actively practicing UCLA Health physicians who have placed at least one specialty referral to cardiology, pulmonology, or gastroenterology within six months during the baseline period. Randomization will be conducted by the study team prior to trial initiation using a computerized procedure. The study team will generate 100,000 candidate random allocations of providers to one of three study arms in approximately equal proportions. For each candidate allocation, we assess balance across the following five metrics derived from the baseline period: referral volume to each of the three target specialties (cardiology, GI, and pulmonology) during the baseline period, assessed separately for each specialty (using ANOVA F-test p-values); physician's modal target specialty (using chi-square p-values); physician's department group (using chi-square p-values). The top 1% of allocations achieving the best simultaneous balance across all five metrics (maximin criterion) are retained, and the final allocation is selected at random from this set. This study does not employ traditional blinding. Physician participants are aware of the content of the order panel they encounter, as the intervention is delivered through their routine EHR workflow. However, physicians are not informed that they are participating in a research study prior to the trial, to avoid influencing referral behavior. The research team conducting outcome analyses does not interact with participants and outcomes are derived from objective EHR data, minimizing the risk of ascertainment bias. Analysis Plan: Primary Analyses: The investigators will estimate intent-to-treat effects using ordinary least squares (OLS) regressions models with heteroskedasticity-robust standard errors to predict outcome variables. All analyses will be conducted at the physician level. * The primary dependent variable will be the physician-level rate of low-value specialty referrals per 100 referral relevant encounters during the six-month intervention period. A referral-relevant encounter is defined as a clinical encounter in which the physician bills an ICD-10 diagnosis code associated with one of the target medical conditions for which referral criteria have been developed: hyperlipidemia or hypertension (for cardiology), cough, asthma, COPD, or incidental lung nodule (for pulmonology), and GERD, constipation, acute diarrhea, or chronic diarrhea (for gastroenterology). Whether a referral meets the pre-specified criteria will be assessed using a combination of three approaches: (1) structured EHR data, including laboratory values, medication records, imaging orders, and diagnosis codes; (2) physician-entered data captured within the order panel; and (3) LLM-assisted feature extraction of clinical notes, used for criteria that cannot be reliably identified from structured data alone (e.g., red flag symptoms for GI problems, medication intolerance). The approach that we plan to take for each referral condition and each criterion is listed below. * The primary regression model will include a binary indicator for assignment to Arm 3 (Information + Review + Checklist), with Arm 1 (Control) serving as the reference group. * Control variables include: Physician's department/division; physician's modal referral-target division in the baseline period; baseline referral volume in each target division; baseline low-value referral rate across three target divisions. As robustness checks, the investigators will estimate specifications that weight observations by the number of referral-relevant encounters. Cardiology Hyperlipidemia - General Cardiology referral - low-value if criterion is unmet: \- High-intensity statin tried \>3 months OR not tolerated ≥2 different statins: medication prescription records; statin intolerance via allergy/intolerance records and LLM-assisted coding of clinical notes Hyperlipidemia - Lipid Clinic referral - low-value if no criterion is met: * Suspected familial hypercholesterolemia (FH) and/or LDL \>190 mg/dL: ICD-10 codes for FH and structured lab data for LDL * Premature ASCVD, recurrent CV events, or strong family history: ICD-10 codes and LLM-assisted coding of clinical notes * Severe hypertriglyceridemia (triglycerides \>500) or complex lipid problem: structured lab data for triglycerides; LLM-assisted coding of clinical notes for complex lipid problem * Elevated Lp(a) \>90 mg/dL: structured lab data Hypertension - low-value if neither criterion is met: * BP not controlled on ≥3 antihypertensives at maximally tolerated doses, one of which is/was chlorthalidone or hydrochlorothiazide: medication records; presence of chlorthalidone/HCTZ confirmed from medication list; blood pressure trends from vital signs flowsheet * Requires 4 or more antihypertensives to achieve blood pressure control: medication records; blood pressure trends from vital signs flowsheet Pulmonology Cough - low-value if neither criterion is met: * Cough duration \>8 weeks: physician-entered response in order panel (all arms) * Concern for or evidence of chronic underlying lung disease: ICD-10 codes and LLM-assisted coding of clinical notes Asthma - low-value if no criterion is met: * Requires high-dose ICS or long-term oral corticosteroids to control symptoms: medication records * Frequent need for rescue inhaler despite ICS or ICS-LABA (more than 1-2 days per week, or more than 1 episode waking with asthma symptoms per month): medication refill records and LLM-assisted coding of clinical notes * Significant side effects or intolerance of asthma medications: LLM-assisted coding of clinical notes; allergy records * 2 or more asthma exacerbations per year requiring oral corticosteroids: medication records and ICD-10 codes * Severe asthma exacerbation requiring hospitalization: hospitalization records; LLM-assisting coding of clinical notes that mention hospitalization outside of UCLA Health * Fixed airflow obstruction (post-bronchodilator FEV1/FVC abnormally low): pulmonary function test results * Diagnostic uncertainty: LLM-assisted coding of clinical notes to assess for constructs such as "rule out", "query", or "possibly" a diagnosis. COPD - low-value if no criterion is met: * Significant respiratory symptoms despite medical therapy (GOLD Group B), irrespective of FEV1: LLM-assisted coding of clinical notes * 2 or more exacerbations in a year, or 1 hospitalization due to exacerbation (GOLD Group E), irrespective of FEV1: hospitalization records; LLM-assisting coding of clinical notes that mention hospitalization outside of UCLA Health * Severe obstruction by PFT (FEV1 \<50%, GOLD Stage 3 or 4): pulmonary function test results * Requires supplemental oxygen: vital sign flowsheets * Pulmonary hypertension or other significant comorbid pulmonary disease: ICD-10 codes and LLM-assisted coding of clinical notes * Diagnostic uncertainty: LLM-assisted coding of clinical notes to assess for constructs such as "rule out", "query", or "possibly" a diagnosis. Lung Nodule - low-value if referral is placed for incidentally detected nodule and no criterion is met: * Patient age younger than 35: EHR demographics * History of cancer, immunosuppression, or high clinical suspicion for infection: ICD-10 codes and LLM-assisted coding of clinical notes * Solid nodules \>8mm, part-solid or ground-glass nodules \>6mm, multiple nodules, or other complex scenarios: radiology reports Gastroenterology GERD - low-value if neither criterion is met: * Red flag symptoms (dysphagia, vomiting, weight loss, hematemesis, anemia): ICD-10 codes and LLM-assisted coding of clinical notes * Failed a PPI trial: medication prescription records Constipation - low-value if neither criterion is met: * Red flag symptoms (rectal bleeding, family history of colon cancer, weight loss): ICD-10 codes; family history; and LLM-assisted coding of clinical notes * Failed a fiber supplementation trial: medication records and LLM-assisted coding of clinical notes Acute Diarrhea (\<14 days) - low-value if no red flag criterion is met: * Immunocompromised status: ICD-10 codes; immunosuppressant medication records; LLM-assisted coding of clinical notes * Rectal bleeding: ICD-10 codes and LLM-assisted coding of clinical notes * Pregnancy: ICD-10 codes Chronic Diarrhea (\>14 days) - low-value if criterion is unmet: \- Red flag symptoms: bloody diarrhea, signs of fat malabsorption, unexplained weight loss, hypoalbuminemia, anemia, family history of IBD/CRC/Celiac, immunocompromised status, older adult with multiple chronic illnesses and/or medications, or relevant travel history: structured lab data for hypoalbuminemia and anemia; ICD-10 codes and immunosuppressant medication records for immunocompromised status; LLM-assisted coding of clinical notes for all remaining criteria Secondary Analyses: The investigators will conduct secondary analyses to better understand the mechanisms through which the interventions operate: (a) comparing Arm 2 vs. Arm 1 to assess the effect of highlighting referral criteria and the likelihood of being reviewed, and (b) comparing Arm 3 vs. Arm 2 to assess the effect of prompting deliberation with a checklist, using the same modeling approach as the primary analysis. In addition, the investigators will analyze measures related to the hypothesized mechanisms (referral knowledge, perceived accountability, and perceived procedural frictions) using physician post-RCT survey measures. These analyses will examine whether physicians' referral knowledge, perceptions, and experiences are associated with referral behavior, and whether patterns across arms are consistent with the hypothesized mechanisms. Each mediator will be operationalized from the post-RCT survey as follows: * Referral knowledge will be measured as the number of clinical vignettes (out of 6) for which the physician's referral-appropriateness judgment (Yes/No) matches the evidence-based criterion. Each physician evaluates six hypothetical scenarios (two from each target specialty), of which three meet evidence-based referral criteria and three do not; the order of vignettes are randomized. Each vignette is scored as a binary correct/incorrect response (Yes/No against the criterion), and the items are aggregated into a single knowledge score ranging from 0 to 6. * Perceived accountability will be measured with a single item asking how much the physician cares whether their referrals will be considered appropriate by receiving specialty physicians, rated on a 7-point Likert scale (1 = Not at all, 7 = Very much). * Perceived procedural friction will be measured with a single item asking the extent to which the referral order panels prompt physicians to consider referral appropriateness during decision-making, rated on a 7-point Likert scale (1 = Not at all, 7 = Very much). These analyses will be conducted as follows: * As a first step, the investigators will compare mean values of each measure (referral knowledge, perceived accountability, and perceived procedural friction) across study arms using the same OLS regression framework as the primary analysis, to assess whether each survey measure differs across arms in the direction the intervention was designed to produce. * The investigators will then conduct statistical mediation analyses using the product-of-coefficients method with bootstrapped confidence intervals for the indirect effects (Preacher \& Hayes, 2008). In these models, the independent variable is an indicator for arm assignment (e.g., Arm 3 versus Arm 1 for a comparison between these two arms), the outcome is the physician-level rate of low-value referrals per 100 referral-relevant encounters, and the mediators are the three physician survey measures described above. The indirect effect represents the portion of the total intervention effect on low-value referral rates that is statistically attributable to each mediator.The investigators will first examine each mediator separately in independent models, and then estimate a parallel mediation model including all three mediator measures simultaneously to assess their relative contributions while accounting for intercorrelations among them. The investigators will also conduct exploratory analyses examining whether the effect of Arm 3 (vs. Arm 1) varies by baseline pre-RCT survey measures of referral knowledge, perceived accountability, and perceived procedural friction. For each of these baseline survey measures, the investigators will estimate an OLS regression to predict the primary dependent variable on assignment to Arm 3 (vs. Arm 1), the baseline measure, their interactions, and control variables mentioned above. These analyses will assess whether physicians with lower baseline knowledge, lower perceived accountability, or lower baseline tendency to consider referral appropriateness show larger treatment responses. Heterogeneity Analyses: Although physicians are the enrolled study participants and the primary unit of analysis, the intervention may affect referral behavior differently depending on patient characteristics. We will therefore conduct pre-specified heterogeneity analyses by patient gender, race/ethnicity, and insurance type. Specifically, we will estimate treatment effects separately for patients who are female versus male, non-Hispanic White versus all other race/ethnicity categories combined, and insurance type (Traditional Medicare, Medicare Advantage, Medicaid, Commercial, other/unknown; if we can receive more detailed insurance information to further differentiate commercial plan types-e.g., PPO vs. HMO-we will compare major commercial plan types). For each patient subgroup of interest, the investigators will compute physician-level rates of low-value referrals restricted to referral-relevant encounters involving patients in that subgroup (e.g., separate rates for encounters involving female versus male patients). Each physician will then have multiple observations, capturing outcomes for different patient subgroups. The investigators will estimate a single regression model that includes the patient subgroup indicator, physician's treatment assignment indicator, and their interaction, with standard errors clustered at the physician level. The interaction term tests whether the treatment effect on low-value referral rates differs across patient subgroups. Per funder requirements, the investigators will check whether the effects differ by physician gender and race/ethnicity, though no significant differences are expected. Missing Data: The investigators will handle missing data as follows. Missing covariates will be handled with mean imputation and missing indicators. Missing or undefined physician-level outcomes may arise if a physician does not have referral-relevant encounters during the intervention period. * The investigators will assess whether the frequency of such situations differs across randomized arms; no detectable differences are expected. * If these situations are balanced across arms, the investigators will exclude physicians without referral-relevant encounters and analyze observed data without imputation. * If there is imbalance across arms, the investigators will instead assign each affected physician their baseline value of a given outcome.