Health technologies being assessed
The eFI+ will be developed using components of the original eFI, supplemented with additional routine primary care EHR data, and guidance on the added benefits of implementing simple clinical measures in routine primary care practice. The eFI+ will be suitable for rapid implementation in UK primary care EHR systems, building on existing close links with system suppliers (SystmOne/EMISWeb/Vision/Microtest).
The investigators will develop, then internally and externally validate the eFI+ using the Secure Anonymised Information Linkage (SAIL) databank, the ResearchOne database, and the Leeds Data Model (LDM).
In addition, the investigators will analyse Community Ageing Research 75+ (CARE75+) cohort study data (CI Clegg, n≈1,200) as the only national cohort study to include eFI scores, to investigate how simple measures that can be assessed in primary care, but are not available in routine EHR data (e.g. gait speed, timed-up-and-go test; activities of daily living; loneliness) may improve prediction.
Study design
Prognostic model development, internal validation and external validation using routine primary care research data (ResearchOne), linked datasets (SAIL databank and LDM) and cohort study data (CARE75+), with integrated Decision Analytic Modelling, including health economic analysis.
Databases
1. Secure Anonymised Information Linkage (SAIL) databank
Anonymised records from around 5 million people in Wales, with linked primary care, ED attendance, hospital admissions, outpatient data, social care, Welsh Care Homes Dataset, and ONS mortality data. SAIL includes eFI summary scores and individual components.
2. ResearchOne
Nationally representative, de-identified data from around 6 million UK primary care electronic health records on the TPP SystmOne clinical system. ResearchOne includes eFI summary scores and individual components.
3. Leeds Data Model (LDM)
Anonymised, linked primary, secondary, community and social care data from 810,000 patients across 108 practices in Leeds, including eFI summary scores and individual components.
4. Community Ageing Research 75+ (CARE75+) cohort
National prospective cohort study (n≈1,200) collecting detailed sociodemographic information, frailty measures (including eFI scores), simple instruments suitable for use in primary care (e.g. gait speed, timed-up-and-go test; activities of daily living; informal care; loneliness), and key outcomes at six, 12, 24 and 48 months. CARE75+ is a very rich dataset that provides a highly efficient method to investigate how simple instruments might augment eFI performance.
Eligible population
Patients ≥65 years with moderate frailty (eFI score 0.24 to 0.36) or severe frailty (eFI score \>0.36) and registered with a ResearchOne, SAIL or LDM practice on 1st April 2018.
All CARE75+ participants with moderate frailty (eFI score 0.24 to 0.36) or severe frailty (eFI score \>0.36) will be eligible.
Outcomes for risk prediction (all 12 months)
* New or increased home care package
* ED attendance/hospitalisation with fall or fracture
* Nursing home admission
* All-cause mortality
Predictors
Components of the eFI, supplemented with variables available within routine primary care EHR data and clinical assessment measures practical for use in primary care.
Prognostic models
Each prognostic model will be developed and internally validated in just one of the databases, and then externally validated in a second database
Sample size for prognostic model development
SAIL and ResearchOne extracts will each include ≈600,000 patients aged 65 or over, with an estimated 72,000 having moderate frailty, and 24,000 severe frailty. LDM extract will include ≈150,000 patients aged 65 or over, with an estimated 18,000 having moderate frailty and 6,000 severe frailty.
For model development, a key indicator of the effective sample size is the number of outcome events. Previous research into the outcomes of interest, and feasibility estimates using CARE75+, ResearchOne and SAIL, inform estimates for anticipated number of events within 12 months.
* New or increased home care package: Anticipated 15,864 events in SAIL, based on 14.9% 12 month incidence in moderate frailty group (10,080 events), and 24.1% 12 month incidence in severe frailty group (5,784 events).
* ED attendance/hospitalisation with fall or fracture: Anticipated 8,064 events in SAIL, based on 7.4% 12 month incidence in moderate frailty group (5,328 events) and 11.4% incidence in severe frailty (2,736 events).
* Nursing home admission: Anticipated 2,160 events in ResearchOne, based on 2.0% 12 month incidence in moderate frailty group (1,440 events) and 3.8% 12 month incidence in severe frailty group (720 events).
* All-cause mortality: Anticipated 12,216 events in ResearchOne, based on 10.6% 12 month incidence in moderate frailty group (7,632 events) and 19.1% 12 month incidence severe frailty group (4,584 events).
Therefore, even when taking the lowest estimate of incident events by 12 months (for nursing home admission), for each outcome the investigators would expect at least 2,160 events in each of ResearchOne or SAIL. This enables us to robustly estimate a prognostic model for each outcome even with up to 108 predictor parameters, corresponding to 20 events per potential predictor parameter (2160/20). This exceeds 'rule-of-thumb' recommendations of 10 or 15 events per predictor parameter.
Furthermore, conservatively assuming the new models will have a Nagelkerke R-squared of 15%, Riley's sample size formula suggests that at least 7.5 events for each predictor parameter will ensure overfitting and optimism are minimized, when the outcome proportion is 3%. When increasing outcome proportion to 20% (home care package), 9% (fall/fracture), or 15% (mortality), the minimum sample size required is 18, 11.5 and 15 events per predictor parameter, respectively. The investigators exceed all these, due to the large datasets available.
Sample size for external validation
Current recommendations are that at least 100 events and 100 non-events (ideally 200) are required for prognostic model external validation. Our estimates indicate considerably more than this, such as 2160 events for the least prevalent outcome of care home admission in SAIL and ResearchOne, and 540 in LDM (which will only be used for external validation of models).
Missing data
Handled using multiple imputation and Rubin's rules, under a missing at random assumption, including outcome in the imputation model, accounting for practice clustering.
Analysis plan
i) Prognostic modelling
The investigators will build 4 separate prognostic models within the development datasets to predict risk of our 4 key stated outcomes in individuals with moderate or severe frailty as the startpoint.
For each outcome, for those with moderate frailty (eFI score 0.24 to 0.36) or severe frailty (eFI score \>0.36) the investigators will develop and internally validate a prognostic model containing just eFI (as a whole as it currently stands) and then containing components of eFI (included as predictors) along with additional routine primary care EHR data. The regression model will be logistic regression or flexible parametric survival, for binary or time-to-event outcomes (as appropriate when the investigators observe the database coding and censoring etc.), to produce outcome risks by 12 months.
Due to the large sample size, overfitting is expected to be small, but the investigators will adjust for it using penalisation via a global shrinkage factor estimated via bootstrapping. Where variable selection is considered important for parsimony, the investigators will rather use penalisation via elastic net. Internal validation will use bootstrapping of the entire development dataset, and optimism-adjusted estimates of predictive performance produced for calibration (e.g. calibration-in-the-large, calibration slope, Observed/Expected), discrimination (e.g. C-statistic) and overall (e.g. Nagelkerke R2) performance of predicted risks. Continuous variables will not be categorised and potential non-linear effects examined using splines or fractional polynomials. Non-proportional hazards for predictors will also be examined with interaction terms with time.
All models will be externally validated in a different database. Predictive performance statistics will be derived as described above (e.g. C-statistic, calibration slope), alongside calibration plots showing agreement between observed and predicted risks, across the spectrum of predicted risks, using a loess non-parameter smoother.
Separately, the investigators will use the CARE75+ dataset to investigate the additional predictive power of clinical assessment measures of prognostic factors identified from the reviews that are practical for use in routine primary care.
ii) Decision modelling
Prognostic models will be translated into a framework to guide clinical decision making by identifying relevant thresholds of predicted risk, above which implementation of our stated interventions is warranted. This will allow us to generate a decision analytic model (DAM), which will be examined using decision curves and net benefit in the external validation datasets.
iii) Health economic evaluation
The health economic evaluation will be conducted in two stages. The objective of the first stage is to provide a short-term, 12-month comparison of the cost-effectiveness of the scenarios identified by the DAM. For the second stage, the investigators will extend our analysis to a long-term cost-effectiveness evaluation of these scenarios.
