The Clinical Impact of POCUS Use in General Practice

In 2023/2024 the POCUS IN PRACTICE (PIP) study, a large stepped wedged cluster randomized clinical trial (SW-CRT) will be conducted in general practice in Denmark. The overarching aim of this study is to evaluate the implementation of point-of-care ultrasound (POCUS) in Danish general practice. An educational intervention for this study has been developed and tested in a previous study (Evaluating scanning competence following a structured POCUS training program for general practitioners: A hybrid effectiveness-implementation study: Clinical trials registration: NCT05274581). Since then, the intervention has been further developed and in line with the medical research council's framework for developing and evaluating complex interventions , the intervention is ready for the next stage of evaluation. We use the effectiveness-implementation hybrid trial design typology proposed by Curran et al which has been found suitable for preparatory studies in relation to implementing complex interventions . This is a hybrid type 2 effectiveness-implementation trial3, with simultaneous testing of both the effectiveness of an Ultrasound course (the intervention) and the feasibility of the implementation strategy. We previously tested other effectiveness related outcomes (the educational intervention's ability to give participating GPs' scanning competence) in a hybrid type 1 effectiveness-implementation trial and the present study builds on the evaluation of implementation-related outcomes from the previous study. This study will be conducted in office-based general practice in Denmark with the educational sessions taking place at two different ultrasound educational facilities. Outcomes to assess the effectiveness of the intervention: Following the use of POCUS in the general practice consultations the GPs will follow normal procedures for registration of activities in general practice using remuneration codes, fee-specific codes for using POCUS, and referrals in the medical record system. Prior to this study 11 fee-specific codes for 11 different POCUS examinations will be installed in the medical record system. The Primary Sector Data Provider Platform (PLSP) will develop an algorithm that allows for the following data extraction on each participating GP in a given time frame: (1) number of POCUS examinations performed, (2) number of referrals, and (3) number of consultations (outcomes E1-3). PLSP will deliver monthly aggregated data for each participating GP from the first teaching seminar to 24 months after the first teaching seminar. These continues data will be summarized to demonstrate variation over time. For the registration of the clinical effect of POCUS use in the consultation (outcome E4), we will use an adapted version of a questionnaire used in a previous cohort study . The GPs will be asked to specify the following for each POCUS examination: (1) organs/structures examined with POCUS (drop down menu or free text), (2) whether POCUS use entailed a change in diagnosis (yes/no), (3) whether POCUS use entailed a change in referral (yes/no), (4) whether POCUS use entailed a change in treatment (yes/no), (5) whether POCUS use entailed a change in diagnostic certainty for the GP (yes/no), (6) whether POCUS use was helpful for the GP in terms of making a decision for patient management (5-point Likert scale), (7) whether POCUS use was helpful for the GP in terms of arriving at a diagnosis (5-point Likert scale). GP will assess the questionnaire immediately after performing POCUS during consultations in clinical practice. These categorical data will be collected for each participating GP from the first teaching seminar to 12 months after the first teaching seminar.. For the registration of unwanted cases (outcome E5), we will use a questionnaire giving the GPs the possibility to explain in free text using their own words to explain (1) what was scanned, (2) circumstances surrounding the scan, (3) what happened, and (4) reflections after the scan. Data will be collected from at all time during the study. We will report the aggregated total number of monthly registrations for each participating GP. The free text elaborations will not be reported, but passed on to the safety committee (see section 5d). The assessment of unwanted cases for the study does not substitute the GP's registration of untoward incidents required by Danish law (https://stps.dk/da/ansvar-og-retningslinjer/vejledning/rapporteringspligt/). Outcomes to assess implementation strategy effects: For the assessment of implementation strategy effects (outcomes I1-4), we will use the register data (fee specific codes) collected by PLSP on the monthly use of POCUS for each participating GP (outcomes I1 and I4) and the activities logged (pre-post tests, webinars, assignments, attendance on seminar days, logbook registrations) in the online platform used in the educational intervention (outcomes I2-3) to estimate the frequency of use and the percentage of adherence, respectively. The continues data (outcomes I1and I4) will be collected prospectively from the first teaching seminar to 12 months after the first teaching seminar, while the binary data (outcomes I2 and I3) are collected retrospectively at the end of the educational intervention including registration made in the online platform during the educational intervention. Outcomes to inform further development of the implementation strategy For outcomes related to the development of the implementation strategy (outcomes D1-D11), we will use a questionnaire distributed to participants three months after the first teaching seminar. A reminder will follow after 2 weeks. The questionnaire will include the following questions with a 5-point Likert scale: (D1) Did the educational intervention meets local needs? (D2) Was the educational interventions agreeable in terms of content and delivery? (D3) Was the educational intervention suitable for implementation? (D4) Was the educational intervention in line with your values, mission, priorities? (D5) Are you satisfied with the amount of support and resources allocated to you? (D6) Was the educational intervention too difficult or complex to implement? (D7) Did any organisational political, economic or social factors influence the implementation of the educational intervention? (D8) Did the setting, including organisational values or norms influence the implementation of the educational intervention? (D9) Did you have the capacity (e.g. knowledge and skills) to implement the educational intervention? And the following questions with a free-text answer category: (D10) How must did you have to invest financially to implement the educational intervention? (D11) How much time and resources did you have to invest to implement the educational intervention? Outcomes to assess the feasibility of trial methods For the assessment of the feasibility of trial methods (outcomes F1-F8), we will evaluate the data quality by calculating the proportion of missing or incomplete registrations, recruited participants in relation to possible number of participants as well as participant retention and adherence. The steering committee will discuss the results and perform a qualitative evaluation based on predefined progression criteria (see study protocol). For outcome F8, we will pilot test validated scales to determine GPs' perceived stress (Cohen's 10-item Perceived Stress Scale), GPs' job satisfaction (Warr-Cook-Wall Job satisfaction Scale) and GPs burnout symptoms (Maslach Burnout inventory). Prior to the study, these questionnaires were tested for face validity in four focus groups with 18 POCUS using GPs (unpublished materiel). The GP questionnaire will be distributed to participating GPs prior to the third teaching seminar (day 37, at the end of the educational intervention). For each scale, we will calculate a mean sum score with SD and a median sum score with IQR. Participants characteristics On the first teaching seminar the following participant characteristics will be collected: Age (years), gender (M, F, other), previous use of ultrasound (number of months with regular use), previous ultrasound courses of minimum 1 day duration (yes/no), scanner type (low range, mid-range, high end), type of practice (collaboration, partnership, solo), location of practice (urban, rural, mixed), number of patients assigned to the practice, number of GPs working in the practice. From PLSP, we will receive the following aggregated data on patients (age, gender) calculated based on patient ID in the medical records in the clinics (CPR numbers). These data are extracted and transferred to CAM AAU anonymously. Time schedule We will invite participants from two PLO-e ultrasound course to participate: Teaching seminars course 1: August 31st and September 28th and November 30th 2022 Teaching seminars course 2: September 6th and October 10th and November 29th 2022 Sample size This is accompanying research and as such no formal sample size calculations are made. Our potential study population is 36 GPs, who have signed up to participate in an ultrasound course organized by PLO-e. We expect a participation rate of 66% corresponding to 21 GPs. Previous cohort studies from Danish general practice have shown that GPs' average use of POCUS corresponds to once a day. With a data collection period of 24 months for participants corresponding to 200 working days, we expect approximately 8400 POCUS registrations in this study. Recruitment Two ultrasound courses delivering the educational intervention have been included in PLO-e's course catalogue for 2022 and as such all GPs working in Danish general practice has had the opportunity to sign-up for the courses. PLO-e has handled the participant recruitment and registration based on a first-come-first-served principle as well as the financial and practical aspects of organizing and executing the courses. The two ultrasound courses have a total of 36 available spots and all 36 participants are offered the opportunity to participate in the data collection for this study. However, not all participants will be offered financial compensation for participating in the study, as a collective agreement between the Danish regions and the association of GPs only allows for those GPs with in longest distance to a radiology department to have a free course attendance and a fee for performing POCUS during the data collection. The offer to participate in the study is made upon registration to the course and repeated at the first webinar. At the webinar participants are informed about the research project and written information about the project including a consent form is distributed to participants. The signed consent forms are collected from participants on the first training seminar. Data collection Register data: Data regarding different POCUS examinations, number of referrals, number of consultations will be delivered by PLSP (outcomes E1-3, I1 and 4). The data are registered prospectively by GPs in the medical records during the consultations from the first teaching seminar until 24 months after the first teaching seminar. We will receive these data retrospectively from PLSP, who will use an algorithm to collect monthly aggregated data for each participating GP. PLSP will also deliver aggregated data on patients age and gender independent from other study data. GP registrations: For the GPs' registration of POCUS effect in the consultation (outcome E4), GPs will use paper registrations in logbooks during the educational intervention (months 1-3). Here the registrations are made for the GPs own purpose as part of the learning process. Following the educational intervention, we will ask the GPs to continue making these registrations with the purpose of research, using a QR code to access an online questionnaire. The paper registrations will be collected by the research team at the third teaching seminar, while the online-questionnaire data are collected continuously from untill 12 months after the first teaching seminar. For the reporting of anonymously unwanted cases, GPs will have a QR code and a link for an online questionnaire (outcome E5). These data are collected prospectively continuously during consultations. GP questionnaire: For the cross-sectional questionnaire exploring burnout, stress and job satisfaction at 3 months (outcome E6), an online questionnaire will be distributed by email to participants following the third seminar day and a possible reminder will follow after two weeks. The questionnaire collecting background characteristics on participating GPs will be distributed on the first teaching seminar. Online platform: For the evaluation of participant adherence (outcome I2) and reach (outcome I3) we will use the registrations on the online platform (Microsoft Teams) supporting the educational intervention. These are collected retrospectively at the end of the educational intervention (three months after the first teaching seminar). Process evaluation questionnaire: For the evaluation of implementation of the educational intervention (outcomes D1-11) a questionnaire will be distributed by email to teachers and participants following the educational intervention (three months after the first teaching seminar). Data management The data management will follow the rules of the General Data Protection Regulation (GDPR) and the Danish Data Protection Agency. All study data will be stored on a secure server at Aalborg University. Questionnaire data will be collected using the online platform SurveyXact (Rambøll, Aarhus, Denmark). A data management agreement has been made between Rambøll and Aalborg University. At the end of the data collection, data will be moved from SurveyXact to Aalborg University's server. Register data will be collected by PLSP. Data management agreements between PLSP and CAM AAU will be made prior to the collection of data. PLSP will receive data from the different IT suppliers for the general practice clinics. Data management agreements between the IT suppliers and PLSP as well as the GP clinics and the IT suppliers already exists. For this specific study, we will develop an instruction for the GPs to incorporate in the existing data management agreement prior to the collection of data. All patient-related data will remain in the GPs clinics until the end of the educational intervention (three months after the first teaching seminar). No data will be transferred to CAM AAU before legal approval by the health authorities has been obtained. Statistical methods All statistical analyses will be performed using STATA version 17 (StataCorp, Texas, USA) and analyzed according to a predefined statistical analysis plan, which will be uploaded to clinical Trials.org prior to the analysis of data (24 months after baseline). Data will be summarized and reported using descriptive statistics. For effectiveness related outcomes E1-E5, frequencies and variance are reported as mean and 95% CI or median and IQR depending on the distribution of data. We will use graphic illustrations to demonstrate change over time. To test if changes over time are significant, we will use repeated measures ANOVA analysis. For outcomes related to the implementation strategy effects (I1-I4) and feasibility of trial methods (F1-F7), we will report proportions as frequencies, while outcomes related to the implementation strategy (D1-D9) will be reported with an average score and outcomes D10-D11 with a summarized estimate on costs. For fesability-related outcome F8,we will report the questionnaire response rate and the corresponding 95% CI. Missing data Missing values, owing to e.g. premature termination of the questionnaire, are considered completely random and will be declared for each outcome. Research ethics approval The study will be performed in accordance with the Declaration of Helsinki. The project was notified to The North Denmark Region Committee on Health Research Ethics. (Den Videnskabsetiske Komité for Region Nordjylland, reference number 2022-000764) who responded that according to Danish Law (komitélovens § 14, stk. 2), no ethical approval is needed for this project. As we are collecting patient-related data for the purpose of research, we will seek permission by the health authorities. All patient-related data will be collected retrospectively after this approval is obtained. We will honor our information obligations to patients about the project by making project information available in the GP's clinic as hand-outs and posters. The project has been registered and conducted according to the regulations of the Danish Data Protection Agency (registration number ID 242-3). Harms The participating GPs will be obliged to report any undesirable effects including Suspected Unexpected Serious Adverse Reactions (SUSARs), Serious Adverse Events (SAEs) and Adverse Events (AEs) related or possibly related to the use of POCUS to the safety committee. Consent or assent Informed consent form the participating GPs will be collected by the principal investigator prior to the study. We will not seek consent from patients, as no collected study data are directly linked to the specific patient. Dissemination policy We aim to publish positive, negative or inconclusive results of the study in a peer-reviewed journal. The project group will also present results at conferences. We aim for two scientific publications: Article 1 will include effectiveness-related outcomes (E1-E6), while Article 2 will report the development of the intervention and implementation-related outcomes (I1-4, D1-11 and F1-8).