Is Betwixt App Effective and Acceptable?

Study Background Information and Rationale Mental health problems are one of the main causes of overall disease burden globally (7.4%) (more than HIV/AIDs and tuberculosis, diabetes, and transport injuries), and their prevalence continues to rise. Globally, 792 million people (10.7%) experience mental health problems, and anxiety is the most prevalent (284 million/3.8%), followed by depression (264 million/3.4%). One in six adults in England have a common mental disorder (depression or anxiety), which is approximately one woman in five or one man in eight. Hence, mental health problems have a significant public health impact, nationally and globally. National Context Within the UK, the NHS Long Term Plan (NHSLTP) outlined a new service model to overcome concerns of funding, staffing, increased inequalities, and pressures, by providing more options, better support, and coordinated care at the right time, in the optimal setting. The associated Mental Health Implementation Plan (NHSMHIP) outlined the priorities of the NHSLTP in mental health, including digitally enabled mental health care. This involved NHS England and NHS Improvement supporting the development of applications ("apps"), digitally enabled models of therapy, and online resources to support mental health and recovery. Digital mental health interventions have been found to be effective and acceptable means of delivering mental health support on a large scale. The NHS previously had a library of approved apps which closed in 2021 and now they only recommend the NHS app and the COVID-19 app. As a result, NHS England created the Digital Technology Assessment Criteria for Health and Social Care, an assessment tool for ensuring digital health technologies meet standards of legislation and good practice. There is currently no database of recommended apps and there is an onus on healthcare organisations to assess each app at the point of procurement. The Mental Health Million project surveyed over 45,000 respondents and found that 50% of individuals with a clinical level of mental health risk did not seek help. The reasons were not knowing what kind of help to seek, thinking it would not make a difference, and a preference for self-help. Hence, individuals may need support with identifying what help is available, including both facilitated and self-help options. The NHS Five Year Forward View first referenced the importance of 'empowering patients' and 'personalised care' and the NHSLTP then stressed 'personalised care'. A recent study noted significant challenges in digital mental health interventions, particularly regarding a lack of personalisation. In summary, there is a need for digitally enabled mental health care which is personalised, includes self-help, and is supported by good quality, contemporaneous evidence, and although the NHS has criteria for assessing digital technology, they no longer have provision for recommending apps to the public. Emotion Regulation (ER) Technology ER is defined as: "shaping which emotions one has, when one has them, and how one experiences or expresses these emotions". It entails regulating positive and negative emotions, dependent upon goals. Since the 1990s, there has been vast research into the field of ER, for example, it was proposed that ER is necessary for daily functioning. It has also been found that deficits in ER appear to be relevant to the development, maintenance, and treatment of mental health conditions (such as, depression, borderline personality disorder, substance use, eating disorders, somatoform disorder, and other symptoms), hence, ER is transdiagnostic. It has been outlined the importance of cognitive reappraisal in ER. Numerous studies have researched digital technology and ER, such as, a meta-analysis which outlined the need for learning ER as an experiential process. The importance of rigorously designed theory-driven studies on digital technologies offering personalised, timely interventions has been emphasised. It has been deduced that there is "encouraging evidence that digital technologies may be beneficial for enhancing ER skills and providing personalised care remotely". Finally, a systematic review of mental health apps and concluded that they have "promising outcomes", however, few apps specifically promote ER. Hence, there is evidence that ER is a transdiagnostic factor in the development, treatment, and maintenance of various mental health conditions. Research has found that ER can be learnt experientially, that personalised and timely digital technology can support ER, but more theory-driven studies are required, and mental health apps have "promising outcomes", yet few apps specifically target ER. Betwixt Betwixt is a narrative-based game which aims to improve ER. It has been argued that digital tools are currently being developed without consideration for theory. However, Betwixt is theory-driven and built upon the foundations of self-determination theory (human motivation, volition, and personality in social contexts) and social cognitive theory (learning within a social context, people are active agents who can influence, and be influenced by their environment). It is also based upon two ER skills: cognitive reappraisal and self-compassion, prominent features of cognitive behavioural therapy (CBT) and mindfulness-based interventions respectively. Cognitive reappraisal and self-compassion are core processes that underpin various evidence-based interventions, such as, CBT and compassion-focused therapy. Betwixt was privately funded and the National Institute for Health Research (NIHR) emphasised the importance of industries creating their own innovations to improve the health and care system, thereby saving money, and improving quality within the NHS. A proof of concept/acceptability study of Betwixt found positive results in a general population sample, such as, 73.1% of participants found Betwixt informed their way of thinking. It is worth noting that this study had a small sample size (n = 26), and hence, the results may not be generalisable. A general population randomised control trial (RCT) of Betwixt was completed and found significant and large reductions in depression, stress, and self-reflection. However, further research of Betwixt is required as the previous literature studied a general population (not a clinical population). In summary, Betwixt is a theory-driven app with foundations in ER, CBT, and mindfulness. It is based on recommended treatments that have been proven to be effective. In research, Betwixt has been found to be effective and arguably acceptable in a general population. Summary and Study Rationale In conclusion, there is a need for digitally enabled mental health care which is personalised, includes self-help, and is supported by good quality, contemporaneous evidence. ER is a transdiagnostic factor in various mental health conditions and ER can be learnt experientially. Personalised and timely digital technology can support ER, and mental health apps have "promising outcomes", yet few apps specifically target ER. To potentially overcome this, Betwixt is a theory-driven app with foundations in ER, CBT, and mindfulness and it is based on recommended treatments that have been proven to be effective. It has indications of acceptability and proven effectiveness in a general population. In addition, it has been argued the importance of reach: An intervention that can be accessed by a large number of people (e.g., an app) (even if it has a smaller effect size) has a larger impact on public health than an intervention with a large effect size that is available to fewer people. The gap in knowledge this research is going to address is whether the Betwixt app improves ER in a clinical population, and if it is acceptable to service users. Regarding impact within clinical psychology, if Betwixt is found to be effective and acceptable, it could be a standalone, or waiting list intervention which could have health economic benefits for the NHS, and it could increase psychological mindedness/readiness for therapy or improve symptomatology. This study may also deduce information about improving Betwixt or inform feasibility for an RCT of Betwixt in a clinical population. This proposed research also aims to test the underlying processes of Betwixt to support psychological health and wellbeing: as Betwixt is theory-driven and has underlying mechanisms of action, evaluating the app also enables broader questions about the value of its foundations. Study Aims In considering the clinical applicability of a theory-based emotion regulation app (Betwixt) for individuals awaiting treatment for common mental health difficulties, this study has three aims: A: Evaluate the effectiveness of the Betwixt intervention on emotion regulation, self-compassion, and cognitive reappraisal in a clinical population. B: Investigate whether changes in processes targeted by Betwixt result in improvements in clinical outcomes. C: Explore the acceptability, and putative processes of Betwixt within a clinical context. In addressing these three aims, quantitative and qualitative data will be integrated, through a mixed-methods single-case experimental design (SCED) (outlined below). Intervention Betwixt is a mental health game, designed to improve emotion regulation. It consists of different stories, which are displayed to the user in a narrative format, with background sound. The user is shown portions of the narrative, before being given options for proceeding further (e.g., 'cover your ears', 'look at the sky', or 'look down'). The narrative then continues, taking account of the option chosen, and further narratives and options are provided. The participants will need to download the Betwixt app from their phone app store. They will then need to create a log-in, before starting to use the app. Betwixt is not designed to provide medical advice, diagnosis, or medical treatment. It is purely a narrative-based game, designed to improve emotion regulation. Study Design Epistemology This study will be approached from the epistemological position of pragmatism, which developed due to dissatisfaction with polarised paradigms, and a desire for combining them. Pragmatism enables mixed-method designs to be employed, which include quantitative and qualitative elements, hence, this is the position that the current study will be approached from. This study aims to investigate the effectiveness and acceptability of Bewtixt within a clinical population. Effectiveness research (study aims A and B) is investigated by comparing numerical data (often psychometric outcome measures) to quantify the effects of an intervention. Acceptability research (study aim C) is investigated through narrative data from semi-structured interviews (by an independent interviewer), due to the scope and depth of understanding of this method (and openness to being experience-led, as opposed to assumption-led). Semi-structured approaches to understanding acceptability allow for more nuanced appreciation of any issues and are likely to be more comprehensive (when compared to having structured questions based on the researchers' a priori expectations regarding issues that might arise). Hence, mixed-method research is the most appropriate paradigm to investigate this and this study will entail a sequential explanatory design (quantitative then qualitative phases). Study Configuration To investigate the effectiveness of Betwixt (quantitative element of this study, related to study aims A and B), a SCED series will be undertaken. Qualitative data from post-intervention feedback semi-structured interviews will also be used to gain an in-depth account of participant-reported changes and change attributions, which will be integrated with quantitative SCED data (identifying points of convergence, divergence, and elaborative context). SCEDs evaluate intervention effectiveness on a case-by-case basis and each participant acts as their own control condition. SCEDs provide valuable information about the mechanisms of therapeutic change, the effectiveness of interventions (via systematic manipulation of the independent variable) and enable empirical approaches to be translated into naturalistic clinical settings. This study will entail an AB design (comparing the baseline and Betwixt intervention), with multiple baselines across participants. This involves the systematic and repeated measurement of dependent variables (process measures of emotion regulation, self-compassion, and cognitive reappraisal) against the independent variable (measurement of usage of Betwixt, from app data). ABA designs (baseline, intervention, baseline) are more statistically robust; however, this would entail forgetting all learning from Betwixt and there would be the ethical consideration of withdrawing an intervention. To make inferences from the data, a stable baseline is required, however, this can delay treatment. Given practical and ethical concerns, the intervention will not be delayed in the event of an unstable baseline; however, baseline trends will be adjusted for in analysis. Qualitative semi-structured interviews will be undertaken with the dual purpose of understanding change (triangulating findings from the quantitative studies, aims A and B) and exploring the acceptability of the intervention (aim C). Framework analysis will be used to structure the qualitative data analysis. In conclusion, this study design is the most appropriate and pragmatic, regarding epistemology, methodology, time, money, and resources. Study Management The Chief Investigator has overall responsibility for the study and shall oversee all study management. The data custodian will be the Chief Investigator. Victoria Harper (student) will undertake the practical aspects of the study, and report to Jacob Andrews (Chief Investigator), Nima Moghaddam, and Sam Malins. Duration of the Study and Participant Involvement Study duration: 24 months. Participant duration: Six to seven weeks for each participant. Eight months in total. End of the study: The end of the study will be the interview or survey of the last participant. Selection and Withdrawal of Participants Recruitment. Participants will be adults on the waiting list for a specific Improving Access to Psychological Therapies (IAPT) service. An IAPT service was chosen as this is the first study into the use of Betwixt in a clinical population and these services support the lowest level of mental health severity and risk. In addition, a proof of concept/acceptability study into Betwixt and participants reported that the app has the potential to positively influence those that have "mild mental health concerns". Betwixt will be delivered as a waiting list intervention, due to ethical considerations, so as not to delay treatment for individuals in a clinical population. The recruitment strategy entails clinicians/gatekeepers within the IAPT service offering the intervention to eligible individuals during their assessment appointment. The clinician will provide information to the potential participant about the study (Participant Information Sheet (PIS)) and if they wish to participate, the clinician will check eligibility and get their permission to pass their information onto the research team. The research team will then contact the participant about the study, explain what participation entails and discuss consent (Consent Form). The research team will also provide training and ongoing support to the clinicians to enable them to approach potential participants and to not place unnecessary burden on IAPT clinicians. The participants who take part in the study will then receive the Betwixt intervention whilst waiting for their intervention with IAPT. It will be explained to the potential participant that entry into the trial is entirely voluntary and that their treatment and care will not be affected by their decision. It will also be explained that they can withdraw at any time, but attempts will be made to avoid this occurrence. In the event of their withdrawal, it will be explained that their data collected so far cannot be erased and we will seek consent to use the data in the final analyses where appropriate. Expected Duration of Participant Participation. Study participants will be participating in the study for six to seven weeks each. Removal of Participants/Participant Withdrawal. Participants may be withdrawn from the trial either at their own request or at the discretion of the Investigator. The participants will be made aware that this will not affect their future care. Participants will be made aware (via the information sheet and consent form) that should they withdraw the data collected to date cannot be erased and may still be used in the final analysis. Informed consent. The clinician from the IAPT service will share information about the study at the participant's scheduled assessment appointment. The clinician will provide information to the potential participant about the study (Participant Information Sheet (PIS)) and if they wish to participate, the clinician will check eligibility and take their consent to pass their information onto the research team. The research team will then contact the participant about the study, explain what participation entails and discuss consent (Consent Form). All participants will provide written informed consent. The informed consent process will entail prospective participants interacting with Victoria Harper via video call (Microsoft Teams), to further explain the study and address any queries, ensuring that the participant has sufficient time to consider participating or not. Victoria will share her screen during this video call to receive consent, and she will also be in contact with each participant throughout the study (in the form of the weekly check-in calls) to verify continued participation from the person who consented. The Investigator will answer any questions that the participant has concerning study participation. This will also be traceable and verifiable back to the referring clinician as Victoria will have set up the consent interview using contact information shared by this clinician. The Informed Consent Form will be signed and dated by the participant before they enter the trial. Informed consent will be collected from each participant before they undergo any interventions related to the study. A digital copy of this will be kept by the participant, and a paper copy will be kept by the Investigator. Should there be any subsequent amendment to the final protocol, which might affect a participant's participation in the trial, continuing consent will be obtained using an amended Consent form which will be signed by the participant. Sample Size and Justification. Published guidance recommends a minimum of three cases for SCEDs to achieve sufficient data and the published average is six. Eight participants is deemed to be a sufficient sample size (with replacement participants if they do not contribute sufficient data to the intervention phase), to allow for attrition. Due to the nature of small sample sizes, attrition can have a significant effect, hence, more participants will be recruited than the recommended minimum and published average to mitigate this. In addition, participants who drop out will be asked to complete a semi-structured exit interview/survey to provide additional feedback about acceptability and usability. As a result of the small sample size in SCEDs, replication with the same subject and across multiple subjects improves the external validity of such studies. Hence, a sample size of eight (with replacement participants if they do not contribute sufficient data to the intervention phase) will be sufficient and feasible within the timeframe of the study and to enable replication of effects measured. This sample size should also enable saturation in the interview element of the study. Data For the SCED phase of the study, quantitative data will be captured based on process measures throughout the SCED, and pre-/post-intervention outcome measure scores, via QuestionPro. This data will be stored in a password protected Microsoft Excel form, on secure university systems. Participants will also be pseudonymised, so that study data (outcome and process measures) can be identified by the researcher, but not by others. The password-protected, encrypted file will include a pseudonymisation key. For the semi-structured interviews, the qualitative data will be recorded online and saved on and stored in a password protected Microsoft Excel form, on secure university systems. This data will be coded based upon themes (see Data Analyses section for more detail). The interviews will be conducted by an independent interviewer (student at the University of Nottingham, not involved in this study) and take place online, via Microsoft Teams. These interviews will be video and/or audio recorded, dependent upon each participant's preferences as to whether they wish to be video recorded. The interviews will be transcribed using the University of Nottingham's automated transcription service. The researcher will then review each recording and transcription to ensure accuracy. Study Regimen Preparation Prior to participants being recruited, the research team will provide training to clinicians within the IAPT service to inform them of the study and its rationale, and to train them in how to assess suitable participants, describe the study to participants, gain permission to pass on contact details, and the process for passing on details. Data Collection Recruitment. The IAPT clinicians will lead on the first part of the study whereby suitable participants will be identified and their details will be passed onto the researcher. At the start of the study, each participant will be assigned an identification number for pseudonymisation. Once they have consented and started the study, all of their quantitative data will be stored on a Spreadsheet titled with their number and their qualitative data will be stored in a digital file titled with their number. The research team will then contact potential participants, provide information, and receive informed consent. The researcher will support participants to complete measures, engage in the baseline phase and the intervention phase of the SCED. This research will include a staggered participant intervention (concurrent multiple baseline) to prevent an event in time affecting the results. Staggering the intervention for different participants is also more naturalistic considering how referrals are made to IAPT services. This is a waiting list intervention so it will not delay the individual's intervention with IAPT. SCED Phase (six or seven Weeks). The length of the baseline phase will be variable at either two or three weeks to strengthen the experimental design (to mitigate time/instrumentation effects as an explanation for phase change). During the baseline phase, participants will complete measures every second day. This is in keeping with a finding that the minimum number of data points required for a baseline phase is three to twelve (modal number of three to four). All participants will be in the intervention phase for four weeks (whereby they will complete a dream or a chapter (20-60 minutes) on Betwixt and complete measures every two days); hence, half of the participants will be randomly allocated to spend six weeks in the SCED (two-week baseline) and half to seven weeks (three-week baseline). Weekly check-in calls will be completed by the researcher to troubleshoot technical issues, overcome obstacles and as a cue for participants to engage in Betwixt. A log of check-in calls will be kept, calls will be recorded, and an adherence check will be completed, which will be included in the final report. It is acknowledged that providing weekly check-in calls could complicate understanding of any Betwixt-specific change processes as the participants will interact with the researcher and the intervention will be guided self-help (not self-directed). It will be documented whether the content of the calls was a check-in/orienting to the app or pseudo-therapy. This scaffolding will be put into place to support engagement and in implementation in an NHS service, service users may receive weekly check-in calls from Assistant Psychologists, for example. There is evidence that compliance increases with check-in calls. In addition, this study has a relatively small sample size and attrition could have a significant impact. The interviews may also provide information about real world implementation and whether these calls were helpful. No blinding will be required for this study as all participants will be taking part in the intervention. Interview Phase. The researcher will then organise semi-structured acceptability and change interviews which should take 30 minutes. These will be conducted via Microsoft Teams by an independent interviewer (student at the University of Nottingham, not involved in this study). to enable participants to feel comfortable to be fully open and honest about their experience of Betwixt. If a participant withdraws at any time, the independent interviewer will conduct a semi-structured exit interview with the participant. If they do not respond to the interview request, a survey will be sent. If they do not complete the survey within four weeks, their study involvement will end. The study will be terminated when eight participants have completed the study. If participants fail to comply with the protocol at any time, this will be discussed in the weekly check-in call. If this continues, it will be offered that they can withdraw. App Data. Elitsa Dermendzhiyska (Director, Mind Monster Games Limited and co-creator of Betwixt) is a collaborator in this study. She is able to access app activity/general usage data via an individual's email address or user ID. Activity feed data available includes whether the participant has signed up to the app, opened the app, engaged with the app, or received a notification. A user chooses what to be called in the app, so Betwixt does not have access to the participant's name, unless they provide it. Other data collected includes user ID; purchase history; the content users write in the app; and crash logs. The data will be sent from Ellie to Victoria Harper, via password-protected email, and stored on secure university systems. Dissemination Following analysis, the results will be written into a report and submitted to a journal for publication. The results will also be shared with the IAPT team involved, the local NHS trust, and at regional conferences. The report may highlight areas for further research. There will be service user and carer involvement throughout this research project. Measures Quantitative Please refer to outcome measures section. Qualitative There is a topic guide (for both the change and exit interviews), and an exit survey (contingency for participants who withdraw and do not respond to the exit interview request). To structure the semi-structured change and exit interviews, the Theoretical Framework of Acceptability (TFA) and Client Change Interview Schedule (CCIS) have been amalgamated. To deduce the acceptability aspect of the study, the constructs of TFA (affective attitude, burden, perceived effectiveness, ethicality, intervention coherence, opportunity costs, and self-efficacy) and the CCIS sections (general questions, changes, change ratings, helpful aspects, attributions, resources, problematic aspects, limitations, and suggestions) have been combined. To structure the change-focused aspect, the putative change processes underpinning Betwixt (outlined in Appendix C) have been used. In addition, feedback will be gathered regarding the impact of the weekly check-in calls. These interviews are intended to be semi-structured.