Background:
The residency promotes learning, improves the development of skills, competencies, practices and attitudes characteristic of each different medical specialty. Just as the health of the children reflects the health and future of a population, the university and the residency are likely to reflect the quality of the health system.
Collapsed and overloaded health systems rely on residents as a low-cost human resource. The residency program should be characterized by a supervised learning system to promote progressive independence in decision making and clinical reasoning. This learning experience can be affected by many factors like stress, sleep deprivation, poor learning environments, relocation issues, vertical structure, excess of administrative tasks and work overload among others. This characteristics of the learning/working environment have an individual direct impact on safety, processing, learning and decision-making capacity and a group effect on safety and quality of patient's care.
Residents are a population of high risk for depression, burnout and social isolation. The well-being of residents is a key aspect of their training that can be monitored as a security event, managed and actively promoted. While several strategies are used to measure the degree of work overload and its impact on training residents, none seems to be holistic and appropriate to understand the dimensions of this complex phenomenon. The aim is to generate and validate a tool to measure the well-being of residents considering their multidimensional features.
Research Question: Is it possible to built a valid instrument to measure well-being in residents of medical residency programs?
Instrument description: It is planned to develop a paper multidimensional, self-administered, questionnaire, relative brief, which considers the most important domains of well-being. The main features of the instrument are shown in figure 2 (domains, format, administration, score).
The whole process of generation and validation of the instrument of well-being measurement in residents will be developed according to study protocol. All study participant will give freely informed consent to participate. The protocol and its informed consent will be evaluated and approved by an independent research ethics committee before starting the study. The protocol will be registered in argentinian public database of research protocols and in clinicaltrials.gov.
Development and validation
1. Well-being instrument development
1.1. Conceptual framework development: The aim of the first meeting is to construct a simpler conceptual framework about the medical resident well-being considering the literature review, thoughts, expertise and experience of each participant. In group of experts in education, residents and medical specialists will develop a group conceptual framework using the Nominal Group Technique (NGT). This technique is a solving method designed to provide each member of a group with an equal voice in the resolution of a particular problem. A complete description of the original technique may be found in Van de Ven AH, Delbecq AL. The nominal group as a research instrument for exploratory health studies. Am J Public Health. 1972; 62: 337-342 7.
1.2. Domain selection The domains will be selected according to consensus in a group of experts in education, residents and medical specialists considering the conceptual framework defined for the problem in previous meeting. Potential domains to include are physical, mental, social, role/functional, fatigue/sleep, burnout, work environment, future potentiality, mistreat/abuse, learning experience/environment, self care, and resilience.
Previous instruments will be considered, conceptual framework, experience and opinion of the participating experts, and bibliography on the well-being of residents to decide the main domains to be represented in the instrument. NGT will be used to make this grupal decision.
1.3. Item selection to access each domain Each domain will consist of between 5 and 10 items. Likert or pseudo-Likert scales will be used with 5 standard options with a neutral central value. All the items will be affirmations or statements obtained from previous instruments that measure similar constructs, previous literature, experience of the consensus group integrated by residents, specialists in education, medical specialists.
Initially, all the items that the group considers pertinent per domain will be included. For the generation of the list of items per domain, the NGT will be used again. For the design of the items, previous instruments, conceptual framework, experience and opinion of the participating experts, and bibliography on the well-being of residents should be considered.
The list of items per domain, will be reduced to contain a minimum of 5 items and a maximum of 10 items according to group discussion and consensus. The NGT will be used again to define a group order according to importance Each participant of the consensus group will order the items of each domain according to order of importance. The items will be reduced according to the redundancy, difficult for the translation to other languages, awkward of phrases, excessively idiomatic expressions, difficulty in understanding or less relative importance with respect to the rest of the items.
The selected items will be refined. They will be phrased in plain language to avoid ambiguities. With a single concept by affirmation, in short sentences. A 7th grade or complete primary education (according to the argentinian education system) level will be used. The use of slang confusing words will be avoided. The time frame for each item to increase measurement precision and avoid ambiguity will be discussed.
The items will be sorted in order to keep more general and less personal items at the beginning, and more private items at the end. The items referred to the same time frame or similar concepts will be order in proximity. The order of the items will not respect the domains, but will be designed to increase responsiveness, comprehension and precision. The options will always be evenly arranged to avoid errors in filling.
1.4. Scale and scoring structure design For the scaling of each item on the Likert or pseudo-Likert scale, a score from 0 for the worst lack of well-being, and 4 for the best well-being option will be used. In all cases, the highest score will be assigned to the response that represents the greater well-being situation and the lower score to the lower well-being.
The results of each questionnaire will be presented as one score per domain and a global score. Each domain will have a maximum score of 100 and a minimum of 0, as well as the global score. Each item will be assigned a weighted score according to group consensus on the importance of the items in each domain. The score of each domain will be obtained by adding the weighted scores of each item within the domain. Each domain will be assigned a weighted score according to the global importance of the domain in the whole construct decided by group consensus. This weights assigned to each domain will be used to sum a global score for the whole questionnaire.
Once the validation period of the instrument has been finished, the scale will be applied to a larger population of residents using networks like the Sociedad Argentina de Medicina (Argentinian Medicine Society). With these results, the averages and standard deviations will be obtained. This tables will be used to calculate a z-score (standard deviation units) according to sex and age for each domain and the global score.
1.5. Preliminary version and pretest A preliminary version that will be pretested with a group of residents will be presented, medical specialists and specialists in medical education. Clear instructions should be added in the header on how to complete the questionnaire and how to solve it in case of mistakes during the answers. A final thank-you message will be added.
2. Validation
2.1. Content validity and face validity: To evaluate face validity and content validity the final version of the questionnaire produced during the development phases will be used. Face validity will be evaluated asking to different participants whether the questionnaire seems to be valid to represent the construct well-being.
Content validity will be evaluated with instrument inspection and discussion by a group of experts and residents as describe in the last section of the development phase. The group discussion will be focused on whether the instrument incorporates the appropriate components and facets for the measurement of the construct that is intended to represent. Each participant will answer if all the items included in the questionnaire are essential to represent the construct resident well-being.
2.2. Criterion validity Criterion validity will be evaluated assessing concurrent validity and predictive validity using the same sample of residents.
There is no appropriate gold standard to evaluate the concurrent validity as I have not found any other instrument designed and validated to represent the construct well-being of residents with the chosen perspective. Two imperfect approximations will be used as gold standard: the correlation of the global score with the SF36 \[20\] as a tool to access global quality of life and the WRQoL scale.
A sample of residents will complete the three questionnaires (SF36, WRQoL and the well-being in residents questionnaire). Pearson's correlation coefficient will be used to evaluate the association of the global scale with SF36 and WRQoL. Even though this two scales are not gold standards for this construct the investigators expect them to be highly correlated to the global scale of the new instrument.
To evaluate predictive validity the same sample of residents will be used to access if the new questionnaire is able to predict the occurrence of future events: medical error, level of stress and depression, alcoholism, drug abuse, and residence abandon.
2.3. Construct validity The evaluation of construct validity implies the accumulation of evidence that the questionnaire actually measures the construct. To evaluate the construct validity of the residents well-being questionnaire the convergent and discriminant validity will be evaluated (Multitrait-Multimethod approach), test hypothesis of association between the questionnaire scale and different situations and evaluating the behaviour of known groups. With all this evaluations, the investigators are planning to gather information about the performance of the questionnaire scale to the conceptual model of hypothesized relationships.
Convergent and discriminant validity will be tested by measuring in a sample of residents simultaneously related and not related scales and the well-being questionnaire. To access convergent validity, physical activity, depression, anxiety will be used. To access discriminant validity use socio-economical status will be used, SF36 and subscales and the domains of BMI. The Multitrait-Multimethod Approach will be used to build a matrix of correlations between the different scales accessed and each domain and global scale of the well-being questionnaire.
Additionally it is expected to see more well-being related to some specific rotations like clinical research, and elective rotations compare to night shift or during the excess of workload of vacation period. The measurements of this instruments will be compared in the same residents during this specific rotations.
