Idea Transcript
Factors that impact on medical student wellbeing -‐ Perspectives of risks
Individual Support Programme School of Medicine Cardiff University June 2013 Dr Debbie Cohen Sarah Winstanley Paula Palmer Joanna Allen Sophie Howells Giles Greene Dr Melody Rhydderch Individual Support Programme & Centre for Psychosocial & Disability Research
CONTENTS
1.0
Executive Summary………………………………………………. 4
2.0 2.1 2.2 2.3 3.0
Background…………………………………………………………… 6 Impact of medical training..……………. ……………………. 6 Wellbeing…….…..…………………………………………………... 9 Models….…….…..……………………………………………………. 10
4.0 4.1
4.2 4.3
Methods……….………………………………………………………. 13 Quantitative..………………………………………………………... 13 4.1.1 Questionnaire development………………………. 13 4.1.2. Outcome measures……………………………………. 14 4.1.3 Data collection………..…………………………………. 15 4.1.4 Data validity checks……………………………………. 15 4.1.5 Quantitative data analysis………………………….. 16 Qualitative….……….………………………………………………… 16 4.2.1 Recruitment……………………………………………….. 16 4.2.2 Group structure…………………………………………. 17 4.2.3 Qualitative data analysis……………………………. 17 4.2.4 Integrating the data…………………………………… 17 Evaluation data – exploring face validity………………… 18
5.0 5.1 5.2 5.3 5.4
Results…………………………………………………………………… 19 Quantitative results……………………………………………….. 19 5.1.1 Descriptive analysis…………………………………….19 5.1.2 Raw scores……………………………….................... 20 5.1.3 Regression models…………………………………….. 21 5.1.4 Wellbeing correlations………………………………. 23 5.1.5 School comparisons and tool development.. 24 Qualitative results………………………………………………....26 5.2.1 Focus group findings………………………………….. 26 5.2.2 Open comment findings…………………………….. 27 Participating medical schools’ feedback………………… 29 Summary of results………………………………………………… 31
6.0
Discussion and conclusion..……………………………..…… 34
Aim of Study…………………..…………………………………….. 12
7.0 8.0 8.1 8.2 8.3 8.4 8.5 8.6
References.………………………………………..…………………. 38 Appendix.……………………………………………………………… 41 Questionnaire, information sheet and consent form Domains and outcome measures Focus group information sheet and consent form Focus group matrix Sample feedback report Evaluation survey
1.0
EXECUTIVE SUMMARY Wellbeing is known to have a major impact on health and performance amongst medical students internationally. This study set out to understand in more depth medical students’ perspectives of the factors that impact on their wellbeing during training. The Individual Support Programme (ISP) at Cardiff University was established in 2001 and sits within the Centre for Psychosocial and Disability Research, School of Medicine. As well as providing a support service for medical students and doctors, the ISP has a proven track record of undertaking research into the relationship between performance, health and wellbeing. This study was developed to look at medical students’ perspectives on risk factors that impact on their health and wellbeing during training. The objective was to develop a formative tool for UK medical schools that could be used as a basis for enhancing student wellbeing using quality improvement principles. In summary, these principles suggest the importance of non-‐judgment, respecting different starting points and encouraging each school to take one step in the right direction with the aim being to continuously improve its processes to proactively support student wellbeing. This was a mixed method study. A questionnaire was designed in collaboration with medical students at Cardiff University, and consisted of 47 items based on an occupational health risk assessment model known as the DETTOL model. D.E.T.T.O.L. is an acronym that represents the known major work related risk factors: demands, environment, timing, travel, organisational and layout (Cohen, Khan and Sparrow, 2012). Questionnaires were distributed across six UK medical schools. Focus groups were also conducted across 4 medical schools to strengthen and support the findings. The aim of the qualitative analysis was to triangulate the findings from the questionnaire data. Feedback reports were provided to the participating medical schools and an evaluation of the impact of the feedback was conducted using a simple evaluation questionnaire and by seeking views via telephone interviews. 2,735 questionnaire responses were received, equating to approximately 6.7% of the total UK medical school population. Analysis confirmed that this was a representative sample. The questionnaire was analysed across eight ‘domains’ that together encompassed the various aspects of studying medicine: work-‐life balance, safety, culture, acquisition of knowledge and skills, perceived support for academic issues, perceived support for health/personal reasons, demands of the course, and travel and orientation. Analysis explored from a student’s perspective how well the medical schools functioned across the eight domains. It examined how these impacted on the outcome measure, which in this study was student wellbeing. The results showed that all of the medical schools that participated in this study function very well in some areas, such as facilitating the acquisition of knowledge and skills, and much less well in others, such as ‘travel and orientation’. The results also suggested that the biggest gain in wellbeing could be achieved through the domain of ‘culture’. Focus groups conducted alongside the questionnaire across four of the medical schools provided insight into students’ views on potential solutions to the factors impacting on their wellbeing. Evaluation data from the medical schools 4
suggested that using the questionnaire provided a valuable addition to processes that they already had in place. The study has allowed the development of a simple formative tool to understand how different risk factors may impact on students’ wellbeing. Based on quality improvement principles it enables medical schools to review key areas of risk and provides an opportunity to learn from other schools’ experiences and best practice.
5
2.0 BACKGROUND 2.1 The impact of medical training on students It is recognised that training for medical students requires processes and procedures that differ from those for many other university students. The literature highlights a number of factors specific to studying medicine that may cause increased stress in students compared to the general population (Dyrbye et al., 2005). It is well recognised that medical students’ workload is considerably higher than that of many other students at university. Academic pressures identified include issues such as overwhelming burden of knowledge, differing learning styles and the impact of the learning environment (Vitaliano 1988; Dunn et al., 2008; Tyssen et al., 2000; Firth–Cozens, 2001). Medical students are presented with large amounts of information to process and retain (Yiu, 2005; Holm, et al., 2010). The relentless nature of the examination system leaves little time for hobbies or interests outside medicine (Radcliffe & Lester, 2003). Performance anxiety is in itself well recognised and the objective structured clinical examination (OSCE) which is a core method of examining medical students has been identified by some as causing students significantly high levels of stress. (Radcliffe & Lester, 2003; Dyrbye et al., 2005). Many students find themselves in direct competition with their peers and friends, which may add to their stress (Radcliffe & Lester, 2003). 2.1.1 The clinical environment Academic stress may vary across the year groups and is related to differing factors such as clinical practice versus lecture-‐based learning (Dahlin 2005). The types of stressors shift as students move through their training (Guthrie 1998, Dahlin 2005). As students move into the clinical years of training they frequently rotate to different hospitals and new working environments (Dyrbye et al., 2005) and often become separated from their friends. One study describes how the transition into the third year of medical training brought about many new challenges. Students described feeling ‘useless’ and unable to contribute to patient care. They felt they had insufficient knowledge or skills to take an active role and spent much of their time in year three ‘waiting for something to happen’ on the ward, rather than performing a function (Radcliffe & Lester, 2003). Students also described their need to be seen as a competent clinician (Chew-‐Graham et al., 2003). Developing a professional persona, particularly during the clinical years, is frequently cited as a contributor to undergraduate stress (Radcliffe & Lester, 2003). The medical school environment presents students with ethical conflicts, exposure to death and human suffering and the need for developing quick decision making when faced with emergency situations (Mahajan, 2010; Tyssen et al., 2000). Many medical students feel inadequately prepared to communicate with dying patients and their families, leaving them feeling fearful, anxious, and hesitant of these interactions (Dyrbye et al., 2005). 2.1.2 The working environment Clinical placements undertaken by medical students have much in common with the working environment experienced by their qualified colleagues. Work-‐related factors have been seen to have 6
an independent contribution in explaining deterioration of mental health in young doctors (Tyssen et al., 2000). This may be due to the long working hours, the learning environment and the interactions with their colleagues (Dyrbye et al., 2005). Some junior doctors face additional stress due to the poor attitudes and unethical behavior of their senior colleagues, coupled with the use of teaching by humiliation and embarrassment (Paice et al., 2002; Radcliffe & Lester, 2003). This behaviour can lead to confusion, distress, and anger in young doctors (Paice et al., 2002). Many students may find observing this behaviour towards their junior doctor colleagues and themselves as students distressing. However it has been reported that inappropriate behavior towards them decreases by the final year as they begin to behave more like doctors than students and are accepted more by senior doctors into the medical profession (Radcliffe & Lester, 2003). 2.1.3 Transitions Periods of transition can be particularly hard for medical students (Niemi & Vainioaki, 2006). Much of the relevant literature suggests that doctors and medical students are ‘under-‐prepared’ for transitions (Kilminster et al., 2011). The transition from school to medical school can be particularly stressful due to the changes in teaching styles and the adjustment to competing with people of similar or greater intellectual ability (Dunn et al., 2008; Radcliffe & Lester, 2003). In addition, students have to cope with other changes at this time including leaving home for the first time, making new friends and experiencing new freedoms (Radcliffe & Lester, 2003). 2.1.4 Personal stressors Medical students can feel isolated from other non-‐medical students due to the significant differences in their training, including the long hours, the length of the course and the nature of the work (Radcliffe & Lester, 2003). This is compounded by the need for students to travel and spend time away from home, which can impact on social and personal activities and relationships (Yiu, 2005; Holm et al., 2010). This lack of continuity can leave some students feeling vulnerable and anonymous; this is particularly felt by those who neither excel nor fail, feeling like they are unnoticed somewhere in the middle (Radcliffe & Lester, 2003). Medical students will also experience many personal life stressors common to others in their age group (Dyrbye et al., 2005). Students may face illness, bereavement, injury of themselves or family members as well as dealing with personal relationships and in some cases pregnancy and child-‐ rearing. Children add a level of complexity to students’ lives and may affect female students’ health; in one study of second-‐year medical students, female students were more likely to be depressed if they had children, whereas no such relationship was observed among their male parent colleagues (Dyrbye et al., 2005). Even after adjusting for children and work hours, females show higher levels of stress related to the work-‐home interface than males (Tyssen at al, 2013). Many medical students suffer financial hardship. Travel to and from placements expected of students, coupled with demands such as text books, appropriate clothing and medical equipment have a financial implication for students. The length of the medical course, the long academic year and lack of regular free time that would allow students to supplement their training with outside work adds to significant financial debt (BMA, 2010; Dyrbye et al., 2005). The BMA calculates that 7
students who began their degree in 2006 can expect to graduate with debt of up to £37,000 (£46,000 in London) (BMA, 2010). 2.1.5 Managing health Many studies describe mental ill health and stress related ill health in medical students. Medical students display high levels of depression and anxiety (Nieme & Vainioaki, 2006). The prevalence of depression and anxiety disorders are described by some as being significantly higher in both doctors and medical students than in the general population (Schneider, 1993; Firth-‐Cozens, 1987; Kash, 2000; Bellini, 2002). However more recent longitudinal studies suggest that although depression is present the prevalence may not be as high as reported previously (Quince et al., 2012). Whilst many health issues arise independently, other health issues, particularly mental health issues, for medical students are as a direct result of trying to cope with difficult personal, social or learning environment related factors during their studies (Cohen et al., 2012). A further factor is that medical students, like doctors, are particularly poor at managing their own health (Hooper et al., 2005). There are many reasons why students avoid seeking appropriate help, including concerns over confidentiality, fear of stigma and the concern it may impact on career progression (Chew-‐Graham et al., 2003; Fox et al, 2011). Students and doctors tend to manage their own health through ad hoc corridor consultations, self-‐medication and personally initiating investigations, referrals or treatment (Fox et al, 2011; Hooper et al., 2005). Medical students also fail to use health services; in one study it was estimated that less than a quarter of first and second year medical students who were depressed were using mental health services (Givens & Tjia, 2002). 2.1.6 Culture Culture has been defined as “a pattern of shared basic assumptions that a group or organisation learn as it solves its problems of external adaptation and internal integration, that has worked well enough to be considered valid and, therefore, to be taught to new members as the correct way to perceive, think, and feel in relation to those problems” (Schein, 1992). Organisational culture is a powerful driver of the behaviour of individuals who exist within it. It has both positive and negative aspects. On the positive side, a strong culture where people know how they should interpret situations and react, particularly in a high risk environment like healthcare, is important. On the negative side, one of the most powerful aspects of culture are the unspoken rules, which often exert a stronger influence over student behaviour than other aspects of organisation, such as its espoused values. In the medical education literature, the unspoken rules are often described as the ‘hidden curriculum’. One particularly influential unspoken rule regards how students behave in a learning culture where illness demonstrates weakness and doctors should be strong (Fox et al., 2011). Working arrangements such as being pressurised not to miss shifts reinforce the culture in which distress is overlooked and seeking help discouraged, (Fox et al., 2011). This in turn fosters presenteeism. Presenteeism is defined as being in work when unwell and is well recognised as a major contributor to performance issues across all health and social care professionals. Hull and colleagues (2008) report how doctors often cite workload, stigma and fear of harming future career prospects, as reasons for 8
remaining in work when unwell. The financial impact of presenteeism is well recognised where within the NHS presenteeism costs health care organisations more than sickness absence (Boorman, 2009). 2.2 Wellbeing There is no consensus around a single definition of wellbeing, but there is general agreement that as a minimum, wellbeing includes the presence of positive emotions and moods (e.g. contentment, happiness), the absence of negative emotions (e.g. depression, anxiety), satisfaction with life, fulfillment and positive functioning. The Foresight Mental Capital and Wellbeing Project (2008) describes wellbeing as “a dynamic state in which the individual is able to develop their potential, work productively and creatively, build strong and positive relationships with others and contribute to their community. It is enhanced when an individual is able to fulfill their personal and social goals and achieve a sense of purpose in society”. Thus, wellbeing is more than the avoidance of ill health; it is about the nurturing of positive attitudes and decisions about lifestyle and social interactions. Wellbeing is based on the broader construct of the biopsychosocial model, which recognises the important interplay between all three of these areas. Wellbeing in the workplace or an educational environment therefore requires a culture that actively assists individuals to fulfil their own potential rather than just promote reactive management of ill health or adverse situations. It requires an environment that supports physical, mental, social and spiritual development and understanding. It is more than ensuring a culture that limits harm to individuals; wellbeing is the promotion of a corporate responsibility to positive attitudes to work, lifestyle and social interactions both within and outside the working environment. It is partnership between the individual and the organisation and requires meaningful dialogue and a flexible response to need. Organisational wellbeing is a broad term but in essence engenders meaningful and productive activities in a safe and healthy environment. To achieve this requires a value based working environment, that allows for open dialogue and discussion where individuals feel listened to, clarity of purpose and structures, and good team working. Employee wellbeing is about good working relationships with team members and line managers or supervisors. It includes recognising the importance of good physical and mental health balanced with motivation and clarity of goals, self respect and resilience and a network of support and development that is flexible to employees varying needs. In the context of medical training, it is the balance between the medical school educational and clinical demands and the medical students response to learning alongside a healthy lifestyle and social interaction that are central to wellbeing (Cohen & Rhydderch, 2013) and that requires further exploration. 9
2.3 Models This project was based on well-‐recognised models of risk validated for use in organisational contexts. 2.3.1 Models of risk The model of risk D.E.T.T.O.L. was developed through collaboration with Professor Sayeed Khan and Dr Debbie Cohen at Cardiff University. The model developed methods for GPs and secondary care doctors to undertake simple risk assessments of their patients’ health in relation to their work. The D.E.T.T.O.L. model of risk assessment is detailed in Figure 1 below where each of the six letters in the acronym represents an area of potential risk. Figure 1: D.E.T.T.O.L. model (Cohen, Khan, Allen & Sparrow, 2012)
§ Demands: physical, intellectual § Environment: wards, lectures, (e.g. dusts, chemicals, size of rooms) § Timing: shift work, early start, long hours § Travel: between sites, long distances, lone travel § Organisational: timetables, teaching, support § Layout: ergonomics, work equipment
Further ‘dynamic’ models from occupational psychology were also employed to further understand risk and effects of risk on a student population. Figure 2 below illustrates the dynamic model of risk developed, adapted from the Occupational Stress Indicator (Cooper, 1988). 10
Figure 2: Dynamic Model of Risk Sources of Risk Acquisition of Knowledge & Skills
Demands
Travel & Orientation
Safety at Work
Characteristics
Organisational
Strategies
Organisational Support
1. Culture 2. Processes
Skills development
Individual Effects
Academic Performance
Student Engagement
Physical / Mental Health School Performance
Individual
1. Attributes 2. Circumstances 3. Expectations
Organisational Effects
Behaviour Work-‐Life Balance
The Occupational Stress Indicator is based on the idea that stressors do not influence everyone in the same way. That view is applied in this current study on perceptions of risk. Therefore, the importance of medical students’ perceptions along with their interpretations of the learning environment, the process of cognitive appraisal and the effect of personality characteristics and demographic factors is emphasised. The OSI model argues that work pressures lead to negative outcomes (lowered job satisfaction and mental and physical health) and that this relationship may be moderated by individual variables. In this study, it is argued that perceptions of risk are moderated by individual characteristics such as personality and background health, as well as organisational characteristics such as processes in place to support student wellbeing. In addition, sources of risk are moderated by strategies used by students in their day-‐to-‐day lives such as their approaches to revision and maintaining a healthy work-‐ life balance. As a result, the same level of a particular risk may have a different impact on different individuals. The impacts within the model are described as individual effects and organisational effects.
11
3.0
AIM OF STUDY
This study was developed to look at medical students’ perspectives on risk factors that impact on their health and wellbeing during training. The objective was to develop tool for UK medical schools that could be used as a basis for enhancing student wellbeing. The tool aimed to provide medical schools across the UK with a method of understanding and enhancing student support specific to their own students’ needs and concerns.
12
4.0
METHODS
This was a phased mixed method study. Phase 1 included the development of a questionnaire to medical students at Cardiff and Leicester medical schools. In addition, focus groups were conducted with all year groups at both medical schools. Phase 2 was an extension of this study commissioned by the GMC in June 2012. The study was expanded to cover a wider group of medical schools. Imperial, Brighton, Bristol, Hull and York, and Peninsula medical schools were recruited to the study, to gain perspectives from medical schools of different sizes and styles of programme. The questionnaire was distributed to these five additional schools and further focus groups were conducted. Ethical approval was sought and approved at each medical school. Theoretical models to understand and measure wellbeing, and workplace risk and support were used to underpin the work. 4.1 Quantitative Methods – Exploring construct validity 4.1.1 Questionnaire development The questionnaire was designed in collaboration with medical students at Cardiff University. Sophie Howells, a Cardiff medical student, undertook this work as part of her research project. It consisted of 47 items based on the risk assessment model D.E.T.T.O.L. The questionnaire was then tested for face and content validity through a pilot and cognitive debriefing with a group of 10 medical students. Debriefing involved recording whether or not each of the items was reported to be problematic in terms of the comprehension of the concept, the wording of the question, or the response options. The response selected was recorded along with any suggestions for improvements made by the respondents, such as a more appropriate vocabulary. The research team reviewed the responses and concerns that arose during the debriefing process and potential solutions were recommended. The questionnaire was then further reviewed to confirm appropriate changes had been made. A copy of the questionnaire is available in the appendix 8.1. The information sheet and consent form for the use of the questionnaire is contained in 8.2. Three versions of the introduction and description of the questionnaire were created to respond to the varying ethical requirements at each medical school. All items and demographic questions in the questionnaire were identical. 13
4.1.2 Outcome measures Following completion of the questionnaire the 47 items were further analysed and restructured into 8 ‘domains’. This is shown in Figure 3 below. The items corresponding to each domain are detailed in appendix 8.2. Figure 3: Questionnaire Domains Work-life Balance Safety Culture Acquisition of Knowledge & Skills 47 item questionnaire designed using D.E.T.T.O.L.
Perceived Support: Academic Perceived Support: Personal/health Demands Travel & Orientation
As outlined previously, organisational culture is a powerful driver of behaviour. A positive organisational culture is deemed to be inclusive and supportive and have a strong positive impact on the individuals within it. Therefore, for the purpose of this report, the domain of ‘culture’ focuses on two questionnaire items. Firstly, question 29 which relates to isolation, i.e. a sense of feeling excluded and secondly, question 42 which relates to the student expectations of the need to be resilient. Figure 4: Questionnaire Domain of Culture Medical school fosters a sense of anonymity Q.29 and feeling of isolation among the students. ‘Culture’ domain Q.42 I feel there is an expectation from the medical school for me to be resilient whilst on placement As well as constructing domains, a proxy outcome measure of wellbeing was chosen. This was a composite of two questionnaire items that focused on ‘feeling respected’ and ‘valued’. 14
Figure 5: Questionnaire Proxy Outcome Measure of Wellbeing Q.42
The medical school treats me with respect
Proxy measure of Wellbeing Q.43
The medical school makes me feel valued
The definition of wellbeing as described previously in this report is wide ranging. However, we were constrained by the need to design a brief questionnaire (constructed using the DETTOL concept) to minimize the data burden collection upon medical students. We therefore chose to focus our proxy measure of wellbeing on two items: value and respect. We chose these two constructs, as they are considered fundamental by the theories of Maslow (1970), Deci and Ryan (2000), and Ryff and Keyes (1995). The medical school makes me feel valued: A recent survey conducted by the American Psychological Association (APA, 2012) found that feeling valued at work is linked both to performance and wellbeing. The medical school treats me with respect: Tay and Deiner (2011) found that respect was one of the core indicators of subjective wellbeing. 4.1.3 Data collection The questionnaires were made available to access through the online survey software ‘Bristol Online Survey’ (BOS). The method of disseminating the link to the relevant survey differed slightly between medical schools to comply with their ethical requirements. This included: the virtual notice board ‘Blackboard’, emails direct from medical school staff, and links placed in student newsletters. Reminders went out approximately two weeks later, with a third and final reminder targeting medical schools with low response rates a week after that. Paper copies of the questionnaire were also distributed. The exact nature of the distribution varied between medical schools, with some schools allowing researchers access to lectures (collecting questionnaires in break times) and others encouraging their own staff to distribute the questionnaires in tutorials. Students were requested to only complete one format of the questionnaire. 4.1.4 Data validity checks Paper responses were input in to BOS manually by a member of the research team. The data from the paper questionnaires entered manually were subject to the following checks: 10% 15
of questionnaires entered were checked, and if an error was found, 100% of the field containing the error was subsequently checked. 4.1.5 Quantitative data analysis A descriptive analysis was undertaken to explore the response rates to the questionnaire. The demographics associated with the respondents to the questionnaire broken down by medical school were also explored. Both of these analyses were conducted to assess the generalisability of the results. The remainder of the quantitative data analysis was designed to address issues related to the construct validity of the questionnaire. Construct validity refers to the degree to which inferences can legitimately be made from the operationalisations in a study to the theoretical constructs on which those operationalisations were based. Each of the eight domains can be considered as separate conceptual constructs that together make up the overarching construct known as ‘risk factor domain’. Although demonstrating construct validity is an ongoing process, the analyses described below allowed for an initial exploration of how each risk factor domain is influenced by variables such as medical school, year group and type of course. Exploring the influence of the domains on wellbeing provides an opportunity to explore the arguments highlighted in the introduction that risk factors have the potential to positively and negatively impact on medical student wellbeing. The quantitative data was therefore analysed as follows: 1. An initial overview analysis was undertaken by calculating raw mean scores and related f scores for each of the risk factor domains broken down by medical school. 2. Following a rescaling of the raw scores to produce 1-‐5 mean values, a regression analysis was undertaken for all year groups as well as for style of course (Problem based learning and traditional). 3. A comparison of medical schools on each of the risk factor domains was undertaken by calculating median scores. 4. Finally, multilevel modeling was undertaken to analyse risk factors and their relationship to wellbeing. The impact of improving a score (1-‐5) by 1 on each risk factor domain on wellbeing was calculated. 4.2
Qualitative methods: Exploring content validity
To explore content validity of the questionnaire, a qualitative approach to understanding how risk factors potentially impact upon wellbeing was undertaken. This was felt to be fundamental to achieving a better understanding of students’ perceptions of risk and how they may impact upon their wellbeing. The aim of the qualitative analysis was to triangulate the findings from the questionnaire data. 4.2.1 Recruitment 16
Focus groups were conducted with each year group at Cardiff and Leicester medical schools in Phase 1. We also aimed to purposefully select a year group from each of the five additional medical schools, but due to poor weather and exams, we were unable to recruit at all 5 schools. We did complete focus groups at each of Brighton and Bristol medical schools in Phase 2. However, no new themes emerged and so we did not pursue any additional focus group data. Students were recruited by sending out recruitment emails targeting specific year groups, and displaying posters at each medical school. Places were allocated on a first-‐come first-‐served basis. Incentives (a voucher, memory stick and lunch) were offered to those volunteering to take part. 4.2.2 Group structure An average of 12 students per group took part in a total of 12 focus groups. The nominal group technique (Gallagher, 1993) was employed to enhance engagement. This approach combines quantitative and qualitative data collection in a group setting and allows the researchers to overcome some of the problems inherent in running focus groups where participants may encounter concerns around hierarchy. The focus groups lasted 50 minutes each over lunchtime slots. They were audio recorded and field notes were taken. The focus group tasks included stating the top 5 ‘demands’ of being a medical student, and solutions for key challenges. These solutions were collated into a matrix contained in appendix 8.4. Participants remained anonymous. The flip charts and other materials to aid the ranking process and discussion data collected from the focus groups was later analysed alongside the audio recordings. 4.2.3 Qualitative data analysis The focus group data from phase 1 and 2 along with the 250 open comments from the survey were analysed thematically using framework analysis (Smith & Firth, 2011). Initial analysis identified and described themes, beginning with initial reading and re-‐reading of a selection of transcripts by two members of the research team. These were discussed and codes identified to provide the basis of a coding framework. Data was then systematically coded with two members of the research team independently coding a sample of transcripts. Discrepancies were checked, discussed and clarified. Data was stored and coded using NVivo. Following an initial thematic analysis, further in depth analysis was conducted using an iterative process and drawing upon relevant theory where appropriate (Kelly, 2010). 4.2.4 Integrating the quantitative and qualitative data Finally a comparison of the quantitative and qualitative data was undertaken with each being interrogated from the perspective of the other. 17
4.3 Evaluation data – Exploring face validity Following the data collection and analysis phase, reports were produced for each medical school (see appendix 8.5). The medical schools were then asked to complete an evaluation questionnaire to elicit feedback on the usefulness and utility of the questionnaire and accompanying feedback report as a intervention to prompt quality improvement in the area of student wellbeing using the risk factor model (appendix 8.6). Finally, telephone interviews were arranged with stakeholders in a subset of the medical schools to follow up any issues arising from the questionnaire.
18
5.1
5.0
RESULTS
Quantitative results
5.1.1 Descriptive analysis Response rates 2,766 responses were received, giving an overall response rate of 42%. The response rate from Imperial College was only 2%, therefore as the sample was likely not to be representative, the Imperial College sample was removed from further analyses. The remaining sample of 2,735 equates to approximately 6.7% of the total UK medical school population and a 48% response rate. Figure 6: Questionnaire Response Rates
Imperial, 31 (2% response rate)
NUMBER OF RESPONSES Peninsula, 324 (30% response rate) Bristol, 322 (26% response rate) Brighton, 397 (57% response rate) Hull & York, 477 (64% response rate) Leicester, 506 (67% response rate) Cardiff, 709 (47% response rate)
19
Demographics Table 1 provides the demographic profile of the questionnaire sample. Comparison to GMC data on the present UK medical student population suggested that a representative sample had been collected. Table 1: Demographic Profile of Questionnaire Sample School (N=2,735)
N (%)
Year of study (N=2,725)
N (%)
1 2 3 4 5 6 Gender (N=2,734) Female Male Age (N=2,733)
397 (14.52) 322 (11.77) 709 (25.92) 477 (17.44) 506 (18.50) 324 (11.85)
1 2 3 4 5
755 (27.71) 572 (20.99) 527 (19.34) 470 (17.25) 401 (14.72)
First degree (N=2,735) No Yes Ethnicity (N=2,729)
541 (19.78) 2,194 (80.22)
18-21 22-25 26+ Marital status (N=2,735)
1,560 (57.08) 896 (32.78) 277 (10.14)
White Black Asian Mixed
2,014 (73.80) 75 (2.75) 401 (14.69) 84 (3.08)
Single Married Rather not say Children (N=2,734)
2,449 (89.54) 255 (9.32) 31 (1.13)
Chinese Other
64 (2.35) 91 (3.33)
No Yes First language English (N=2,734)
2,690 (98.39) 44 (1.61)
Christian None Other
1,126 (41.41) 1,076 (39.57) 442 (16.26)
No Yes
338 (12.36) 2,396 (87.64)
1,751 (64.05) 983 (35.95)
Religion (N=2,719)
5.1.2 Raw scores Table 2 shows the raw mean scores for each of the domains and the related f scores. The raw scores are domain specific, due to the fact that each domain had differing numbers of questionnaire items contributing to it. Therefore a comparison of raw scores across the 8 domains is not possible. However, the raw score enables the reader to view how medical school responses differed descriptively within each domain. For example, whilst medical school C achieved a raw score of 11.57 on the domain known as travel and orientation, medical school D achieved a raw score of 21.21 on the same domain. 20
However, it is possible to make one comparison across the domains using the f score. The f score is generated from a one way ANOVA, a technique used to compare means of two or more samples. The f score allows comparison of variability across the domains. The f score relates to the differences in variation of scores of the different samples within a domain with a higher score representing a greater degree of difference or variation. The f scores in this analysis are all highly significant apart from the ‘demands’ domain, which is still significant. However this result does reflects to some extent the large population sampled. It should be noted at this point that the raw scores are not controlled for size of the medical school, gender etc; if these are controlled for, the f score still remains significant or very significant, but at about half the value shown in Table 2. Table 2: Raw mean (SD) scores for each domain from each school. F from univariate one-‐way ANOVA Acquisition Work-life Demands Travel & Safety Culture Perceived Perceived Balance
F score
of Knowledge & Skills 64.95***
Orientation
Support: Academic
24.00***
72.99**
126.25***
114.67***
62.20***
36.13***
Support: Personal/ health 26.12***
A
22.75 (4.38)
11.07 (3.55)
34.66 (6.01)
19.10 (5.76)
18.28 (5.04)
14.12 (2.86)
11.32 (2.15)
17.64 (5.11)
B
19.18 (6.85)
10.65 (3.75)
30.46 (8.35)
11.22 10.03)
14.82 (6.56)
12.19 (2.92)
9.60 (3.20)
14.98 (5.90)
C
19.58 (6.36)
10.75 (3.58)
30.99 (7.33)
11.57 (9.39)
14.00 (5.48)
11.79 (2.93)
9.89 (2.80)
14.68 (5.43)
D
24.58 (4.23)
11.52 (3.52)
36.72 (5.97)
21.21 (5.58)
20.86 (4.17)
13.31 (2.90)
11.19 (2.33)
17.32 (5.57)
E
21.35 (5.15)
9.14 (3.27)
29.53 (7.03)
11.48 (10.05)
16.27 (5.95)
11.05 (3.18)
10.42 (2.87)
14.48 (5.56)
F
23.94 (5.27)
10.94 (3.51)
34.03 (6.45)
18.43 (5.67)
19.83 (5.07)
12.92 (3.02)
11.55 (2.25)
15.91 (5.58)
Total
21.71 (5.85)
10.64 (3.60)
32.55 (7.37)
15.10 (9.22)
17.02 (5.99)
12.44 (3.14)
10.59 (2.73)
15.71 (5.65)
***=p