This article aims to critically evaluate the use of observed structured clinical examinations (OSCEs) to assess the clinical practice of post-registration nursing students training for advanced roles. It will demonstrate and analyse a rationale for the appropriateness of this pedagogy in education practice today. Reference will be given to the aptitude of this method when identifying areas for student development, with a focus on advanced nurse practitioners (ANPs) who require advanced assessment skills as part of their role.
The Multiprofessional Framework for Advanced Clinical Practice (Health Education England (HEE), 2017) stated that advanced practitioners need a high quality of training to provide enhanced patient care. The Royal College of Nursing (RCN) (2018a; 2018b) states that ANPs are experienced nurses who should be educated to master's level. ANPs must be able to evidence additional core capabilities across the four pillars: clinical practice, leadership and management, education and research (NHS Employers, 2021). NHS England's (2017) five-year plan to provide sustainability and partnership transformations identifies the importance of competence in clinical skills, which must be assessed, achieved and measured. However, Sastre-Fullana et al (2017) indicated that there is ambiguity within advanced roles and many different conceptual approaches. They argued that direct clinical assessment from superiors, peers and structured educational programmes must be used to assess learning outcomes.
The use of the OSCE approach is integral to clinical skills training and is essentially a method of simulating practice to empower students to improve autonomy and develop complex understanding within a safe, controlled clinical environment (Aronowitz et al, 2017). Therefore, an evaluation of OSCEs as a means of assessment for ANPs will help to assess their suitability for recommendation in future.
Background
OSCEs were primarily introduced in medical education as a means of assessing the skills and ability of surgical students, rather than a test of factual knowledge (Cuschieri et al, 1979). The use of OSCEs is seen as a robust way of preparing students for practice; this simulated approach is thought to closely link theory to practice (Parkin and Collinson, 2019).
Although such methods may have derived from the medical profession, Liddle (2014) stated that OSCEs also provide the perfect opportunity to assess nursing students. Students' cognitive, psychomotor and affective skills can be evaluated in the simulated setting using structured tools to gauge their performance. This links into nursing theories that support the ‘novice to expert’ model of learning, and the transition of the learner to proficient practitioner (Benner, 1984). Therefore, the exploration of this form of assessment with its transition and transferability to the nursing profession is of importance. In order to review the information and critically analyse the OSCE as a form of assessment for clinical skills for ANPs a literature search was carried out.
Literature search
Key terms were searched within the library catalogue and databases for this basis of this article. The search engines used were CINAHL, ProQuest, PubMed, and EBSCOhost.
Due to the large volume of studies that were available the main search terms used were: ‘ANP and OSCE’ ‘Advanced practitioner and OSCE’, ‘clinical education and OSCE’ ‘OSCE and clinical skills’, among other narrower terms. The exclusion criteria were broad to enable advanced clinical practitioners (ACPs) from other professional backgrounds to be included, and non-health sciences or education-related literature to be excluded. Despite the ANP and OSCE being the focus, much can be learnt from interprofessional learning, which can improve the health, safety and patient focus through collaboration (World Health Organization, 2010; HEE, 2017).
Educational theories
The literature makes reference to Dale's (1969) pyramid (or cone) of learning (Figure 1), which is a learning principle based on the classification of learning processes. The teacher and learner must move across the processes to achieve learning, with each principle being quantified by percentages. In Dale's theory, learners retain more information through what they ‘do’ rather than what they hear, read or see.
This model has some correlation with the principle of Maslow's (1943) hierarchy of needs, where each level of his pyramid becomes more complex, and can directly relate to the lifelong learning of higher education pathways (Schulte, 2018).
Bloom's taxonomy (1954) is arguably one of the most influential processes underpinning the fundamentals of education (Kreitzer and Madaus, 1994). It enables educators to systematically evaluate the learning of students and changes in their behaviour, which can be applied across diverse learning environments. Masters (2013) questioned the credibility of Dale's pyramid. Increasing evidence suggests that there is variability in much of the literature that cite this model, which raises questions about its influences on learning practices.
The pyramid concept was taken further by Miller (1990) to assess the development of clinical competency in medicine. Four levels of skills enveloping student behaviours and student performance are evaluated in Miller's model. The student moves from ‘knows’ to ‘knows how’ to ‘shows how’ to ‘does’. Some might contend are similar to the cognitive, affective and psychomotor domains that nurses learn (Hagler and Morris, 2015).
Current literature states that an OSCE is a valid, flexible, and reliable method of competence assessment which has been used universally at many educational levels (Daniels and Pugh, 2018). It can be adapted by examiners to suit local needs, which accounts for its general popularity in higher education and clinical practice (Harden et al, 2015). However, Scott (2020) claimed that an OSCE is essentially a test of recall and argued that the student could memorise and systematically perform a practical task, yet not understand the theory that underpins it. Biggs and Tang (2011) described the OSCE as a form of superficial or surface learning, which ultimately could have adverse effects on patient care. A deeper understanding of a subject is needed in which the student develops strategies to enhance and create meanings in the subject studied (Asikainen and Gijbels, 2017). It could be argued that this is the preferred type of learning required to achieve advanced practice.
Scott (2020) stated that educational programmes should not rely on student recollection as an assessment method at the expense of other forms of learning and development. It could be argued that OSCEs should be used as an adjunct to the wider learning curriculum for ANPs, with timely feedback and other coaching mechanisms in place, alongside reflective practice. An example of this could be a partnership approach to learning and assessing, through the use of a learning agreement, which may focus on personal outcomes, or the use of the ‘teach back’ strategy to contextualise recall (Bastable et al, 2019). In addition to this, Jasper et al (2013) noted that reflective practice as a learning method should be integral to all nursing practices and assessment feedback, because it helps practitioners to refer back to the cause-and-effect mechanism of learning by doing as advocated by Dewey (1938).
Application to practice
It must be acknowledged that evaluating learning for an ANP is more complex than the mere assessment of information recall to fulfil an OSCE checklist, and more about applying learning to practice. The relationship with social and environmental factors is of great importance within theory and practice. Morrow and Torres (1995) argued that theories will evolve and progress and educational structures are not static. When applied to methods of assessment such as the OSCE, parts of the structure may be adapted to suit an assessor's or a profession's needs. It can be reasoned that reflection and other methods of assessment should be employed for ANPs given that they are experienced nurses and need to build on their existing skills, whether behavioural, clinical or social, in order to advance their practice. A multifaceted assessment system rather than using the OSCE could encourage a clear focus on experiential learning, which is embedded in adult learning and has become popular in nurse education in recent years (Hughes and Quinn, 2013). It can also link to the practice of ‘scaffolding’, which refers to using different formats of learning materials, and social and linguistic experiences to support a student's learning, reasoning and participation in education activities (Sawyer, 2006).
With the increasing shift toward the OSCE style of assessment in nursing, educators must acknowledge that significant resources are needed when planning clinical skills courses. OSCEs are often resource intensive and the cost incurred requires communication to providers to inform resource allocation (Brown et al, 2015). ANPs are required to have professional registration and tend to be experienced practitioners, therefore costs could impact on the recruitment of those who are self-funding and may also impact on NHS budgets with the plans for recruitment into advanced roles a key part of the NHS Long Term Plan (NHS England, NHS Improvement, 2019). The Department of Health (2012) signalled a shift towards a market style of education procurement involving workforce planning and higher education, arguably making students and lecturers stakeholders in the field of nursing education and commissioning (Gates and Statham, 2013).
The OSCE has maintained its position as the preferred teaching method for skills as part of the advanced practice curriculum. Taylor and Quick's (2020) study of radiology students found that OSCE participants perceived this form of assessment and learning as a valuable, genuine and immersive experience. Their study was an enquiry approach focusing on cascade training from senior peers to junior peers to evaluate performance. This experience may have been less stressful but one could question whether it was robust and its learning outcomes transferable across professions.
Fuller et al (2017) were critical about the assessment and evaluation processes of institutions. They recognise there is a duty to ensure quality and validity throughout the assessment process, citing the behaviour of individual examiners, the design of OSCE stations and assessment outcomes as variables that can impact upon the decision to pass or fail a student. They stated that consideration should be given to the adoption of standardised measurement instruments to assess clinical communication skills in medical or clinical education. They provide recommendations for future research to focus upon evaluating the potential impact of adopting such instruments (Fuller et al, 2017). Harden et al (2015) echo the view that variables in OSCE design require consideration, because the format, activity structure, duration and number of stations, can all affect the assessment and outcome for the students. Different institutions have different feedback systems, rubrics or checklists to assess OSCEs, which may favour a standardisation instrument and improve OSCE moderation.
Adie et al (2013) stated that, in higher education, the process of moderation is commonly governed by agreed university-led policies and procedures. Moderation is the practice of engagement whereby the members of the teaching team foster a shared understanding of what is required for a level of assessment, similar to delivering care in nursing practice. This ensures the assessment connects to specific outcomes and standards, championing processes that are equitable, reliable, valid and evidence based. Daniels and Pugh (2018) stated that such quality assurance processes should be ongoing to ensure the assessment meets its objective. Critically, these processes may need to be more robust for ANP assessments because, the certification and proof of performance for advanced practice is vital to evidence the role. The absence of a place on a professional register to regulate advanced practitioners is thought to be a key factor in an inconsistent approach to the role and educational development strategies (Peate, 2019).
From a quality assurance perspective, Pugh et al's (2016) small international study of medical educators, found OSCEs to be reliable and valid in helping then to identify poor performers who were not progressing to the expected level of clinical competence early on in the assessment process. Their model of competency-based education meant that formative assessments were consistently repeated and students, given feedback, were able to refocus their learning on a specific area to improve. The moderators were also able to ensure quality and consistency by the repetitive nature of the assessment. The feedback remained individualised and of good quality, which is another advantage of an OSCE approach.
Although quality is key, the use of peers should not be dismissed. Collaboration with a capable peer links into Vygotsky's constructivist theory (Reiber and Carton, 1987) and the zone of proximal development, which supports the notion that any teacher can advance the development of a learner when they work together with a more capable peer. This study was based on the learning of children, therefore its validity can only be grounded in the assumption that the theory extends into adult learning. However, Khan et al (2017) argued that, although peer assessment in higher education has been studied, its role within OSCEs has not been systematically reviewed. They used electronic databases to screen and review 13 peer-based, independently reviewed OSCE assessments, by just two reviewers. Their study identified limitations whereby peers awarded students higher grades in comparison to faculty assessors, and checklist grades from them were unreliable, causing concern. It was suggested that, if the students were formally trained to perform peer assessment during the process, then this could promote learning for all concerned, including examiners (Khan et al, 2017). This potentially could be used within an ANP training framework to reinforce learning and use trained ANPs as a future resource group, while benefiting the trained practitioner with regards to professional revalidation (Nursing and Midwifery Council, 2019; 2021).
Within the four pillars of practice, ANPs are required to be nurse educators. Experienced nurses are often experienced in practice but novices within teaching and education, requiring more exposure to gain experience (Sorrell and Canegalosi, 2016). In this way, taking part in peer assessment would benefit ANPs.
Iblher et al (2015) also advocated and encouraged the use of students as assessors in medical OSCE exams, but in formative testing as opposed to summative assessments. Their reasons for this were to prevent legal ramifications or perhaps disputes over results. It must be noted that their standardised structured testing criteria were in a medical specialty, which may not be generalisable to nursing. In support of a stringent testing criteria, Setyonugroho et al's (2015) study focused on consistency in evaluating communication for medical undergraduates and noted that, regardless of the assessor, consideration should be given to the implementation of a standardised measurement tool such as a checklist or rubric to assess skills. This brings reliability and validity to the assessment process.
Challenges
The disadvantages of OSCEs as a method of assessment may become apparent with students who require supportive learning plans. L'Ecuyer (2019) highlighted the increase in numbers of nursing students with learning difficulties across institutions. She also identified a lack of knowledge about how such students can be helped in the clinical environment or educational setting. An OSCE may not be suitable for those with specific learning needs such as dyslexia for example. Gibson and Leinster's (2011) study compared OSCEs, extended matching questions (EMQ) and short answer questions (SAQ) across five cohorts of medical students, to identify differences in the performances between those with and without dyslexia. They reported that any differences between these students disappeared in later years, with no mean test differences in scores apparent. Limitations of this study were the small sample size, studying in only one medical school and the focus solely on dyslexia as a learning need. Other learning needs or a combination of learning needs may have had different outcomes. Although the findings of this study are interesting a low OSCE score could impact a student's education and career path.
Most universities need to accommodate students with disabilities under a ‘reasonable adjustments’ strategy, although Kendall (2018) highlights difficulties with this because it is often unclear what adjustments are required. It is clear that specific learning needs and disabilities cannot be overlooked in higher education. Difficulties with learning could be compared in some respects to language difficulties. Schoonheim et al's (2007) study suggested a link between clinical competency assessment and language proficiency in dental students undertaking clinical assessments. They perceived communication as a main issue, which was improved by extra tuition and supported practice to improve performance. Although this is an older study, the link between learning needs and communication in OSCEs is clear and performance may be adversely affected for ANPs with communication difficulties.
Disability can take many forms and exam nerves can often be considered disabling. Barratt's (2010) study focused on ANP students in the UK, in a qualitative education research study using a focus groups. They acknowledged that students often feel nervous when preparing for, and taking part in OSCEs, therefore alternative approaches to learning and needs were identified. The students took part in video recorded simulated skills with many positive aspects. The method was flexible, less resource and time intensive, it enabled streaming to university web-based platforms, students could re-record and videos could be reused for multiple cohorts. Limitations were not made explicit; however, students who were less able technologically may have been disadvantaged and the face-to-face teaching, the live evaluation and direct assessment of skills may have suffered. Massey et al (2017) noted that video exemplars improved students' confidence, their understanding of their performance and reduced anxiety. However, it is interesting that it failed to improve the overall OSCE results.
OSCE exemplars therefore, could be used to improve staff capacity because students would require less feedback from staff as they would see where they need to improve from watching the video recordings. Using videos could also improve the quality of a student's engagement and experience, which is the aim of higher education institutions (HEIs). In the current COVID-19 pandemic, such virtual learning environments may be beneficial to address students' academic, emotional and social prerequisites, as well as helping to establish some of these methods within the blended, distance approach to learning (Darling-Hammond and Hyler, 2020).
An alternative to the traditional OSCE for ANPs is the use of time-constrained scenario-based practical exams (TSPEs). This method is adapted from OSCEs and may be advocated as an alternative to the traditional method of assessment in healthcare (Hall et al, 2019). This article is from a veterinary nurse perspective, but one could argue that this technique could be transferable to the assessment and evaluation of ANP students in a higher education setting. TSPEs focus on longer cased-based scenarios, which require a greater awareness of planning, time management and decision-making, which are all key requirements for the foundation of evidence-based, proficient and safe patient-focused clinical care (Gardner et al, 2014). This method lends itself to decision-making and autonomy, which are integral to the advanced practice standards, (HEE, 2017).
Conclusion
OSCEs have a proven record for reliability and validity. The literature has noted that there are areas requiring improvement within this method, namely standardisation, but it still upholds as a robust method of competency-based assessment. Advanced clinical practice includes ACPs from many different professional backgrounds with a wealth of experience and knowledge in their own right. Building upon the scaffolding theory for this group of professionals, the OSCE format may benefit from modification. Recommendations from this article are that OSCEs could be accentuated with other assessment approaches to prevent assessment in isolation. Higher educational institutions must continue to focus on the challenge of educating advanced practitioners using an interprofessional approach to learning while incorporating the OSCE method. The use of digital technology to enhance current OSCE methods and improve the outcomes for students with disabilities and anxieties is key, and the virtual learning environment should be explored. OSCEs are just one part of the various assessments that HEIs can use to ensure the high-quality education of ANPs.
KEY POINTS
- The objective structured clinical examination (OSCE) is a mainstream method for assessing clinical competence within the advanced practice arena
- The OSCE's foundations stem from the assessment of medical professionals
- Educational theory upholds OSCE as a robust pedagogy for use within nursing education
- The literature suggests other methods can be used in collaboration with OSCE to enhance the experience and outcomes for advanced nurse practitioners, to improve competence in clinical practice
CPD reflective questions
- What are your experiences of OSCEs? Do you think OSCEs are a robust method of assessing clinical competency?
- What alternatives to OSCEs do you think are best suited to assessing competency?
- Should there be a standardised approach to assessment across higher education institutions for advanced practice?