References

Bombard Y, Baker GR, Orlando E, Fancott C, Bhatia P, Casalino S, Onate K, Denis JL, Pomey MP. Engaging patients to improve quality of care: a systematic review. Implement Sci. 2018; 13:(1)

Bowling A. Evaluating Health Services: multidisciplinary collaboration, 4th edn. New York (NY): Open University Press; 2014a

Bowling A. Data collection methods in quantitative research: Questionnaires, interviews and their response rates, 4th edn. New York (NY): Open University Press; 2014b

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006; 3 https://doi.org/10.1191/1478088706qp063oa

British Society of Urogynaecology and Royal College of Obstetricians and Gynaecologist. Joint guidance on management of urogynaecological conditions and vaginal pessary use during the COVID-19 Pandemic. 2021. https://bsug.org.uk/budcms/includes/kcfinder/upload/files/BSUG-Guidance-on--mm-of-Urogynae-conditions-v4.pdf (accessed 9 June 2023)

Cooper J, Annappa M, Quigley A, Dracocardos D, Bondili A, Mallen C. Prevalence of female urinary incontinence and its impact on quality of life in a cluster population in the United Kingdom (UK): a community survey. Prim Health Care Res Dev. 2015; 16:(4)377-82 https://doi.org/10.1017/S1463423614000371

Collins S. Explanations in consultations: the combined effectiveness of doctors' and nurses' communication with patients. Med Educ. 2005; 39:(8)785-96 https://doi.org/10.1111/j.1365-2929.2005.02222.x

The Independent Medicines and Medical Devices Safety Review. 2018. https://www.immdsreview.org.uk/ (accessed 9 June 2023)

Curtis EA, Redmond RA. Survey postal questionnaire: optimising response and dealing with non-response. Nurse Res. 2009; 16:(2)76-88 https://doi.org/10.7748/nr2009.01.16.2.76.c6763

Dumoulin C, Hunter KF, Moore K Conservative management for female urinary incontinence and pelvic organ prolapse review 2013: Summary of the 5th International Consultation on Incontinence. Neurourol Urodyn. 2016; 35:(1)15-20 https://doi.org/10.1002/nau.22677

Farage M.A., Miller K.W., Berardesca E. Psychosocial and societal burden of incontinence in the aged population: a review. Arch Gynecol Obstet. 2008; https://doi.org/10.1007/s00404-007-0505-3

Hatchett R, McLaren S., Corrigan P, Filer L. Congenital heart: nursing evaluation. Int J Nurs Pract. 2015; 21:556-565 https://doi.org/10.1111/ijn.12300

Health Research Authority. Is my study research? Defining research. 2022. http://www.hra-decisontools.org.uk/research (accessed 9 June 2023)

Hess R, Huang AJ, Richter HE Long-term efficacy and safety of questionnaire-based initiation of urgency urinary incontinence treatment. Am J Obstet Gynecol. 2013; 209:(3)244.e1-9 https://doi.org/10.1016/j.ajog.2013.05.008

Hunt K, Lathlean K. Sampling, 7th edn. In: Gerrish K, Lathlean J, Cormack D. Hoboken (NJ): Wiley-Blackwell; 2015

Jones M, Rattray J. Questionnaire Design, 7th edn. In: Gerrish K, Lathlean J, Cormack D. Hoboken (NJ): Wiley-Blackwell; 2015

Jones G, Brennan V, Jacques R, Wood H, Dixon S, Radley S. Evaluating the impact of a ‘virtual clinic’ on patient experience, personal and provider costs of care in urinary incontinence: A randomised controlled trial. PLoS One. 2018; 13:(1) https://doi.org/10.1371/journal.pone.0189174

Kelley K, Clark B, Brown V, Sitzia J. Good practice in the conduct and reporting of survey research. Int J Qual Health Care. 2003; 15:(3)261-6 https://doi.org/10.1093/intqhc/mzg031

The King's Fund. Admin matters: the impact of administration on patient care. 2021. https://www.kingsfund.org.uk/publications/admin-matters-nhs-patient-care (accessed 9 June 2023)

McGlynn B, White L, Smith K A service evaluation describing a nurse-led prostate cancer service in NHS, Ayrshire and Arran. Int J Urol Nurs. 2014; 8:(3)166-180 https://doi.org/10.1111/ijun.12049

Moule P, Aveyard H, Goodman M. Survey Design and Questionnaires, 3rd edn. London: Sage; 2017

National Research Ethics Service. Defining research. 2014. https://www.clahrc-eoe.nihr.ac.uk/wp-content/uploads/2014/04/defining-research.pdf (accessed 9 June 2023)

National Institute for Health and Care Excellence. Urinary incontinence and pelvic organ prolapse in women: management (NG123). 2019. https://www.nice.org.uk/guidance/ng123 (accessed 9 June 2023)

NHS England/NHS Improvement. The NHS Long Term Plan. 2019. https://www.longtermplan.nhs.uk/publication/nhs-long-term-plan/ (accessed …)

NHS Surveys. The outpatient survey. 2011. http://www.nhssurveys.org/Filestore/documents/OP11_national_tables.pdf (accessed 9 June 2023)

Nursing and Midwifery Council. The code: professional standards of practice and behaviour for nurses, midwives and nursing associates. 2018. https://tinyurl.com/gozgmtm (accessed 9 June 2023)

Oliver R, Thakar R, Sultan AH Urogynecology triage clinic: a model of healthcare delivery. Int Urogynecol J. 2009; 20:913-917 https://doi.org/10.1007/s00192-009-0878-x

Pizzol D, Demurtas J, Celotto S, Maggi S, Smith L, Angiolelli G, Trott M, Yang L, Veronese N. Urinary incontinence and quality of life: a systematic review and meta-analysis. Aging Clin Exp Res. 2021; 33:(1)25-35 https://doi.org/10.1007/s40520-020-01712-y

Randall DM, Fernandes MF. The social desirability response bias in ethics research. J Bus Ethics. 1991; 10:805-817 https://doi.org/10.1007/BF00383696

Serna-Gallegos T, Ninivaggio CS. A lasting impression: telemedicine in urogynecology during the coronavirus disease 2019 pandemic. Curr Opin Obstet Gynecol. 2020; 32:(6)456-460 https://doi.org/10.1097/GCO.0000000000000662

Siedlecki SL, Butler RS, Burchill CN. Survey design research: a tool for answering nursing research questions. Clin Nurse Spec. 2015; 29:(4)E1-8 https://doi.org/NUR.0000000000000134

Sinclair AJ, Ramsay IN. The psychosocial impact of urinary incontinence in women. Obstetrician Gynaecologist. 2011; 133:143-148 https://doi.org/10.1576/toag.13.3.143.27665

Solomon DJ. Conducting web-based surveys. Practical Assessment, Research, and Evaluation. 2001; 7 https://doi.org/10.7275/404h-z428

A specialist service evaluation: a cross-sectional survey approach

22 June 2023
Volume 32 · Issue 12

Abstract

This article presents the findings of a service evaluation of a specialist urogynaecology service and highlights the implications for nursing practice. Aims: To evaluate the overall patient experience and the degree to which individual needs were being met, and make improvements to service delivery. Method: A cross-sectional survey questionnaire was sent to a random sample of women (n=350) who were registered on the electronic patient record system from January to June 2020. Two items on the questionnaire related specifically to communication with patients during the first COVID-19 lockdown in March 2020. Routine service data were collected and included in descriptive statistical analysis. Qualitative themes were thematically analysed. Results: The response rate was 39% (n=137). More than 90% of respondents felt listened to during consultations with nursing and medical personnel, and had trust and confidence in their clinical expertise, felt involved with decisions about care and felt comfortable with intimate examinations. Negative responses related to poor communication with administration and rescheduling appointments. Conclusion: Findings resulted in a departmental review of admin processes, which resulted in improvements to administrative systems, staff training, communication and information giving, and directly benefited patients.

Service evaluations (SEs) adopt rigorous, scientific methods to collect data about the effectiveness of services in a systematic way (Bowling, 2014a). They seek to understand the value and worth of a service, and the extent to which it is meeting its aims. The findings of these evaluations generate evidence to improve quality of care delivery and inform local decision making (National Research Ethics Service, 2014). As a SE does not involve randomisation or a change in routine care or treatment ethical approval is not required (Health Research Authority, 2022). At the heart of SEs is patient engagement, a key component to improving healthcare services and appraising quality (Bombard et al, 2018). Nurses can consider SEs as a useful tool in their practice. Evidence generated facilitates quality improvement initiatives and directly impacts patient care.

Urogynaecology services provide expert multidisciplinary investigations and treatment for women with lower urinary tract and pelvic floor disorders. Conditions treated include stress urinary incontinence (SUI), overactive bladder (OAB), pelvic organ prolapse (POP), bladder pain syndrome (BPS), bowel dysfunction (BD) and urinary tract infections (UTIs). A population-based postal survey in the UK (n=1415) found that 40% of women suffered with some type of UI (Cooper et al, 2015). Furthermore, POP was identified in 50% of parous women on physical examination (National Institute for Health and Care Excellence (NICE), 2019). Risk factors for developing UI include age, parity, obesity, menopause and health conditions such as diabetes, neurological disorders and spinal injury. While not life-threatening, UI is strongly associated with a negative effect on physical and emotional wellbeing (Sinclair and Ramsay, 2011). Feelings of embarrassment relating to UI can lead to social isolation, depression, and anxiety (Farage et al, 2008). Women seeking help from specialist urogynaecology services require a sensitive, holistic and individualised approach to care.

The urogynaecology service at Kingston Hospital provides diagnostic, surgical, pharmacological and conservative treatments. During the study period, the service was applying for accreditation with the British Society of Urogynaecology (BSUG). The intensive accreditation process focuses on surgical data, outcomes and compliance with national guidelines (NICE, 2019). However, a significant amount of the diagnostic and conservative care in our service is delivered by specialist nurses and pelvic health physiotherapists in the outpatient setting and was therefore not included in the BSUG accreditation process.

A gap was identified to evaluate the overall patient experience and quality of service provision encompassing all aspects of care. Conservative management includes behavioural modifications such as bladder training, fluid management, weight loss and pelvic floor exercises. Treatments include pharmacotherapy, vaginal pessaries for POP, percutaneous tibial nerve stimulation, bladder instillations and teaching intermittent self-catheterisation (Dumoulin et al, 2016).

Study objectives

This study aimed to:

  • Establish the degree to which individual needs were being met
  • Evaluate patients' experience of the service
  • Assess overall levels of satisfaction
  • Identify areas for improvement
  • Make recommendations for future service delivery.

Method

Design

A cross-sectional survey design was adopted. Data collection was conducted from July to August 2020. The questionnaire generated quantitative data and qualitative free-text responses. Descriptive statistical data on service activity such as referrals, waiting times, appointment bookings and caseload demographics were included. Routine service data provide insight into scale, effectiveness and activity levels, contributing to the body of evidence from which judgment on the value of a service can be made.

Data collection tool

Questionnaires collect objective data about a population in a standardised way (Siedlecki et al, 2015). They capture individual perceptions and are suited to measure patients' experiences of health (Jones and Rattray, 2015). Questionnaires need to be acceptable, reliable and eliminate bias (Kelley et al, 2003). Validated questionnaires ensure reliability (Bowling, 2014b; Moule, 2017); however, a literature review failed to identify an existing questionnaire measuring patient experience in urogynaecology, or a comparable specialty such as urology or continence. Therefore, an existing questionnaire approved by the Trust to evaluate patient experience of rheumatology outpatient services was revised. This was a modified version of the NHS Outpatient Survey (NHS Surveys, 2011), which was rigorously developed through focus groups and interviews.

Although differences in population between urogynaecology and rheumatology exist, overlapping themes in questionnaire content revealed notable similarities. Questions focused on quality of life, level of service support and interactions with different members of the clinical team. During the revision process, wording of condition-specific questions and variations in response choices were tailored for urogynaecology. Furthermore, novel questions around communication, impact of care and aspects of service delivery were devised to meet the specific needs of the evaluation.

A draft questionnaire was disseminated to the urogynaecology clinical team and subjective feedback received contributing to content validity (Jones and Rattray, 2015). The questionnaire was approved for pilot testing by the patient experience team and sent to a convenience sample of five urogynaecology patients. Amendments comprised a minor alteration in terminology and omission of one question deemed unnecessary. During the study design phase, the COVID-19 pandemic broke out with the first period of national lockdown enforced by the UK government. Two additional questions relating to communication and impact of care during lockdown were therefore incorporated into the questionnaire.

Closed questions consisted of categorised variables, ranked questions and rating scales. Open-ended questions with free-text answer boxes followed the more complex quantitative questions. An introductory paragraph explaining the purpose of the evaluation was included on the front sheet with contact details for the department.

Sampling strategy

The target population was all women registered on the electronic patient record (EPR) system from January to June 2020 (n=960) to ensure that participants' experiences of the service were recent, limiting reliance on recall (Hunt and Lathlean, 2015). Patient records were populated from the EPR into an Microsoft Excel spreadsheet and stored securely in a password-protected file on the hospitals' network. New referrals were removed (n=254) to select those having had a minimum of two consultations, receiving enough care to reflect on their experiences. A computer-generated random sampling sequence selected the end sample (n=350) to minimise risk of bias and obtain an estimated (n=100) responses. This is in line with reported national average response rates to online and postal surveys, which are estimated at 15% and 30%, respectively (Solomon, 2001; Curtis and Redmond, 2009).

Questionnaires were mailed to participants in handwritten envelopes. Printed returns labels with the name and department address with stamped envelopes were included. A 4-week return time frame was stated. Paper responses were transcribed by the evaluator into Survey Monkey for analysis.

Data analysis

Descriptive statistical analysis of quantitative data was performed using Microsoft Excel and presented in tables, graphs or written text with simple numbers or percentages. A thematic analysis of free-text answers was undertaken adopting Braun and Clarke's (2006) 6-step process (Table 1). Initially, the entire data set was read, reviewed and coded. Codes were used to identify common or explicit components from which broader themes were extrapolated. A thematic map further refined and named themes prior to analysis within the context of quantitative results.


Table 1. Six phases of thematic analysis
Phase Examples of procedure for each step
1. Familiarising oneself with the data
  • Transcribing data
  • Reading and re-reading
  • Noting down initial codes
2. Generating initial codes
  • Coding interesting features of the data in a systematic fashion across the data-set
  • Collating data relevant to each code
3. Searching for the themes
  • Collating codes into potential themes
  • Gathering all data relevant to each potential theme
4. Involved reviewing the themes
  • Checking if the themes work in relation to the coded extracts and the entire data-set
  • Generate a thematic ‘map’
5. Defining and naming themes
  • Ongoing analysis to refine the specifics of each theme
  • Generation of clear names for each theme
6. Producing the report
  • Final opportunity for analysis selecting appropriate extracts
  • Discussion of the analysis
  • Relate back to research question or literature
  • Produce report
Source: Braun and Clarke, 2006

Results

Demographic data

The total urogynaecology population from July 2019 to June 2020 was n=1690; 44% of whom were aged 65 years or over, 32% were aged 36-55 years, and 9% were under 36 years old. However, the highest number of responses (72%) were from the 65 and over age groups, see Table 2.


Table 2. Study population: age groups
Age group (years) Total population n (%) Random sample n (%) Respondents n (%)
Under 26 19 (1) 3 (1) -
26–35 132 (8) 21 (6) -
36–45 263 (16) 40 (11) 10 (8)
46–55 268 (16) 48 (14) 11 (8.)
56–64 269 (16) 50 (14) 17 (12)
65–74 393 (23) 98 (28) 53 (39)
75 and over 346 (21) 90 (26) 46 (33)
Total n (%) 1690 (100) 350 (100) 137(100)

Of the ethnicity data on the EPR system at that time, 19% was missing. Of those recorded, 67% were of white, white Irish or white other ethnicity, 6% Asian ethnicity, and less than 1% Black African or Black Caribbean ethnicity, reflecting the general population served by the Trust. Referral data showed that 77% were GP referrals, 21% were internal consultant referrals and 1% from other sources. Data on patient pathways revealed that 21% of patients were discharged during the 12-month period. Of these, 11% did not attended their first appointment and were discharged, 36% visited 1 to 3 times, 28% had 4 to 6 appointments, 16% had 7 to 9 appointments and 9% had 9 or more attendances. Of the cancelled appointments, 20% were cancelled by the hospital, 14% by the patient and 9% did not attend.

Quantitative findings

During the study period, from January to June 2020, 137 survey responses were received, equating to a 39% response rate: 131 were postal and 6 electronic; 100% of respondents met the inclusion criteria of receiving a minimum of 12 weeks of care. Of the respondents, 83% (n=112) were with the service for 1 year or more, with 37% (n=50) receiving care for over 3 years.

Communication during the COVID-19 pandemic

During the first national lockdown (March-June 2020), all face-to-face appointments were postponed or converted to telemedicine. Changes were communicated to women via telephone call or letter; 74% (n=97) of respondents rated communication as ‘very good or satisfactory’.

Urogynaecological conditions treated

The survey asked what urogyneacological conditions were treated: 44% (n=60) self-reported POP, 31% (n=42) reported a combination of POP and other urogynaecological conditions, 28% (n=39) had SUI. OAB was reported in 27% (n=37) and 45% (n=62) reported being treated for two or more conditions. No data were collected on other urogynaecological conditions such as BPS, recurrent UTIs or problems relating to childbirth. Additionally, 64% of respondents received conservative management with 35% either planning to or had had surgical treatment for UI or POP in the past.

Experience of clinical care

The degree to which the individual needs of participants were met was evaluated through (a) experience of consultations with the clinical team, and (b) communication with the service outside of appointments. Participants reported seeing a urogynaecology specialist nurse (82%; n=112), a consultant (64%; n=88), a registrar (26%; n=35) and a physiotherapist (15%; n=21).

Participants were asked to reflect on their most recent experience of a face-to-face consultation and rate different aspects of care in accordance with their individual needs (Figure 1 and Figure 2).

Figure 1. Experiences of face-to-face consultation with urogynaecology consultant or registrar
Figure 2. Experiences of face-to-face consultation with urogynaecology specialist nurse

Communication with the service

Meeting individual needs was further explored through experiences of contact outside of appointments. Of the respondents, 35% (n=48) contacted the service for clinical advice. Of these, 79% (n=38) received advice within 1 working day, 9% (n=4) within 3–5 working days, 4% (n=2) waited more than 5 working days and 9% (n=4) did not respond; 85% (n=40) reported receiving the advice in a timely way. Furthermore, 87% (n=41) had their clinical queries answered ‘fully’ or ‘to some extent’, but 9% (n=4) felt that their questions had not been fully addressed.

Of the respondents, 66% (n=90) contacted the administration regarding appointments. Telephone contact was rated the highest with 66% (n=59) finding it ‘easy’ or ‘extremely easy’; however, 20% (n=18) reported telephone contact as ‘difficult’ or ‘extremely difficult’. In-person contact was reported to be ‘easy’ or ‘extremely easy’ in 53% (n=11) of respondents, 28% (n=6) found it ‘difficult’ or ‘extremely difficult.’ Email contact was rated ‘difficult’ or ‘extremely difficult’ in 51% (n=4) of respondents, 38% (n=3) reported email as ‘easy’ or ‘extremely easy’. Communication via letter was rated ‘easy’ or ‘extremely easy’ by 38% (n=3), and ‘difficult’ or ‘extremely difficult’ by 38% (n=3).

These results suggest that there is not one ideal route of communication for all patients.

The impact of care on participant's physical and emotional wellbeing was assessed using a score of either a positive, neutral or negative impact on areas of life relevant to the participant (Figure 3).

Figure 3. Impact of care on physical and emotional wellbeing. NB only 132 respondents filled in this question

Wishes for future service delivery

Ranked responses for organising future appointments were collected. ‘Organised in person’ was ranked highest (n=61), then ‘Letter’ (n=59) and ‘text message’ (n=13) with ‘online letter’ ranked the lowest (n=13).

The preferred mode of contact for the urogynaecology specialist nurses was a dedicated telephone line (n=110), followed by e-mail (n=54).

The lowest response (n=35) was for both telephone and email. Preference of appointment type if a physical examination was not required revealed participants still ranked face-to-face appointments highest (n=63), followed by phone call (n=48). Video calls were the lowest ranked preference for communication (n=11).

Overall satisfaction

The service was rated as ‘very good’ or ‘good’ by 91% (n=124).

Qualitative findings

There were 306 free-text responses for four open-ended questions. Following thematic analysis adopting Braun and Clarke's (2006) 6-step approach, three main themes emerged:

  • Communication
  • Systems and service delivery
  • Expert clinical care.

Deeper insight into patient experience and the needs of participants were identified through qualitative themes. Additionally, areas for service improvement were highlighted (Table 3).


Table 2. Qualitative themes
Main themes Sub-themes
Communication
  • Communication around changes in care during COVID-19 pandemic
  • Co-ordination of care
  • Clinical information giving
Systems and service delivery
  • Cancelled and rescheduled appointments
  • Environment and equipment
Expert clinical care
  • Teams' knowledge and experience
  • Impact of care on physical and emotional wellbeing
  • Accolade for the specialist nurses
  • Symptom improvement

Table 3. Service improvement ideas
Area for improvement Quality action
Information-giving during clinical consultations: access to additional information and advice about urogynaecological conditions
  • Urogynaecology pages of the Trust website revised to include clear signposting with links to nationally recognised, evidenced-based information
  • Improved access to online and paper copies of patient information leaflets were made available to patients during consultations
Informing patients of how to contact the service outside appointments
  • Mapping exercise undertaken to streamline patient calls: ensure calls reach the right person at the right time
Communication with administration team
  • Restructure of administration team and review of roles and responsibilities undertaken, including level of support required to meet the needs of older, more vulnerable urogynaecology patients
  • Training package for administration teams devised including standard operating procedures to support patient pathways
  • Staff recruitment and retention initiatives introduced to increase morale
IT and outpatient environment
  • Since completion of this service evaluation a new, purpose-built gynaecology outpatient department opened at the Trust

Communication

The essence of this theme was the positive impact and high levels of satisfaction expressed by respondents when they experienced clear and timely communication with the service during and outside of the consultations. Effective communication resulted in feeling reassured when being fully informed about their condition and treatments:

‘Phone calls to check I was OK to come to my appointment and procedure to follow. Face-to-face consultation with excellent guidance given and understanding of my problem. Phone number for specialist nurse given and encouraged to use.’

However, when communication or co-ordination of care was deemed absent or poor, frustration and anxiety was described in a small number of participants.

Systems and service delivery

Processes and the clinical environment were central to this theme. The negative impact felt through changes to, or rescheduled, appointments directly affected participants' experiences of the service. The uncertainty and anxiety resulted in decreased satisfaction, particularly during the COVID-19 lockdown. Awareness of the outpatient environment and IT systems also correlated with lower levels of satisfaction, resulting in suggestions and requests for improvement:

‘The appointment situation could be improved, I turned up for one appointment and was told that I had called to cancel it.’

Expert clinical care

This theme centred around overwhelmingly positive feedback on care and recognition of the clinical teams' expertise. Participants expressed confidence and an appreciation of the knowledge and experience of all disciplines. Overall, a positive impact on physical, social, and emotional wellbeing was reported. Embedded within this was also the notion of kindness, attributed to the team's ability to put respondents at ease. Expert care correlated with a high level of patient satisfaction contributing to the overall quality of the service:

‘I am confident that I have received expert advice from a highly competent team.’

Discussion

Urogynaecology demographic data

During the study period, the service had a large patient caseload (n=960), of whom 59% were over the age of 56 years. This is in keeping with large UK prevalence data reporting a peak age of 55-74 years in women suffering with UI (Cooper et al, 2015). POP is reported to affect 50% of women over the age of 50 years (NICE, 2019). An ageing population in the UK means demand for women's services specialising in pelvic floor dysfunction is anticipated to grow (NHS England/NHS Improvement, 2019). At the time of writing (December 2022), the urogynaecological caseload at Kingston Hospital had increased by 255% (n=4230) from n=1690 (July 2019-June 2020).

Meeting individual needs

Results indicate that the service delivered high standards of clinical care. The urogynaecology medical and nursing teams scored highly in addressing individual needs of patients, both during consultations, and when seeking clinical advice outside appointments. Notably, the specialist nursing team scored particularly high in ensuring that participants did not feel rushed, felt comfortable sharing intimate details about their condition, listening to them and explaining things in a way that participants understood. Conversely, approaches to consultations differ among clinicians. In a qualitative study, Collins (2005) found that doctors approach consultations aiming to identify therapeutic treatments, whereas specialist nurses encourage patient engagement, signposting and listening, thus suggesting differences in their approaches. The correlation between increased patient satisfaction and nurse-led services have been identified by Hatchett et al (2015) and McGlynn et al (2014), who highlighted the importance and value of specialist nurse roles in improving patient care and leading services. In this evaluation, 96% of respondents reported trust and confidence in the whole clinical team. However, 35% contacted the service for clinical advice outside of appointments. This suggested a need to improve provision of condition-specific clinical information during consultations, leading to an improvement in this area of care.

The existing appointment system and communication with the administrative team negatively affected patient experience, some 66% of respondents reported contacting the administration team. Low item response to specific subquestions in this section limited in-depth analysis. However, of those received, 34.5% rated contact ‘difficult’ or ‘extremely difficult’ (mean score for all modes of contact). Poor experiences of communicating appointment changes during the COVID-19 lockdown resulted in anxiety and dissatisfaction.

‘The appointment situation could improve. The latest appointment I received I had to call to check the date etc.’

‘Changes of appointment date make me anxious.’

‘Not replying to phone calls, getting appointments wrong etc causes me to worry.’

The reported benefits when changes to care were communicated effectively during the COVID-19 lockdown, and subsequent instigation of telemedicine were evident. Telemedicine was associated with high levels of satisfaction and was reported as being as effective as face-to-face consultations in meeting individual needs.

‘Telephone call received, excellent advice.’

‘Reassured about coming into hospital, perfect outcome.’

‘During telephone call my treatment was discussed and made plan.’

This was consistent with available evidence. Serna-Gallegos and Ninivaggio, (2020) reported a reduction in exposure to COVID-19 through telemedicine for older more vulnerable urogynaecology patients. An improved patient experience with telemedicine, with no difference in clinical outcomes was found in a randomised control trial examining patient experience with telemedicine versus standard care in urogynaecology (Jones et al, 2018). Beyond the pandemic, virtual delivery of care in urogynaecology was reported to be valuable and safe (Hess et al, 2013). Future directives on the use of telemedicine will require further review. Post-pandemic patients' experience of technology might be different as society has become more used to technology following further lockdowns and home working. However, urogynaecology patients might be at risk of complications such as UTIs or urinary retention necessitating an urgent in-person review (Oliver et al, 2009). Furthermore, physical examination, observation of red flags or patients being at risk of domestic abuse or mental health issues could be missed with telemedicine.

Further exploration in this area is required as participants ranked face-to-face consultations highest compared to telemedicine or video conference even when no physical examination was required. This may indicate the value placed on face-to-face interaction and the relationship with clinicians or is merely reflective of the age of respondents who were more likely to have hearing impairments or were not able to manage virtual consultations. Little evidence currently exists around the effect that administration processes have on NHS patients. However, at the time of writing, The King's Fund was designing a research project examining NHS administration and its effect on patient experience (The King's Fund, 2021).

Impact of care

Quality of life indicators associated with the greatest impact of care were improved physical activity, decreased feelings of embarrassment related to their condition and renewed ability to socialise, suggesting the service was performing well in this area. These elements align with published evidence documenting effects of UI and POP on quality of life. In a systematic review and meta analysis Pizzol et al (2020) found that restrictions on physical and social activities relating to stigma and shame attached to such conditions can result in isolation and depression. The qualitative data collected strengthened findings in this aspect, highlighting particular issues pertinent to urogynaecology patients:

‘Having a vaginal pessary fitted has made an enormous difference to my life, I am lucky to be very active and was considerable hampered before.’

‘I am now able to enjoy dance classes and physical exercise.’

‘The care and medical advice have made me have my life back. I am very grateful to your wonderful team.’

‘Due to the nature of my condition, my whole life has improved. Before there was a constant feeling of uncomfortableness 24 hours a day.’

Quality of life indicators reported as having the least improvement were: ability to remain at work, enjoyment in sexual relationships and relationships with partners. It is difficult to say whether these responses reflected the lifestyle of older respondents who were more likely to have been retired, widowed or not in sexual relationships. Negative feedback from two respondents related to pain and infection associated with previous vaginal mesh surgery. Such complications were acknowledged in the Independent Medicines and Medical Devices Safety review into the safety and efficacy of vaginal mesh procedures (Cumberledge, 2018).

Impact of COVID-19 lockdown

The impact of the study period taking place during the first national lockdown (March-June 2020) is significant. Routine elective and non-urgent outpatient activity was suspended, resulting in large increases in cancelled and rescheduled appointments. This placed significant demands on the administration team. The focus on organisational processes and structure captured in this evaluation exposed areas where improvement was required. The anxiety reported by patients as a result of the lockdown and delays to care during this time was evident. Findings did not merely highlight vulnerability in participants, but reinforced the importance of well-functioning administration systems to being vital in delivering high-quality, reliable services.

With the majority of the clinical team redeployed to areas of higher acuity during the COVID-19 pandemic, the lead urogynaecology nurse was responsible for safety netting and supporting the most vulnerable urogynaecology patients. This included those with recurrent UTIs and vaginal pessaries for POP. Vaginal pessaries were in situ for extended periods and this cohort of had the most contact with the service during this time with 45% (n=60) being treated solely for POP. In response national guidelines were released to support urogynaecology services caring for women with vaginal pessaries during the pandemic (BSUG and Royal College of Obstetricians and Gynaecologists, 2021). Those experiencing bleeding, pain or difficulty passing urine required urgent review and were invited to attend the hospital to see a specialist nurse.

Improvements to service delivery

The results of this service evaluation were presented at the Trust governance, management and patient experience meetings. Findings identified specific areas for improvements to service delivery and resulted in practical changes to patient care. The quality improvements made were rooted in the evidence generated by this study and benefitted patients directly. The complex needs of urogynaecology patients were particularly highlighted with recognition from the Trust that increased administrative support was needed.

Strengths and limitations

The 39% response rate is a strength of the study and above the national average for postal surveys (Kelley et al, 2003). However, 96% (n=132) of these chose to return the paper version, which yielded a lower-than-average response for online completion (Solomon, 2001; Curtis and Redmond, 2009). The volume of free text answers (n=306) generated extensive qualitative data, adding depth to the findings.

Random sampling methods ensured a representative sample was selected. While this approach demonstrates external validity and a robust study design, 72% of responses were from the 65 years and older group and only 9% were under 36 years of age. Therefore, findings can only be considered within the context of age and cannot truly be generalised to the full urogynaecology population. Bias in survey research is well-documented and challenging to eliminate (Kelley et al, 2003). Feasible differences such as language difficulties between responders and non-responders were considered although discounted as routine service data identified the population was predominantly of white British ethnicity. Furthermore, the majority of the data related to POP and UI, omitting patients' experience with other urogynaecological conditions such as BPS, bowel dysfunction and problems relating to childbirth.

Social desirability threatens the validity of self-reported data and is a recognised weakness of survey design (Randall and Fernandes, 1991). Of the participants, 60% had their most recent appointment with a specialist nurse, of which 38% saw the nurses three or more times in a 12-month period. However, regular appointments to change the vaginal pessary every 3-4 months is part of routine urogynaecological care. The development of strong nurse–patient relationships over repeated appointments could be reflected in the overwhelmingly positive feedback about the specialist nurses. Item non-response was low, adding to validity and indicating a high level of acceptability and understandability of the novel data collection tool.

A review of the sampling strategy is required for future service evaluations in order to reach the wider urogynaecology population. For example, convenience sampling through disseminating questionnaires when attending appointments, or electronic distribution could elicit greater responses from all ages and conditions treated.

Data collection and analysis was undertaken solely by the lead nurse. While reflexivity was applied, there was a risk associated with introducing personal assumptions and beliefs in analysis of the qualitative data (Moule et al, 2017) and cannot be entirely excluded.

Trustworthiness and reliability in qualitative and quantitative analysis would be enhanced with more than one researcher. A larger evaluation team comprising all disciplines would broaden the scope of findings and support a truly mixed-method approach including interviews or focus groups. Refinement of those items in the data collection tool with the lowest responses would further improve methodological quality. Advanced quantitative analysis such as cross-tabulation would ascertain whether there was any correlation between satisfaction and aspects such as age, conditions treated and length of time under the service.

Conclusion

Service evaluation is an effective tool that nurses can use to assess the value and quality of new and existing clinical services. The results generated through active patient engagement form part of the evidence base fundamental to make changes to care (Nursing and Midwifery Council, 2018). Service evaluations highlight areas where improvement to local service delivery is required and can develop better understanding of patients' needs. Evidence from this evaluation led directly to quality improvement initiatives, streamlining of administration processes and systems within the urogynaecology service. This co-designing approach between clinical, administration teams and patients is in keeping with results of a recent study into NHS admin processes by The King's Fund (2021).

Predesigned data collection tools for specialist services are not always available. Existing tools can be adapted successfully and tailored to individual services. Team involvement in the evaluation process could facilitate a truly mixed methods approach with a deeper and more rigorous analysis of data. Local Trust quality improvement or patient experience departments offer support with service evaluation and design of data collection tools.

A reassessment of this urogynaecology service is currently being planned. Methodological revisions to items within the data collection tool and dissemination methods are being considered.

Nurses can use service evaluations to showcase specialist and advanced nursing roles, demonstrating their contribution to nursing practice and the impact their role has on patient populations. Furthermore, findings could form part of the evidence base when considering expansion of nursing establishment and clinical services. Positive comments about care received raised morale within the whole team. Administration and clinical staff reported feeling valued as a result of the study.

KEY POINTS

  • Service evaluation is a useful tool that nurses can adopt to assess the value and quality of nursing services
  • Data collection tools can be tailored to individual services.
  • Evidence generated through service evaluation can directly improve patient care
  • Stakeholder involvement and support improves the evaluation process

CPD reflective questions

  • How could service evaluation be used in your specialty?
  • How effective do you feel communication is currently in your service?
  • What could the inclusion of one-to-one interviews or a focus group have added to this service evaluation?