Return to flip book view

Neami National Medicare Mental Health Centres Implementation Co-Evaluation SnapshotsTo contact the research team please email: alive-hub@unimelb.edu.au

What is an Implementation Co-Evaluation? A Snapshot on the Framework and its application in Medicare Mental Health CentresCitation: The ALIVE National Centre. 2024. What is an Implementation Co-Evaluation? A Snapshot on the Co-Evaluation Framework and its application in five new mental health care service innovations in community settings. The ALIVE National Centre for Mental Health Research Translation: Australia. To contact the research team please email: alive-hub@unimelb.edu.auRead more about the Implementation Co-Evaluation at the ALIVE National Centre Website:https://alivenetwork.com.au/our-projects/head-to-health-implementation-co-evaluation/This co-partnership was conducted during 2023 when sites were named Head to Health and in May 2024 the Federal Government renamed them Medicare Mental Health Centres.• An implementation co-evaluation is a collaborative exploration of how service innovations and new models of mental health care are being implemented.• The co-evaluation means that work is conducted as a co-partnership between service-research organisations with a view to understanding the ecosystems of service settings.• Co-evaluations have a commitment to co-learning and the involvement of multiple groups with vested interests.• Co-evaluations are iterative and continuous throughout implementation.• A co-evaluation seeks to elevate experiential knowledge by designing with experiential knowledge at the heart. Therefore lived-experience researchers play a critical role in framing and undertaking the research with embedded co-research essential.What is an Implementation Co-Evaluation?Image caption: The iterative design and development of an Implementation Co-Evaluation Framework.Co-Designers shared that the co-evaluation should ask about –waiting times, how accessible theservice was (physical space,flexibility, appointments), howcomfortable people felt, suggestionsfor Improvements, follow up andhow connected people felt aftersessions and would they come back,if this was the first time attending aHead to Health service, how is theguest getting on with the serviceand the team, the overallexperience.Evaluation/measurement of existing quality is a prerequisite for its improvement: what cannot be evaluated and measured cannot be improved.” 1

Matching– requesting preferred researcher/sFlexibility– time, place, a survey or an interviewFrequency– ask regularly not as a one offChoice– choose to write response or use a scaleRespectful Engagement– careful and over timeEthics approvals were obtained from: University of Melbourne Human Ethics Sub-Committee (ID 26525); Human Research Ethics Committee of NT Health and Menzies School of Health Research (NT HREC: ID 2023-4624); AH&MRC Ethics (ID 2133/23); Aboriginal Health Research Ethics Committee (ID 04-22-1066); NEAMI Research and Evaluation Committee.Implementation Strategies to increase Co-Evaluation Frameworks in servicesIMPLEMENTATION GAP 1 –There can be a narrow understanding of how to evaluate new service innovations and models of care.INDIVIDUAL LEVERS: Standard data collection approaches do not always share what matters most for those most impacted.IMPLEMENTATION GAP 2 – Data collected for evaluations can focus on service performance metrics and overlook experiential data. The YES survey is limited for improvements in new innovations and models of care. ORGANISATIONAL LEVERS: A rethinking of what routine data is collected at service and government level is needed. Early co-evaluations mean that relevant implementation factors can be identified for future embedding.IMPLEMENTATION GAP 3 - Few frameworks exist to guide the design, development and application of co-evaluations between service-research partners and wider communities. COMMUNITY LEVERS: greater awareness of new co-evaluation frameworks will guide co-learning and implementation across multiple levels of community sectors and settings.ACCESSIBILITY High visibility, extended hours; fee free, without referrals or appointments, immediate responses to significant distress and suicidality, an alternative to Emergency Departments. REDUCED BURDEN front of house care and information sharing, a central point for assessment, needs-based service navigation, access to short and medium psychological therapies. PERSON CENTRED improve wellbeing through episode of care model, trained peer workers; adequate supervision, student placement opportunities, interdisciplinary care, and strengths-based innovation. The Co-Evaluation Framework and conduct can be understood through cycles of lived-experience research knowledge translation. For this, foundational experiential knowledge is the basis for methods to enable generative knowledge about guest and staff experiences to form within co-research teams and for co-analysis to produce transformative knowledge for integration and implementation.Initial government goals of the new service models for co-evaluation focusThe cycle of lived-experience research knowledge generation and translationWhat mattered for people living with mental ill-health and distress“Most important is it happens wherever and whenever the person prefers, and this is flexible each time.” (Co-Designer Principles).1Samartzis & Talias. 2020. Assessing and improving the quality in mental health. Int. J. Environ. Res. Public Health 17(1): 249.

Who accesses Neami National Medicare Mental Health Centres? An Implementation Co-Evaluation Snapshot # 1To contact the research team please email: alive-hub@unimelb.edu.au“… (Head to Health) doesn’t make it feel like it’s a place that you go to get a service … and makes me feel like it’s a place that you go to and to have a talk and to feel like an actual person.“ (Guest)Read more about this project at the ALIVE National Website:https://alivenetwork.com.au/our-projects/head-to-health-implementation-co-evaluation/This co-partnership was conducted during 2023 when sites were named Head to Health and in May 2024 the Federal Government renamed them Medicare Mental Health Centres.Image caption: Location of the five Implementation Co-Evaluation sites and who attended in a three month window of data collection 2023-2024.• Over 94% were SATISFIED with their care and being supported by an integrated team of clinically trained and peer trained workers.• Guests experienced ACCESSIBLE support delivered in a calm, relaxed environment where they had space and time to share their stories.• Services were presenting AFFORDABLE and TIMELY mental health support in a new and evolving way that filled gaps within the service system.• Guests said care was NON-JUDGEMENTAL, RELATIONAL and provided HOPE. What worked in the model of care delivered and for whom?

If people were referred, most common sources: Head to Health• Mean Kessler 10* = 34 (very high levels of distress)• 62-70% IAR* = 3 (moderate intensity of services)• Across services 94-99% IAR* 2 – 4 (low to moderate and higher service needs)• Mixed anxiety/depression symptoms noted as most common for people• 17% suicidal risk referral • Average engagement 123 daysUrgent Mental Health Care Centre• 38% Triage level 4 (Semi urgent, see within 60mins)• 70% - Triage Levels 2-4• 18% attended UMHCC with suicidal ideation• Average engagement at services 3.9 hours*Initial Assessment and Referral is a government developed decision making tool to identify level of need (1 mild to 5 severe) and to guide matching of services.*Andrews, G., & Slade, T. (2001). Interpreting scores on the Kessler psychological distress scale (K10). Australian and New Zealand journal of public health, 25(6), 494-497.Ethics approvals were obtained from: University of Melbourne Human Ethics Sub-Committee (ID 26525); Human Research Ethics Committee of NT Health and Menzies School of Health Research (NT HREC: ID 2023-4624); AH&MRC Ethics (ID 2133/23); Aboriginal Health Research Ethics Committee (ID 04-22-1066); NEAMI Research and Evaluation Committee.Implementation Strategies to Improve Care in Medicare Mental Health CentreIMPLEMENTATION GAP 1 - Guests raised the time-limited nature of care as a challengeINDIVIDUAL LEVERS: Implement approaches at the individual level of care to connect people into enduring systems of care across primary, social and community based options.IMPLEMENTATION GAP 2 - Guests reported homelessness far more frequently than reported national ratesORGANISATIONAL LEVERS: Implement service level strategies to connect people across existing ecosystems to respond to areas of structural inequalities andsocial determinants.IMPLEMENTATION GAP 3 - Aboriginal and Torres Strait Islander and LGBTQIA+ guests could feel shame and stigma and/or racism seeking supportCOMMUNITY LEVERS: implement a range of appropriate community level prevention strategies to respond to shame and stigma and racism in seeking mental health support.“it was good to have someone that’sactually been through mental healththemselves not someone thatdoesn't really have theunderstanding ….” (Guest)“They’re not judging you for how youlive their life, your life. They’re tryingto help you to live a better life tounderstand your life and live better. Ifind that amazing.” (Guest)Most guests self-referred (65-85%)28% of survey respondents would not have sought support elsewhere.Most guests were discharged home or had no recorded discharge destination. HospitalCommunityGeneralPractitionerHousingWhat were the pathways in and out of services?What do we know about the needs of people who accessed services?

Who delivers care in Neami National Medicare Mental Health Centres? Implementation Co-Evaluation Snapshot #2Citation: ALIVE National Centre. (2024). Who delivers care in Neami National Medicare Mental Health Centres? An Implementation Co-Evaluation Snapshot # 2.The ALIVE National Centre for Mental Health Research Translation: Australia.To contact the research team please email: alive-hub@unimelb.edu.au“I feel like we kind of fill gaps here.” (Staff)“you need to really make all the members aware of each other's skills and strengths and scope of practice. So, when people understand each other and what they can deliver, that makes it a lot easier” (Staff)Image caption: Pie charts illustrate a point in timehead count of peer and clinical staff at sites.Peer Workers Clinical StaffTownsvillePenrithDarwinGeelongUMHCCThe emergent practice approach in Centres two to three years into implementationStaff identified that- Services provided a welcoming, calm and safe environment for guests (service term for people seeking support).- Guests were feeling supported and welcomed.- Services filled gaps in the service system.- Staff went above and beyond to look after each other and the community.- Staff were committed to person centred care and creating safety with guests.- There were some challenges in integrating within the service system.- The service model and approach is positive but structure and care pathways in, out and through the services needs further development.Geographic Differences based on Census 2021 Data- Three services were in regions with higher than national averages of Aboriginal and Torres Strait Islander residents. Available service data indicated a higher proportion of First Nations guests attended than expected from census data.- In four locations there were higher numbers of people told they had a mental health condition compared to the national average (8.8%), while Darwin was lower than the national average (5.9%).- All sites were located in regional areas or centres across Australia.Read more about this project at the ALIVE National Centre Website: https://go.unimelb.edu.au/69w8This co-partnership commenced after the first year of services operating in 2022 with data collection in 2023-2024 when sites were named Head to Health. In May 2024 the Federal Government renamed them Medicare Mental Health Centres.

Staff responses to surveys* may include service managers and support staffDemographics (N=106)- average age of staff: 39 years- average years worked in mental health: 6.9 years- average months working in service: 14.3 months- 62.5% identified as female- 26.9% identified as LGBTQIA+- 6% identified as Aboriginal and/or Torres Strait Islander people- 21.6% identified as culturally or linguistically diverse background Improvement Areas- 76% agreed the service has effectively connected with the existing service system- 69% agreed they were clear on their role of that of others in the team*Respondents: 33% clinical/ 50% peerImplementation journey conversations with all staff and governance membersOver six months peer workers, clinical staff, and managers/governance met in separate groups where they selected images of how the implementation felt at that time. Examples such as the blue image were used to share views that:- Services were developing and evolving.- Connections were being made internally and across the sector but there was more to do.- Development work was progressing but could be slow.- Changes in staff, leadership structures, and inconsistent training creating process issues.- Things were stabilising, but still be a bit rocky. - Challenges delivering the model in practice with equity and meaningful inclusion between clinical and peer staff.Implementation Strategies for Staff Development in CentresIMPLEMENTATION GAP 1: Peer and clinical staff have unclear scopes of practice. Training levels vary as does understanding of mental health care and specifically the model of care. INDIVIDUAL LEVERS:.Develop clear scopes of practices to define role responsibilities and boundaries. Systematise training and development to the scopes and models of care. IMPLEMENTATION GAP 2: Staff turnover has been high and challenging for consistent care delivery. Development of service cultures have lagged.ORGANISATIONAL LEVERS:Create a culture of staff retention through facilitated support and supervision. Foster whole of team co-learning, and safety to hold challenging conversations.IMPLEMENTATION GAP 3: There was a lack of clarity within the community and sector around what the services are seeking to do, and how they are delivering care. Word of mouth and other services were the most common ways people said that they found about Centres.COMMUNITY LEVERS:There is a need to build community awareness of the service models and points of difference, and where the models sit in the service system for the general public and across service sectors. Ethics approvals were obtained from: University of Melbourne Human Ethics Sub-Committee (ID 26525); Human Research Ethics Committee of NT Health and Menzies School of Health Research (NT HREC: ID 2023-4624); AH&MRC Ethics (ID 2133/23); Aboriginal Health Research Ethics Committee (ID 04-22-1066); NEAMI Research and Evaluation Committee.“..from the outset, lack of policy, lack of framework, lack of role definition, and so much changing.” (Staff)“They [guests] just come in, and we’re just sort of bombarding them with the questions.” (Staff)“People experience connection in a way which is simply not possible in a hospital.” (Staff)“We focus on building a strong, well educated, well supported team so that they are able to provide the best care for our guests.” (Staff)Good experiences for staff Not so good experiences for staff