MARC Cross-Site Evaluation: Executive Summary : simplebooklet.com

Executive Summary The Mobilizing Action for Resilient Communities Initiative MARC is a multisite community initiative funded by the Robert Wood Johnson Foundation RWJF and the California Endowment from October 2015 through December 2017 that uses an ACEs framework to foster trauma informed and resilient communities and overall well being ACEs stands for Adverse Childhood Experiences early traumatic events in a child s life that have been demonstrated through rigorous scientific research to have lifelong effects on health and Over the two years of the behavior demonstration the MARC MARC brought together 14 existing networks across the country that were already using ACEs as a foundation to create change in their communities MARC aimed to foster change in those communities as well as stimulate broader regional and national change by strengthening the individual collaborations and facilitating learning across them The Health Federation of Philadelphia supports and facilitates the MARC community efforts mobilizing support and building collective capacity of the groups to create positive social change The sites also coordinated with an online informational and social networking platform the ACEsConnection Network Evaluation Questions Four main questions guided the evaluation questions 2 and 4 drive most of the final report 1 and 3 were the focus of the interim report networks Grew bringing in more members and sectors Engaged in a variety of activities from strengthening their own networks fostering other networks building awareness training and technical assistance advocating for and informing policy building evidence among others Contributed either directly or indirectly to over 100 outcomes most often involving changes in organizations and systems Used a variety of processes of change such as working through their network members engaging in direct outreach and training efforts providing expertise and serving as a trusted source Provided lessons for others interest in implementing networks and or addressing ACEs and fostering resilience 1 What approaches are MARC communities using to promote resilience and address early adversity violence and trauma What are the characteristics of the networks involved in this work What factors support and foster success in promoting resilience and addressing ACEs and what factors challenge or block success 2 What changes are occurring in the networks over time and what factors facilitate network growth and success 3 To what extent are networks engaging more individuals and organizations in the work What strategies are more or less successful in deepening the community base What factors facilitate or hinder efforts to enhance community engagement 4 To what extent are the networks leading to the following changes in their communities improved trauma informed policies and practices at the organizational and system level increased funding for ACEs and trauma related work increased identification and dissemination of best practices increased knowledge of ACEs trauma informed and resiliency practices and increased data collection capacity for ACEs and resiliency indicators

Page 3

Findings Sites continue to be involved in a range of activities with some differences in emphasis across sites Sites continued to do most of the same categories of activities as described in the interim evaluation though a few sites have shifted their emphasis For example in the first year most sites 12 of 14 engaged in efforts to strengthen their networks only five in the second year reported some activity in this area with two sites Alaska and San Diego having it as a significant activity Similarly fewer sites in Year 2 focused on communicating about their efforts Key activities for most sites 10 included awareness building training and improving trauma informed practices Key activities for subsets of sites numbering five each include network expansion and support policy activities community engagement and evidence and data Smaller numbers of sites were involved in evaluation activities n 4 and other activities such as seeking sustainable funding Some of the differences among sites in activities relate to differences in outcomes and strategies for achieving them The Health Federation of Philadelphia HFP played a role in introducing sites to key programs and initiatives to enhance their work fostering individual connections and bringing in resources to facilitate access to and adoption of new practices In addition HFP placed an emphasis on public policy involvement to guide sites to include those activities and areas of change more in their network work Sites grew larger became more multi sectoral and reported increased collaboration Although increasing the overall size of the network was not an explicit goal for all MARC sites all but one network increased the number of members in their networks during the MARC period In addition most networks increased the number of sectors engaged in their work MARC communities increased the number of members representing Education K 12 more than any other sector Results of Social Network Analysis indicate that the growth in networks increased the number of connections among network members in MARC communities between 19 and 152 In addition the number of network members who collaborate a lot increased from baseline through the MARC period In general network density overall connectivity decreased over time while the average number of connections among members in each network increased As networks grow there are more possibilities for connections so the proportion of all possible connections can decrease but individual members can have more Based on self reports of the network s influence on their own work members of MARC networks noted that the networks had most influence on how they or their staff understand their own ACEs backgrounds and how organizations train their staff Network members in Kansas City and Wisconsin reported an increase in the networks influence on their work in the greatest number of ways Over 100 outcomes were documented across the sites most commonly involving changes in practice Across all 14 MARC sites 115 outcomes were identified using an Outcome Harvesting Approach designed to identify and verify changes that occur at least in part due to the efforts of the sites The number of outcomes per site ranged from three for San Diego to twenty for Illinois with an average eight outcomes per site 1 1 It is important to note that the number of outcomes is an estimate Although we strove for a consistent process across the sites it is possible that in some sites considered parts of a process as separate outcomes whereas others combined these as one Moreover some

Page 4

The majority of outcomes 68 involved trauma informed practices and encompassed a range of changes to foster trauma informed and or trauma sensitive environments including adoption of practices within organizations e g expansion of a Trauma Informed Classroom in 20 schools in Boston adoption of training and training curricula that fostered trauma informed practices e g mandatory ACEs training for all nurses in prominent local hospital in Montana changes to the physical environment e g redesign of an ER room at an Illinois hospital for DV victims and self care practices e g Wellness specialist at Garmin introduced specific resilience practices into the company and its policies Other types of outcomes included establishing relationships n 16 instituting policy changes in organizations n 13 public policy outcomes n 10 funding changes n 4 and data changes n 4 More than two thirds of the outcomes were targeted at the organizational level and approximately a fifth each were targeted at the city county and systems level Approximately 10 each were at the community and state level and very few at the regional and national level Most of the changes occurred in the Education Community Development Civic Engagement and the Healthcare Medical sectors sectors that showed increased representation over time The MARC Networks contributed to change in their communities in a variety of ways both direct and indirect Networks had clear and direct contributions to 69 of the 115 outcomes Common strategies included serving as catalysts for change by conducting outreach to develop new affiliates or foster new networks conducting personal outreach to organizations to convince them to adopt trauma informed practices advocating promoting and championing change such as pushing for public policy change and offering presentations and forums often accompanied by follow up efforts to spark change in organizations For the remaining 46 outcomes networks had less direct or limited contribution working with a number of actors to bring about the change Sometimes the network provided a tipping point especially through offering its input as a trusted source or providing expertise Working through members led to both direct and indirect contributions to change and maximized the ripple effect networks can have Lending expertise either proactively or in response to requests was also a common way for networks to either spark or contribute to change Other common cross site strategies associated with change were training holding presentations and forums and partnering with organizations Outcomes varied in their size who they affected and the environments they impacted The outcomes that appeared to have the greatest likelihood of fostering and sustaining significant change were those that affected organizations in sectors typically reticent to address the issue reflected deepening of practices and more systemic change have the potential of reaching and preventing trauma for large numbers of individual or those most vulnerable to trauma provide funding or other resources to grow and sustain the work changes are large and others are smaller and incremental Therefore we do not focus on the exact number per site but offer it as providing some indication of the outcome activity across the sites

Page 5

influence and train gatekeepers those in prevention positions and those experiencing secondary trauma Outcomes that were less significant for the networks are those in preliminary or developmental pilot stages those that were minor changes or affected small numbers incremental changes outcomes well underway prior to MARC and changes that are one time occurrences for an organization Sites distinguish themselves in the way they approach change The strongest and clearest patterns across sites in the outcomes produced relate to the role that the networks have in their communities in bringing about these outcomes When we examine the nature of the outcomes their type and reach we find few differences across the sites Where the sites distinguish themselves is how they approach change Although many of the sites engage in similar activities such as awareness building and training they cluster into groups based on how they put these activities together and the process they use for enacting change We find the sites fall into five different dominant role categories Trusted source and collaborator at multiple levels Community change partners High profile network working through members Change through active outreach awareness building member initiatives and training Change focused on rebuilding Lessons Relevant to Networks and Addressing ACEs and Fostering Resilience The evaluation is focused on a learning agenda The 14 networks provide a laboratory for understanding the role networks can have in creating more trauma informed policy and practice and fostering resilience through a variety of mechanisms Lessons have emerged that are relevant for networks overall and for networks and organizations addressing ACEs and fostering resilience Some of the lessons for networks reveal challenges that networks experienced and how they can be tackled such as balancing professional vs grassroots membership developing a network configured differently than planned others reflect the reality of how networks contribute to outcomes including both direct and indirect contributions along with a variety of players the role of backbone organizations and leadership and others reflect how networks change themselves often evolving through stages developing more explicit governance structures Lessons related to addressing ACEs and fostering resilience highlight the importance of context in shaping networks and their approaches the difficulty in engaging some reticent sectors like businesses challenges posed by stigma and resource concerns and the value in embracing multiple perspectives on the topic Implications for New Networks The evaluation findings and lessons offer a few key implications for networks embarking on addressing this topic including attending to the culture and context of the community along with the capacity and resources in developing and implementing the network approaching change in a multi step manner that is informed and embraces multiple perspectives developing an explicit leadership and working structure especially as membership grows and building in data and measurement that can track and communicate progress to the membership as well as outside sources such as funders

MARC Cross-Site Evaluation: Executive Summary

Page 1

Page 2

Page 3

Page 4

Page 5