Return to flip book view

MI Federation Protective Factors Training Initiative Evaluation Report July 2023–September 2024 On behalf of the Michigan Federation for Children and Families this report was prepared by Joana Tolchinsky, MSW, Doctoral Student at Michigan State University School of Social Work Elizabeth Henderson, MSW, Director of Policy and Research, MI Federation 620 S. Capitol Avenue, Suite 325 • Lansing MI 48933 Voice-to-email (517) 485-8552 Janet Reynolds Snyder, MBA, Executive Director www.michfed.org Janet Reynolds Snyder, MBA, Executive Director: janet@michfed.org Kadi Prout, LMSW, Associate Executive Director: kadi@michfed.org Elizabeth Henderson, MSW, Director of Policy & Research: elizabeth@michfed.org Jenny Crichton, Director of Member Relations & Events: jenny@michfed.org Rose Homa, Director of Administration & Operations: rose@michfed.org

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 2 Protective Factors Training Evaluation Report July 2023–September 2024 Contents 1. Overview and Background 2. Training Events Provided 3. Average Training Delivery Rating 4. Perceived Change in Knowledge A. Protective Factors for Family Preservation Workers B. Protective Factors for Family Preservation Supervisors C. Protective Factors for Foster Care and Adoption Staff D. Protective Factors Lunch-and-Learns 5. Relevance and Recommendation 6. Participant Impressions A. What specific knowledge will you implement in your practice? B. Positive aspects and opportunities for improvement C. Use of the Children’s Trust Fund Protective Factors training modules D. Additional feedback 7. Trainee Demographics 8. Key Findings and Recommendations

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 3 1. Overview and Background Protective Factors are the strengths that help support families and reduce the likelihood of child abuse and neglect. As an association of community-based agencies administering the Protective Factors Survey and assessing impact and progress in the five Protective Factors domains with families, members of the Michigan Federation for Children and Families came together with a shared goal of increasing knowledge, skills, and consistent application among programs to provide the greatest impact with families. Recognizing this need, the Federation collaborated with its members to conduct an environmental scan, identified existing resources, and developed and sustained the current Protective Factors Training Initiative. This initiative, launched in 2020, has been sustained through generous financial support—initially from Casey Family Programs, followed by The New Foster Care in the second year, and from July 2023 to September 2024, through an investment and contractual agreement with the Michigan Department of Health and Human Services (MDHHS). Through this initiative, 11 private agency professionals became certified by the Children’s Trust Fund Alliance (CTFA)—a national organization with expertise offering a certified training-of-the-trainers curriculum—to deliver the training, “Bringing the Protective Factors Framework to Life in Your Work.” As a result, the following in-depth trainings are offered as live synchronous sessions via Zoom throughout the year: • Introduction to the Protective Factors Framework in Michigan: 1-hour (pre-recorded) session • Protective Factors for Family Preservation Workers: 6 hours in length, offered in two 3-hour sessions • Protective Factors for Family Preservation Supervisors: 3.5 hours in length • Protective Factors for Foster Care and Adoption Staff: 3 hours in length • Lunch-and-Learn Sessions: 1-hour, live interactive sessions providing a community of learning, expanding upon the curriculum, application of the framework, and coaching The partnership between MDHHS and the Federation has allowed the Federation to continue offering training to family preservation teams throughout the state, and expand the training curriculum to all foster care and adoption staff through support of four additional certified trainers with expertise in these program areas. To date, over 600 professionals have been trained by certified trainers in Michigan through the Federation’s initiative. This report presents the data, outcomes, and analysis of the Protective Factors trainings conducted from July 2023 through September 2024, informing stakeholders and decision-makers of the program’s significance and success. Ultimately, the initiative contributes to improving family safety, permanency, and well-being by focusing on the strengths of families. The Federation and training team would like to thank the MDHHS for providing continued funding of the project. In addition, special appreciation goes to MDHHS Director Elizabeth Hertel, Senior Deputy Director of the Children’s Services Agency Demetrius Starling, Director of the In-Home Bureau Tim Click, and Family Preservation Office Manager Nancy Rostoni for their support and partnership.

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 4 2. Training Events Provided Type of Training Session Training Session Dates Total Number of Training Sessions Provided Training Events Total Trainees Attended Evaluation Surveys Returned Protective Factors for Family Preservation Workers 2023: August 30, November 29 2024: January 23, March 27, May 23, July 29 6 5 116 101 Protective Factors for Family Preservation Supervisors 2023: August 24, December 7 2024: March 19, June 13, September 23 5 4 36 19 Protective Factors for Foster Care and Adoption 2024: March 5, April 11, April 22, May 7, June 11, July 23, August 6 7 7 131 96 Protective Factors Lunch-and-Learns 2023: July 20, August 14, September 19, September 26, November 9 2024: January 11, March 12, May 14, July 17, September 18 10 6 245 65 Totals 28 22 528 281 This data described throughout is based on the 281 returned surveys for the trainings conducted. 3. Average Training Delivery Rating All three trainings types—for which participants were asked to rate the delivery on a scale from 1-10—were rated at or above 8.5 out of 10. The Protective Factors Lunch-and-Learns surveys do not contain a question about training delivery. Protective Factors for Family Preservation Workers: 9 Protective Factors for Family Preservation Supervisors: 9.4 Protective Factors for Foster Care and Adoption Staff: 8.7

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 5 4. Perceived Change in Knowledge A. Protective Factors for Family Preservation Workers We asked the 116 participants of the Protective Factors for Family Preservation Workers trainings in November 2023, January, March, May, and July of 2024 to rate their level of understanding regarding four different objectives. We received 101 completed surveys. The following charts illustrate the participants’ self-rated level of UNDERSTANDING of the following: Protective Factors Identification Only 6% of participants felt they had no understanding, and 12% rated themselves as having “minimal understanding” of Protective Factors prior to the training. Both categories decreased to 0% after the training. Ratings of “somewhat understand” decreased from 29% to 3%. And while “moderate understanding” decreased by 4 percentage points, “strong understanding” increased by 47 percentage points. Strengths-Based Theory The ratings for participants’ understanding of Strengths-Based Theory changed as follows: “no understanding” or “minimal understanding” decreased from 5% and 10% respectively to 0%. “Somewhat understand” ratings decreased from 18% to 3%. Moderate understanding decreased by 7 percentage points, but “strong understanding” more than doubled, from 30% to 68%. (Note: only 24 participants rated their prior knowledge on this question, versus 77 on all other questions.)

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 6 Use of Resources to Integrate Protective Factors Post-training ratings of participants' knowledge of available resources to integrate Protective Factors into their practice contained zero “no understanding” or “minimal understanding” ratings, compared to 5% and 15% respectively prior to the training. “Somewhat understand” ratings decreased from 33% to 3% as ratings shifted. 23% of participants claimed “moderate understanding” before the training, which increased to 32% after the training. “Strong understanding” increased more than 2.5 times, from 24% to 65%. Implementation of Protective Factors Prior to the training, participants had a combined low understanding of 18% (7% “no understanding,” 11% “minimal understanding”) on how to implement Protective Factors into their daily practice. This decreased to 0% post-training. As with previous questions, “somewhat understand” decreased drastically by 33 percentage points to 3%. “Moderate understanding” increased from 25% to 33%, and “strong understanding” increased by 43 percentage points to 64%. B. Protective Factors for Family Preservation Supervisors 36 supervisors participated in our four Protective Factors for Family Preservation Supervisors trainings (12/23, 03/24, 6/24, and 9/24) that are included in this in report. 19 participants completed post-training surveys. The following charts illustrate participants’ self-rated ability to PERFORM five different tasks related to the Protective Factors. Identify Positive Habits and Strategies in Daily Practice No participants rated themselves as unable to identify how to develop positive habits and strategies for their daily practice prior to the training. 16% of participants rated themselves as either having “minimal ability” or “some ability” to develop positive habits and strategies for their daily practice prior to the training. Both of these categories decreased to 0% after the training. The “moderate ability” category decreased from over half of the participants (53%) prior to the training to 42% after the training. Ratings for “strong ability” more than tripled after the training, from 16% to 58%.

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 7 Describe Key Strengths of Team Members No participants felt they had “no ability” or only “minimal ability” to describe team members’ strengths before the training. Ratings for “some ability” also decreased to 0% by 32 percentage points after the training. The “moderate ability” decreased from 53% to 42%, while “strong ability” ratings increased by 42 percentage points, to 58%. Engage in Coaching Activities to Build Worker Skills Prior to the training, a combined 27% felt they had either “no ability” (5%) or “minimal ability” (21%) to engage in coaching to build workers’ skills. Post-training, those categories decreased to 0%, and participants’ ratings of “some ability” decreased by 21 percentage points to 5% after the training. The “moderate ability” category increased from 37% to 42%. The ratings for “strong ability” saw the largest increase, with ratings changing from 11% prior to the training to 53% post-training. Outline Case Plans Again, no participants felt they had “no ability” to know what to look for when case planning with families prior to the training. 10% felt they only had “minimal ability” to do so, but that category decreased to 0% after the training. 37% felt they had “some ability” before the training, which decreased to 5% after the training. Another 37% rated themselves as having “moderate ability” prior to the training, which decreased by 5 percentage points post-training. Participants rated themselves as having much more “strong ability” post-training, increasing this category from 16% to 63%. Strengthen the Protective Factors Framework as Leaders Prior to the training, 31% of participants rated themselves as having “no ability” or “minimal ability” to strengthen the Protective Factors Framework as leaders, but both of those categories decreased to 0% post-training. 32% thought they had “some ability” prior to the training, which decreased to 10% after the training. As with other questions, the “moderate ability” ratings increased from 21% to 37%, and the number of participants who felt they had “strong ability” to strengthen the Protective Factors Framework as leaders more than tripled after the training (16% pre, 53% post).

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 8 C. Protective Factors for Foster Care and Adoption Staff Of 131 participants in the new Protective Factors for Foster Care and Adoption Staff training, held seven times in 2024, 96 participants completed the post-training survey. The following charts illustrate participants’ self-rated ABILITY regarding four different skills before and after the training. Identify the Five Protective Factors 23% of participants rated themselves as not being able to identify the five Protective Factors at all, and 16% rated themselves to have “minimal ability” before the training. Those ratings decreased to 1% and 0% post-training, respectively. The 31% rating of “some ability” decreased to 6% in post-training ratings. 15% of participants rated themselves as having “moderate ability” prior to the training, which more than doubled to 34% after the training. An already high 16% of participants who rated themselves to have a “strong ability” to identify the Protective Factors prior to the training more than tripled to 59% post-training. Utilize the Protective Factors Prior to the training, 10% of participants rated themselves as unable to utilize the Protective Factors and 15% answered “minimal ability.” Both of those categories decreased to 0% after the training. While 31% of participants felt they had “some ability” to utilize the Protective Factors before the training, that number decreased to 7% after the training, shifting into the “moderate” and “strong ability” categories. “Moderate ability” increased by 14 percentage points to 41% post-training, and “strong ability” tripled, from 17% to 52%, after the training. Learn a Strengths-Based Approach This question had the highest pre-ratings, with only a combined 7% of participants thinking they could not or could only minimally utilize a strengths-based approach before the training. Post-training, these categories decreased to 1%. 22% of participants reported themselves as having “some ability” prior to the training, which decreased to 4% post-training. Pre-training, 42% rated themselves as having “moderate ability,” which decreased to 30%, making it possible for the “strong ability” category to more than double, from 29% before to 65% after the training.

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 9 Locate Additional Resources Regarding Protective Factors 14% and 16% of participants felt they had either “no ability” or “minimal ability,” respectively, to locate additional resources prior to the training. After the training, these categories decreased to 1% and 0%, respectively. The rating of “some ability” decreased from 27% to 11% post-training. Participants feeling they had “moderate ability” pre-training increased by 9 percentage points post-training. The biggest increase was in “strong ability” ratings, which increased 3.5 times, from 14% prior to 49% after the training. D. Protective Factor Lunch-and-Learns Of the 151 participants in the six Protective Factor Lunch-and-Learns included in this report (11/23, 01/24, 03/24, 5/24, 7/24, 9/24), 65 completed an evaluation survey. We asked them to rate whether their understanding of the topic increased after attending the Protective Factor Lunch-and-Learn. No participants reported they “disagree” or “strongly disagree.” 5% of participants selected the “neutral” category. 38% of participants reported they “agree,” and 57% of participants indicated they “strongly agree” that the Protective Factor Lunch-and-Learn they participated in increased their understanding of the topic covered. 5. Relevance and Recommendation Training Avg. Rating of Relevancy % Strongly Agreed Avg. Would Recommend % Strongly Agreed Protective Factors for Family Preservation Workers 4.7 82 4.7 76 Protective Factors for Family Preservation Supervisors 4.6 82 4.6 82 Protective Factors for Foster Care and Adoption Staff 4.5 66 4.3 57 Protective Factors Lunch-and-Learns 4.9 86 4.6 71 On a scale from 1 (strongly disagree) to 5 (strongly agree), the 281 participants who completed evaluation surveys rated the relevance of the covered topics to their work with ratings ranging from 4.5 (Protective Factors for Foster Care and Adoption Workers) to 4.9 (Protective Factor Lunch & Learns), averaging 4.7 out of 5. When asked if they would recommend the training to a co-worker, ratings for “strongly agree” ranged from 57% (Protective Factors for Foster Care and Adoption Workers) to 82% (Protective Factors for Family Preservation Supervisors), averaging 72%. 5%38%57%Increased Understanding Neutral Agree Strongly Agree0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5RelevantWould RecommendTotal Averages

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 10 6. Participant Impressions At the end of the surveys of all Protective Factors trainings, we asked participants to answer four open-ended questions: A. What specific knowledge or skills from this training do you plan to implement in your professional work with children and families engaged with the child welfare system? B. Please give an example of something we did well in this training and one example of something we can improve in the future. C. Did you utilize the Children’s Trust Fund Protective Factors Training modules available online? Please share information about your experience that will help guide the trainers in any additional resources or training needed in the future. D. Anything else you would like us to know? Following is a sampling of the responses provided, including some key themes: A. What specific knowledge will you implement in your practice? Multiple participants stated they would use their knowledge of Protective Factors themselves in their practice with families, and to correctly document Protective Factors in their case notes. • “Adding the PF in our program progress reports. Even though our program does not require using the PF, we wanted another tool to use to continue to strengthen the families.” • “Talking to families about the PF when doing activities with the families.” • “I plan to use these to better my case notes and the outlook on my consumer.” • “I plan on ensuring that our providers use the protective factors language, when possible (both to be strength-based and for continuum of care with other prevention agencies), and have begun thinking about ways we can better incorporate protective factors in our report writing.” Further, many participants appeared to connect with the concept of parental resilience and approaching families from a strengths-based perspective and with more empathy. • “I plan to use the assessment in order to start having conversations with families around their strengths and ways we can build their supports and resiliency.” • “The practical advice about how to approach talking with a family about the protective factors that they exhibit and the necessary practice of putting it in words that the client will understand. Not writing fancy words for CPS but rather putting it into words the client can understand.” • “Using a strength-based approach with parents and case members. Recognizing accomplishments in treatment plans to help parents feel supported.” • “Recognize bias, look at families with a strength lens, provide better support to families to help meet their needs.” • “How to utilize our resources and be a stronger support for the families. Having knowledge of the protective factors and what action we can do to create a bond with the family and increase their own protective factors, as well.” B. Positive aspects and opportunities for improvement Overall, participants appreciated the breakout rooms and the opportunity to connect and share with people at different agencies who held a similar position. • “Loved the time for listening/learning and also the breakout aspect for applying.” • “Use of breakout rooms was essential to the success of this training! Having a chance to talk to everybody at least once and hear their experiences was very educational.” • “I enjoyed the breakout rooms and getting to hear from others from different jobs, but similar backgrounds.” • “I loved the collaboration and knowledge from everyone. I learned several tricks and tips to teach. Such as glitter jar, Daniel Tiger, and SO many more!

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 11 Participants also pointed out that the materials and presentations were clear, easy to follow, and engaging, with a relaxed atmosphere to the training. • “Creating a stimulating and entertaining learning environment to keep the audience focused was top-notch.” • “I think the activities were very engaging and fun, I also think the videos were entertaining and relevant.” • “Everything and nothing! :-) The course is very informative, inclusive, and thorough... and done in such a fun and friendly way; it is great as is!” Regarding room for improvement, multiple comments touched on needing more time, spacing out the training, or having it offered in person. • “It got overwhelming for me when you were going through all the handouts. Break the training into two parts or make it an all-day training.” • “Something that can be improved on is time. I feel that having more time in this training would be beneficial.” • “Next time have it in person. Most people have a hard time learning virtually.” • “I think it would have been better to lengthen the training sessions over two days and make them 9a-1p or something to allow for more discussion.” Secondly, participants suggested that the training would be even easier to follow if they received handouts or PowerPoint slides prior to the training to take notes on or follow along. • “[I]t’s too much talking, which makes it hard to follow along without the actual PowerPoint slides.” • “Improve would be to use more of the materials handouts as a discussion forum, too.” C. Use of the Children’s Trust Fund Protective Factors Training Modules Of those who answered this question (n=107), 43 had taken the online module and had positive comments: • “Yes, I did utilize the Children's Trust Fund Protective Factors training modules. I found the experience very educational and helpful in my role.” • “[Y]es, very informational and useful information.” • “Yes, it was a great framework to be able to start this training with base knowledge.” Many of the 64 participants who had not taken the online module prior to the training left comments suggesting that information be included about where and how to access the modules and that they be required. D. Additional feedback The surveys administered after the Lunch-and-Learn trainings included the additional question of what topics participants would like to see in future trainings. Many participants stated that they appreciate repetition of content they already heard. They also requested additional resources, specifically regarding housing, and learning more about the Protective Factors and how to utilize them. • “The reminders are nice, so even though I've had these trainings before, it helps to hear it again.” • “Housing/how to access supports for families that are in need of housing—what options are available for families in Michigan.” • “Continuing to expand my knowledge on how to link protective factors to goals, safety plans, and action steps.” The last open-ended question was mostly utilized to provide feedback for the presenters or reiterate that the training was useful and enjoyable: 020406080Yes No4364Online Module Utilization

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 12 • “Enjoyed this presentation.” • “It is very important to continue these trainings, to remind us of all the support available to workers and clients.” • “I enjoyed this training; it helped me understand how I can better help the families using these core strategies and tools I work and serve daily.” • “Presenters were highly knowledgeable and showed great passion for the topic.” • “I enjoyed the information. It will help me in my job immensely.” • “Thank you. I enjoy these Lunch & Learns about Protective Factors. It definitely gives me a better understanding about Protective Factors.” • “This training was engaging, relevant, and well-presented! Thank you!” • “Good activities, engaging trainers, effective videos.” 7. Trainee Demographics Race and Ethnicity 91% of participants answered “Non-Hispanic/Latinx,” 6% identified as “Hispanic/Latinx,” and 2% preferred not to answer. The vast majority of participants (72%) were White/European American, followed by Black/African American (16%). 5% were multiracial, and 2% were Asian American and Native American. Gender Overwhelmingly, participants were female (92%). 6% of all participants were male, 1.5% of data was either missing or participants preferred not to answer this question, and .5% were non-binary. Age Age ranges of participants had the most spread of all demographics, ranging from under 25 years of age to over 65 years of age. However, only 2 out of all survey participants were over age 65. 36% were under age 30, and 25% of participants were between 30 and 39 years of age, 23% were between 40 and 49 years of age, and 16% were over age 50 but under 65 years of age. 91.7%6.3%0.5%1.5%GenderFemale Male Non-binary Prefer not to anser/Missing0.0% 5.0% 10.0% 15.0% 20.0%>6555-5945-4935-3925-29Age Range0.0%20.0%40.0%60.0%80.0%WhiteBlack/African AmericanMultiracialPrefer not to answer/MissingAsian AmericanNative/IndigenousRace

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 13 Positions Held Participants in the Family Preservation Worker, Supervisor, and Lunch-and-Learn trainings held many different positions. 21% were Families First Workers, followed by Families Together Building Solutions Workers (17%). A quarter of participants answered “Other,” which included but was not limited to Safe Care Workers (9 + 1 supervisor), Residential Treatment Staff (6), and Parent Partners/Coaches (5). Participants of the Protective Factors for Foster Care and Adoption training mostly held positions as Foster Care Caseworkers (34%), Adoption Caseworkers (16%), and Foster Care Supervisors (14%). Years in Position Roughly 50% of non-supervisors had held their position for less than one year. Only one participant had been in their position for more than 20 years. Fewer participants (40%) in the Protective Factors for Family Preservation Supervisors training had been in their positions less than Family Preservation Worker training participants. Degrees Held Non-Supervisors The majority of non-supervisor participants had a bachelor's degree (BA/BS = 58%), followed by participants with a bachelor's in social work (BSW = 19%), and participants with a master’s degree (MA/MS = 11%). 7% of non-supervisor participants answered “other,” which included two master of social work (MSW), a BS, and an associate's degree in progress. Supervisors While the majority of supervisors also held a bachelor’s degree, they were fewer than half (43%). With 29%, the percentage of supervisors holding a master’s degree was almost three times higher than for non-supervisor participants (11%). Additionally, 7.1% of the supervisor participants held an MSW, compared to only 5% of non-supervisor participants. 0.0%10.0%20.0%30.0%QRTP supervisorHomebuilders workerQRTP direct care workerBuilding Community…Family preservation…Family Reunification…Families Together…Families First workerOther:Positions Held0.0%10.0%20.0%30.0%40.0%50.0%Lessthan 1year1 - 2years3 - 4years5 - 10years11 - 15years16 - 20years21-25yearsYears in Position0.0%20.0%40.0%60.0%BA/BSBSWMA/MSMSWHigh school…OtherDegrees Held Non-Supervisors0.0%50.0%BA/BS MA/MS BSW Other MSWDegrees Held Supervisors

Michigan Federation for Children and Families • Protective Factors Training Evaluation Report • Page 14 8. Key Findings and Recommendations The Protective Factors Training opportunities were designed to enhance participants' understanding of Protective Factors that promote resilience and well-being in individuals and families. Throughout the sessions, participants were actively engaged in discussions, activities, and interactive learning and coaching experiences that aimed to deepen their knowledge and improve their ability to apply the Protective Factors Framework within their professional settings. Data collected from post-training evaluations consistently reflected high levels of satisfaction, with participants expressing enthusiasm about both the content and the facilitation of the sessions. The data also indicates a significant increase in participants’ perceived knowledge across all training objectives, demonstrating that the training methods were both effective and impactful. Participants repeatedly emphasized the relevance of the training to their specific roles, acknowledging its practical value and applicability. Many indicated a strong intention to incorporate the Protective Factors Framework into their work with children and families, signaling a lasting impact on their professional practice. The results of this evaluation clearly demonstrate the meaningful impact the training has had on participants’ understanding and approach to working with Protective Factors, further validating the importance of continuing this initiative. The Protective Factors Training Initiative directly supports the Michigan Department of Health and Human Services (MDHHS) Keep Kids Safe Action Agenda, contributing to the state’s overarching efforts to promote the safety and well-being of children within their family environments. In addition, the training initiative aligns closely with the focus of a two-generation (2Gen) whole-family approach, included in the MiFamily Together pilot program. The 2Gen approach emphasizes addressing the needs of both children and caregivers, fostering overall family stability and reducing the risk of child maltreatment. By providing professionals with the tools and knowledge needed to apply the Protective Factors Framework, this training initiative plays a crucial role in supporting Michigan’s mission to build and grow strong and resilient communities. Given the positive outcomes and strong participant feedback, this report supports the continuation of the Protective Factors Training Initiative and recommends its expansion to reach new audiences and incoming staff. The ongoing collaboration with MDHHS represents a key strategy for maintaining alignment with the department’s priority of keeping children safe within their families while fostering a positive, measurable impact on communities across Michigan.