PrEPArATION FOr ADULTHOOD– SUPErvISING FOr SUCCESS

The Preparation for Adulthood–Supervising for Success (PASS) training program was developed by the National Resource Center for Family-Centered Practice and Permanency Planning at the Hunter College School of Social Work, in collaboration with our subcontractor, the National Foster Care Coalition, along with state and city partners the Oregon Department of Human Services, State Office for Services to Children and Families, the New York City Administration for Children’s Services and the Mississippi Department of Human Services, Division of Family and Children’s Services. The state/city partners were chosen in order to develop materials that could be used in diverse settings e.g. state and locally administered systems, rural and urban, as well as state’s with a strong child welfare provider community. The goal of the Preparation for Adulthood–Supervising for Success training program was to generate a supervisory training curriculum and on-line transfer of learning guidebook to impart best practices of youth development, youth permanency, and collaborative decision-making, using innovative training strategies that could be duplicated by supervisors during their ongoing supervision. In this way both the content and the strategies of the training transferred directly into the workplace.

tem have very unsatisfactory outcomes including high rates of homelessness, incarceration and unplanned pregnancy, and low rates of education and employment (Courtney, Piliavin, Grogan-Kaylor, & Nesmith, 1998;Nixon, 2005;Reilly, 2003); Second, Title IV-E and John H. Chafee program resources appear not to have had the positive effects expected to ameliorate these poor outcomes (Nixon, 2005); And, third there was an identified need for training to assist child welfare supervisors in understanding youth development (Mallon, 2005).
The project's focus on positive youth development was designed to combat prevailing systemic beliefs against reunification or adoption possibilities for older youth (Mallon, 2005;Nixon, 2005). Training for child welfare supervisors has predominately focused on supervising staff to meet generalized permanency needs while focusing on the family as a whole. Most of the work is still done in the context of family-centered services that build on family strengths and meet family needs. There is limited attention given to assessing problem situations from the youth's perspective and preparing a youth for independence and/or transitioning out of foster care or toward another permanency option.
The purpose of this program was to provide what Nixon (2005) called a "cognitive and practice-related reorientation toward a positive youth development approach that includes an understanding of a young person's needs for family connections and a social network, supplemented by skills and competencies" (p. 573). The program was also developed to focus on strategies for supervising the child welfare worker to identify the specific needs of these youth and to develop a plan for achieving goals to meet those needs regardless of other permanency work being done in the family unit.
Four core principles have been identified as essential in order for adolescent transitional living programs to be successful-positive youth development; collaboration; cultural competence; and the creation of permanent life-time connections (Muskie School of Public Service, 2004). The PASS program was designed to incorporate these four core principles into practice, since it is likely that youth will be more successful when these four principles are honored regardless of the type of services provided. In addition, specialized skills are essential to work effectively with older youth. Mallon (2005)

in his book Toolbox No. 3 Facilitating Permanency for
Youth made the case for the development of these skill sets. Child welfare supervisors need training to understand youth development principles and strategies, to focus on giving young people age-appropriate opportunities to exercise leadership, to build skills, and to become involved in decision-making about their future. Furthermore, direct line staff need assistance from their supervisors about how to effectively engage youth in age appropriate ways as they go about developing youthfocused assessments.
Therefore, the program and its curriculum was developed to ensure that child welfare supervisors work with child welfare workers to understand and utilize a positive youth development philosophy; and to encourage the use of youthfocused, youth directed, assessments, with age-appropriate intervention planning.

Developing the Curriculum
The purpose of the initial phase of the "Preparation for Adulthood-Supervising for Success" project was to develop core perspectives that then informed the development of a training curriculum for child welfare supervisors to facilitate the effective delivery and management of Federal Independent Living Services for Youth in Foster Care. One of the earliest steps in the project was the development of a program logic model that served as both a blueprint for the program activities and for the evaluation ( Figure 1). As can be seen from Figure 1, the logic model clearly laid out the feedback loop utilized for curriculum development and design as well as the plans for developing sustainability.
The logic model illustrates that a key first step in the process was to gather data about experiences of success from a range of constituents (supervisors, workers, and young people) using a variety of methodologies, including survey, phone interviews, focus groups and consensus workshops. Data were gathered from all three of the project sites (Mississippi Department of Human Services, New York City Administration for Children's Services, and Oregon Department of Human Services, State Office for Services to Children and Families). A full description of the methodology is available in the evaluation section.
Another key part of the early work was a review of all the preparation of adulthood curricula previously developed during federally funded training projects, as well as those developed by our state and city partners (Fordham University, 2003;Muskie School of Public Service, 2003;The Jordan Institute for Families, 2003). These curricula highlighted strengths-based practices, youth development, and emotional intelligence. Our city partner had developed a framework for practice and supporting worker curriculum, An Integrated Approach to Working with Youth, which focused on the intersection of youth development, permanency, and preparation for adulthood skills.
An essential ingredient in this early process was developing relationships with key stakeholders, including a foster care manager and an assistant commissioner, training directors, university partners, and independent living coordinators at our partner sites to foster investment in the project. During the initial visit to each site, we developed core teams that took responsibility for coordination for that location. In our meetings we learned about the various innovative programs they had implemented and used those practices as a foundation for the project framework. In order to insure that different voices were heard we conducted focus groups with child welfare workers, child welfare supervisors and young people who had been involved with the child welfare system. The focus group feedback was then analyzed for key themes.  These various data sources, including curriculum, policy and procedure review and stakeholder focus groups, converged to create a surprising degree of overlap in response to the questions posed. Many important issues were raised and a range of themes were evident. Combining the feedback with available literature (CWLA, 2005;Muskie School of Public Service, 2004;NRCFCPPP, 2005; University of Minnesota, 2005) six core perspectives emerged. These core perspectives were then used to drive the development of the PASS training program. The six core perspectives identified were: • Develop and maintain positive permanent connections between youth and caring adults.
• Actively engage youth in developing life skills that will prepare them for successful adulthood.
• Relate to youth as resources rather than recipients of the services in the child welfare system.
• Create and maintain environments that promote physical and emotional safety and well-being.
• Value the individual strengths and uniqueness of each youth.
• Involve a diverse array of stakeholders in the development of a comprehensive continuum of services and supports for youth transitioning out of the For each core perspective we utilized the National Resource Center framework to develop corresponding guiding principles and best practices (NRCFCPPP, 2005). The following is an example of the core perspective on youth permanency:

Core Perspective: Youth Permanency
Guiding Principles: • The development of positive, meaningful relationships that foster a sense of belonging and connectedness over time is encouraged and supported.
• Adults and youth are consistently and actively engaged together in activities and experiences.
• Cooperative experiences that build trust and foster honest and open communication are developed and supported. Practices: • Provide youth with opportunities to create, maintain and strengthen sup-portive and sustaining relationships with birth families including siblings, fictive kin, foster and adoptive families, and significant others.
• Provide opportunities for youth to develop connections to peers and mentors.
• Provide opportunities for youth to be engaged in youth/adult partnerships.
• Provide intentional recruitment for permanent adult connections.

The PASS Training Model
Once the content of the trainings had been determined then content delivery methods had to be developed. Utilizing existing supervisory training programs (Collins-Comargo & Groeber, nd;Copa, Lucinski, Olsen, & Wollenburg, 1999;Fone, 2006;Osmond & Darlington, 2005), and feedback from the focus groups and interviews we developed a supervisory skill set that would guide the development of our training materials. Specifically, we expanded the idea of a three-step process of discovery, engagement, and integration developed by the Portage Project (Copa, Lucinski, Olsen, & Wollenburg, 1999) to create a reflective practice model to disseminate the principles outlined in the core perspectives (See Figure 2).
In this reflective practice model the first step is discovery where administrators, supervisors, and workers can use the website to download relevant information in preparation for the learning circles, connect with information, and easy to use resources that can be utilized in the day to day management of cases. We heard from supervisors that some of their best professional development experiences had been in smaller training seminars and over an extended period of time. We used this information in our second step; engagement where supervisors participate in six learning circles. These learning circles are small, facilitated focused discussion groups which take place over time. During these circles participants develop a cohesive learning community and provide a structure for high quality peer learning. The learning circles define real-world challenges specific to the supervision of adolescent cases and identify incremental action steps that address issues raised by participants. During these learning circles participants are asked to take charge of their own professional development through active participation in the learning community. In order to assist in the transfer of learning to case workers we created a one-day supervisory overview training with the focus on supporting educational supervision.
In this reflective practice model the last step is integration. Supervisors can download on the job activities for use in their supervision, which integrate the concepts and strategies discussed in the learning circles.
To emphasize the collaborative nature of the project and to maximize the strengths of the different participating agencies, both the core principles and the conceptual framework were developed incorporating contributions from each project partner. We enhanced the core perspectives from curriculum materials de-veloped at NYC ACS (Morse, 2007), the learning circle model was developed by the University of Mississippi as Clinical Supervision Learning Laboratory Projects (Collins-Comargo & Groeber, nd) and the template for the Supervisory on-thejob activities was contributed by the Child Welfare Training Partnership at Portland State University (http://www.cwpsalem.pdx.edu/). During our curricula review and focus groups we heard the growing need to have youth voice as part of the training project. In order to honor the voices of youth in foster care, we created digital stories with our partners: the Oregon Department of Human Services, State Office for Services to Children and Families and the New York City Administration for Children's Services. According to Leslie Rule of the Digital Storytelling Association (nd), "digital storytelling is the modern expression of the ancient art of storytelling. Digital stories derive their power by weaving images, music, narrative, and voice together, thereby giving deep dimension and vivid color to characters, situations, experiences, and insights" (electronicportfolios.com/digisotry/ index.html). Digital storytelling is a structured group experience that allows individuals an opportunity for self-expression by morphing words and narration with images created through the computer and Internet. The final outcome is an often powerful multi-media mix of images and voice. As part of the project we have created twenty stories each one told from personal points of view and reflecting issues of permanency and preparation for adulthood services, supports, and opportunities. The digital stories enhance the quality of the supervisory learning circles by incorporating the voices of young people, child welfare workers and supervisors to highlight critical practice issues. A number of Digital Stories are available for use Discovery Independent Study: Using a wide range of web-based tools located on the project website Engagement: Six Supervisory Learning Circles using Facilitated Discussion and Skill-Based Practice

Integration:
Web-based Supervisory On-the-Job Activities: To ensure transfer of learning in supervision and training on the project website at (http://www.hunter.cuny.edu/ socwork/nrcfcpp/pass/digital-stories/index.htm).

Structuring the Training-The Use of Ritual
During the first year of the project we created the framework, core perspectives, supervisory skill set, and materials for the learning circles. We also developed a series of learning circle ritual activities that would take place regardless of content to provide familiarity and to anchor participants to the continuity of the circles. First the learning circle always begins with a series of focus questions that are answered in small groups and shared with the large group. The focus questions ground participants in the topic to be discussed and also bring supervisory issues to the forefront. Then, we show digital stories that are reflective of the core perspective that is being discussed. During each learning circle we present promising practices and resources and discuss their integration into supervisory practice. Next, we have the participants engage in an experiential activity such as the permanency maze, foster care pipeline, broken squares, and bridge building to further illustrate core perspective concepts. Each learning circle concludes with an action plan for a short-term change effort developed by the supervisor and shared with the group. These action plans focused on incorporating new knowledge, skills, and attitudes into practice. Action plans are then reviewed at the beginning of each learning circle to assess the extent to which the plan has been successfully accomplished and, to identify factors that facilitated or obstructed completion.

Project Evaluation Plan
During the first two years, the development of the curriculum generated data relevant to process evaluation, with feedback both on curriculum content and training delivery methods. Feedback from the process evaluation has been actively used throughout to make adjustments to the program. We have also conducted limited outcome evaluation related to execution of action plans. In the third year, we are conducting a more comprehensive outcome evaluation including knowledge pre-and post-tests, and follow-up focus groups to discuss the impact of curriculum content on practice. The process evaluation also continued through the third year.
The first year evaluation focused on both quantitative and qualitative data to inform curriculum development and core competencies to be included in the training. Quantitative data was gathered using a 73-item self-administered questionnaire that was developed to determine how supervisors and child welfare workers viewed the relative importance of a range of factors in Transition, Independent Living, and Self-Sufficiency Services (TILSS). Respondents were asked to rank the importance of items representing CWLA's (2005) Standards of Excellence for TILSS on a scale from (1) not important to (10) very important. Respondents were also asked how important they felt it was for TILSS supervisors to be competent in a range of areas, and to have particular values, knowledge, and skills using a 4-point scale ranging from unimportant to very important.
Qualitative data was gathered through focus groups with supervisors, child welfare workers and foster youth. Three different focus group guides were developed, with a similar focus, but with questions tailored to be appropriate for each distinct group. The questions asked participants to reflect on successful child welfare cases that they have been involved with as well as challenges they have faced in the delivery of services. Participants were asked to evaluate services (delivered or received), the types of training currently available, future trainings they desired, and whether or not the development of a website would be useful. Participants were also asked what content they would like made available online. Telephone interviews were used when key participants could not attend focus groups and to supplement the information gathered from focus groups. The phone interviews followed the same question guide as the focus groups.
First year results included analysis from sixty-six surveys gathered from across all three sites, including 29 in New York City, 19 in Mississippi, and 18 in Oregon. Eight focus groups, two with supervisors, three with caseworkers, and 3 with young people. The three groups conducted in Mississippi were with 8 supervisors, 9 caseworkers (all from Southern Mississippi), and 8 young people. In New York City the 3 groups included 5 supervisors, 8 workers, and 6 young people. In Oregon the two focus groups conducted were with 4 caseworkers, and with 11 Foster Club All Stars youth who represented Oregon (2), Oklahoma, Michigan, Washington State, South Carolina, California, Hawaii, Georgia, North Carolina, and Nebraska.
The qualitative results overwhelmingly raised positive relationships as the key point reinforced under success stories by the supervisors, the workers, and the youth. The development of positive and sustaining relationships fostered success. The workers and youth spoke of the importance of connection and of investment in the individual relationships. There was a sense of a caring relationship that went beyond 9-5 that was not about a job, but about authentic connection. Worker creativity was mentioned in relation to exceeding expectations and creating possibilities for the young people.
As outlined in the outputs section of the logic model, the second year evaluation focused on pilot-testing the curriculum for both content relevance and training delivery methods. Data were gathered for different purposes during the secondyear of the program. Self-administered questionnaires were used to gather participants' reactions to the learning circles in which they participated. A short 24-item questionnaire was developed to assess how learning circle participants viewed the curriculum and training style. The feedback questionnaire began with 10 quantitative questions on a five-point scale, ranging from very low, low, average, high, to very high, that asked about the trainer's knowledge, preparation, and organization; teaching effectiveness; and responsiveness to participants. Other questions focused on content, asking whether the training content supported their job duties; materi-als were useful and available for on-going use; stated objectives were achieved; and the event helped improve knowledge, skills and abilities. An overall session rating was also included, as was a question asking whether they thought their supervisor supported the use of the skills taught. The participants were also asked whether they would use the information obtained to train their own staff, and whether they thought that they would incorporate the new information and skills into their own practice. Qualitative questions explored the content and training style issues further asking for most and least valuable features of the session, and what other topics the participants would have liked to see included. Demographic data was collected relative to their length of time in child welfare, whether they currently supervise workers, how long they have been supervising, and their highest level of education.
Data were also gathered relative to changes in practice that occurred as a result of the learning circles. After each learning circle, participants developed Action Plans and shared them with the group. The Action Plan is a 3 X 3 table that focuses on the knowledge, skills and attitudes acquired during the session. Participants are asked to identify what new knowledge, new skills, and new attitudes were acquired during the training. Participants are then asked how they will apply the new knowledge, new skills, and new attitudes to their work-translating learning to practice. Prior to the next learning circle participants were asked to review their results. The circle facilitator noted all plans and they were reviewed at subsequent learning circles.
The digital story component of the project serves multiple functions. First, creating the stories serve as an emotional and empowering experience for the workers and youth who participated. Second, when analyzed, the content of the stories offers additional data to shape the development of the core principles-the main focus of the project's first year. And third, once completed the digital stories can be incorporated into the trainings adding the voices of both workers and youth into the supervisory training sessions. The successful completion and production of 20 stories, including 9 from child welfare workers and 11 from former foster care youth is a key achievement of the project so far, it is anticipated that 10 more stories will be created before the end of the project. The digital stories were completed in time to be used in the learning circles on safety and well-being in both New York and Oregon and in the well-being and strengths-based learning circles in Mississippi. Whenever they were used, they featured prominently on the list of things that the participants found to be most helpful. The narratives explored within the stories confirm many of the core principles identified by this project. The very special importance of connections or relationships was highlighted both by the youth and the workers, the idea of going the extra mile and staying connected was key. Many of the youth highlighted the importance of a worker, a teacher, or a foster parent someone who served as a mentor and provided a stable connection. The digital stories also highlighted the trauma created by sibling separation and the importance of sibling reunions. Collectively the stories were a testament to the potential for both individual and social change. They are a very powerful addition to the curriculum materials.
In the third and final year of the project the evaluation is focusing on outcome evaluation both in terms of knowledge acquisition and practice change. In addition to the content and delivery feedback survey that has been administered from the beginning, quantitative data will be gathered through the administration of a 20item knowledge survey. The survey will assess knowledge from each of the learning circles to be administered pre-and post-participation in the learning circles. The data will be analyzed to assess changes in the knowledge of circle participants from pre-test to post-test. In addition, qualitative feedback will be gathered from training participants via voluntary participation in focus groups or telephone interviews regarding the extent to which content and skills or activities have been incorporated into their practice. Focus group and interview questions inquire about participant's experiences with the format of the learning circles, with the action plans, the extent that they have incorporated course content into their practice, and what impact that has had. In addition, questions address identifying any obstacles encountered when implementing the content and techniques introduced into the training.

Barriers to Success
In today's child welfare community there are competing training projects, policy agendas, and political issues. We found ourselves struggling with all three during the project's tenure. There was initial enthusiasm from our project partners which was strengthened as we built our core teams in each site. However, in two of our sites the core teams experienced a complete staff turnover. In two teams the department leadership changed which meant developing the teams all over again. We have been able to do this in one site but with competing training and political agendas the other site could not continue participation (however, they will remain active partners in our follow-up evaluations).

Sustainability and Dissemination
The major vehicles to support sustainability and to promote the dissemination of information to our partners in the child welfare community have been the development of the project website and the creation of reusable resources (www. hunter.cuny.edu/socwork/nrcfcpp/pass). The creation of Training for Trainers programs has also added to the potential for sustainability and dissemination. The project website contains links to valuable curriculum resources that supervisors can use in their practice. Examples of resources available include information on the core perspectives, learning circle content, and digital stories. We have found much interest in using the digital stories as vehicles for recruitment of foster parents, advocacy, and professional development with all three partners. The digital stories as well as the learning circles provided unique opportunities for the participants and provide opportunities to integrate these practices into youth development work for those who review these materials on the website. Beyond Oregon, New York, and Mississippi, others around the country have found these products to be innovative and useful in their practice with youth and staff. In fact, the long-term outcomes identified in the logic model have actually been made manifest as one of the goals was to increase usage of the products developed.
In addition, we are holding training for trainers with our New York City partners and working with the Adolescent Services Resource Network at the Hunter College School of Social Work to develop training expertise in the PASS model so it can be part of the ongoing array of training offered in their project. We are working with Oregon Department of Human Services, State Office for Services to Children and Families to conduct training targeted to both trainers and supervisors. We feel that supervisors can convene their own learning circles once versed in the framework at their area offices. This would empower them to continue to build learning communities throughout the state.

Conclusion
The project experience so far has been very successful in achieving its goal of designing and delivering an innovative training program for supervisors. We feel that this model could be used with any training content and provide a different learning experience for participants. We are optimistic that through train the trainer sessions and continued use of web-based resources the projects usefulness will sustain beyond the immediate participants. We are also optimistic that the learning circle training model, action plans, and digital stories developed during this project could be used to deliver content on other key areas for supervisors and supervision.