| [Previous] | [Table of Contents] | [Next] |
Our analysis framework of program and evaluation criteria produced a range of diverse youth programs for review, some of which may be described as positive youth development, some as promotion programs, and others as primary prevention. Readers may question how the label of positive youth development fits a particular program with a prevention focus. The goals of this project were to analyze what programs do, and what their evaluations measure, rather than focusing on how they were labeled. We found that a number of programs traditionally considered primary prevention interventions incorporated many of the same positive youth development constructs as programs usually viewed as positive youth development programs. As will be demonstrated in this chapter, many programs with a "primary prevention" label were in fact embedded with numerous positive youth development strategies, and measured positive youth outcomes in addition to changes in problem behavior.
In this first section we describe the organization of the chapter and summarize the criteria used to exclude evaluations. The remaining sections of the chapter describe the settings, content, research design, and results of 25 well-evaluated positive youth development programs. These 25 programs incorporated positive youth development constructs into universal or selective approaches with youth between the ages of six and twenty. Their evaluations used a strong research design (experimental or quasi-experimental with viable comparison groups) and an acceptable standard of statistical proof, provided adequate methodological detail to allow an independent assessment of the study's soundness, and produced evidence of significant effects on youth's behavioral outcomes.
Program material in this chapter is ordered according to how many social domains a program incorporated into its youth development framework. The number of social domains a program addressed is part of the definitional criteria for determining whether the intervention could be considered "comprehensive." Well-evaluated programs in a single social domain are presented first. This section includes eight programs, two in the community domain and six in the school domain. Programs operating in multiple social domains are presented next, beginning with programs in two domains and concluding with three domain programs. The two-domain section has eight programs, including seven that combined family and school, and one program with school and community components. The three-domain section has nine programs, seven of which combined family, school, and community, one addressing family, church, and community; and a program working with school, community, and the work place. The description of each study presents the positive youth development constructs that were addressed, program curricula and strategies, the evaluation's research design, and the posttest and follow-up (if available) results of the intervention. Some descriptions also feature implementation and cost-benefit information.
From the original database of 161 programs identified as potentially within the scope of this study, 77 positive youth development programs had evaluations that appeared to meet the initial criteria for the analysis. Eight-four programs were not included for one of the following reasons:
Eight positive youth development programs targeted a single social domain. Two of these, Big Brothers/Big Sisters and Bicultural Competence Skills, operated in the community domain. Six programs focused on children in the school domain: Growing Healthy, Know Your Body, Children of Divorce, Life Skills Training, The PATHS Project, and Project ALERT.
Two well-evaluated positive youth development interventions in communities demonstrated significant youth behavior outcomes. Both programs used experimental research designs and random assignment of children to intervention and control groups. One, Bicultural Competence Skills, used a skill and competence-based curriculum; the other, Big Brothers/Big Sisters, was a mentoring program without a skills component. Bicultural Competence Skills included follow-up results. Although Big Brothers/Big Sisters did not include long-term follow-up, it provided a sustained intervention exposure (18 months) and measurement period.
Both programs sought to build bonding, competence, and positive identity, but their approaches to promoting these constructs were very different. Each addressed healthy bonding relationships in its own way: Big Brothers/Big Sisters with individual adults, and Bicultural Competence Skills through strengthening the bonds that bicultural children have to both majority and sub-group cultures. While Bicultural Competence Skills addressed competence directly through a skills training curriculum, Big Brothers/Big Sisters took the approach that the primary mechanism of changes in competence (social, behavioral, emotional) are based on the development of a consistent adult-child bond in a mentoring relationship.
In both evaluations, program strategies had a measurable impact on students' outcomes. Positive youth outcomes included greater self-control, assertiveness, and healthy and adaptive coping in peer-pressure situations (Bicultural Competence Skills program), and improvements in school attendance, parental relations, academic performance, and peer emotional support (Big Brothers/Big Sisters). Problem behaviors were also reduced or prevented. Substance use was lower in the experimental groups for both interventions, and hitting, truancy, and lying were reduced as a result of participation in Big Brothers/Big Sisters.
Tierney, Grossman and Resch (1995) evaluated the Big Brothers/Big Sisters program, a mentoring intervention in the community domain. The study addressed nine positive youth development constructs, including social, emotional, cognitive and behavioral competencies, positive identity, bonding, resiliency, self-efficacy, and prosocial norms.
The intervention model featured positive youth development strategies that did not include a specific skills training component, but rather, targeted systemic change in the child's social domains as a function of bonding with a healthy adult. The core strategy in Big Brothers/Big Sisters is to have youth use the program structure and resources to establish a mentoring relationship with prosocial adults. The minimum time commitment required of mentors was several hours, two to four times a month for at least a year. More than 70% of matches in this study met at least three times a month for more than three hours each time, and nearly half met once a week and had an average total exposure of 11 months. The one-to-one mentoring was based on careful matching of adult mentors and children on backgrounds, preferences, and geographic proximity. National standards were implemented for volunteer and youth screening, training, matching, meeting requirements, and supervision.
The Big Brothers/Big Sisters evaluation used an experimental study design that randomly assigned participants to the intervention condition or a wait-list control group. The 18-month wait list corresponded to what is often the usual waiting period for matches in this program. Each agency implemented random assignment procedures until it met the sample size goals. Sites were chosen based on large active caseloads, waiting lists, and geographic diversity. No long-term follow-up data were collected. All data reflected pre-post measurement from interviews conducted at baseline and at immediate posttest 18 months later. Attrition analyses showed that the original sample of 1138 youth, ages 10-14, experienced attrition of 179 participants, resulting in a final sample of 959 youth. Participants were from eight BB/BSA program sites in Phoenix, Wichita, Minneapolis, Rochester, Columbus, Philadelphia, Houston and San Antonio during the study period in 1992-93. Approximately 60% were minority group members (predominantly African-American and Hispanic). More than 40% received food stamps or public assistance.
The evaluation reported significant outcomes at the p<.10 level, but this review acknowledged only those results which met the more rigorous p<.05 standard. The evaluation demonstrated results on behavioral and attitude measures in six impact areas: antisocial activities, academic performance and other school indicators, relationships with family, relationships with friends, self-concept, and social/cultural enrichment. Overall program and control group differences included decreases in drug use (45%, p<.05), hitting (32%, p<.05), the number of times a youth skipped class (37%, p<.05) or a day of school (52%, p<.05), and the number of times a child lied to his parents (37%, p<.05). Significant increases were found in the perceived ability to complete schoolwork (71%, p<.01), and improved parental relationships reflected as increases on indicators of trust (64%, p<.05). Specific sub-group effects showed the program had the greatest impact on substance abuse reductions for minority boys (67.8%, p<.05). Measures of cognitive or academic competence showed larger impacts for minority girls in the treatment group, whose perceived scholastic competence scores were 10% higher (p<.01) than those of control group minority girls. Caucasian Little Brothers also significantly increased their scholastic competence scores (7%, p<.05). Minority male children in the intervention group experienced the highest levels of increased peer emotional support (6%, p<.05) compared to children in the control group. No overall significant effects on measures of self-concept (differences on global self-worth, social acceptance or self-confidence) were noted, but the sub-group of Caucasian treatment group boys scored significantly higher on the social acceptance scale than their control group counterparts (p<.01).
Schinke, Botvin, Trimble, Orlandi, Gilchrist and Locklear (1988) evaluated the Bicultural Competence Skills program, an intervention based on bicultural competence theory and social learning principles. The program addressed 11 positive youth development constructs, including social, emotional, cognitive, behavioral, and moral competencies, positive identity, bonding, self-efficacy, recognition for positive behavior, opportunities for prosocial involvement, and prosocial norms.
A core component of the program philosophy was helping children develop a positive identity based on "bicultural fluency," or becoming socially competent in two cultures. The specific skills training strategies used by the program ranged from practice and rehearsal techniques (fostering opportunities, recognition, and promoting competence), such as role-play, to positive peer-based cultural strategies (promoting positive identity, prosocial norms, and bonding), such as homework assignments on communication and coping strategies associated with bicultural efficacy. Two Native American counselors led the 10-session intervention.
The study used an experimental pretest, posttest, follow-up design in which subjects were voluntarily recruited, then randomly divided after pretesting by reservation site into prevention and control conditions. All subjects completed four outcome measures before, immediately following, and at 6-months post intervention that analyzed culture-relevant peer influences on tobacco, alcohol and drug use, levels of substance abuse, and changes in substance abuse knowledge and attitudes relevant to Native American culture. The unit of assignment matched the unit of analysis (individual). The attrition analysis showed that attrition averaged nine percent across the sample at 6-month follow-up with no dropout differences identified between conditions. The sample of 137 Native American youth, whose average age was between 11 to 12 years old, came from two western Washington reservation sites in a population drawn from tribal and public schools. The evaluation established group equivalence at pretest on the children's household composition, level of acculturation, and current place of residence.
The evaluation found significant results at immediate posttest measurement and at 6-month follow-up in favor of the program group. The analysis showed main effects for the intervention (F(1,124) = 8.28, p<.005) and for measurement occasion (F(1,124) = 7.22, p<.01). At posttest, program students were significantly more knowledgeable about substance use and abuse and held less favorable attitudes about substance use in the Native American culture than their control group counterparts. The intervention group youth also had significantly higher ratings for self-control, the ability to generate alternative suggestions to peer pressure-based encouragement to use substances, and assertiveness. The outcomes on substances used in the previous fourteen days showed that intervention youth reported lower levels of smokeless tobacco use (Program group (P) mean = 2.38, Control group (C) mean = 3.77, p<.05), less alcohol (P mean = 3.63, C mean = 4.71, p<.05) and marijuana use (P mean = 2.12, C mean = 3.79, p<.05). At the 6 month follow-up, intervention participants continued to score higher on measures of knowledge of substance abuse (P mean =17.2, C mean =11.5, p<.05), self-control (P mean =1.89, C mean =.87, p<.05), alternative suggestions (P mean =1.14, C mean =.43, p<.05), and assertiveness (P mean =1.38, C mean =.90, p<.05), and reported significantly less use of smoked tobacco (P mean =1.41, C mean =2.37, p<.05), smokeless tobacco (P mean =2.56, C mean =4.11, P<.05), alcohol (P mean=3.76, C mean = 4.92, p<.05), marijuana (P mean =1.97, C mean = 4.02, p<.05), and inhalants (P mean =.94, C mean =1.32, p<.05) in the last 14 days.
Six programs set in the school domain demonstrated effectiveness. These included Growing Healthy, Know Your Body, Children of Divorce Intervention Program, Life Skills Training, the PATHS Project, and Project ALERT. The first three programs focused on health promotion, and the latter three addressed competence promotion through skills training.
The analysis of single-domain school-based interventions identified six programs that were well evaluated and showed significant effects on youth behavior. These positive youth development programs set in schools can be generally divided into two types: health promotion-focused interventions, and competence promotion-focused interventions. This analysis identified three health promotion and three competence promotion programs in which positive youth development constructs and strategies were successfully incorporated and changed behavioral outcomes for children. In all six programs, the primary emphasis of the intervention was on children's acquisition of skill-based learning to produce the desired behavioral changes. Strategies in these programs relied on opportunities for children to absorb new information and knowledge, and practice specific skills (e.g., coping, decision-making, self-management, frustration tolerance, impulse control, refusal/resistance, life skills, and academic mastery).
Five of the six programs (Growing Healthy, Know Your Body, Life Skills Training, The PATHS Project, and Project ALERT) were multi-year interventions; the exception was a relatively short-term intervention (Children of Divorce) that used developmentally-focused strategies to promote mental health protective factors in a specific at-risk population (children of divorced parents). These programs all used a strong research design, with five of the six employing an experimental design and random assignment of children to intervention and control groups. In all three evaluations of the school-based competence promotion programs, long-term follow-up was designed into the evaluation framework, although only two of the three demonstrated continued effects at follow-up (PATHS and Life Skills Training). Although the health promotion program evaluations did not include follow-up, two of the three (Growing Healthy and Know Your Body) were multi-year trials that provided sustained intervention exposure and measurement periods.
All programs produced evidence of significant changes in children's positive or problem behavior. Among the improvements in positive youth outcomes that resulted from these interventions were better personal health management attitudes and knowledge (Growing Healthy), practices (Know Your Body, Growing Healthy); greater assertiveness, sociability, problem-solving, and frustration tolerance (Children of Divorce); increased acceptance of prosocial norms having to do with substance use (Life Skills Training and Project ALERT); increased interpersonal skills and decision making (Life Skills Training); and a higher capacity for managing one's reactions and behavior in social and emotional situations, greater self-efficacy with creating new solutions to problems, and increased empathy (PATHS). These interventions also had a significant impact on the reduction or prevention of problem behaviors in children. One of the greatest areas of impact for several programs involved successfully changing knowledge, attitudes and/or behavioral practices around cigarette smoking (Know Your Body, Growing Healthy, Life Skills Training and Project ALERT). Two single domain programs also improved youth attitudes and practices around substance use and abuse (Life Skills Training and Project ALERT). Other favorable changes in youth problem behaviors included changes in aggressive and conflict behavior (PATHS).
Walter, Vaughan and Wynder (1989) evaluated the impact of Know Your Body, a school-based health promotion intervention that addressed eight positive youth development constructs including social, emotional, cognitive, and behavioral competencies, self-efficacy, recognition for positive behavior, positive identity, and prosocial norms.
The program strategies encompassed a range of social, cognitive and behavioral skills training designed to promote children's competencies and self-efficacy on health and self-management issues. These included educating youth on the connections between smoking-related decisions and self-image, values, anxiety, and stress (positive identity, prosocial norms); skills training in stress management, decision-making, communication and assertiveness (competence, self-efficacy, recognition, positive identity); and increased awareness of social influences in the initiation of smoking (prosocial norms). The curriculum-based intervention was based on the principles of social learning theory and the health beliefs model. The primary goal was to modify personal health behaviors related to the future development of cancer, including cigarette smoking, diet, and physical activity.
The study used an experimental design, with eight schools (485 students) randomly assigned to the intervention group and seven schools (620 students) randomly assigned to the control group. There were only two measurement points over six years, at baseline and at six-year posttest. Both the attrition analysis and pretest measures of group equivalence showed that the 65% of original participants who remained at posttest measurement were not significantly different from those lost to the study on baseline levels of risk factors, knowledge, or behaviors. The evaluation used schools as the unit of analysis and unit of assignment. Implementation quality was assured, though not systematically measured, with teacher adherence to special teaching protocols monitored by research staff. The study targeted a population of all fourth grade children (n=1105) in 15 New York city elementary schools serving Hispanic/Latino, Caucasian, African-American, and Asian ethnic groups, although the largest group represented was primarily middle class Caucasians. Of the eligible subjects, 911 (82.4% overall, 92% of intervention school students and 75% of nonintervention school students) participated in the baseline measures. By the end of the study, 593 (65%) of the sample had measurement data recorded at both baseline and at termination six years later. In the baseline sample, the mean age was 8.9 years at baseline, and 79.3% were Caucasian, 13.8% were African American, 2.2% were Hispanic and 4.7% were another ethnic identity (mostly Asian). The authors did not state the ethnographic composition of the final sample at posttest in this evaluation. The intervention provided two hours of curricula per week taught by the children's regular teachers throughout the school year over a six-year period.
Behavioral outcomes were reported through a combination of physiological measures (e.g., serum tests to detect nicotine levels, examining and weighing school lunch portions), parent or caretaker reports, and 24-hour dietary recall interviews. The results at six-year posttest showed that the program was effective in modifying two major risk factors associated with the development of cancer -- smoking and diet. Ninth grade students (both females and males) in the intervention schools had 73.3% (p<.005) lower rates of smoking initiation than students in the nonintervention schools. Also by the ninth grade, students (both females and males) in intervention schools had 19.4% (p<.05) lower levels of saturated fat, a 9.8% decrease in total fat, and a 9.5% (p<.05) net increase in consumption of carbohydrates. Significant effects among sub-groups showed that males increased their consumption of total carbohydrates, while females showed net decreases in total and saturated fat plus increases in total carbohydrates and crude fiber.
The analysis of the Growing Healthy program (known originally as the School Health Curriculum Project, or SHCP) used information from several evaluations (Connell, Turner & Mason, 1985; Connell & Turner, 1985; Smith, Redican & Olson, 1992). The study of main interest was an evaluation of the cumulative effects of the intervention. This was a sub-study of the larger School Health Education Evaluation (SHEE), an extensive study of more than 30,000 children, grades 4 to 7, in 1071 classrooms in 20 states. Growing Healthy addressed nine positive youth development constructs, including social, emotional, cognitive, and behavioral competencies, self-efficacy, opportunities for prosocial involvement, recognition for positive behavior, positive identity, and prosocial norms.
The intervention was a comprehensive school health education curriculum that addressed emotional and social aspects of a child's growth and development. The program strategies were implemented by regular classroom teachers, who were trained along with other educational staff in a team training format over three to five days. Students were exposed to between 43 and 56 lessons over a one or two year period (depending on intervention condition). Strategies targeted youth's skills and educational innovations in the classroom. Skills training covered a range of social, cognitive, and behavioral instruction that promoted various positive youth development constructs (e.g., competence, self-efficacy) and enhanced self-image and school and home behavior (positive identity, competence, self-efficacy). Educational innovations were introduced to positively change teacher practices (promoting recognition for positive behavior, opportunities for prosocial involvement, and prosocial norms).
The Growing Healthy/SHCP study used a quasi-experimental design with comparison groups. Four school districts (one mid-Eastern urban district, one mid-Southern suburban district, and two Western rural districts) were selected on the basis of meeting these conditions: all had a pool of children who had received one grade unit (43-56 sessions) of instruction in the preceding year; had a second pool of children who had received no instruction; had both pools take part in pre- and posttesting in the previous year; and had scheduled implementation for the next higher grade level after the first intervention year. The study was designed to assess the cumulative effects of the intervention on 65 classrooms whose students fell into two groups, prior program exposure and no exposure. The unit of analysis was partial classrooms defined by previous health instruction. This resulted in eight groups of students (n=1397) in 130 partial classrooms with varying exposure levels: two groups receiving two units or doses of the curriculum, four groups receiving one unit, and two groups receiving no units. In the 1982-83 school year, fourth and fifth grade students were enrolled in classrooms which either did or did not receive the program. The performance of these groups at the beginning of the year was assessed on tests of knowledge, attitudes and practices by comparing 01 vs 03 and 05 vs 07. Initial testing for effectiveness was done through comparing 02 vs 04 and 06 vs 08. Youth from these classrooms were then promoted to the next higher grade: grade four SHCP and unexposed students were promoted to grade five and grade five SHCP and unexposed students were promoted to grade six. Pretest measures of group equivalence were conducted and showed that groups were comparable at baseline measurement points. The youth consisted of 47% who had been previously exposed to SHCP and 50.3% who were male. Their ethnic identities were 40% African-American, 1% Hispanic, 57% Caucasian, and 1% from unspecified ethnic backgrounds.
The results indicated significant effects for the intervention groups in both years compared to those with no exposure to the curriculum, shown by differences at posttest for knowledge, attitudes, practices, and smoking variables. On all knowledge measures (growth and development, mental health, personal health, nutrition, family life, disease prevention, substance use/abuse, safety/first aid, consumer health, community health), attitude measures (accepting personal responsibility, maintaining a healthy body, safeguarding the environment, respecting the rights and roles of self and others), and measures of practices (decision-making skills, personal healthy practices, social adaptability), groups with two units of exposure performed better than those with one unit, while both groups scored higher than groups with no exposure to the curriculum. At the 1982-83 posttest, intervention classrooms outperformed unexposed classrooms by 9% on the knowledge tests, and 5% on the attitude and practices scales (all findings significant at p<.01). The 1983-84 posttests found significant effects for knowledge by exposure and grade, and for attitudes by exposure group. Significant effects for practices were found by exposure and grade, with the fifth grade having higher practice scores than the sixth. Effects on smoking variables showed both exposure and grade differences. Although the fifth grade children showed no significant differences for percent smoking, regardless of exposure (fifth grade students reported little smoking activity), students who had received two units of exposure indicated less intention to smoke in the future than unexposed groups. Among sixth grade groups, both exposure groups were lower for both current smoking (2 units = 2.8%; 1 unit = 3.7%, no exposure = 9.6%, p<.05) and for future intentions to smoke (2 units = 9.2%; 1 unit = 8.4%; no exposure = 17.1%, p<.05) than the unexposed group.
Pedro-Carroll and Cowen (1985) evaluated the Children of Divorce Intervention Program, an intervention for elementary school age children that addressed 10 positive youth development constructs, including social, emotional, cognitive, and behavioral competencies, bonding, self-efficacy, resiliency, prosocial norms, opportunities for prosocial involvement, and recognition for positive behavior.
The strategies in this study combined health and competence promotion techniques based upon a child development theory that hypothesized how children respond to parental separation. The 10-session framework corresponded to an initial stage (two sessions) for building group cohesion and trust, (bonding, prosocial norms), an interim stage (four sessions) for structuring practice and rehearsal opportunities (competence, self-efficacy, opportunities, recognition) and a final stage (four sessions) for enhancing mastery and the generalization of skills to the youth's environment (resiliency, self-efficacy). Specific strategies included skills for effective problem-solving, communications, decision-making, and anger and anxiety management.
The intervention used an experimental design in which children were randomly assigned to an experimental or control condition within their own schools, and were matched by sex, grade, length of time since their parents' separation, and eight of 10 pre-adjustment measures. There was no attrition in the study. A posttest was done for all measures two weeks after the intervention ended. No follow-up data were collected. The study sample of third through sixth grade Caucasian students included 75 participants (42 boys and 33 girls) from four suburban schools who had been recruited from program descriptions sent to the parents. Another program evaluation (Pedro-Carroll, Alpert-Gillis & Cowen, 1992) was reviewed, but the 1985 study is reported here because of its stronger research design. We note that the later quasi-experimental study showed similar patterns of results.
Results showed significant improvements at the two-week posttest for the experimental group compared with the control group on a number of behavioral outcomes. Teacher ratings indicated that children in the experimental group had greater reductions in anxiety (31.5% vs 34.16%, p<.02) and learning problems (23.07% vs 27.75%, p<.05), as well as on an overall index of classroom adjustment problems (F(3,66) = 8.49, p<.001). The children in the intervention group were rated higher by teachers on a total competence score (t = 6.50, p<.001), and on specific competencies including peer sociability (F(5,64) = 17.59, p<.001), frustration tolerance (F(5,64) = 6.04, p<.05), compliance with rules (F(5,64) = 7.72, p<.01), and adaptive assertiveness (F(5,64) = 5.10, p<.05). Group leaders rated significant increases in skills for personal problem solving (t = 3.85, p<.001).
This review analyzed two evaluations of the Life Skills Training (LST) in 56 New York State public schools, one covering immediate posttest results of a three-year intervention (Botvin, Baker, Dusenbury, Tortu & Botvin, 1990) and the other, follow-up data collected three years after the end of the intervention (Botvin, Baker, Dusenbury, Botvin & Diaz, 1995). Both evaluations reflected studies conducted in the school domain, although we note that later replications of this intervention model also target the community domain. The three-year intervention addressed six positive youth development constructs, including social, emotional, cognitive, and behavioral competencies, self-efficacy, and prosocial norms.
The curriculum emphasized the practice of personal and social skills, including decision making, refusal and resistance, anxiety management, communication, and assertiveness (competencies). Other goals were to increase youth's information and awareness of substance use and abuse, media and advertising influences, and the capacity for self-directed behavior change. The program was 15 sessions for two sessions per week in the first year, 10 booster sessions in the second year, and five booster sessions in the third year. Teachers implementing the program received one day of instruction and a detailed program manual.
The study used an experimental pre-post-follow-up design that divided the schools into three levels based upon a survey of existing smoking levels (high, medium and low). Within these levels, schools were divided by geographic region (eastern New York state, central New York state and Long Island), and then randomly assigned to one of three groups: a prevention program with one-day formal teacher training and implementation feedback (E1), a prevention program with videotaped provider training and no feedback (E2), or a no intervention control group C. There were an uneven number of schools (the first prevention condition had 18 schools, the second prevention condition had 16 schools, and the control condition had 22 schools), which evaluators noted was due to an uneven number of schools in the original assignment blocks, loss of one school after randomization but before intervention, and intentional oversampling of control schools. The attrition analysis showed greater overall losses in the control group. The evaluation established equivalence among the groups at pretest on the variables associated with the evaluation's behavioral outcomes Of those students in all groups whose pretest measures indicated substance use, smoking, or marijuana use, those in the control group who used marijuana at pretest had higher attrition rates. Over the six years of the long-term study, 40% of the original sample were unavailable for follow-up due to absenteeism, transfers, and drop-outs. There were 5954 students in the original sample drawn from a universal population of students in 56 New York State public schools in grades seven to nine. The samples used for the posttest analyses included 4466 students who took both pretests and posttests (referred to as the full sample), and 3684 students who were identified as having received a nearly complete implementation of the program (referred to as the high fidelity sample). The samples used for the long-term analyses included 3597 students who completed both pretests, posttests, and follow-up measures (full sample), and 2752 who received higher fidelity implementation (high fidelity sample). Across studies, most participating youth were white (91%) and lived with both parents (83%).
The study addressed implementation through teacher training and manuals, student guides, staff observation, feedback, and reinforcement. Implementation was measured by dividing objectives covered during each session by the total number of actual curriculum objectives for the particular session observed.
The evaluation of immediate posttest results for Life Skills Training (three years after baseline) reported significant changes in youth's knowledge, attitudes, and behavior. Both intervention groups showed significantly lower cigarette (F(2,3678) = 5.72, p<.003) and marijuana (F(2,3678) = 4.04, p<.01) smoking rates compared to the control group. The second intervention group (taped teacher training) showed a lower frequency of alcohol intoxication (F(2,3678) = 3.25, p<.04) compared with controls. Both prevention conditions showed improvements for knowledge of substance use. The first intervention group (workshop teacher training) showed significantly higher scores than the second intervention group on knowledge of smoking consequences, and both intervention groups had significantly lower expectations than the control group concerning adult smoking norms. The first intervention group had significantly lower expectations about norms for adult and peer marijuana use. Both prevention groups had significantly higher interpersonal skills knowledge than the control group.
The long-term follow-up (Botvin, Baker, Dusenbury, Botvin & Diaz, 1995) measured the participants six years after baseline when they averaged 18 years of age, and reported results separately for the full sample and the high fidelity sample. This evaluation reported only long-term behavioral outcomes associated with substance use. The authors noted "Carbon monoxide levels in expired air were significantly correlated with self-reported cigarette smoking (r=.35, p<.001) across groups, providing presumptive evidence for the validity of the self-report data" (Botvin et al., 1995:1109). The long-term follow-up results for the full sample (n=3597) showed significant decreases in monthly smoking rates (E1 = .27, p<.05, E2 = .26, p<.01, C= .33) for both prevention groups compared to the control, and lower weekly rates for both intervention groups as well (E1 = .23, p<.01, E2 = .21, p<.05, C = .27). The prevalence of heavy cigarette smoking (full sample) was lower for the second prevention group (E2 = .09 vs C = .12, p<.05) compared to the control. Problem drinking rates (full sample) decreased for both prevention groups compared to the control group youth (E1 = .57, p<.05, E2 = .55, p<.01, C = .59, p<.05). Polydrug use rates showed lower combined monthly cigarette smoking and alcohol use for both prevention groups (E1 = .21, E2 = .21, C= .29, p<.01), lower combined weekly tobacco and alcohol use for the second prevention group (E2 = .10, C = .14, p<.05), lower combined weekly tobacco and marijuana use for both prevention groups (E1 = .04, E2 = .04, Cl = .08, p<.01), and lower weekly levels of combined alcohol, tobacco and marijuana use for both prevention groups (E1 = .03, E2 = .03, C = .06, p<.05).
The long-term follow-up results for the high fidelity sample (n=2752) of youth also demonstrated continued effects of the intervention. Six years after baseline, the sample who had received a relatively complete implementation of the program showed significantly lower rates for both intervention groups compared to the control group on measures of monthly (E1 = .24, E2 = .23, C = .33, p<.01), weekly (E1 = .20, E2 = .19, C = .27, p<.05), and heavy cigarette smoking (E1 = .09, E2 = .08, C = .12, p<.05). Both intervention groups had lower levels of weekly (E1 = .24, E2 = .20, C = .29, p<.01) and heavy (E1 = .53, E2 = .52, C = .59, p<.01) drinking, as well as intoxication (E1 = .31, E2 = .28, C = .40, p<.01). The second intervention group had lower levels of monthly drinking (E2 = .54, C = .60, p<.01) and the first intervention group had lower monthly marijuana rates (E1 = .10, C = .14, p<.05). Both groups had lower weekly marijuana rates (E1 = .05, E2 = .05, C = .09, p<.05) compared with controls. Polydrug use measures showed lower rates for both intervention groups for monthly combined cigarettes and alcohol (E1 = .19, E2 = .1, C = .27, p<.01), monthly combined cigarettes and marijuana (E1 = .05, E2 = .07, C = .12, p<.01), weekly combined alcohol and marijuana (E1 = .03, E2 = .04, C = .07, p<.01), weekly combined cigarettes and marijuana (E1 = .02, E2 = .03, C = .08, p<.01), and weekly combined cigarettes, alcohol, and marijuana (E1 = .02, E2 = .02, C = .06, p<.01). The first group showed lower polydrug rates of combined monthly alcohol and marijuana (E1 = .09, C = .13, p<.05) and combined monthly cigarettes, alcohol, and marijuana (E1 = .05, C = .10, p<.01). The second intervention group showed lower rates of combined weekly cigarettes and alcohol (E2 = .09, C = .13, p<.01).
Greenberg (1996) and Greenberg & Kusche (1997) evaluated the impact of The Providing Alternative Thinking Strategies (PATHS) program, a school-based skills training program thataddressed 10 positive youth development constructs, including social, emotional, cognitive, behavioral and moral competencies, bonding, self-efficacy, resiliency, recognition for positive behavior, and prosocial norms.
The PATHS curriculum (Kusche & Greenberg, 1994) included strategies for self-control and coping; self-management of feelings; and interpersonal problem-solving. The three-step self-management process centered on the use of red, yellow and green stoplights as techniques for the child to guide herself in monitoring and managing behavioral and emotional impulses. Program principles presume that recognition, labeling, and understanding of emotional states are necessary for effective behavioral self-control, positive peer relations, and good problem-solving abilities. The self-control unit is a modified version of the stoplight model used in the Yale-New Haven Middle School Social Problem-Solving Program (Weissberg, Caplan & Bennetto, 1988).
The study used a pre-post-follow-up experimental design comparing the intervention vs control groups by normally adjusted vs.high risk/special needs students. For the regular needs subsample, four schools were randomized to either the intervention or control group. For the special needs subsample, 14 special education classrooms from three school districts were randomly assigned to intervention or control condition. Posttesting and two follow-up assessments were conducted over the next four years. At pretest, the participants were in the first and second grades. At posttest they were in the third and fourth grades. The program was conducted with an original sample of 426 six- to eleven-year-olds. The final group of 286 subjects were those who completed all of the individual tests as well as both teacher interviews. This sample was comprised of all ethnic groups: 165 Caucasians, 91 African Americans, 11 Asians, seven Filipinos, seven Native Americans, one Hispanic, and four children of unknown ethnic identity. Of the 286, 130 received the intervention (83 regular education; 47 special education) and 156 were in control classrooms (109 regular education; 47 special education.) The evaluation measured the program over one school year, with classes taught three times a week for 20-30 minutes by the regular teachers. Teachers underwent a three-day training workshop and received weekly consultation and observation from project staff. The original intention of the study was to assess the effects of one versus two years of intervention. However since only about 30% of children received two years of intervention, there was insufficient power to detect differences in those analyses.
The posttest results for the normally adjusted subsample showed significant improvements in the intervention group children's abilities to make important discriminations among internal emotional states (the ability to generate positive and negative feeling words (p<.01), to define five complex feelings (p<.01), and to provide appropriate personal examples of different feelings (p<.001); in children's sense of self-efficacy and beliefs that they can manage their feelings (p<.01); improved reasoning with respect to the feelings of others (p<.01); and how feelings change (p<.001). There were general group improvements in solutions to interpersonal problem-solving, including a higher percentage of prosocial solutions, higher scores for total effectiveness of solutions, less likelihood to show aggressive solutions, and a greater likelihood to show prosocial solutions in second and third choices for solutions (no p values were provided). On cognitive competence measures there were two significant improvements for the general group: non-verbal reasoning on a test of cognitive skills (p<.01), and visual-spatial reasoning (p<.03). Improvements in social competence were found only for the boys in the intervention group compared to the control group boys.
Among the special needs youth at posttest, there were self-reported improvements for the intervention group in the ability to generate positive and negative feeling words (p<.01), to provide appropriate personal examples of different feelings (p<.001), in their efficacy/belief that they can hide, manage, and change their feelings (p<.01), in improved reasoning about others' feelings (p<.01),and how feelings change (p<.001). There was a higher percentage of non-confrontational solutions and a lower percentage of aggressive solutions. Significant changes in social competence were noted, including specific improvements in internalized symptoms (p<.01), frustration tolerance (p<.01), assertiveness (p<.01), peer social skills (p<.01), and on a combined measure of social competence (p<.001). Teacher ratings of these children showed improvements in empathy, the ability to stop and calm down, to resolve peer conflicts, to define feelings, and to identify problems.
The follow-up evaluation of the general group showed continued effects for the intervention on measures of emotional understanding and interpersonal problem-solving skills. At one year, there were lasting effects for greater self-control and prosocial solutions among the intervention group children, plus sustained improvement in the quality of planning on a task of social planning (p<.05), and a test of concentration and focus (p<.05). Lower aggression and passivity levels were also sustained (no p values reported). At the second year, there were significant improvements on measures of externalizing behaviors and social/school functioning (p<.05), and lower rates of student-reported conduct problems (year 2).
At one-year follow-up, the special needs group showed continued effects for the quality of social planning (p<.05), focus and concentration (p<.05), less aggression and passivity, more self-control and prosocial solutions, lower rates of student-reported conduct problems and somatic complaints (p<.05), and lower rates of depressive symptoms (p<.01).
Ellickson, Bell ,and Harrison (1993), Ellickson, Bell ,and McGuigan (1993), and Ellickson and Bell (1990) evaluated the immediate posttest and long-term impact of Project ALERT, a school curriculum that addressed five positive youth development constructs, including social, cognitive and behavioral competencies, self-efficacy, and prosocial norms.
This multi-year, school-based program was provided to students in grades seven to eight and used refusal and resistance skills development strategies (competence, self-efficacy) seeking to motivate young people to resist pro-drug influences (prosocial norms) and to give them the capacity for doing so (competence, self-efficacy).
The evaluation of Project ALERT used an experimental pre-post-follow-up design in which 30 schools were chosen from eight school districts in California and Oregon. These districts represented a range of community environments, racial and ethnic groups, and socioeconomic levels. The schools were blocked by district, and assignment was restricted to a subset balanced among experimental conditions on characteristics such as school test scores, language at home, drug use among the schools' eighth graders, and the ethnic and income composition of school districts. Schools were then randomly assigned to one of three conditions: control, experimental-teacher-led or experimental-teacher-led with assistance from teen leaders. The posttest evaluation was conducted 15 months after baseline after the students had received both the 10-session program in the seventh grade and the three-session booster program in the eighth grade. The baseline sample consisted of 6527 students. The final sample (n= 3852) at the end of the eighth grade intervention represented 60-64% of the original sample. Of the missing students, about one half moved after baseline and one half were absent or failed to supply the relevant information at one or more surveys. Attrition analyses showed the experiment's internal validity was not affected and attrition rates did not differ significantly across experimental conditions. However, external validity was affected due to students omitted from the end of eighth grade survey being more likely to have before-treatment characteristics often cited as risk factors (e.g. family disruption and early drug use). Posttest analyses used three sub-groups based on baseline risk levels, with risk level 1 being students who had tried neither cigarettes or marijuana, risk level 2 students who had tried cigarettes but not marijuana, and risk level 3 were those students who had previously tried marijuana. The demographic composition of the final sample was 49% male, 71% White, 9% Hispanic, 8% African-American, 10% Asian, and 3% Native American and another ethnic identity. Twenty-five percent of the sample had a C average or lower grades, 20% had a father who wasn't a high school graduate, 33% were not living with both birth parents, 48% had prior use of cigarettes, 75% had prior use of alcohol and 14% had prior use of marijuana. Nine of the schools had minority populations of at least 50% and 18 drew from neighborhoods with household incomes below their state median.
The program showed short-term positive effects through the end of eighth grade on behavioral measures and measures of cognitive risk factors associated with substance use. Sub-group analyses based on baseline risk level showed that cigarette use reduced significantly for baseline experimenters in measures of past month use (E2 = 23.6, C=32.3, p<.01), in monthly use (E1=16.5, C=22.4, p<.05), in weekly (E1=5.7, C=11.1, p<.01) and daily (E1=2.3, C=5.1, p<.05) use. For smoking marijuana, there were significant improvements in prevalence estimates (29% vs 36%, p<.01), expectations of using, and perceived consequences. The impact on marijuana beliefs was slightly more effective among those who had not tried either marijuana or cigarettes (risk level 1) than among those who had tried cigarettes but not marijuana (risk level 2), particularly for students in the teen leader group; this latter effect, however, diminished over time. For cigarette and marijuana knowledge and attitudes, changes were particularly positive for baseline non-users of cigarettes and marijuana. The program also had a significant impact on each item making up the social consequences scale. Program effects were smaller for alcohol; effects on youth beliefs were limited to positive changes in youth prevalence estimates at 15 months (p<.01), and the perceived ability to resist an offer of alcohol on a date (p<.05). Expectations of future use changed more for marijuana than for the other target substances. Fewer students in both intervention conditions anticipated using marijuana in the future, but the program had no impact on cigarette and alcohol expectations during the second year. Compared to the control group, intervention students were more likely to believe that using cigarettes and marijuana can bring immediate and negative social consequences and can result in becoming dependent on the substance. The program reduced the intervention group estimates of how many students use each target drug and helped those in the teen-leader schools believe that resisting cigarettes and marijuana can bring greater respect from one's friends.
Ellickson and Bell (1990) reported negative outcomes for baseline smokers at the end of the intervention. Characterizing these as "boomerang effects," they noted these negative results were stronger in the teen leader schools (where many of the other robust positive effects were measured). By 12 months, baseline users had increased smoking by 20% in these schools (p<.05), which grew to 30% after exposure to the booster program (p=.004). The authors noted that this boomerang effect for baseline smokers has been found in other anti-smoking programs (e.g., Biglan et al, 1987). They offered this interpretation (p.1304): "The results...suggest that early cigarette smokers need a more aggressive program than that offered by the social influence model alone... For these more confirmed smokers, being told that most of their peers do not smoke and exposing them to nonsmoking teens appears to be irrelevant at best and counterproductive at worst."
Results of long-term follow-up measured at grades 10 and 12 showed that all intervention effects diminished by the end of high school. Once the lessons stopped, the program's effects on drug use stopped. Significant effects on cognitive risk factors (belief that drug use has negative personal consequences and that resistance brings respect from one's friends) persisted for a longer time (many through grade 10) in the teen leader schools (p<.01), but did not produce corresponding reductions in use. By the end of high school, the intervention no longer had a significant effect on behavior. Its earlier positive impact on cigarette and marijuana use had completely disappeared as had the negative boomerang effect for baseline cigarette users.
Eight programs combined two social domains or components. Seven effective youth development programs were conducted in combined family and school domains: the Child Development Project, Fast Track, Metropolitan Area Child Study, Reducing the Risk, the Seattle Social Development Project, the Social Competence Program for Young Adolescents, and Success for All. These programs successfully changed youth outcomes, promoted positive youth development constructs and strategies, and incorporated parent or family involvement. One program, Teen Outreach, combined school and community domains.
The analysis of school- and family-based interventions identified seven effective youth development programs that were well evaluated and showed significant effects on youth behavior. These primarily school-based programs successfully changed youth outcomes, promoted positive youth development constructs and strategies in the school setting, and used a variety of methods to incorporate parent or family involvement. Five programs were multi-year interventions that used strong quasi-experimental research designs; two used an experimental design (Fast Track and Metropolitan Area Child Study).
Positive youth development programs set in school and family domains typically promote competence within, and bonding to, the family while promoting these positive youth development constructs in the youth. The school is usually the primary setting for implementing youth strategies while a combination of approaches are employed to engage the family. These include direct parent training or education strategies, often conducted at the school; program implementation strategies in the home setting to enhance the child's acquisition of new skills and learning (e.g., parent involvement in homework assignments generated from the school-based component, or home visits); or parental participation in the program design or organizational strategies. Metropolitan Area Child Study and Seattle Social Development Project used the first strategy, direct parent training. Child Development Project, Reducing the Risk, and Social Competence Program for Young Adolescents used the second parent strategy, bringing parents into the implementation of the program. Fast Track used the first two, training and home visits. Success For All used all three, training parents, and involving them in both the implementation and organizational aspects of the intervention. These school-family programs all used methods to assure the quality of program implementation, and nearly all programs measured implementation. Those measuring implementation included Child Development Project, Fast Track, Reducing the Risk, Seattle Social Development Project, Social Competence Promotion Program for Young Adolescents, and Success for All.
Positive youth development programs set in schools and families generally tried to introduce protective factors into both settings. While the children are learning skills or other youth development strategies, parents are frequently the focus of efforts to foster family competence, parent self-efficacy, bonding between child and family and between parent and intervention, and to promote prosocial norms in the family. Only a few of the interventions systematically measured parent changes or outcomes, e.g., Fast Track measured parenting competencies.
Each program produced evidence of significant changes in youth's positive or problem behavior. Improvements in positive youth outcomes included greater social acceptance by and collaboration with peers (Child Development Project; Fast Track); improved communication with parents and greater self-efficacy around contraceptive practices (Reducing the Risk); higher achievement and school attachment (Seattle Social Development Project); increased social acceptance by, involvement and cooperation with peers, problem-solving and creative solutions (Social Competence Promotion Program); improved cognitive competence and academic mastery (Success for All); and improvements in acceptance of authority, classroom atmosphere and focus, and appropriate expression of feelings (Fast Track). These interventions also had a significant impact on the reduction or prevention of problem behaviors in children, including alcohol (Child Development Project and Seattle Social Development Project) and tobacco (Child Development Project) use. Rates or frequency of delinquency or aggressive behavior decreased in four programs (Fast Track, Metropolitan Area Child Study, Seattle Social Development Project, and Social Competence Promotion Program). Youth attitudes and practices around contraception or initiation or prevalence of sexual activity were reduced in two programs (Reducing the Risk and Seattle Social Development Project).
Battistich, Schaps, Watson and Solomon (1996) evaluated the Child Development Project, an intervention that addressed 13 positive youth development constructs, including social, emotional, cognitive, behavioral, and moral competencies, bonding, resiliency, self-efficacy, recognition for positive behavior, positive identity, opportunities for prosocial involvement, prosocial norms, and self-determination.
The Child Development Project is characterized by a specific positive youth development goal, to help schools become caring communities of learners. The intervention was designed to become part of the children's overall school experience. The theoretical foundation of the program was based on social learning theory, learning and motivation theory, prosocial development theory, and bonding and attachment theory. The program curricula were integrated with teacher training and new practices to reduce exposure to "old practices." Parents were targeted to build stronger connections between home and school (bonding). The two primary strategies included bringing parents into community-building projects with their children and teachers, and "homeside" activities (bonding, competence). The core program used a comprehensive approach to instructional practices. These consisted of classroom practices such as cooperative learning (opportunities, bonding, competence, self-efficacy, self-determination), a values-based reading and language arts program (prosocial norms, competence), and developmental discipline techniques (self-determination, competence, opportunities). Another component included classroom and whole school elements such as classroom and school community-building activities (bonding, prosocial norms, opportunities).
The study used a quasi-experimental research design, specifically a cohort sequential method involving two demonstration and two comparison schools in each district, at a total of 24 elementary schools from six school districts around the United States. Demonstration schools were selected based on the likelihood they would implement the program; comparison schools were then matched on characteristics of the student populations, and willingness of the school administration to participate. Beginning with baseline assessments in spring 1992, successive cohorts of upper grade students were assessed annually in the spring of each year using group-administered questionnaires. Two sets of analyses examined program effects on student drug use and delinquency. The first set examined condition differences and the second set examined these results in the context of program implementation levels. In the second set of analyses, the 12 demonstration schools were classified into three groups of four schools each -- high, moderate or low implementation -- based on changes from baseline in six observational measures of program practices. School districts included three large urban districts, one small urban district, one suburban district, and one rural district. The authors described the student populations at these schools as varying greatly. The range of students receiving free or reduced school lunch was from 2% to 95%; the sample's ethnic composition over three years of measurement ranged from 39-54% for Caucasian children, 17-23% for African-Americans, 21-27% for Hispanics, 5-10% for Asians, and 2-3% "other" youth (defined as Native American, Alaskan Native, Pacific Islander, and "other"). Between zero and 32% of participants spoke limited-English or no English, and average achievement levels ranged from the 24th to the 67th percentile on standardized tests. The overall sample (n = 1645) included slightly fewer boys (48%) than girls (52%). The overall program included students from the third through the sixth grades; assessments for this evaluation included the fifth and sixth grade cohorts, with average age 11 to 12 years old.
Assessments of drug use and delinquent behaviors were limited to students at the top elementary grade (fifth or sixth) at each of the schools. The student questionnaire results showed that alcohol use significantly declined (4%) at the demonstration schools during the first year of the intervention and stayed the same during the second year (F(2, 4510) = 4.19, p <.02). In contrast, alcohol use increased significantly at the comparison schools during the same period (2-3% increases each year; t(4715) = 2.49, p<.02, ES= .12). Changes in the prevalence of marijuana use and vehicle theft showed similar trends but were not statistically reliable. Tobacco use declined among students at both demonstration and comparison schools over the course of study (F(2,4524) = 11.52, p<.001). The authors also presented results of an extensive implementation analysis of the 12 demonstration schools, which were classified into low/medium/high implementation based on changes from baseline in six observational measures of program practices: teacher warmth and supportiveness, emphasis on prosocial values, encouragement of cooperation, elicitation of student thinking and expression of ideas, use of extrinsic control, and student autonomy and influence. Differences in youth outcomes associated with implementation level were discussed at length, and conclusions about effectiveness were linked with the high implementation schools. However, most of these could not be determined to be statistically significant due to, among other things, reduced sample sizes. There were, however, significant decreases for program students in high implementation schools compared with controls in marijuana use (F(2, 1368) = 8.56, p <.001), carrying weapons (F(2, 1375) = 7.28, p <.002), and vehicle theft (F(2,1368) = 7.11, p <.002). In addition to these outcomes for this large sample, another evaluation (Battistich et al., 1989) of an earlier, smaller (n=521) sample of Child Development Project participants reported that program children were significantly more accepting of, and accepted by, their classmates, were less lonely, and were lower in social anxiety than children in comparison schools.
Greenberg (1998) and The Conduct Problems Prevention Research Group (1997) evaluated the Fast Track Prevention Project, a multi-site, multi-year intervention that addressed 10 positive youth development constructs, including social, emotional, cognitive, and behavioral competence, self-efficacy, resiliency, bonding, recognition for positive behavior, opportunities for prosocial involvement, and prosocial norms.
The comprehensive intervention is guided by developmental theory on youth competence and the multiple influences through which antisocial behavior manifests. Fast Track is based on a unified model of prevention that consists of both universal and selective components, with some of the selective components individualized to the needs of the child and the family. The universal component is the PATHS teacher-led classroom curriculum. The selective intervention components were administered only to the high-risk sample, and included PATHS, parent training groups, home visits, child social skill training groups, child tutoring in reading, and peer pairing in the classroom. The Fast Track curriculum in first grade had 57 lessons of the main curriculum, taught approximately three times a week with each lesson lasting 20-30 minutes. Forty percent of the lessons focus on skills for understanding and communicating emotions, thirty percent focus on skills for increasing positive social behavior; and thirty percent focus on skills for self-control and social problem-solving. Parent and child training groups were weekly in grade one, biweekly in grade two, and monthly thereafter for the rest of the project. Teacher training included a two and a half day workshop and weekly consultation and observation with project staff.
Approximately 13 schools within each of four sites (Durham, North Carolina; Nashville, Tennessee; Seattle, Washington; and rural central Pennsylvania) were selected based on community risk factors and invited to participate in the intervention. Schools were randomized into intervention and control participants (after obtaining building consensus to participate) and the intervention was conducted in three successive years with three cohorts of first graders. There were 201 classrooms and 184 matched comparisons across the three cohorts. For the high-risk sample, a multi-stage screening of all kindergarten children based on teacher and parent ratings yielded the top 10% of high-risk children. They were then contacted for pre-intervention assessments and upon completion of these, included in the total sample with the condition based on the school they entered for the first grade. This resulted in three successive yearly cohorts for a total of 448 intervention and 450 control group children. The high-risk sample was approximately 47% Caucasian, 51% minority students, generally African-American, and two-thirds male. Fifty-five percent of children received free or reduced lunch and the mean reading percentile was in the 49th percentile. No significant differences existed between intervention and control schools on any of these characteristics.
The evaluations reported findings for both the universal and high risk samples. The full study results were reported through teacher, peer, and independent observer ratings. Results for the high risk sample were reported through teacher and parent measures. In the full study, the teacher ratings showed outcomes in favor of the intervention group on measures of accepting authority (F(1,333) = 3.6, p<.04), and a higher mean liking score (F(1,333) = 3.8, p<.05). Peer ratings showed that intervention classrooms had lower peer aggression scores (F(1,377) = 5.42, p=.02), and less hyperactivity (F(1,377) = 4.66, p=.03) than controls. Observer ratings of positive classroom atmosphere were significantly higher for the intervention classrooms (F(1,279) = 5.6, p=.01). Sub-ratings of these scales showed improvements for students' appropriate expression of feelings (F(1,279)=4.3, p<.05), classroom enthusiasm (F(1,279) = 8.2, p<.01), and the classroom's ability to stay focused and on task (F(1,279) = 9.5, p<.005).
Analyses of the high risk sample using growth curve models showed the impact of the intervention on two conduct problem measures provided by parent daily report and teacher observations of classroom adaptation. Outcomes showed that the control group increased in disruptive behavior at school compared with intervention children, while intervention group children showed greater reductions in conduct problems at home compared with controls. Although the intervention and control groups started at the same level of disruptiveness, the control group increased on this dimension while the intervention group stayed the same or declined slightly. Parent data showed similar patterns of greater decline in conduct problems at home for the intervention group youth.
Analyses of nine mediating variables measuring the mediating role of child and parent competencies (six child and three parent competency measures, tested as a set) were conducted using the average of the end of second and third grade assessments. Investigators used structural models linking the intervention condition to each youth behavior measure and found significant results for the mediating variables: improvements in both child and parental competency factors did in fact mediate the decreases in child conduct problems. The results of the growth curve data underscore an important aspect of positive youth development, namely that intervention effects are gradual and cumulative and operate along a developmental (time-based) trajectory.
The analyses examined the effects of dosage and used four ratings of the quality of implementation: teacher's rated skill in teaching PATHS concepts, skill with which teachers modeled and generalized PATHS concepts throughout the day, the quality of teacher classroom management skills, and teacher's openness to consultation with the educational coordinator. For dosage there was a significant main effect on the sociometric rating of prosocial behavior. The teachers' rated skill in teaching and modeling PATHS concepts, the skill with which teachers modeled and generalized PATHS concepts, and classroom management skills were all significantly related to scores for authority acceptance, cognitive concentration, and social competence. Classroom management skill was also related to the sociometric score on prosociality. Openness to consultation was significantly related to cognitive concentration, social competence, prosociality, and to hyperactivity.
The overall results from this intervention suggest some interesting questions for future studies of positive youth development. As the authors (Conduct Problems Prevention Research Group,1997:13) said: "The interpretation of these findings is a bit complicated by the fact that the universal intervention was delivered along with the selective intervention for children with early behavior problems. Although analysis with and without the high-risk children showed similar patterns, the classroom scores may not be entirely free of the impact that the selective intervention may have on the high-risk children. Improvements in the high-risk children may reduce teacher stress, improve peer relations and thus also possibly affect outcomes."
Eron, Guerra, Henry, Huesmann, Tolan and Van Acker (1997) evaluated the impact of Metropolitan Area Child Study, a multi-year prevention field trial that addressed 10 positive youth development constructs, including social, emotional, cognitive, behavioral and moral competencies, bonding, resiliency, self-efficacy, opportunities for prosocial involvement, and prosocial norms.
This intervention is a combined universal and selective prevention program designed to influence norms affecting aggression in a general population, and to lower aggressive behaviors in a high-risk sample. The intervention goal was to modify the youth's cognitive system while influencing aspects of her social systems (the classroom, the school and the family) shown to be key in the learning of aggressive behaviors. The intervention used different combinations of positive youth development strategies across conditions. In the first condition, the classroom intervention integrated three components including a teacher education program, collaborative support strategies from project staff, and a social-cognitive curriculum called "Yes I Can." The educator strategies included the development of prosocial behavior in students (prosocial norms, recognition), cultural sensitivity (competence, bonding), and proactive behavior management (self-efficacy, competence, resiliency). The curriculum strategies consisted of 40 one-hour lessons taught by teachers over two years, and included teacher manuals and student workbooks. Program themes included self-understanding (self-efficacy, resiliency), relationship of self to others (bonding), and moral beliefs (competence, prosocial norms). The second condition was a classroom-based general enhancement intervention, and an intensive small-group component in which youth received social-cognitive and peer-relationship training in a small group. These strategies targeted children's cognitive attributions, beliefs and scripts, and gave them modeling, role-play and leadership opportunities to practice new scripts in a non-threatening setting. Small groups met during the regular school day once a week for 28 weeks over two years (12 weeks in first year and 16 weeks in the second year). The third condition included the classroom program, the small group intervention, and a family intervention. The family relationship component emphasized parenting management skills, enhanced communication (bonding, competence), and family cohesiveness strategies including family problem-solving, monitoring, reframing and coping skills (competence, bonding, prosocial norms, opportunities).
The intervention used an experimental design. Two urban communities (Chicago and Aurora) with many high-risk children were selected within a larger metropolitan area. Schools were recruited through an extensive screening process, and 16 elementary schools participated (from an original group of 130 that applied). These were randomly assigned to each of four experimental conditions including three intervention levels and a control condition, as described. Four schools received each of the three interventions and there were four control schools that received none. Schools were blocked on the basis of their ethnic composition, geographic location and aggressiveness rating level of the children. The four blocks were Chicago Hispanic, Chicago African American, Aurora Hispanic, and Aurora Mixed. One block of four schools was randomly assigned to each condition. Two schools (one in Chicago, one in Aurora) were replaced because of administrative changes before the first year, and one school was replaced after the third year of the intervention due to administrative issues. The attrition analysis showed that from baseline to posttest, attrition was about 44% (range of 39% in Level C to 50% in Level A), yielding a final sample of 789 children. Initial aggression scores indicated that aggression was significantly higher for the group that dropped-out. The overall sample was studied in three two year waves: 1991-1993, 1992-1994, and 1993-1995. This included an early intervention consisting of four cohorts with second-third or third-fourth grade students, and a late intervention consisting of two cohorts of fifth-sixth grade students. The sample was 40% African-American, 40% Hispanic and 20% Caucasian. Two-thirds of the children qualified for a free lunch program, with significant variance across ethnic groups (African Americans were the mostly likely to qualify, Caucasians were the least). Low socioeconomic status was significantly correlated with aggression levels at baseline. The baseline aggression scores of the study sample were high compared with national averages. There were no significant differences in ethnic composition, SES, levels of aggression, and beliefs about aggression across the four conditions.
Published results for the Metropolitan Area Child Study showed mixed outcomes. Pretest aggression scores were used as a covariate. Children who were already moderately prosocial and received the small group intervention were found to have significantly increased prosocial behavior at posttest (F(3,361) = 4.04, p<.01). Prosocial behavior was found to have a significant moderating effect on posttest aggression scores for the early intervention condition. In the early intervention condition, control group participants who were low in prosocial behavior and in aggression at pretest were found to have high levels of aggression at posttest; this was not true for intervention children, which the authors suggested meant the program prevented the development of high levels of aggression in this sub-group of children. Late intervention (grades 5-6) children who were high in pretest aggression had significantly lower posttest aggression scores than control children (F(3,257) = 4.8, p<.003). However, the late intervention children who scored as moderately aggressive at pretest scored worse on the posttest than comparable control group children.
In addition to these published evaluation data, more recent unpublished evidence showed significant effects for decreased aggression for the full sample, delivered early (second and third grade). Results showed the early intervention improved the program group's on- task behavior and stopped deterioration of prosocial behavior. Both the early and late versions prevented normal deterioration in academic achievement, compared with control youth. These effect sizes ranged from .15 to .33 (Tolan et al.,1998).
Kirby, Barth, Leland and Fetro (1991) evaluated Reducing the Risk, an intervention that addressed nine positive youth development constructs, including social, emotional, cognitive, behavioral and moral competencies, self-efficacy, opportunities for prosocial involvement, prosocial norms, and self-determination.
The program strategies combined cognitive behavioral skills training (competence, self-efficacy), a social inoculation focus (self-efficacy, self-determination), teacher and peer role modeling (opportunities, prosocial norms), and organizational strategies designed to influence students' motivation and peer group variables associated with delaying sexual activities and effective contraception. The curriculum structured parental involvement in children's learning and skills acquisition through multiple homework discussion strategies. The program exposure was 15 class periods and some parent-child discussion of an unspecified amount, with teachers attending a three-day training session. Implementation was assessed through observations and ratings by the evaluation team, and student and teacher perceptions of the curriculum usefulness. The evaluation collected data from pretest to posttest and over an 18-month follow-up period.
The evaluation used a quasi-experimental design with partial random assignment and comparison groups. Forty-six classes of students taking a mandatory health education class were assigned to either treatment or comparison groups. Some of the classes were randomly assigned, and others were not. Approximately half of the program and control classes that were taught by the same teachers; the other half were divided as program classes taught by one group of sexuality educators and comparison classes taught by a different group. Thirteen urban and rural schools in California participated with a total of 1033 students in 23 classes (586 in the intervention group; 447 in the comparison group) surveyed at pretest, and 758 completing both the posttest and follow-up questionnaires. The authors (Kirby et al., 1991: 256) note: "Because most comparison classes were receiving whatever sexuality instruction those teachers were already providing, this evaluation measures the impact of the RTR curriculum relative to other sexuality curricula, not the overall impact of RTR alone." The sample consisted of 53% female youth and 47% male youth, of whom 62% were Caucasian, 20% were Latino, 9% were Asian, 2% were Native American, 2% were African American, and 5% were of unspecified ethnic identity. The attrition analysis showed a 27% rate for both the intervention and comparison groups, and those lost to attrition differed in several ways from the remaining participants. At pretest they were more likely to be older, male, sexually experienced, not live with both parents, have poor grades, not have talked to their parents about abstinence, and failed to use birth control because sex was unplanned.
The initial study results were based on posttest outcome measures at six months of contraceptive knowledge, perception of peer sexual prevalence, sex initiation, and unprotected sex. These indicators were assessed by the students, teachers and parents. At posttest measurement, the intervention group had significantly greater contraceptive knowledge than the control group (76% vs 65%, p<.001). There were greater increases among the intervention students in communicating with their parents about abstinence (66% vs 61%, p<.01) and birth control (53% vs 41%, p<.01). The comparison group believed at posttest that more of their peers were having sex than at pretest (3.1% vs 3.3%, p<.01), while there was no change in this belief for the intervention group.
Results for the long-term follow-up showed sustained behavioral effects for the intervention. Significant results were found in lower rates of initiation of intercourse among intervention participants who had never had intercourse (29% vs 38%, p<.05). Measures of contraceptive practices showed no significant overall differences between the two groups for increases in contraceptive use from pretest to posttest, but there were significant differences in some sub-groups. Only 4% of females and 3% of low-risk teens in the program group had unprotected intercourse, compared with 16% of females and 11% of low-risk youth in the comparison group (p<.001). Among all lower risk youth independent of previous sexual experience, youth in the program group were significantly less likely to have had unprotected sex at last intercourse, and less likely to have had unprotected sex all or most of the time. At pretest, 11% of both intervention and comparison groups had engaged in unprotected sex; at 18 months only 13% of the intervention group were not using contraception, compared to 23% of the comparison group (p<.05). Other continued effects seen at 18 months for the students in the program group included greater levels of contraceptive knowledge (p<.001) and communication with parents about abstinence (p<.05).
Hawkins, Catalano, Kosterman, Abbott, and Hill (1999) evaluated the Seattle Social Development Project, a multi-year interventionthat addressed nine positive youth development constructs including social, emotional, cognitive and behavioral competencies, bonding, self-efficacy, opportunities for prosocial involvement, recognition for positive behavior, and prosocial norms.
The intervention sought to promote academic achievement and other competencies, and to prevent a range of negative youth outcomes including crime, teen pregnancy, drug abuse, school failure and dropout, by to increasing prosocial bonding, which is hypothesized to be a mediating construct of the desired outcomes. Components included teacher training in methods of classroom instruction and management, parent training in developmentally appropriate methods of parenting, and social competence training for youth. In the first and second grades, parents in the full intervention condition were offered the seven-session "Catch 'Em Being Good" curriculum for child behavior management skills. In the spring of second grade and again in third grade, these parents were offered the four session" How to Help Your Child Succeed in School" curriculum. In grades five and six, full and late intervention parents were offered the five-session Preparing for the Drug Free Years curriculum. In grade six, students in the full and late treatment groups received four hours of training from project staff in recognizing and resisting social influences to engage in problem behaviors and generating positive alternatives for staying out of trouble. Teachers in intervention classrooms received five days of inservice training per year on proactive classroom management, interactive teaching, and cooperative learning methods to use in managing and teaching their classes.
The study used a quasi-experimental design with comparison groups. The population overrepresented schools located in high-crime, high-poverty neighborhoods, but youth within classrooms in these schools were treated universally. To assess the effects of full intervention and late intervention, a nonrandomized controlled trial with three conditions was created. The full intervention group received the intervention package from grade one to six. The late intervention group received the intervention package in grades five and six only, and the control group received no special intervention. This design was created in 1985 by nesting an intervention study begun in 1981 at first grade entry within the longitudinal panel study. For the present study, schools were assigned to conditions in the fall of 1985 and from that point, all fifth grade students in each school participated in the same interventions. The full intervention group consisted of all students who were randomly assigned to intervention classrooms in grades one through four in eight elementary schools participating in the earlier experimental study, and who remained in schools assigned to the intervention condition in grades five and/or six in the present study. The late intervention group consisted of students in intervention schools who were in intervention classrooms in grades five and six only, some of whom were controls in the earlier intervention study. The control condition consisted of students in schools assigned to receive no intervention in grades five and six and who were not in intervention classrooms in grades one through four. The final sample included 643 fifth grade students assigned to three conditions whose parents consented to their involvement in the follow-up study, which was conducted six years after intervention at age 18. Of these 643 students, 598 were successfully tracked and interviewed (93%). The demographic and ethnographic composition among three experimental conditions ranged from 47-53% male, 42-46% Caucasian, 21-30% African-American, 18-24% Asian, 3-7% Native American, and 1-5% of other ethnic identity. Participants' eligibility for free lunch averaged 56%. At age 18 the three study groups did not differ significantly on residential stability, socioeconomic status, or proportions who were male, Caucasian, non-Caucasian, or from single-parent households. Implementation quality was assured by teachers in both sets of classrooms being observed for 50 minutes on two different days in the fall and spring of each year, and given scores rating fidelity to intervention practices. Data were also collected using questionnaires for youth, parents and teachers; data on delinquency charges in King County Juvenile Court, California Achievement Test results, grade point averages, and school disciplinary action reports through age 17 from the Seattle School District.
The findings reported when the students were 18 (six years post intervention) showed positive effects for the full treatment intervention group compared to the control group on many of the school, delinquency and sexual behavior outcomes. Students in the full intervention reported significantly stronger attachment to school (p=.006), improvement in self-reported achievement (p=.01) and significantly less involvement in school misbehavior (p=.02) than controls. While no effects were shown for either the full or late intervention groups for lifetime prevalence of cigarettes, alcohol, marijuana or other illicit drug use at age 18, significantly more subjects in the control group than in the full intervention group had committed violent acts (48.3% vs 59.7%, p=.04), reported heavy alcohol use in the past year (15.4% vs 25.6%, p=.04), had engaged in sexual intercourse (72.1% vs 83%, p=.02), or had multiple sex partners (49.7% vs 61.5%, p=.04).
Analysis of the interactions between poverty and intervention condition showed that the full intervention was significantly more effective for poorer children in positively affecting attachment to school and in reducing having to repeat a grade. The intervention had significantly greater effects for working and middle class youths in reducing the lifetime prevalence of pregnancy or of having or fathering a baby. Most observed intervention effects did not differ by gender; however, the full intervention had significantly greater effects on preventing males from repeating a grade and engaging in sexual activity.
Weissberg and Caplan (1998), and Caplan, Weissberg, Grober, Sivo, Grady and Jacoby (1992) evaluated the Social Competence Program for Young Adolescents, which addressed nine positive youth development constructs including social, emotional, cognitive and behavioral competencies, bonding, prosocial norms, self-efficacy, recognition for positive behavior, and resiliency.
The program's six units were designed to enhance personal and interpersonal effectiveness (social-emotional competence and bonding) and to prevent the development of maladaptive behavior. The intervention did this through teaching and reinforcing students' developmentally appropriate skills and information (cognitive and behavioral competence, self-efficacy, recognition), fostering prosocial and health-enhancing values and beliefs (prosocial norms) and creating environmental supports (all forms of competence, bonding, and resiliency). The 12-week intervention was a 16-session competence promotion program that used highly structured, scripted 45-minute lesson plans in conjunction with applications to promote competence and prevent substance abuse.
Implementation strategies included training for teachers and undergraduate aides in 10 90-minute workshops before program implementation, and continuing throughout the 16-week intervention. They also received on-site consultation and coaching during classroom lessons, and following the 16-week program received three monthly two-hour training to focus on ways to help maintain SCP skills by students. Two observers independently rated the quality of program implementation in each classroom.
The evaluation used a quasi-experimental research design with comparison groups. The program was described to teachers at four middle schools who were then given the opportunity to voluntarily participate. Teachers who volunteered had their classrooms assigned to a control or program condition based on scheduling and comparability of academic-ability grouping levels across conditions. Separate analyses were conducted to compare the outcomes associated with high versus low implementation fidelity. Tests of the groups' equivalence at baseline indicated they were comparable to one another on all but eight (five favoring the intervention group, three favoring the control group) of 28 variables at pretest. The sample consisted initially of 447 fifth to eighth grade students from 11 program (n=238) and nine control (n=183) classes in four urban, multi-ethnic schools. The final sample (n=421) of 210 male and 211 female students from low- and middle-income families included 178 Caucasian students, 167 African-American, 72 Hispanic, and four students of other, unidentified ethnic identity. The authors provided analyses of group differences showing that control and program groups did not differ with respect to grade level, gender or race and that students had comparable academic achievement levels. However, the groups did differ on 8 of 28 variables at pretest. The attrition analysis showed high retention levels across groups, with the study losing only 26 students (5.8% of the sample) whose parents withheld permission.
Teacher observations were made by both primary and secondary teachers (who were hypothesized to be blinded to treatment condition) to help deal with the potential of inherent teacher bias. After the main program impact analyses, a related set of analyses were performed to assess whether the quality of program implementation influenced student outcomes by comparing pre-to-post changes for students from well-implemented program classes, less well-implemented program classes, and the control group. Three assessment instruments yielded 16 measures of students' problem solving skills, conflict resolution attitudes, and self-reported assertiveness in response to hypothetical situations.
The evaluation found significant positive effects on minor delinquency (36.8% vs 2.8%, (F(1,399) = 4.37, p<.05) and increases in or maintenance of positive behaviors including improved behavioral conduct (F(1,399) = 8.57, p<.01), involvement with peers (F = 6.02, p<.05), and social-acceptance ( F = 4.17, p<.05). Significant changes were noted for the intervention group for social problem-solving (F(9,168) = 4.92, p<.001), and improvements in attitudes for conflict resolution strategies (F(6,171) = 2.97, p<.01). Program participants improved more than controls in generating a greater number of cooperative solutions (F(2,192) = 16.14, p<.01), more effective solutions (F(2,192) = 5.52, p<.01) and more planful solutions (F(2,192) = 8.08, p<.001). Program students generated significantly lower percentages of aggressive (F = 4.94, p<.05) and passive (F = 11.29, p<.01) solutions from pre to posttest when compared with control group students. Program students liked peers who resolved conflicts in assertive ways significantly more than control students (F(1,176) = 4.07, p<.05); endorsed cooperative strategies more than control students (F(1,175) = 4.24, p<.05); and were shown to respond more assertively rather than passively or aggressively to hypothetical situations (F(1,311) = 7.06, p<.01). Primary teacher ratings showed that relative to controls, program students improved significantly in behavioral conduct (F = 17.54, p<.001) but not in social acceptance by peers.
Independent of implementation quality, program students showed significant gains relative to controls in solution effectiveness, number of planful solutions and percentage of cooperative solutions, and decreased more in the percentage of passive solutions. Students in well-implemented program classes improved significantly more than those in lower-quality implementation classes and control classes on the number of alternative solutions generated to problem situations (F(2,192) = 13.01, p<.001), the extent of endorsing assertive (F(2,193) = 3.58, p<.05) and cooperative (F(2,193) = 4.09, p<.05) conflict-resolution strategies, and on secondary teacher ratings of behavioral conduct (F(2,366) = 8.95, p<.001) and social acceptance (F(2,366) = 15.88, p<.001).
Slavin et al. (1996) evaluated the impact of Success for All, an intervention that addressed eight positive youth development constructs, including social, cognitive, and behavioral competencies, bonding, self-efficacy, opportunities for prosocial involvement, recognition for positive behavior, and prosocial norms.
The program has been described elsewhere as based on reading achievement, but its focus on academic success is grounded in a more comprehensive philosophy of what it takes to promote positive youth development in children. The program authors refer to their concern with a focus on the success of every child, and a philosophy of "neverstreaming," i.e., not separating special needs students from the mainstream. The program formula links academic success and cognitive competence to developing a child's overall capacity for self-respect and self-efficacy, concluding that in their absence, the ingredients necessary for successful positive youth development disappear: "When a child fails to read well in the early grades, he or she begins a downward progression" (Slavin et al., 1996: 42). The program addressed cognitive competence in several ways, through reading performance achievement, and through rehearsing strategies for self-assessment and self-correction. Other positive youth development strategies included one-to-one tutoring, cooperative learning, assessments every eight weeks, vision/hearing screenings, working with parents and social service agencies to ensure attendance, parenting skills workshops, providing medical services and eyeglasses, and helping with behavioral problems. This six-year intervention targeted Grades K-5 in a sample of 110 schools. At the time of evaluation the program was being implemented in 300 schools in 23 states.
The quasi-experimental research design was a 'multi-site replicated experiment' design in which groups were matched rather than randomly assigned, and each grade level cohort was specified as the unit of analysis. For specific site studies the unit of analysis was the individual student; however, the comprehensive evaluation reported here chose the multi-site replicated experiment method of analysis which specified each grade level cohort as the unit of analysis. Each intervention school was matched with a comparison school based upon variables including poverty-level percentage of students qualifying for a free lunch, achievement level, and ethnicity of the student body. Individual children in program schools were matched with children in non-program schools based on either scores on district-administered standardized tests or scores on the Peabody Picture Vocabulary Test. The overall findings for 23 evaluated schools constituted 55 experimental and 55 control cohorts, with each cohort made up of 50-150 students. The sample of almost all African-American students in first grade were 75-96% eligible for a free lunch program.
The results showed significant differences in favor of the intervention group for reading scores at all grade levels. The Success For All program yielded statistically significant positive effects on every outcome measure at every grade level (p<.001), which averaged around half a standard deviation per grade level. Students in the lowest 25% of their grades showed effect sizes averaging one standard deviation. These effect sizes progressively increased with each year of implementation of the program. The intervention also showed significant positive gains for Asian (mostly Cambodian) students learning English as well as Spanish speaking students who were provided with a Spanish version of the Success for All curriculum. The overall comparison of Success for All to Reading Recovery, a similar reading program based upon one-to-one tutoring, showed only slightly better results for Success for All when looking at results for all children. However, an analysis of only special education children showed substantial differences in favor of Success for All students (effect size = .77).
The analysis found only one well-evaluated, effective youth development program, Teen Outreach, that was conducted in combined school and community domains. This primarily school-based intervention promoted positive youth development constructs and strategies in the school setting by providing community service opportunities for young people, and produced positive behavioral outcomes on school performance and reduced teen pregnancy.
The analysis showed that positive youth development programs using a community domain component can be described as organizing or applying their strategies in one of three ways: through being an implementation resource or site, in which for example, children perform community service or volunteering, or meet for group intervention sessions at a community center; through program designers addressing specific community risk or protective factors; or through direct involvement of the community, as in community organization and mobilization strategies that target changes in community-level policies and practices. This program is an example of the first type of community domain application. The school was the primary setting, while community resources and opportunities were used to support outcomes.
Allen, Philliber, Herrling, and Kuperminc (1996) evaluated the Teen Outreach Program, which addressed twelve positive youth development constructs including social, emotional, cognitive, behavioral and moral competencies, bonding, self-efficacy, opportunities for prosocial involvement, prosocial norms, positive identity, belief in the future, and self-determination.
Although the focus of program evaluation and measurement was on preventing teen pregnancy and academic failure, the theoretical focus was a developmental approach based on Helper Theory (defined as "empowerment") and Social Development Theory (defined as "promotion of autonomy and identification with others"). The evaluation predicted that students would be empowered by having the opportunity to be help-givers, rather than help-receivers. The program involved adolescents in volunteer activities; classroom discussions centering on various age-appropriate issues and service experiences (understanding yourself and your values, life skills, dealing with family stress, human growth and development, and issues related to social and emotional transitions from adolescence to adulthood); and class activities promoting program goals (group exercises, role plays, guest speakers and informational presentations). The volunteer component included such activities as working in hospitals, peer tutoring, and participation in walkathons. Students were required to provide a minimum of 20 hours per year of volunteer experience, but averaged 45.8 hours over the course of the program, with the median participant performing 35 hours of service. Positive youth development strategies included skills training in coping, decision making, self-management and life skills (competence, self-efficacy, opportunities for prosocial involvement); tutoring (bonding and competence); and techniques for and shifting peer group perceptions and norms (prosocial norms). Goals included promoting children's empowerment (self-determination), autonomy and identification with others (positive identity and bonding). Classroom discussions occurred at least once a week throughout the year.
The study used an experimental design in which 25 schools nationwide were randomly assigned to conditions from 1991 to 1995. The evaluation did not include a follow-up period; all data were measured at immediate posttest following a one-year (school year) intervention. Prior to this evaluation, a number of evaluations of Teen Outreach had suggested promising results, but each was subject to design limitations. One purpose of this study was to address concerns generated from the limitations of the previous research. Students were randomly assigned to either Teen Outreach participation or the control condition either at the student level or occasionally, at the classroom level. At the student level, sites had more students sign up for the intervention than could be accommodated in the program and participants and controls were randomly selected by picking names out of a hat or choosing every other name on an alphabetized list. Students entered the program in various ways, as part of their health curricula, as an academic elective, through teacher/guidance counselor suggestion, or as an after-school activity. Approximately 10% of sites contacted regarding the random assignment procedure participated; the remaining sites did not want to participate in random assignment or did not have enough interested participants. There were 342 program participants and 353 control group participants at study entry, all in grades nine through 12. The program group was 86% female and the control group was 83% female. The intervention group was 17% Caucasian, 67.7% African American, 12.9% Hispanic, and 2.4% of another ethnic identity. The control group was 20.4% Caucasian, 66.6% African American, 9.6% Hispanic, and 3.4% of another ethnic identity. Students were assessed at the start of the school year and at program exit during late spring. Attrition analyses showed 5.3% attrition among program participants and 8.4% among comparison students. Students who dropped out of the study were not significantly different from those who remained either in history of class failure, ethnicity, parents' educational levels, or household composition. However, those who left were more likely to have had or caused a prior pregnancy, to have been suspended, to have been younger, and to have been male. Although three sites were dropped from the analyses due to significant differences in entry characteristics or failure to recontact large numbers of youth in the control groups, the authors conducted substantial alternative analyses which lent confidence to their findings.
Self-report questionnaires provided information on changes in problem behaviors, including school failure, suspensions and pregnancy. The same questions were asked at pre and posttest, except the pregnancy question referred only to one year and a question was added to identify students who had dropped out or intended not to return to school in the prior year. Significant decreases were found for the experimental group on measures of school failure (31% vs 37%, p<.001), school suspension (16% vs 21%, p<.001), and teen-pregnancy (3.2% vs 5.4%, p<.01), compared with the control group. The authors reported cost data, stating that a full academic year of the program to a class of 18 to 25 students cost approximately $500 to $700 per student.
Nine effective programs combined their strategies across three settings. Seven programs were conducted in combined family, school, and community domains: Across Ages, Adolescent Transitions Project, Midwestern Prevention Project, Project Northland, Responding in Peaceful and Positive Ways, Valued Youth Partnership, and Woodrock Youth Development Project. These multiple-domain programs successfully promoted positive youth development strategies in school, incorporated parent or family involvement, and used community strategies or settings. One program, Creating Lasting Connections, combined family, church, and community, and one program, Quantum Opportunities, combined school, workplace, and community.
The analysis of school, family and community-based interventions identified seven programs that were well evaluated and showed significant effects on youth outcomes. The family-school-community programs promoted positive youth development constructs and strategies across the three domains, incorporated parent or family involvement, and used resources or opportunities from the local communities in which the children lived. Five of the seven programs (Across Ages, Adolescent Transitions, Project Northland, Responding in Peaceful and Positive Ways, and Woodrock) used experimental research designs, assuring confidence in the internal validity of the observed outcomes.
The school-family-community interventions were frequently based in schools, used units of assignment tied to the schools (e.g., classrooms), and used the school component strategically to tie in the family and community components. These programs typically placed emphasis on the careful integration and monitoring of individual and group strategies across all three domains. For example, in Across Ages, the quality and structure of the interactions between the child's mentor and the child's parents were considered as important to successful outcomes as the school-based curriculum. Programs generally tried to introduce protective factors into all three settings. While the children were being taught skills, or other youth development strategies were addressed in the program's youth development framework, parents were the focus of efforts to bolster family competence, parent self-efficacy, bonding and alignment with prosocial norms, and local communities were the focus of efforts to use community assets, resources, and partnerships to enhance the success of the other strategies. In ways similar to those described in the school- and family-domain programs, parents in these programs were typically engaged either through direct parent training or involvement in program implementation or organization. However, unlike the two-domain school-family interventions, these programs were generally based on program principles that stressed the importance of addressing community risk and/or protective factors as an integral part of producing successful youth outcomes. As in the school-community domain programs, these interventions incorporated communities either through using their social, economic or physical resources, or targeting specific community risk factors, or attempting to influence community-level policies and practices. More than half of these programs (Across Ages, Midwestern Prevention Project, Valued Youth Partnerships, Woodrock) emphasized the development of strategic relationships or partnerships with the community.
These programs produced improvements in positive youth outcomes including more positive attitudes about older people and higher levels of community service (Across Ages); higher levels of social skills learning (Adolescent Transitions) and school attendance (Across Ages); greater self-efficacy with respect to substance use refusal (Project Northland); higher reading grades and cognitive competence (Valued Youth Partnerships); and improvements in race relations and perceptions of others from different cultural or ethnic groups (Woodrock). These interventions also had a significant impact on the reduction or prevention of problem behavior in children. Four programs changed attitudes and practices related to substance use (Across Ages, Midwestern Prevention Project, Project Northland, and Woodrock). One program successfully changed negative family interaction patterns and reduced levels of family conflict (Adolescent Transitions). Two programs reduced either school suspension or drop-out rates (Responding in Peaceful and Positive Ways and Valued Youth Partnerships). Two programs reduced aggressive and violence-related behaviors and/or attitudes (Adolescent Transitions, Responding in Peaceful and Positive Ways). Three programs reduced levels of cigarette, marijuana, and/or alcohol use (Midwestern Prevention Project, Project Northland, Woodrock).
LoSciuto, Rajala, Townsend, and Taylor (1996) evaluated Across Ages, an intervention that addressed 11 positive youth development constructs, including social, emotional, cognitive, and behavioral competencies, bonding, resiliency, self-efficacy, recognition for positive behavior, positive identity, opportunities for prosocial involvement, and prosocial norms.
The program's theoretical foundation integrated positive youth development, youth identity development, social problem-solving, and the social development model: strengthening protective factors in the individual, the family, the school, the peer group, and the community/neighborhood was expected to increase resiliency in children. The goal of Across Ages was to demonstrate the impact of an intergenerational mentoring approach to drug prevention for high risk sixth grade students. Program components included mentoring for at least two hours twice a week for the school year (bonding, opportunities for prosocial involvement) by adults 55 years old or older; one hour every two weeks of community service activities with the mentor (opportunities for positive involvement); 26 sessions of exposure to the Social Problem-Solving model (competencies) as used by Weissberg and Caplan (1988) in the Positive Youth Development Curriculum; parental involvement and strengthening parent-child bonds by coaching parents in more effective parenting styles (opportunities for prosocial involvement, recognition for positive behavior, bonding, competencies) during Saturday workshops. The program also focused on developing positive relationships between mentors and parents.
Workshops were held for teachers on the use of the Social Problem Solving Model. Teachers rated students on how much they participated in the positive youth development curriculum and community service sessions, and project staff rated mentors
on the level of involvement with each student. Parental participation was not assessed due to its being "sporadic" and not providing sufficient information for analysis.
The study used an experimental pretest, posttest research design. At the time of evaluation, data had been collected for three years and there were two years remaining in the project. Experimental and control group classes were selected randomly from among sixth grade teachers in three schools who had indicated a willingness to participate. Three classes in each school were selected randomly from the remaining pool of sixth grade classes and assigned to one of three groups, two intervention conditions and a control condition. One intervention group (PS) received the Positive Youth Development Curriculum, community service and parent workshop components, and the other intervention group (MPS), received those components and mentoring from older adults. Mentors were carefully recruited, screened, trained and matched with the youth. The program was successful at retaining mentors, with two-thirds of the mentors at evaluation having been in the program since its inception four years earlier. Data were collected for the 1991-92, 1992-93 and 1993-94 academic years and combined in this evaluation. Pre-intervention group equivalence was established. Attrition rates were similar for all groups (Group C, 23%; Group PS, 22%; and Group MPS, 25%). For the three evaluation years, a total of 729 students completed the pretest. The final sample use in the evaluation consisted of 562 students who completed both the pretest and the posttest (77% of those originally pretested). The sample of 562 children were sixth grade students attending three public middle schools in Philadelphia neighborhoods. The target population included African American (52.2%), Asian (9.1%), Latino (9.0%) and Caucasian (15.8%) students, 180 of whom were served by Across Ages each year for three years. The neighborhoods were characterized by poverty, a high incidence of substance abuse and drug related crime, and a significant number of abandoned houses. In each school, student achievement was low and attendance poor. Many children were living with grandparents or other relatives (percentages not specified). Fifty-three percent of students who completed both the pretest and posttest were female. Approximately equal numbers of students completed both the pretest and posttest in each of the three experimental groups (189 in the control group, 193 in Group PS and 180 in Group MPS).
The evaluation reported significant outcomes using ANCOVAs that compared two groups at a time. The authors hypothesized that one of the two intervention conditions, the combined mentoring-problem solving (MPS) group, would show more positive outcomes compared with the control group, and also when compared with the other intervention condition. The reported results showed significant effects for the combined group condition when compared with the control group on a number of measures. The combined condition also showed several significant improvements compared with the other intervention. The significant outcomes for the combined MPS group compared with the control included increased positive attitudes (F(1,316) = 4.34, P=.038) on four dimensions (school, the future, elders, and older people); increased knowledge about older people (F(1,313) = 7.04, p=.008); improved reactions to situations involving drug use (F(1,271) = 4.17, p=.042); and higher levels of community service (F(1,208) = 5.10, p=.025). The combined mentoring-problem solving condition also reported significantly better results compared to the other intervention condition for attitudes toward older people (F(1,316) = 6.36, p=.012). Having a mentor and participating in the MPS intervention significantly improved school attendance for youth in that intervention condition, compared with both control and the other intervention group (F(2,447) = 4.58, p =.01). The problem-solving condition significantly improved their knowledge about older people compared with controls (F(1,368) = 5.32, p =.022). Within-group analyses of the mentoring group on mentor/student bonding measures showed that level of mentor involvement was positively associated with improved school attendance (F(2,138) = 25.03, p = .000).
Andrews, Soberman and Dishion (1995) evaluated the Adolescent Transitions Project, a program that addressed eight positive youth development constructs including social, cognitive, and behavioral competencies, bonding, self-efficacy, recognition for positive behaviors, opportunities for prosocial involvement, and prosocial norms.
The intervention used a parent and youth skills training model designed for substance use and problem behavior prevention. The evaluation reviewed two deliveries of the program, one in a community mental health setting and one in a school setting, but presented outcome data on only the first delivery. The main intervention components in both versions were the Parent Focus and Teen Focus conditions. Intervention groups targeted predictors of problem behavior, including family management practices, communication, limit setting, problem solving, goal setting, and dealing with negative peer influences. Parent consultants were used to assist group leaders. The Parent Focus curriculum combined three sets of family management skills, including prosocial fostering, limit-setting, and problem-solving. The Teen Focus component was developed from a Botvin & Wills cognitive-behavioral curriculum created in the mid 1980s. The curriculum emphasized behavior modeling by employing a peer counselor who had successfully completed the modeling component. Adolescents learned self-regulation skills including realistic and incremental goal-setting and problem-solving. The program included 12 sessions of curriculumwith 18 hours of contact total. The evaluation contained comprehensive implementation data on participation, program retention, parent and teen engagement, and satisfaction with the program.
The evaluation used an experimental, pretest posttest design. A cluster sampling approach was used first and followed by random assignment to conditions. Parents were recruited through newspapers, community flyers, school counselors, and other community professionals. Following parent inquiry, a telephone screening was conducted using a 10 question instrument based on risk factor research (Bry, McKeon & Pandina, 1982). If the child was assessed as at risk on at least four dimensions, the family was randomly assigned to one of four intervention conditions: Parent Focus, Teen Focus, self-directed change, and a control group. Interventions were conducted over two years, with four cohorts of approximately 30 families per cohort. Each intervention group consisted of approximately seven to eight families. Boys and girls received assignments separately to assure equal distribution of gender across conditions. The attrition analysis showed program retention was 90%; this meant 143 of 158 families were represented in the termination assessment. The data revealed no significant differences between families who stayed and those who left. The sample of 143 families resulted in a youth sample of 83 boys and 75 girls between ages 10 and 14, enrolled in middle schools containing sixth through eighth grades, with the mean grade level for the sample being seventh grade. Ninety-five percent of the sample was European-American. Other demographic information was not provided in the evaluation. Approximately one fourth of the families had an annual income of less than $10,000 and more than half the families received governmental financial assistance. Approximately 50% of mothers and 45% of fathers had some college education.
Results were assessed through child behavior inventories, audiotaped problem-solving scenarios and videotaped family problem-solving sessions. Outcomes were reported for participant engagement and satisfaction, skills acquisition, family interaction patterns, and changes in family conflict. Both the Parent and Teen components were shown to be effective in engaging both parents and teens, teaching the targeted skills, and reducing parent-child conflict. The Parent component had a short-term effect on reducing teen problem behavior in school. The Teen Focus condition did not demonstrate a significant effect on the students' problem behavior in school, but youth in the Teen Focus condition demonstrated higher social learning scores (F(1,140) = 5.76, p<.05). Youths' negative engagement was reduced significantly when parents participated in the Parent Focus intervention, while control group youth increased in negative engagement (F(1,107) = 5.27, p<.01). The same pattern of effects occurred in the Teen Focus intervention, with a reduction in their negative engagement with parents compared to the inactive intervention conditions (F(1,114) = 4.89, p. < .05). Fewer than one half of fathers participated in the videotaped family interaction, and analyses were restricted to mothers' behavior. Mothers who were exposed to both the Teen (F(1,113) = 6.51, p<.01) and Parent (F(1,106) = 7.1, p<.01) Focus conditions showed less negative engagement in the family interaction, compared to mothers in the control condition who tended to increase their negativity. Those in the Parent Focus condition reduced conflict levels (F(1,101) = 10.95, p<.001)from pretest to posttest. There was a decrease in family negative events for families in this intervention condition while negative events increased for those in other conditions. The Teen Focus group also yielded significant reductions in family conflict (F(1,118) = 8.1, p<.005) compared with the control group, and significantly fewer family negative events (F(1,107) = 6.54, p<.01). Teacher ratings of child behavior revealed effects specific to the Parent Focus condition. Teens with parents in this program condition reduced their rate of aggression, while teens in the inactive intervention conditions were rated as more aggressive at posttest (F(1,106) = 5.33, p<.05).
Pentz, Dwyer, Johnson, Flay, Hansen, MacKinnon, Chou, Rohrbach and Montgomery (1994), Pentz, et al. (1989), and Pentz, Trebow, et al. (1990) evaluated the implementation and five-year follow-up results of the Midwestern Prevention Project - Project STAR - Kansas (MPP). The multi-year intervention addressed seven positive youth development constructs including social, cognitive, and behavioral competencies, self-efficacy, recognition for positive behaviors, bonding, and prosocial norms.
In the 1984-1985 school year, the MPP was initiated in 50 public middle/junior high schools and 15 communities. The components of this comprehensive intervention included mass media programming, a school-based educational program for students, parent education and organization, community organization and health policy. The STAR Program consisted of five components implemented over five years at the average rate of one per year while mass media program was used in all years of the intervention. The school program involved 20 hours of direct contact with students and their parents in the first and second years (10-session school program for resisting and counteracting drug use influences and 10 hours prevention practice homework activities with parents). Community organization activities created metropolitan task forces against drug use in the third through the fifth years. Mass media coverage included 16 television, 10 radio and 30 print media events throughout the program delivery period. Two hours of training were provided to television station managers. An average of 31 mass media programs per year occurred in the first through the third years, decreasing to 10 per year in the fourth and fifth years. This entailed news clips, commercials, talk shows, press conferences and articles covering baseline drug use and STAR goals in Kansas City, introduction of each program component, skills demonstration, and public recognition of participating students. Tobacco and alcohol policy changes occurred in the fourth and fifth years.
The evaluation used a quasi-experimental, partial randomized control trial that varied the intervention condition (prevention program or delayed program control with health education as usual) and the school grade of the children's initial intervention (grade six or seven, depending on which represented the transition year from middle or junior high school). The MPP measurement design consisted of annual assessments of several different youth samples. The universal population of students was initially selected from 50 schools in 15 communities, but the study design led to only eight out of 50 being randomly assigned (schools were assigned to conditions based on school administrator scheduling flexibility after the start of the school year). The rest of the assignments used demographic matching and relied on the flexibility of schools to implement the intervention. Twenty-four were matched and assigned non-randomly to program schools and 18 were matched and assigned non-randomly to control schools. For measuring longitudinal effects of the intervention on the entire population, two of these samples were merged: a panel consisting of all students from the 1984-1985 entering grade cohort in eight schools who were tracked individually over time (n = 1607) and a 25% sample of students from the 1984-1985 cohort in the remaining 42 schools, who were cross-sectionally and randomly sampled by classroom each year (n = 3771 in 1984-85). The middle or junior high school was the unit of assignment for the initial intervention. Of an average of 4664 students targeted for assessment, 94% received annual parental consent for participation. The final sample at posttest had an ethnic group representation that was 76.6% white, 19.2% African-American, 49.3% female, 60.9% low SES, 22.6% in sixth grade, and 60.8% urban. At two years into the intervention 63.6% of the sample were enrolled in schools different from the schools of origin.
The Midwestern Prevention Project conducted extensive implementation methods and measures. Teachers were selected to implement the intervention based upon whether they had primary responsibility for teaching health education courses or, in schools where formal health education was not offered, responsibility for teaching courses in which health education matter could be most easily assimilated; and course loads which reached all students in the target grade. Program teachers received an initial three-day workshop and a one-day refresher workshop on school and booster program implementation, including drug prevention skills and program delivery. Program teachers provided training to two to four peer leaders per class in a one-hour training session. Training was followed by program staff making monthly phone calls to teachers and periodic meetings with principals. The parent program was implemented as a series of planning meetings chaired by the principal and attended by two to four parent representatives and two STAR program students. An annual Parent Skills Night was offered for all parents of STAR program students. Principals, parent representatives and student representatives received an annual one-day workshop on parent program implementation. In addition to training parents in parent-child communication and prevention practice support skills, multiple methods were used to incorporate parents at an organizational, system-wide level. Parents were mobilized to change school policies about institutionalizing drug prevention curricula and restricting drug use in and around schools.
Implementation analyses reported data from parent phone calls and meetings of STAR program staff with principals; observations of program sessions and content analyses of media events by an MPP project archivist; program implementation evaluations completed by teachers (self-report surveys) and parent group members (phone survey interviews); and a consensus rating of overall program implementation quality by three STAR program staff members. The investigators used three operational definitions for implementation: adherence (was the intervention only well implemented in the experimental and not the control conditions?), exposure (what was the amount of the intervention that was delivered?), and reinvention (how much does the implementation in question differ from the program standard that is being tested?). Sixty-five teachers from 27 program schools were trained to implement the program, and in questionnaires administered immediately post training, all teachers reported that they had been "very adequately" or "moderately adequately" prepared to implement the program. Program implementation effects were estimated for prevalence rates of drug use, with school as the unit of analysis. To generate school level data on implementation, individual teachers' ratings were averaged within each school. Teacher reports showed that all twenty-seven schools assigned to the program condition implemented the program during the school year as planned. Staff reports confirmed implementation by all program schools, and also confirmed that the twenty-three control schools adhered to the control condition design. The number of sessions implemented by program schools ranged from three to 10 with a mean of 8.76 sessions (2.06 SD). Average length of time per session was forty minutes. Program exposure ranged from 2.75 to 9.17 hours with a mean of 6.47 (1.74 SD). Of the teachers, 100% responded that they had not deviated substantially from implementation as designed (32% deviated "not at all" and 68% deviated "slightly").
The program was evaluated as an entire intervention package; however, it should be noted that one of its components, the community organization and mass media coverage, were available to both program and control students and therefore limits the external validity of the program. Outcome data came from self-report survey of substance use, biochemical measure of smoking, and school records. Results showed the intervention significantly reduced monthly, weekly, and heavy cigarette, marijuana, and alcohol use through three year follow-up (p<.05). There was some decay of effect at four year follow-up which corresponded to a decrease in control group use. By the five year follow-up the increase in the proportion of students reporting use continued to be significantly higher in the control group than the program condition on all of the monthly drug use measures as well as for weekly cigarette use. The exceptions were the effects on daily cigarette use and heavy marijuana use. Results assessing the impact of varying implementation rates on substance use were highly significant. Schools with a high level of implementation had little or no increase in rates of weekly use of substances and a decrease in the use of cigarettes in the last month (compared to increases for low implementation and control groups). Six years after MPP began, the increase in substance use prevalence rates for the 1984 cohort of intervention school students continued to be lower than control schools, with average reductions of 8.4% in monthly use, 5.7% in weekly use, and 4.9% in heavy use for cigarettes, alcohol, and marijuana.
Perry, Williams, Veblen-Mortenson, Toomey, Komro, Anstine, McGovern, Finnegan, Forster, Wagenaar and Wolfson (1996) evaluated Project Northland, a family, community and school-based intervention that addressed eight positive youth development constructs including social, emotional, cognitive and behavioral competencies, bonding, self-efficacy, opportunities for prosocial involvement, and prosocial norms.
The intervention was conducted over a three year period, beginning in the sixth grade in fall of 1991 and continuing through seventh and eighth grade. Although the program's focus was on preventing substance abuse, particularly seeking to influence children's choices about using alcohol and cigarettes, Project Northland used many strategies that simultaneously promoted positive youth development. Strategies were designed to influence psychosocial factors such as peer influence, self-efficacy, child-parent communication, and perceived ease of access to substances. The study used a multi-level, multi-component, community-wide approach. Students received many forms of skills training intended to enhance their competence in dealing with their parents, and with peer pressure and normative expectations about alcohol. In addition to specific skills development, the intervention addressed community-level changes in alcohol-related programs and policies. The school component used a social-behavioral curriculum, homework, peer leadership training, parental involvement/education, and community-wide task force activities ("Slick Tracy," "Fun Night," "Amazing Alternatives," "PowerLines").
The study employed an experimental, delayed control group design. Twenty combined school districts were blocked by size (small, medium, large, very large) and randomized to either an intervention or delayed condition (N = 10 in each group). This was made possible by combining four of an original 24 districts with other districts to generate an adequate sample size in each unit to be randomized. Annual surveys were used to measure alcohol use, tobacco use and psychosocial factors. Comparison group schools were allowed to use other programs such as DARE or Project Quest until 1994 when they implemented Project Northland. In a 1992 survey, over 90% of these students reported having taken part in DARE (40% in intervention districts) and 21% had taken part in Quest (compared to 2% in intervention districts). The study appropriately matched unit of analysis and assignment (school district). Tests were performed to assess group equivalence and showed pretest equivalency on a number of key variables. Group differences showed that, at baseline, more students in the intervention districts reported alcohol use, were slightly older (0.1 years older), and had fewer white students than in the reference districts. Of the 2,351 students measured at baseline, 93% (N = 2,191), 88% (N = 2,060) and 81% (N = 1,901) were surveyed at the end of the sixth, seventh and eighth grades respectively. The attrition analysis showed that of 450 (19%) lost to follow-up at the end of the eighth grade, 231 (51.3%) were in the intervention condition and 219 (48.7%) were in the control condition. No significant differences were noted in baseline alcohol use between those who were lost to follow-up in the intervention vs the control conditions, or between those who were lost to follow-up and those who remained.. Of the 450, 278 (62%) had moved, 31 (7%) were absent, 42 (9%) had moved to a Project Northland school in a different treatment condition, 87 (19%) refused or were not allowed by their parents to participate and 12 (3%) were deleted from the analyses due to three or more inconsistent responses. The final sample consisted of 1901 students in grades six to eight from 20 schools in northeastern Minnesota. Students belonged to two ethnic groups, with 94% Caucasian and 4.5% Native American.
Implementation was extensively addressed and measured. Teachers and peer leaders received training before the program was implemented. Implementation was measured through exposure and participation indices completed by parents and teachers.
The primary effects of the intervention were measured with self report questionnaires given to students and parents. Other forms of measurement included observations of alcohol purchase attempts by underage buyers, telephone surveys of alcohol merchants, and interviews with community leaders. However, only the self-report questionnaires were included in the evaluation. The analyses were based on comparisons among all students, and among baseline non-users and baseline users.
For all students in the intervention districts, there were significantly lower scores on the alcohol scale by the end of eighth grade than control district students (16.0 vs 17.5, p<.05, on a 8-48 scale where 8 was no tendency to use, and 48 was high levels of use). The scale score was also significantly lower among baseline non-users in the intervention districts compared with the control districts (13.8 vs 15.3, p<.01). For all students, the percentages who had lower alcohol use in the past month (23.6% vs 29.2%, p<.05) and past week (10.5% vs 14.8%, p<.05) were significantly better in the intervention districts compared with the control districts. In the intervention districts, there were lower onset rates for baseline non-users in the past month (15.3% vs 21.2%, p<.05), and the past week (5.3% vs. 9.8%, p<.01) by end of eighth grade, and lower past year use for baseline non-users at the end of seventh (21.1% vs 29.1%, p<.05) and eighth (30.4% vs 41.6%, p<.006) grades. Non-users had lower rates of marijuana use (3.1% vs 6.2%, p<.05) and cigarette use (15.5% vs 20.7%, p<.05) in the intervention districts. There were no significant differences between conditions for all students in cigarette use, smokeless tobacco, or marijuana use. Positive changes for alcohol-related knowledge and attitudes were linked to resisting peer influence. Among all students, those in the intervention districts had significantly lower scores on peer influence by the end of eighth grade (24.6% vs 27.0%, p<.05). Intervention students were significantly more likely to report being able to resist alcohol in social settings even though the self-efficacy scale showed no significant differences between groups. Among baseline non-users, students in intervention districts had significantly lower scores by eighth grade on peer influence (22.8% vs. 25.4%, p<.05) and higher scores on self-efficacy (21.6% vs 20.4%, p<.05). By the end of sixth grade, intervention district students were significantly more likely to report that their parents talked with them about drinking related problems. Among all remaining psychosocial variables for all students, only one finding was significant, the greater likelihood of being disciplined by school for consequences of driving after drinking (2.5% vs 2.2%, p<.001). Baseline non-users in intervention districts at the end of eighth grade were more likely to say they had influence in their communities on alcohol-related issues than baseline non-users in the control condition.
Two evaluations of The Richmond Youth Against Violence Project (Farrell & Meyer, 1998, 1997) assessed the effectiveness of a multiple-domain, school-based intervention called "Responding in Peaceful and Positive Ways" (R.I.P.P.). The intervention addressed 10 positive youth development constructs, including social, emotional, cognitive, behavioral and moral competencies, bonding, prosocial norms, self-efficacy, opportunities for prosocial involvement, and recognition for positive behavior.
This review analyzed two evaluations of the R.I.P.P. curriculum, one using 1993-94 data and the other, 1995-96 data. The intervention described in the evaluation of 1995-96 outcome data represented an expanded 25-session version of an earlier 16-18 session program originally developed from Prothrow-Stith's (1987) violence prevention model and concepts from "The Friendly Classroom for a Small Planet" (Children's Creative Response to Conflict, 1988). R.I.P.P. is an ongoing, multi-year intervention that the program authors described as a "developmentally-anchored health promotion model" (1996:13). The initiative of which R.I.P.P. is part began in 1991 as a broad collaborative effort between the Richmond Public Schools, the Richmond Behavioral Health Authority, and Virginia Commonwealth University. In 1993 this collaboration was expanded through a cooperative agreement with the National Center for Injury Prevention and Control within the Centers for Disease Control and Prevention.
The expanded 25-session curriculum was based on the results of an evaluation of the earlier 18-session model. The expanded curriculum used standardized manuals and sought to address gender differences at the level of curriculum development and personnel training; to increase intensity in order to produce a stronger effect; to have clearly stated objectives; to have a firm base in research and theory about adolescent violence; to be sufficiently standardized in order to minimize differences in interpretation of the program's content; and to emphasize how students can use the skills they learn in the program outside of school and throughout their lives. The components included adult role modeling (prosocial norms, bonding), peer mediation (opportunities, bonding and competence), team-building activities (bonding, competence, and opportunities), relaxation techniques (emotional competence), small group work (bonding and opportunities), role plays (opportunities and competence), and cognitive restructuring methods such as mental rehearsal. The program also provided staff development and parent training in non-violence and conflict resolution methods. Students worked with a trained prevention specialist who implemented the R.I.P.P. curriculum once a week, modeled and reinforced appropriate non-violent behavior in the schools, promoted a caring community of students and adults, and supported prosocial norms and expectations. The specialist also implemented a school-wide peer mediation program available to all students at each school.
The intervention used an experimental design conducted at three middle schools in Richmond during the 1995/1996 school year. The intervention was implemented with half the sixth grade students at each participating school and in each school the classes were randomly assigned to either the intervention or control groups. The initial sample included 295 intervention and 307 control group members. Data from 23 intervention students were removed when they were determined to have missed more than a third of the sessions. Analyses showed that these students had significantly higher rates of suspensions, lower grade point averages and attendance, and higher frequencies of violent behavior. The final sample of 579 sixth grade students at three urban middle schools consisted of even numbers of boys and girls in the intervention (n=135 boys, n=137 girls) and control (n=154 boys, n=153 girls) groups. Students ranged in age from 10 to 15, and 96% were African-American. There were no significant differences between the intervention and control groups on ethnicity, gender or age. Complete data were ultimately available on 455 students, and analyses compared the 124 students with incomplete data. Although there were no gender or ethnicity differences, students with incomplete data were significantly older, had lower grade point averages and school attendance, had more violations for fighting and weapons, and more suspensions.
Data included school code violations for fighting, assaults, weapons in school, and school suspensions, and self-report measures of self-restraint, nonviolent responses, violent behavior, and attitudes supporting violence. All weapons violations except one occurred in one school, so this school alone was used in that analysis. Results at immediate posttests showed that program participants had significantly fewer disciplinary code violations in the last quarter of the year for fighting (2.2% vs 5.2%, p<.05) and carrying weapons (1.9% vs 7.4%, p<.04), and lower rates of in-school suspensions (1.5% vs 5.5%, p<.01) compared to the control group. Program participants also improved their knowledge of the intervention material (F(1,299 = 56.66, p<.001), used their school's mediation program more frequently (42% vs 30%, p<.05), and reported significant reductions in fight-related injuries relative to the control group (4% vs 9%).
Unpublished data (Farrell, 1998) for all three time points (pretest, posttest and follow-up) were available for 353 students (R.I.P.P. n=169; control n=184). Significant effects for posttest to 6-month follow-up changes were found for the knowledge test, the violent behavior frequency scale, the suppression of anger scale, and the frequency of threatening a teacher. Several gender-specific effects were also found. For boys there were significant effects for improved impulse control, frequency of drug use and an item that asked students how frequently they skipped school due to concerns for their safety. For girls there was a significant positive effect on the problem situation inventory. School disciplinary data at follow-up also showed significant sustained effects, including fewer in-school (4.8% vs 11.7%) and out-of-school suspensions (16.5% vs 21.3%) compared to the control group.
Cardenas, Montecel, Supik, and Harris (1992) evaluated the impact of the Valued Youth Partnership Program, which addressed 11 positive youth development constructs, including social, emotional, cognitive and behavioral competencies, bonding, self-efficacy, recognition for positive behavior, positive identity, opportunities for prosocial involvement, belief in the future, and prosocial norms.
Comprehensive positive youth development was at the core of the program philosophy, which emphasized the integration of individual and social environmental strategies to help transform a child's self-concept (positive identity). The components of the program included bilingual instruction for limited-English proficient students; a cross-age tutoring component (bonding, competence); classroom enhancement activities; school-business partnerships (belief in the future); increased student recognition of accomplishments and talents (recognition, positive identity); parental involvement in school activities; staff development; leadership models; and self-paced and individualized instruction curriculum. Tutors received classes once a week to develop their tutoring skills, engaged in tutoring at least four hours per week, participated in at least two annual field trips, and attended various presentations by role models. Tutoring groups were generally conducted at a one to three (tutor to learner) ratio. Although specific implementation varied by site, critical elements were identified that all sites adhered to, including weekly classes for tutors with a minimum of 30 sessions per school year, a minimum age and grade difference of three years between tutor and tutee, provision of a stipend, and a flexible curriculum based on students' tutoring and academic needs. The specific goals of the program were to reduce dropout rates, enhance students' basic academic skills, strengthen students' positive perceptions of self and school, decrease student truancy, reduce student disciplinary referrals and form school-home-community partnerships to increase the level of social and emotional support.
The evaluation used a quasi-experimental design in which a pool of students was identified based upon being limited-English-proficient as defined by the State of Texas guidelines and reading below grade level on a standardized achievement test. The tutoring group was selected from a pool first based upon scheduling and availability, and then the comparison group was randomly selected from the remaining pool of at-risk students. The sample of 194 participants was drawn from a largely Hispanic population of at-risk limited-English-proficient middle school students on four campuses in two public school districts in San Antonio, Texas. A total of 101 secondary school tutors and 93 comparison group students, average age 12 years old, participated in the program. The ethnic group composition reported for tutors was 61% Hispanic and 2% Caucasian (remaining percentages and ethnicities not reported); for the control group, 69% Hispanic and 0% Caucasian (remaining percentages and ethnicities not reported). While 33% of the tutors reported having been retained a grade at baseline, 42% of the control group had. Baseline data showed no significant differences between tutors and comparison group on age, average grade in reading, quality of school life and self-concept scores, ethnicity or retention. The only statistically significant difference between the two groups at baseline was eligibility for school-lunch program with the tutor group having lower socioeconomic status than the comparison group. At posttest, only 63 of 101 tutors and 70 of 93 control students had data for reading, self-concept and quality of school life.
Implementation teams were given clear roles and customized guidelines for each implementation team member. The evaluation component of the program had a structured implementation component to measure program operations and develop corrective action as needed.
Posttest data were collected on student grades, disciplinary action referrals, absenteeism, self-concept, and quality of school life. The Valued Youth Partnership study showed significant impacts for tutors, particularly in reducing dropout and improving reading grades. Results of the reading grades data were analyzed only for those students who had data for reading, self-concept and quality of school life (63 tutors and 70 comparison group students). Being in the tutor group led to significantly higher reading grades after the first year, which continued in the second year: the tutors scored on average nearly three points higher than the comparison group (p<.05). Tutors' self-concept (p<.05) and attitudes toward school (p<.01) also improved, but their gains were seen only in the first year, leveling off during the second year. The drop-out rate decreased, with one tutor out of 101 (1%) of the tutors having dropped out by the end of the second year of the program compared to 11 of the 93 comparison group students (12%) (p value not provided).
The study included information on costs: the general budget for a program of 25 tutors and 75 tutored students was approximately $25,000 plus transportation and per diem, or $250 per student served. Half of this budget went to tutor stipends and the rest to evaluation ($3000-5000) and training/technical assistance ($5000).
LoSciuto, Freeman, Harrington, Altman and Lanphear (1997) evaluated the Woodrock Youth Development Project (WYDP), an intervention which addressed eleven positive youth development constructs, including social, emotional, cognitive and behavioral competencies, bonding, resiliency, self-efficacy, recognition for positive behaviors, prosocial norms, positive identity, and opportunities for prosocial involvement.
The program is based on Problem Behavior Theory and Social Inoculation Theory, and emphasizes life skills and social competence training while also promoting an anti-drug message and providing broad systems support across all three domains. Intervention components included human relations classes, peer mentoring, extracurricular school activities, and structured interactions between students and teachers, and children and parents. Human relations classes were designed to promote positive self-perceptions (self-efficacy, positive identity) , to raise awareness of the dangers of alcohol, tobacco, and drug use and promote healthy attitudes about not using substances (prosocial norms), and to aid in the development of an appreciation of other ethnic and cultural traditions (positive identity). Peer mentoring involved high school students as mentors who tutored younger students and actively engaged them in individualized projects designed to help develop talents and critical thinking skills (competencies, bonding, opportunities for prosocial involvement, recognition for positive behavior). Extracurricular activities included clubs and weekend retreats designed to improve academic performance, provide creative outlets, widen horizons and provide a "space for students to develop positive interpersonal relations" (opportunities for prosocial involvement, competencies, bonding). The school domain involved having youth advocates meet regularly with teachers to help monitor progress and develop goals (opportunities for involvement, recognition for positive behavior). The family domain involved support through home visits designed to build communication and establish parental trust in the program; and parenting classes covering such topics as communication, parent and school relations, and planning summer activities for children (opportunities for prosocial involvement, recognition for positive behavior, competencies). Although the community domain component was not extensively discussed in the evaluation, program materials documented the strong anti-drug norms (prosocial norms) that the Woodrock program seeks to promote in the Kensington community of Philadelphia.
The experimental design was a randomized pretest, posttest control-group design. Classrooms within four Philadelphia schools were randomly assigned to program or control conditions. Questionnaires were administered in November, 1994, and again in June, 1995, to students from ages six through nine (n=170); and students from ages 10 through 14 (n=197). Attrition analyses indicated that there were no gender differences or differences in baseline levels of outcome measures between those who dropped out and those who remained. There was a significant difference in age, with older students being significantly more likely to leave. Dropouts were most likely to be in the program group for the older sample and in the control group for the younger sample. The sample of 367 students (130 experimental; 237 control) remained after 19% of the original sample of 453 students from ages six through 14 (161 experimental; 292 control) was lost to attrition. Participants in the final measurement sample were 46.9% female, 44.4% Latino, 19.9% Caucasian, 11.4% African American, 11.2% Asian, 9.3% mixed or "other" ethnic identity and 1.9% Native American.
The evaluation reported outcomes for two subsamples of different age participants in the Woodrock program. More outcome data have been collected on the total sample and these will be published soon. Our analysis only had the published report on the two subsamples available. In the published evaluation, the authors reported significant positive differences between the intervention and control groups for the younger sample, and mixed results for the older sample, with one statistically significant (non-behavioral) outcome in the wrong direction.
Younger Group. These children were ages six to nine. For three of the five dependent variables (outcomes as a function of the intervention) the results showed statistically significant improvements for intervention group compared to the control group. The intervention group showed a significant change on two drug related measures: use in the last year (F(1,165) = 4.75, p<.05) and in the last month (F(1,166.) = 11.70, p<.001). There was also significant positive change for younger students' relationships with and perceptions of students of races other than their own (F(1,166) = 4.02, p<.05).
Older Group. For the older group, there was one positive, statistically significant behavior change for the intervention group compared with the control group: reduced levels of drug use in the last month (F(1,193) = 8.86, p<.003). There were no significant differences for drug use in the last year, self esteem or attitudes about race relations. The finding in the wrong direction was on a scale measuring attitudes toward drug use, which for the intervention group participants of the older subsample significantly worsened compared with the control group (F(1,193) = 10.12, p=.002).
Johnson, Strader, Berbaum, Bryant, Bucholtz, Collins and Noe (1996) evaluated Creating Lasting Connections, an intervention that addressed 14 positive youth development constructs, including social, emotional, cognitive, behavioral, and moral competencies, bonding, resiliency, self-efficacy, spirituality, recognition for positive behavior, positive identity, prosocial norms, opportunities for prosocial involvement, and self determination. Evaluators also noted the teaching of cultural competence in the interventions.
Creating Lasting Connections was a five-year demonstration project implemented in five church communities for one year to reduce the onset and frequency of substance use and abuse among high-risk 12-14 year olds. The goal was to positively influence resiliency factors in three domains, specified as "church community," "family," and "youth." The program had two integrative components that incorporated system and individual level strategies. System components used mobilization strategies to involve communities in prevention efforts targeting substance abuse. Individual client components used parent and youth strategies for substance abuse education and communication skills training. Youth skills targeted communication and social self-management with their peers (competence, self-efficacy, bonding), and practice sessions with their parents (bonding and competence). The system component addressed the community domain in a multi-phase strategy for identification, recruitment, selection, formation, and training of families. These processes were implemented by Church Advocate Teams (CATs) "empowered" to participate by helping retain families and evaluate program effects and quality. To address the family domain, the program promoted an increase in parent knowledge and beliefs about substance use (competence); development of family management skills and communication skills (competence, bonding, self-efficacy, opportunities); increased awareness of the impact on youth of family role modeling of alcohol use; parents' self-reported involvement in community activities with their youth; and use of community services when personal or family problems arose. To address the youth domain, the program promoted youth resiliency by targeting their communication and refusal skills, bonding with family, parent/youth involvement in community activities, and use of community services as resources when the need arose. The intervention had a strong family and community orientation in its methods; parents received more intervention hours than youth. This program included 15 hours of skills training in six sessions of approximately two and one half hours with peer groups and then with parents. Volunteer service providers (CATs = Church Advocate Teams) received an average of 18 hours of training in seven sessions of two and a half hours over eight to 10 weeks. Parents received a total average of 55 hours of training in 22 sessions divided over three objectives: one (AOD Issues Training) of 12-16 hours, one (Not My Child) of 16-20 hours, one (Straight Communications Training -- adapted from Say It Straight) of eight to 12 hours with other parents and youth. There was also follow-up consultation and continued support for at least one year, defined as bimonthly phone calls or home visits plus referral service as needed for five to six months.
The evaluation used an experimental, randomized block design with a comparison group and three repeated measures. Data were collected in three waves, at baseline, six to seven months after parent and youth training, and after the follow-up case management services were delivered, one year after initiation. Two evaluation designs were used to test nine hypotheses, eight of which were empirically tested. The overall hypothesis was that effects on youth resiliency are produced by the intervention as church communities become more empowered to prevent substance abuse or as the family increases its resiliency and ability to prevent substance abuse. Three addressed direct program effects on family and youth resiliency, three addressed moderating effects on resiliency, and three addressed moderating effects on AOD use among youth. For the eight hypotheses corresponding to program effects on family and youth outcomes, a randomized block design with repeated measures used church community as the blocking variable to control for site differences. Recruited families were randomly assigned to the intervention or the comparison condition in five church communities after completing a baseline interview. The evaluators used co-variates to further assure group equivalency. An extensive attrition analysis was conducted of individual and family characteristics, risk and resiliency factors from the family and youth domains, and substance use measures among youth. The analysis showed no differential attrition. The sample of 217 African-American and Caucasian youth, ages 12-14, was drawn from a population of families in five church communities in Louisville, Kentucky, and surrounding communities that encompassed both Catholic and Protestant churches with ethnic representation from Caucasian (77%) and African-American (23%) populations. In two pilots and one replication, 246 parents and 131 youth across nine sites were participants in the program. The program was implemented six times among five sites, requiring recruitment of 24 families at each site (12 for the intervention group and 12 for the control group). Data in the final analyses were from 97 parents and 120 youth. Three types of family and youth outcome measures were used to determine program effects on family and youth resiliency: overall direct effects, church community direct effects, and moderating effects. Direct effects included direct effects of CLC on family and youth resiliency outcomes, risk and resiliency factors, and AOD use measures. Interview and questionnaire items came from a standardized battery of AOD items and psychosocial items from the Personal Experience Inventory (PEI), developed by the Chemical Dependency Adolescent Assessment group in St. Paul, Minnesota (Winters & Henley, 1989), and a battery of communication skill items developed by Englander-Golden and Satir (1990).
Three youth resiliency measures showed significant effects for the experimental group over the control group. Intervention youth were significantly more likely to use community services as needed when personal or family problems arose (75% vs 51%, p<.001), to take more action based on the service contact (53% vs 43%, p<.001), and to perceive that the action accomplished something helpful (44% vs 26%, p<.001). There were no significant effects on measures of substance use behaviors. All other significant outcomes were reported as moderating effects divided into family resiliency moderators and youth resiliency moderators. Results of the analyses of family resiliency moderators showed that the onset of all substance use was delayed for intervention group youth for one year as parents positively increased their substance use knowledge and beliefs (-.38, p<.03) and there was decreased parent-youth conflict (.25, p<.05.) The onset of alcohol use among youth was delayed for one year as parents increased their AOD knowledge and beliefs (-.36, p<.04), family communication improved (.30, p<.05), and parent-child conflict decreased (.34, p<.01).
Hahn, Leavitt and Aaron (1994) evaluated the Quantum Opportunities Program, a four-year comprehensive intervention that addressed 13 positive youth development constructs, including social, emotional, behavioral, and cognitive competencies, bonding, resiliency, self-efficacy, recognition for positive behaviors, positive identity, opportunities for prosocial involvement, prosocial norms, self-determination, and belief in the future.
The program was designed to begin in ninth grade and follow participants through the end of high school. Program strategies included a range of education activities such as peer tutoring (competence, bonding) and computer-assisted instruction; service activities such as community service projects (prosocial norms, positive identity, opportunities), jobs, and helping at public events (positive identity, competence, self-efficacy); and youth development activities, such as mentoring (bonding), life and family skills (competence, bonding, self-efficacy), college and job planning (belief in the future, opportunities). Participants were provided with an adult mentor who provided tutoring and cultural enrichment. Financial incentives were given for participation and milestone completion. Bonding was a major emphasis of the program, typified by mottos like "take up" (others coming behind), "once in QOP, always in QOP," and youth being considered part of the program whether or not they attended. Three-fourths of participants in four demonstration sites participated over 500 hours. The average was 1300 total hours over the four year program.
The evaluation used an experimental design in which control and intervention groups were randomly selected from a pool of students going into ninth grade, living primarily in households of single-parent, minority families on welfare. High-schools from which the pools were selected were based upon the proximity to program offices. Fifty students at each site were randomly selected and assigned to either the control or intervention groups. Quantum directors were not allowed to recruit students who had pre-screened themselves into the program, but instead were told to see how many of the 25 youth assigned to the experimental group could be encouraged to join Quantum. Baseline data included demographic characteristics, work experience, school experiences, health knowledge, personal attitudes and opinions, academic skill levels, and functional skill levels. Analysis of the two groups at sample entry indicated that groups were largely free of systematic differences. The attrition analysis showed no significant differences between control and intervention participants who were evaluated in the autumn after program completion (88 remained of 100 members in the experimental group and 82 of 100 in the control group). The sample of 170 ninth to twelfth grade students consisting of 52% female, 75% African-American, 14% Caucasian, 7% Hispanic, 1% Asian, and 2% of another ethnic identity. On average, 94% of the sample had no children, 88% lived with one or both parents, and 78% had a mother or father who had graduated from high school.
The evaluation found significant changes in important youth outcomes over a four year period, most notably in the increases in positive outcomes that favored the experimental group over the control group. Intervention group members had significantly higher high school graduation rates (63% vs 42%, p<.01). Their rates of subsequent college or post-secondary school attendance rates were larger (42% vs 16%, p<.00), and they received more honors/awards than the control group students (60% vs 12%, p value missing from report.)
The evaluation included an extensive cost/benefits study which showed that $3.68 was gained for every dollar spent if QOP college students earned a degree. If only one-third of QOP college students ultimately earned degrees, the estimated benefit cost ratio was $3.04 for every dollar spent.
[ ASPE Home Page | HHS Home Page | NICHD Home Page ]
| [Previous] | [Table of Contents] | [Next] |