Imagine you're an educator in a bustling school in Midtown, USA. You've been asked to help with selecting a prevention program or strategy that will improve the school environment and help your students learn the knowledge and skills they need to resist involvement with alcohol, tobacco, and other drugs as well as other risky behaviors. The stakes are high: your choices could have a great impact on your students' future.

Knowing a bit about effective practice, you and your team begin by assessing the needs at your school. You gather information about teen drug use, violent behaviors, and related trends. You interview students, parents, teachers, the principal, the school board, PTA members, counselors, school health personnel, law enforcement officials, and local prevention specialists. You investigate resources such as time, money, personnel, facilities, materials, and expertise. You hold a community forum to help you identify the problems and needs of the students, as well as their interests and values.

Now what? There are hundreds of programs available purporting to meet the needs you've identified. What process can you use to assess their effectiveness?

Recognizing that this is a typical scenario faced by schools across the country, the U.S. Department of Education's Safe and Drug-Free Schools Program (SDFSP) teamed up with the Department's Office of Educational Research and Improvement (OERI) to establish an expert panel to develop criteria for analyzing program effectiveness and identify promising and exemplary programs that promote safe, disciplined, and drug-free schools. Recently, the Department released the findings of this expert panel.

"Our goal was to expand the current knowledge base on what works," said Bill Modzeleski, SDFSP director. "We recognize this is the first step and we need to quickly move beyond identifying successful programs to identifying broadly based strategies that work."

The Expert Panel Process

In order to identify the most effective prevention programs, the Department pulled together a 15-member team of educators, researchers, evaluators, and program developers, as well as representatives from local and state education agencies, businesses, institutions of higher education, and the medical and legal communities.

Working together as a team, the expert panel members established rigorous criteria to review and evaluate prevention programs. The criteria were as follows:

• The program reports relevant evidence of efficacy/ effectiveness based on a methodologically sound evaluation;
  
  
• The program's goals with respect to changing behavior and/or risk and protective factors are clear and appropriate for the intended population and setting;
  
  
• The rationale underlying the program is clearly stated, and the program's content and processes are aligned with its goals;
  
  
• The program's content takes into consideration the characteristics of the intended population and setting and the needs implied by these characteristics;
  
  
• The program implementation process effectively engages the intended population;
  
  
• The program is integrated into schools' educational missions; and
  
  
• The program provides necessary information and guidance for replication in other appropriate settings.
  
  

In order to be designated as "exemplary," the panel decided that a program must have had at least one evaluation demonstrating its effect on substance abuse, violent behavior, or other conduct problems, and receive "adequate" or "strong"ratings on the other criteria.

In order to be designated as "promising," a program must have had at least one evaluation demonstrating its effect on substance abuse, violent behavior, or other conduct problems OR one or more risk or protective factors that are major predictors of these problems. Promising programs must also receive at least "minimally acceptable" or "adequate" ratings on other criteria.

"Our panel is aware that there are many great, innovative prevention strategies being employed by schools all across the country that don/t as yet have the research behind them to be identified as promising or exemplary," said attorney Mary Jo McGrath, chair of the expert panel. "We are committed to encouraging those that donÕt yet have the evaluation evidence to work toward having that happen and giving them a better understanding of what the standards of efficacy might look like."

With the criteria in place, 132 programs applied for voluntary review. Applicants provided descriptions of their programs, information on their evaluation methods, and evidence that their programs met their intended goals.

Reviewers with expertise in research and evaluation conducted an initial review to assess applicants' evidence of effectiveness. Applications were then reviewed by a second panel for program quality, usefulness to others, and educational significance. Reviewers' ratings were forwarded to the expert panel for final assessment. Through this careful process, the expert panel identified nine exemplary and 33 promising programs that promote healthy students and safe, disciplined, and drug-free schools. (See article in this issue, "What Can Schools Do With the Expert Panel Findings?")

A detailed report summarizing the 42 programs will be published soon and information about the publication's availability will be posted on the SDFS Web site. The Safe and Drug-Free Schools Program plans to use the lessons learned from the expert panel process to develop an ongoing system for identifying effective strategies, approaches, and programs for strengthening the health and well-being of all youth.