Designing a Cost-Effective Field Trial for an Intervention in a Messy System With Multiple, Interacting Variables
Soon after the 4th of July, 2002, and for the past six years, fifteen to twenty adolescents have come to the Chemistry Department at Hunter College to take an intensive, six week chemistry course culminating in a three hour state exam, the Chemistry Regents. All students had taken the exam in June and had failed it. Somehow they had word about the Hunter College course, which asked them to come to school four days a week for seven hours a day. The course promised success on the Regents exam - something that had eluded the students before. In a city that has an annualized 35% passing rate on the chemistry Regents exam and a 6% passing rate on the exam in August, the small summer school at Hunter College boasted 80-100% passing each year on the August exam.
Based, in part, on the summer success, the MSPinNYC was funded to scale up the summer experience to mathematics and biology and to find the parameters critical to its summer success that could be transported into the normal academic year. Working strictly with NYC under-performing schools in high poverty districts, the MSPinNYC went to work reproducing the chemistry results in biology and mathematics. For the past three summers we have more than quadrupled the standard summer school performances. Clearly something in the summer works and works with low achieving kids. But the summer was filled with multiple variables known to contribute to student success - collaborative teaching, peer and undergraduate tutoring, student-centered instruction, practice exams, inquiry labs, student-teacher dialog, and an overt and meaningful commitment to an environment in which all kids were supported (by teachers, tutors, and all instructional staff) in their quest to succeed.
While the MSPinNYC had a successful summer program it was too expensive and complex to transfer directly to the schools. With multiple variables that could have confounding effects, how could the essential factors be identified? Would it be possible (scientifically and financially) to identify cause and effects using a field trial methodology?
