Is There Research on that Intervention?

15 January, 2014
0 Comments

Teacher question:

I am a reading specialist working in an urban school district with struggling readers in K-5. Do you have any suggestions on intervention programs that you find the most beneficial to students? Currently, we are using LLI (Fountas and Pinnell), Sonday, Read Naturally and Soar to Success, at the interventionist's discretion. Is there any research supporting or refuting these programs? Is there another program that you find more effective? We also use Fast Forward and Lexia as computer-based interventions. What does the research say about these tools?

Shanahan response:

The best place to get this kind of information is the What Works Clearinghouse (WWC). This is a kind of Consumer Reports for educators that will tell you if commercial products have been studied and how they did. The benefit to you is that all the information is in one place, it is being provided by the U.S. Department of Education so it won’t be biased towards some company, and they vet the research studies to make sure the information is sound.

What Works Clearinghouse Reviews of Intervention Studies

Some things to be aware of when you seek this information:

Don’t read too much into the fact that there is no evidence on a program. This happens a lot. Instructional programs aren’t like drugs; no one is required to prove that they work before they can be sold. While some companies do commission studies of their products, most do not. The key thing to remember is that a lack of research on a product does not mean that product doesn’t work. In such cases, I usually look to see if a product is as thorough or demanding as those that do have evidence.

Don’t overestimate programs that do have direct research support. Programs do not have automatic effects. A positive result tells you that this program can work under some conditions and with some students. It means that in those circumstances this program did better than… whatever the control or comparison group did. It is good to know that someone was able to get a positive result with such a program (that should help teacher confidence), but often a program that works may not work in your circumstances or with your teachers or with your students. Just because something worked, that doesn’t mean that you could make it work.

A basic ethical obligation of a researcher is to report the results of their studies, even when the studies don’t come out the way they wanted. Commercial companies don’t have this same obligation. What that means is that if a company commissions a study and it gets a positive result, they will allow it to be released; but that isn’t necessarily true when the results don’t show that their product worked. That means available research on a particular program or product may be overestimating the impact. (That’s one of the reasons that I like that WWC is so strong on the evidence: they can’t know about studies that got lost in a file drawer, but they can certainly make sure the available studies meet the highest evaluation standards).

Pay attention to the control group. In medicine, there are standards of care. Typically, a new treatment is compared with the standard of care so that you know that if it “worked” it would be better than what you are already doing. In education, we have no shared standards of instruction, so you need to pay attention to what the intervention did better than. It might have done better than what you are already delivering, and that would certainly encourage you to change programs, but it might be doing better than instruction that you, too, are already outperforming.

Comments

See what others have to say about this topic.