RtI: When Things Don't Work as You Expected

For a great example of why we should proceed with caution when it comes to intervention, read "WHY THE NEW ZEALAND NATIONAL LITERACY STRATEGY HAS FAILED AND WHAT CAN BE DONE ABOUT IT. Evidence from the Progress in International Reading Literacy Study (PIRLS) 2011 and Reading Recovery Monitoring Reports", Massey University Institute of Education, July 2013. It's quite an indictment of Reading Recovery.

11/10/15

Dr. Shanahan - I agree that comprehension, as a whole, is more complex than can be fully captured by CBM, but I do think - like many things - that comprehension does involve a series of more discrete skills, even if there is a "whole" that's more than the sum of its parts. I think we can at least "get at" some of those skills through CBM, even if they aren't 1-minute probes. Still, to you point, I'm not one to suggest that DIBELS/CBM can be everything to everyone, so we should broaden our assessment base.

Second, and this gets back to your first point about using DIBELS/CBM to assess progress: you're right in that the SEM may be higher than weekly ROI scores, but ORF scores were never meant to be interpreted on a weekly basis by looking at one data point and comparing that single data point to baseline scores. ORF scores are meant to be analyzing as most interrupted time series data are - as trends (mean, slope, overlapping points, etc. - referencing Kazdin 1982/2010 here). This is no different from any other form of assessment in which single data points aren't meant to be analyzed in isolation, but are aggregated and analyzed as a sample (e.g., mean of data points).

11/11/2015

Dr Shanahan....it would seem that RTI, in general, has been overblown or teachers simply are not effective reading instructors. The data is pretty clear on the lack of progress for our country on reading achievement.

My question is...how can we help teachers to teach for comprehension? Are we not asking our kids to engage deeply with the texts? Do students not recognize "what you reads defines how you read?"
11/11/2015

Thank you for your article. I have long had a problem with RTI. There should be A specific amount of time should be allotted for trying A specific program, which after 6 weeks it should be clear if it is working or not. I do feel it is important to test before and after the 6 weeks. As a parent with a dyslexic child I know Reading Recovery does not help at all. I had to argue and provide research to the so-called experts, that this was an inappropriate technique. Further they kept my child, and other children who struggle with literacy for not just one year or two years before they consider the parents assertion that their child appears to have a potential learning disability. It is educational malpractice, & a clear injustice 200, families and society in such a way. RT I should go no longer then 3 months, whereupon they should do a test and if progress is not being made then they should give an educational test to test for suspected learning disability. I would like to look for your PowerPoint and share with others. Thank you for your great assessment

11/11/2015

Decoding Dyslexia--

Thanks for your letter. I'm sorry to hear about your child's difficulty. On the basis of the research, I disagree with you about Reading Recovery. It has been found to be an effective, but expensive, approach to teaching reading. However, that does not mean that program (or any other research-based program) necessarily works with everybody (in fact, the research on RR shows that as well--that's why so many kids are discontinued). To know if a program is "working" or not takes longer than 6 weeks. Children's normal growth rates are too slow and current tests are too unreliable to determine with any reliability whether a child is learning in that time from a program. Three months is a more reasonable time frame. You are correct that RtI would delay identifying kids as learning disabled--it takes longer to see if a child is responding to teaching than to just test him/her.

11/12/15

Rob--

I'm sorry to say that far too few students are asked to engage deeply with texts. Reading comprehension instruction is often superficial... often kids are asked to read texts not worth reading and what is done with those texts may not focus much on meaning beyond getting the surface meaning. There is a lot of comprehension skills instruction--which means the reading routines are not fit to the texts, but are seen as transferrable to all texts. There is not a big emphasis on developing knowledge from reading and there is not a big emphasis on teaching students to handle the language demands of text. I've been conducting a study of nearly 1000 classrooms (preK-grade 3) nationwide and it is surprising to me how rare these things are.

tim

11/12/15

Here's a link to a letter by over 30 researchers outlining the weaknesses of Reading Recovery.

"In this open letter, more than 30 international reading researchers expressed concerns about the continued use of Reading Recovery. These experts urged policy makers, educational leaders, researchers, and federal research organizations to acknowledge the weaknesses of Reading Recovery.

They concluded, "Reading Recovery leaves too many students behind." - See more at: http://www.wrightslaw.com/info/read.rr.ltr.experts.htm#sthash.cAPhT6C9.dpuf

11/12/15

Harriet--

And many of the signers of that document are dear friends... and, yet, I will always believe the research rather than the researchers. The research that I have formally and publicly reviewed (I was the one who first identified many of the problems that document complains about) and that the What Works Clearinghouse has reviewed. Again, Reading Recovery clearly delivers a learning advantage to kids; but it is an expensive approach to accomplishing that and it doesn't meet the needs of all young struggling readers. Like my colleagues who signed onto that document, I believe that more explicit teaching of decoding will move kids along more efficiently and powerfully than the ways RR has handled that (Marie Clay originally created RR to address the needs of kids who were struggling despite receiving daily explicit decoding instruction); it doesn't seem to work as effectively when it isn't addressing a gap in kids' learning experience. In situations where kids aren't getting sufficient amounts of quality decoding instruction, RR should be adjusting to meet that need.

11/12/15

Yes, I think your point about "quality decoding instruction" goes to the heart of the matter. Our district has just purchased Leveled Literacy Intervention (LLI)by Fountas and Pinnell, and I recently went through two days of training. Fountas and Pinnell base much of their work on the work of Marie Clay, and in this training it was clear to me exactly how both LLI and Reading Recovery "leave too many students behind". There is far too much emphasis on guessing and not enough on explicitly teaching students how to decode so that as the years progress they can attack all words, including multi-syllabic ones. We know that our weakest readers are the ones who guess, so this isn't a strategy we should be promoting. I applaud the emphasis that both LLI and Reading Recovery have on reading connected text rather than doing "drill and kill" with isolated phonics worksheets, but I can see why New Zealand has not had the results Reading Recovery promised because the program doesn't address the needs of the seriously struggling reader, the one who simply can't read the words. Not to beat a dead horse, but this is why I only focused on sounds in my kindergarten class--I know how crucial it is that kids learn to decode quickly and efficiently and I don't want to do anything to impede that progress. Thanks for your reply! 11/12/15

Dr. Shanahan, have you had a chance to review the research cited in WHY THE NEW ZEALAND NATIONAL LITERACY STRATEGY HAS FAILED AND WHAT CAN BE DONE ABOUT IT. Evidence from the Progress in International Reading Literacy Study (PIRLS) 2011 and Reading Recovery Monitoring Reports, Massey University Institute of Education, July 2013. If so, I would appreciate your thoughts, which may be similar to your response to the open letter. You can imagine how frustrating it is for those of us in the trenches to receive messages from dueling researchers because we really are trying to use best practices based on the current research, but so often we rely on what researchers tell us because we simply don't have time to examine the research for ourselves. Thank you!

11/12/15

Dr. Shanahan - As always thanks for your willingness to engage. First, I'd love a link or other direction to a good text/source of info for current thoughts on reading comprehension in the field along the lines of what you're discussing, particularly when you say that there is no measurable difference in performance across question types - when you say this, are you saying that research suggests that kids tend to do equally bad or good across all question types - from fact recall ("where did the cat go after leaving the house") to higher level inferential questions? If so, you're suggesting that the power or engine that enables kids to answer ALL forms of comprehension questions comes from a more singular, whole understanding of the text?

My experience with comprehension is that it can be broken down into different skill categories, and can't be summed into a singular whole. My experience, and understanding, is that comprehension involves everything from activation of working memory to language comprehension, background knowledge, text engagement (e.g., note-taking, rereading), etc. The target of comprehension could also vary widely, from "comprehending" basic facts of a text to being able to utilize information from the text in external applications. "Comprehension" as a term is sort of like the term "learning" - it means different things depending on the task demand.

So, my sense is that we may be both right - that comprehension can't be fully captured by an assessment of tasks such as factual sequencing and identification of main idea, but that comprehension does involve definable, observable tasks on at least a basic level. Because of that, we can (and should) attempt to get at the skills we can, while certainly continuing to use evidence-based instruction to teach broader comprehension.

Still, would love links to comprehension research you're referencing - always open to changing my mind.

11/13/15

Great Post! I was part of the four original RTI projects where we learned that in urban poverty schools, by 2nd grade more than 50% of students are already Tier 3. Research indicates students that far behind grade level need intense intervention designed for struggling readers. Many RTI educators decided to do kinder "roll ups." Older students were left in the dust. We learned that an RTI paradigm change required the type of explicit language, spelling, vocabulary, & comprehension instruction that Anita Archer writes about. Unless RTI is a dramatic paradigm change, where Tier 3 students are NOT put in humiliating situations reading text at a frustration level, there is no hope of change.
District politics & reading directors worked behind the scenes to sabotage explicit phonics-based RTI projects. Some of the first RTI programs ended as researchers struggled in large districts with reading wars politics. After our RTI kinder students outscored other district students except on the one test Illinois used to test guessing, a district reading director yelled at me in the hall, "This is the saddest thing I've ever seen-how will your kids ever read if they aren't good guessers?" Our kinders, rather than read predictable sentences such as""This is an elephant" tried to sound out the words for the animals. Since it's so difficult to break older students' guessing, we were glad that our kids were using decoding skills. I assured her by the end of 1st grade all of those words would not only be decodable, but would be processed as whole words or word parts.
Originally, we weren't aware that "more isn't always better" but Tier 3 readers didn't go back to classes to receive instruction in Reading Recovery with guessing. However, when the progress stopped of our 2nd grade Tier 3 readers who had been in Tier 3 since kindergarten but were on track to finish the year at a high Tier 2 level, we stayed in that school all day every day to figure out what was happening. Had reading become so difficult they needed more support? On the third day, we learned that a new after school teacher was having all kids read leveled books. These kids still hadn't learned all their phonics skills &resorted to guessing. The guessing bled into reading at other times. We got the after school teacher to read the books & teach vocabulary and comprehension. Then student scores began to increase & they finished the year at a high Tier 2 level. A year later many were Tier 1 readers. Once these students were in 3rd grade, they were ready for the leveled reading that had led to the 2nd grade guessing.
What is critical for a successful RTI program? *An intensive, explicit 90 minutes a day Tier 3 curriculum with proven results for "turn around" for everyone who tests at Tier 3 *A Principal who learns the program, pushes reluctant teachers & honors teachers getting results. *NO FRUSTRATION LEVEL READING where students are humiliated & guess. *Explicit vocabulary, spelling, and comprehension *Mastery, not exposure with core knowledge concepts needed to comprehend higher level text. *A reading coach *NEVER cancelling or reducing reading time. More is related in "Teaching Reading to Students Who Are At Risk or Have Disabilities: A Multi-Tier, RTI Approach." Although our projects were in elem schools, some of the YES PREP charter schools in Houston have successfully used this type of RTI model in 6th-8th grade classrooms demonstrating that it is never too late for struggling readers.

11/15/15

Mary--

Happy to hear things are going well for your students. However, it is important to think about what it means that an RtI program is "working." There are several that are appropriate: (1) since there would be greater attention and emphasis to addressing and preventing reading problems in the classroom--Tier 1--then there should be higher average reading achievement for the schools; (2) since struggling readers would not just be shuttled into Tier 3/Special Education classes, they would receive Tier 2 remedial teaching--students who receive such support would do better than similar kids who are not treated/taught with the Tier 2 intervention, and fewer students should end up in Special Education. Finally, (3) the students who receive Special Education/Tier 3 teaching would do better than such students have in past years (special education kids have usually not done particularly well). To say something worked would require convincing data that 1, 2, or 3 of those goals were actually accomplished beyond chance.

11/16/15

Harriet--

I definitely understand that it is confusing to see that a program works--that is confers a learning advantage on children who are taught with it... and that the same program either overstates its effectiveness (as RR has done since the beginning) or that its use has not led to the overall outcomes intended (such as that RR may have good success with the kids who are taught with it, but that it is not impacting overall school literacy rates in any meaningful way). RR works... kids who receive such instruction do better than control group subjects who do not... however, RR does not work as well as its proponents claim, since they have often failed to consider the performance of students who were dropped from the program for not doing well... etc

11/16/15