Making Decisions about Which Intervention is Best: A Case Study

Teacher question:

I wonder if you could comment on your blog about this crazy idea that the reading specialists should change the program every 12 weeks if a student is not showing growth on the one-minute reading fluency measure. I have second grade student who reads 80 wcpm with 97% accuracy. She made great growth in the fall but has leveled out this winter. She is being removed her from my “program” to Wilson because an outside evaluator said that is what she needs. What do you think?

Shanahan response:

One thing is clear: No matter how I answer this question, somebody is going to be mad at me.

That’s okay with me, I’ll find some way to get out of bed in the morning anyway.

However, the reason these kinds of questions are tricky is because the questioner, from his/her point of view tells me the critical information (critical from his or her point of view)—and the folks on the other side of the argument would have provided me with other information that they think matters.

One time a teacher wrote asking my opinion given the “facts” she had provided. I answered, and she apparently took the answer to the school board. Then the superintendent wrote to me adding some information, and I replied with a somewhat different answer. I’ll do my best to make sense of this question, but don’t be surprised if up the road there is more information that might change my mind.

Should reading specialists change their program every 12 weeks if a student is not showing growth?

I guess that depends on what you mean by “program.”

  If I see that a child is not learning from a given lesson, I’ll make changes to try to correct the problem, adapting my teaching to make sure that it works. That might entail adding explanation or examples or extending the lesson longer. Maybe I’ll come up with a new approach and try the same lesson again tomorrow.

  In this case, you are talking about larger changes than that. If I’ve determined that a young child has a fluency problem and have developed or selected a program to address that need, I’m not going to stop focusing on fluency instruction just because I have a lesson go belly up on me.

  How much time and how much information I’d need to determine whether to make a program change will depend. (By a “program change” I mean moving a youngster from a Title I reading program to a Special Education based intervention; or moving from one commercial program to another; or deciding that a phonics intervention might be better than a fluency-oriented one).

  It will depend on whether I have gained new information about the child that I lacked when the original decision was made. That could be because I’ve worked with the child for a significant amount of time and have seen responses that don’t match with my original decisions or it could be that we collected more test data on the child which changed my mind.

  When it comes to foundational skills decisions with primary grade students… 12 weeks should be a sufficient amount of time on which to make this kind of decision. If a student is not making progress for 12 weeks, I, too, would be concerned and would consider a change. Making program changes of these types once or twice or even three times in a school year is not crazy. (Some folks try to make such decisions more often than that—testing kids weekly, for instance—but that’s foolish, since these tests aren’t refined enough to allow for such fine learning distinctions.)

Should consequential educational decisions be based upon 1-minute oral reading fluency measures?

  If students aren’t making learning progress, it is important to adjust instruction to meet their needs. The idea that kids may be doing poorly now but that they’ll mature and do better later is unwise and has hurt a lot of kids. Kids should not be allowed to languish with no or low learning for extended periods of time.

  However, I’m puzzled about the use of a 1-minute oral reading fluency measure to make this kind of decision. A one-minute oral read won’t provide a reliable estimate of how well a child can read a text fluently. And, if such a test is unreliable, then it is also invalid—since validity depends, in part, on reliability.

  One of the more popular oral reading measures is DIBELS. To estimate kids’ oral reading fluency, DIBELS requires that children perform two one-minute reads; two readings, two passages. That’s where their reliability comes from — from the two minutes of reading performance. The levels of reliability that DIBELS and DIBELS-like testing procedures can obtain are probably sufficient for many instructional decisions.

  In 2001, Sheila Valencia and her colleagues put these kinds of testing procedures to the test. They found big problems with one-minute reads. Significantly more reliable and valid outcomes on such tests required three-minutes of oral reading.

  So, I’m willing to test kids every few months to determine if their foundational reading skills are improving and to make instructional changes if they are not.

  But I wouldn’t make such changes on the basis of a one-minute oral reading. Malpractice!

Should second-graders be assigned to special phonics instruction based upon an oral reading fluency measure?

Typically, one would not make consequential reading decisions based upon any single measure. If I were sending kids to special education or a reading specialist I would want to be both certain that the child was experiencing difficulty in reading and I would want to have some idea of the pattern of reading skills and abilities the child had so we could make sure he/she would learn.

  To accomplish that, one would usually need more than a single measure (even if that were a reliable measure).

  More specifically, if a youngster were struggling with oral reading fluency (as measured by a reliable, valid measure), I would not just assume that the student had a fluency problem requiring explicit fluency instruction—and that is especially true with young students.

  I would, in such a case, want to know about that student’s ability to decode. Some kids with fluency problems really struggle with phonics and their low words correct per minute rates may be due to phonics difficulties. Other kids who don’t read well orally may be pretty accomplished when it comes to phonics (see work by Carol Chomsky, for instance). I would need to know more than that reading accuracy and speed were low before I could determine a sound instructional response.

  If someone assigned students to the Wilson phonics program because of an oral reading fluency test alone, I’d have concerns. Don’t misunderstand, I like the Wilson phonics program and I use oral reading measures. Those aren’t the problems here. The only issue should be about what approach would most likely benefit the student, and even a reliable oral reading fluency test on its own would not be adequate to make such a decision. I’d certainly want to know about this student’s decoding skills (and comprehension, too).

  You evidently don’t agree with the decision made for this student. From my vantage, I can’t tell if it was a good or a bad decision, but if were made on the basis of the information that you provided, there is a very good chance that they have erred.

What would you do with a student who performed at this reading level?

I don’t have a clue.

  I’m very puzzled about the information you provided, and my puzzlement is part of my skepticism.

  You say this student reads at 80 wcpm in the winter of second grade. While I don’t trust that assessment, for the reasons, given above, if I take this score at face value, it would mean that this student is reading at a rate and accuracy only slightly below what an average second-grader would be expected to obtain (Hasbrouck & Tindal, 2016).

  I wouldn’t put an average reader in a special intervention, no matter which department was providing it.

  I wouldn’t put a second-grader with average reading scores in a special phonics program, especially this late in the year. I wouldn’t expect that to improve his/her reading accuracy or rate and if phonics instruction isn’t going to boost those, it certainly won’t facilitate comprehension.

  If a student reads fluently (and a mid-year second-grader who can cold read a second-grade text with 80wcpm is reading fluently), then my concerns would be with reading comprehension and vocabulary instruction, not phonics.