Yes, Virginia, You Can DIBEL Too Much!

  • DIBELS Assessment
  • 20 August, 2009
  • 7 Comments

I visited schools yesterday that used to DIBEL. You know what I mean, the teachers used to give kids the DIBELS assessments to determine how they were doing in fluency, decoding, and phonemic awareness. DIBELS has been controversial among some reading experts, but I’ve always been supportive of such measures (including PALS, TPRI, Ames-web, etc.). I like that they can be given quickly to provide a snapshot of where kids are.

I was disappointed that they dropped the tests and asked why. “Too much time,” they told me, and when I heard their story I could see why. This was a district that like the idea of such testing, but their consultants had pressured them into repeating it every week for at risk kids. I guess the consultants were trying to be rigorous, but eventually the schools gave up on it altogether.

The problem isn’t the test, but the silly testing policies. Too many schools are doing weekly or biweekly testing and it just doesn’t make any sense. It’s as foolish as checking your stock portfolio everyday or climbing on the scale daily during a diet. Experts in those fields understand that too much assessment can do harm, so they advise against it.

Frequent testing is misleading and it leads to bad decisions. Investment gurus, for example, suggest that you look at your portfolio only every few months. Too many investors look at a day’s stock losses and sell in a panic, because they don’t understand that such losses happen often—and that long term such losses mean nothing. 

  The same kind of thing happens with dieting. You weigh yourself and see that you’re down 2 pounds, so what the heck, you can afford to eat that slice of chocolate cake. But your weight varies through the day as you work through the nutrition cycle (you don’t weigh 130, but someplace between 127 and 133). So, when your weight jumps from 130 to 128, you think “bring on the desert” when you real weight hasn't actually changed since yesterday.

And the same kind of thing happens with DIBELS. Researchers investigated the standard error of measurement (SEM) of tests like DIBELS (Poncy, Skinner, & Axtell, 2005 in the Journal of Psychoeducational Measurement) and found standard errors of 4 to 18 points with oral reading fluency. That’s the amount that the test scores jump around.

They found that you could reduce the standard error by testing with multiple passages (something that DIBELS recommends, but most schools ignore). But, testing with multiple passages only got the SEM down to 4 to 12 points.

  What does that mean? Well, for example, second graders improve in words correct per minute (WCPM) in oral reading about 1 word per week. That means it would take 4 to 12 weeks of average growth for the youngster to improve more than a standard error of measurement.

If you test Bobby at the beginning of second grade and he gets a 65 wcpm in oral reading, then you test him a week later and he has a 70, has his score improved? That looks like a lot of growth, but it is within a standard error so it may just be test noise. If you test him again in week 3, he might get a 68, and week 4 he could reach 70 again, and so on. Has his reading improved, declined, or stagnated? Frankly, you can’t tell in this time frame because on average a second grader will improve about 3 words in that time, but the test doesn’t have the precision to identify reliably a 3-point gain. The scores could be changing because of Bobby’s learning, or because of the imprecision of the measurement. You simply can't tell.

Stop the madness. Let’s wait 3 or 4 months, still a little quick, perhaps, but since we use multiple passages to estimate reading levels ,it is probably is okay. In that time frame, Bobby should gain about 12-16 words correct per minute if everything is on track. If the new testing reveals gains that are much lower than that, then we can be sure there is a problem, and we can make some adjustment to instruction. Testing more often can’t help, but it might hurt!

Comments

See what others have to say about this topic.

Angela M.
Jul 02, 2017 01:05 AM

12/7/2012

Hi Dr. Shanahan,

Would you recommend using DIBELS for 5th and 6th graders as a way to benchmark remedial reading students? What would you do for 7th graders throughout the year to monitor growth?

Timothy Shanahan
Jul 02, 2017 01:05 AM

12/17/2012

Let's put it this way, I would definitely monitor students' oral reading fluency up to the point where they can read at an 8th grade level (no matter what their age). Fluency explains about 80% of the variation in reading comprehension in grade 2 (what this means is that if we could make all students fluent, most reading comprehension deficits would disappear). However, fluency diminishes in its importance over the years, mainly because increasing numbers of students reach the necessary levels of fluency to enable comprehension each year. By eighth grade, fluency only explains 25% of the variation in comprehension (thus, even if we made all students fluent at this level, we would still have big comprehension differences). There are various tests of fluency for students in this age group (including DIBELS) and I would use any that provides me with a 2 or 3 minute analysis of student oral reading that considers both accuracy and speed (less than that and you don't get a complete picture of student performance in this area).

M Ruby
Jul 02, 2017 01:06 AM

1/12/15

Tim,
When we look at average growth rates for ORF (increase in cwpm/week, isn't the underlying assumption that the charted data are for students receiving typical classroom instruction?
For children who are significantly below the norm, or considered Tier 3 students, and we assume they are getting evidence-based instruction - more of it - with the intent of closing the gap - can we agree that we are doings onto increase the gain in cwpm/week? If not, how do we close the gap? What is the point of Tier 3 intervention? If we are trying to increase the student's trajectory of growth, wouldn't we want to be progress monitoring with greater frequency?
Thank you

MRuby

Timothy Shanahan
Jul 02, 2017 01:06 AM

1/12/2015

My response: Yes, you are trying to accelerate student growth, but that doesn't make your measuring instrument more reliable. If you test that often you are going to be misled by meaningless noise (scores that look like they are going up or down or even just sitting still, despite what is really happening). If the measure can't tell you what is happening, you don't want to react instructionally to it.

Harriett
Jul 02, 2017 01:07 AM

5/5/2016

Tim, our school district is evaluating its k-6 assessments and has asked for input from teachers. A neighboring district uses DIBELS for its kindergarten assessments. Do you recommend DIBELS for kindergarten with testing every 12 weeks? If not, which kindergarten assessments do you recommend? Thanks!

Timothy Shanahan
Jul 02, 2017 01:07 AM

5/5/2015

Harriet--

I like DIBELS when used appropriately, so I have no problem with that recommendation. Some of my colleagues prefer PALS (and I don't have any objection to that either). The 12 week time span seems reasonable at kindergarten with these skills.

My one concern with these kinds of efforts is that they are able to evaluate some abilities and not others. Thus, you may be testing kids' knowledge of letter names, phonemic awareness, and decoding (all important to monitor in kindergarten), and that will allow you to channel extra instruction to those who struggle in those areas (and that is good), but you will not, from that kind of testing, know whether kids are lagging in language (listening comprehension, vocabulary, syntax, etc.). There are no good quick screeners of those, so I suggest that if some kids are lagging in the tested skills, there is a good chance they will be lagging in language, too, so extra help might be provided to such children not just in the tested skills but in those more complex ones too.

That might seem counterintuitive, but what we see once kids have been caught up in those tested skills, is that they still lag in reading because they are struggling with the untested (and thus, untaught) skills. Your district is going in a good direction with DIBELS or something like it, but you need to not just look where the light is good (the easily monitored skills), but also with those that you will not be able to test like that.

good luck.

Harriet
Jul 02, 2017 01:07 AM

5/5/2016

Very helpful--thank you!

What Are your thoughts?

Leave me a comment and I would like to have a discussion with you!

Comment *
Name*
Email*
Website
Comments

Yes, Virginia, You Can DIBEL Too Much!

7 comments

One of the world’s premier literacy educators.

He studies reading and writing across all ages and abilities. Feel free to contact him.