Improving Achievement... Is It the Tests or the Teaching?

Teacher question:

We are trying to figure out how to help our grade 2 students. Currently, we see a big gap in the percentage of students who are meeting standards in Grade K and 1 compared to the same student results in Grade 2 (more than 90% in Grades K and 1 but only 55% in grade 2) In our assessment. We allow students to have questions read in Grade K but not at the end of grade 1. Why might we see this trend consistently across cohorts? Would you be able to recommend 1 or 2 strategies that we could implement as a district (~74,000 students) to influence these data? 

Shanahan response:

Your question seems to be based on the premise that the problem is in the testing. And, that very well may be.

Scaling texts, questions, and tasks so that a second-grade test is the equivalent of a first-grade test one is a lot of work—work that few school districts usually do. It involves calculating local norms and scaling items across years so that cut scores can be set. That gets so involved that I’m curious about your purpose in doing this. If you are just trying to make sure that grading is consistent or that decisions about retention or placements are equivalent that is what it would.

The more usual purpose is that a district is trying to identify kids who are struggling so that they are on track to do well on an accountability test in Grade 3 and above. In those cases, it is important to follow kids across grade levels to determine what levels of performance are needed in the lower grades to assure later success. If you find, for instance, that 90% of the kids who are meeting standards in grade 3, had scored 75% or higher on the first-grade assessment, then that may be a good cut score to set.  

I would not at all be surprised to find out that your tests aren’t equivalent. That might be fixable by changing an administrative procedure (like whether the teacher will read directions or questions aloud or whether kids are expected to do that themselves). Test passages are more likely the issue if it is a comprehension test—how well do text selections represent different increments of performance… are the second-grade texts as much harder than the first-grade texts as the first-grade texts are above the kindergarten texts?

You definitely could spend a lot of time and resources calculating norms, adjusting tasks, and conducting cut-score determination studies to make sure that the tests are telling you what you want to know.

But you might be wrong about all of that… maybe it isn’t actually a testing problem. Maybe your tests are more equivalent than you think.

Maybe it’s the teaching.

Maybe your second-graders really aren’t doing as well as the younger kids. A lot of districts make a big deal out of giving kids a good start (e.g., beginning reading programs for phonemic awareness and phonics, early interventions like Reading Recovery, efforts to place the best reading teachers in those grades). Second-grade, not so much.

It may be wise to task (and fund) someone in your district with developing better norms and cut scores over the next couple of years. But even if you go that way, I’d strongly encourage you to hedge your bets. While resolve your testing problem—making sure that your assessment plan provides what you really want, whatever that might be—it would be sensible to try to improve achievement. Maybe 45% of your second-graders really aren’t reading well enough.

If you want to do that there are only three things you can do to improve students’ reading achievement:

1.  Increase the amount of literacy instruction and experience that your children receive.

How much reading and writing instruction do your second-graders receive?

Many schools budget a 90-minute block for reading instruction in grade 2. Is that enough? Probably not, given your results. Not all schools necessarily have to spend the same amount of time on reading instruction. Some kids are more challenged than others. I would definitely spend more than this if I were teaching kids with disadvantaged backgrounds, or second-language learners, or kids with certain disabilities. I’d spend more time because it takes them more time to get to the levels they will need.

Even when schools schedule enough time for reading instruction, teachers may not use that time as well as they could. The teacher might be reducing this instructional time by using it to read to kids or to give kids a period of time to read on their own or by just by inefficient teaching (like teaching the same lesson over and over to small groups of children).

I’m not saying that teachers shouldn’t read to kids—I would—but it shouldn’t be considered to be “reading instruction,” since we have no research that it improves reading performance for kids at this grade level. I’m not a big fan of having kids read on their own (without close involvement of teachers whose guidance can increase the amount of learning), but I wouldn’t ban it; I just wouldn’t count it as part of the 90 minutes (same reasoning here—the learning payoffs are just too small to go this way).

One of the things I don’t like about the 90-minute reading block is that it is not a 120-minute reading and writing commitment. Kids can learn a lot about reading from writing instruction and omitting writing can undermine your reading instruction efforts.

In my schools, I’ve argued for 120-180 minutes per day of reading and writing instruction. However much time you agree upon for that, I’d encourage your teachers to try to expand beyond its boundaries, by having the kids reading as part of their science and social studies classes (and by that I don't mean round-robin reading).

2. Make sure instruction focuses on teaching those things that improve children’s reading achievement.

Research has identified a number of abilities that can be taught to second-graders that can improve the children’s reading ability.

I think you need to ask yourselves the following questions:

            Are your teachers teaching decoding?

            Are your teachers teaching oral reading fluency?

            Are your teachers teaching reading comprehension?

            Are your teachers teaching vocabulary?

            Are your teachers teaching writing?

Second-graders should receive substantial amounts of instruction in each of these areas. Again, I have usually argued for devoting roughly equal amounts of instruction for decoding, oral reading fluency, comprehension/vocabulary, and writing. If that isn’t happening—if your teachers are minimalizing some of these because of their philosophies or beliefs, or because the district hasn’t adequately supported them—then I would focus district efforts on correcting these oversights.

Often teachers and principals will downplay one or another of these because they don’t match well with the tests that are to be given. If your reading test doesn’t include decoding, then perhaps they de-emphasize such teaching. The same thing happens with oral reading fluency; “if second-graders are going to be tested on silent reading then why teach oral reading?” the thinking goes. And, that can be the reason why writing gets elbowed out the picture, too. (All of those choices may serve to lower reading achievement.)

3. Make sure the instructional quality is high—that means efficient and effective.

There are many general points that can be made about quality. For instance, studies have shown how critical it is that lesson have clear purposes—purposes that both the children and the teachers are conscious of (that way kids can try to learn, and teachers can be more diagnostic, noticing whether the purposes are being accomplished). Another general example would emphasize the importance of being motivational or encouraging.

There are also quality issues that are specific to each of the instructional components noted above. Teachers may claim, for example, that they are teaching oral reading fluency, but what they actually mean is that they are engaging kids in round robin reading (taking turns reading aloud short segments of text). But that approach usually means children aren’t working with texts that are sufficiently difficult, that no one gets much reading practice, that there is no rereading to try to improve performance.

Similar points could be made for all of the areas noted above. How well are they teaching reading comprehension or decoding or vocabulary?

A careful review of current second-grade instructional practices might suggest actions that the second-grade teachers can take. However, there are things that the schools should be doing as well. For example, is student learning being monitored in the areas noted above, and if so, is extra instruction available to the kids who are not performing well? What are the schools doing to involve parents in their kids’ progress (getting them to listen to their kids read can be a big help, for example)?

I’m certainly not arguing against improving the accuracy of your testing scheme, but I would take this opportunity to give all of your students the best chance to become successful readers. That is more likely to come from increasing the amount of teaching, focusing that teaching on key areas of literacy development, and improving on the quality of that teaching.

Good luck.