Should We Administer Weekly Tests Linked to Standards?

  • 23 February, 2019
  • 22 Comments

Teacher’s question:

My district instituted a weekly "checkpoint" (a short passage and multiple-choice assessment aligned to our standardized test). Teachers are required to give this, and then break it down by standard in a meeting with a coach. I've argued that these tests are likely not measuring what they think they are. They believe that these can tell teachers whether students are mastering certain standards and questions. We have a large proportion of students below grade level. I'm concerned that valuable teaching time, focusing on working with complex texts, is going to be spent on testing, and that the nature of the assessments will lead to skills-focused teaching that won't result in better readers. I've been told that "teachers need something" to know how their kids are doing, and this is what strong districts do? They've asked what I would suggest. How would you answer my admin's question about how best to know whether teaching is resulting in learning, particularly for less-experienced teachers?

Shanahan’s response:

Man, if I had a nickel….

I believe that your assessment of the situation is spot on.

It is not possible to reliably or validly assess those individual reading comprehension standards. That’s why the multi-billion-dollar testing companies that are capable of doing amazing things don’t even pretend to do that. With the approach that you describe kids get less instructional time (to accommodate all of the unnecessary testing), and the testing can’t possibly reveal anything specific that the teachers need to know to improve or shape the intensity or quality of their instruction.

The kids’ ability to answer the questions will be due more to the difficulty levels of the texts they are asked to read in the assessments than to the types of questions on the test… that’s why research repeatedly finds that reading comprehension tests measure a single factor—not all the individual factors that the questions or the standards supposedly represent.

This scheme is a time waster. It serves to make administrators feel good because they feel like they are taking positive action and looking rigorous…. But think about it. The most effective doctors aren’t the ones that prescribe placebos! And, that is just what this approach is; it is a sugar pill that will make you think you are really doing something—but, remember, it is just a sugar pill. It has no therapeutic value. (I find the statement that this is what “strong districts do” to be stunning. I assure you that it is not what those districts have high reading achievement.)

Kids’ ability to answer the questions will likely be due to how well they are able to read the particular texts (and the degree of prior knowledge they might have on the topics addressed in those texts). That means such testing should be done less often and should try to identify the difficulty levels of the texts that kids can and can’t handle, rather than on whether they could answer particular kinds of questions.

If administrators don’t believe this, they should look at their own data to see how reliably the kids perform week-to-week on each item type. If something valid was being measured reliably, then those scores should be pretty consistent—main idea ability or key ideas and details ability shouldn’t bounce up and down. They also might want to make readability estimates of the texts that they use and compare these with how the kids perform on the various sets of questions. What they are likely to see there is the same thing that ACT reports with their tests: if the passages are complex, then kids have trouble answering even the most straightforward or supposedly easy questions, and if the passages are relatively easy, they will be able to answer the supposedly hard questions.

I’d suggest that instead of a weekly test, the district provide an assessment 2 or possibly 3 times per year. What you want to test for isn’t which comprehension skills they do well on, but what levels of text they can handle. From that, you can make a pretty good estimate of who will be able to do well on the state assessment. And, you’ll know which kids you most need to stretch in terms of helping them develop the abilities to read those more complex texts.

The coaches should be supporting the teachers’ efforts to teach vocabulary effectively, to develop fluency, to extend kids’ reading stamina, to handle complex sentences and subtle or confusing cohesive links, and to make use of texts’ structures, rather than focusing on teaching kids to answer particular kinds of questions.

Why would I avoid the practices that you describe? Because they don’t work. Because they hurt kids by wasting their educational time. Because they make teachers, principals, and other administrators look stupid—since they don’t improve achievement.

Why would I take the approaches described here? Because they work. Because research shows that they work. Because my own personal experience as a district administrator tells me that they work on scale.

Comments

See what others have to say about this topic.

Eboni
Feb 23, 2019 07:52 AM

I agree but what you're describing can be a hard sell when you have districts that have an abundance of struggling readers and teachers are told that weekly tests help teachers drive their instruction.

And on another note, this is proably an easy answer-an obvious answer-but what is the single factor reading comprehension tests measeure? Even as I wrote the question it sounded like a setup for a punchline to a joke.

Dr. Gwen Lavert
Feb 23, 2019 08:27 AM

I believe that both can be done; ie@
providing text complexity and weekly test.

Chandra S
Feb 23, 2019 12:09 PM

I have to agree with Dr. Shanahan on this one. There are so many other effective assessments to consider than weekly multiple choice assessments that should be used to help improve students’ comprehension. Someone once said, “Schools spend too much time measuring progress and too little time actually making progress.” These weekly tests inevitably take away precious teaching time because of course these teachers who are forced to do this can’t possibly do anything else for the day other than test the kids. (That’s sarcasm, folks.) But it’s what I see all the time. The students who need the most guidance and instruction from experienced teachers somehow end up receiving less and less instructional time due to “data driven” thinking. I’m starting to believe that the data has “driven” people crazy since they choose to ignore clear research and then find ways to molest the data to justify labeling students and relegating some into homogeneous groups of “low” kids who will be drilled and killed by skills based instruction which does nothing to build background knowledge or improve students’ reading stamina. Instead schools should be “value driven and data informed.” What do we actually value as educators? Do we really want students to be better readers? Or just better test takers who hate reading? If the former is what we value, then schools need to do exactly what Dr. Shanahan has prescribed.

Sam Bommarito
Feb 23, 2019 12:21 PM

You said "I’d suggest that instead of a weekly test, the district provide an assessment 2 or possibly 3 times per year. What you want to test for isn’t which comprehension skills they do well on, but what levels of text they can handle. As you said, " SPOT ON!!!! Sadly I worked for a time in a district that did it the other way. Guess what. Brief gain as the kids got used to the test. Long, long long plateau, because the kids weren't learning anything new. Sad. You way is much more sensible.

Debi Varney
Feb 23, 2019 12:52 PM

If you arenassessming all the standards in small guided groups ,with on level texts it isn't necessary to do weekly one size fits all reading assessment. Our below level amd ELL students are suffering and their self esteem plunging when they don't succeed weekly.

Theresa Seits
Feb 23, 2019 01:20 PM

Benchmark assessments given a couple of times a year provide some useful formative data to use for planning purposes. However, I think it's equally important to track the trajectory of students' reading progress as well. As a teacher, I used running records (with comprehension checks) to regularly progress monitor my students' growth over the year (rather than weekly "standards based" assessments). Students also plotted their growth and shared ownership of their progress. I found this to be a pretty accurate indicator of students' growth as a reader. Is this a better use of instructional time for tracking student progress (rather than assessing mastery of standards)?

Deb B
Feb 23, 2019 03:51 PM

What should drive instruction is teachers’ moment by moment formative assessment of the tasks Dr. Shanahan describes above. In those weekly meetings with coaches teachers can bring student work products (writing, text responses and annotation) as evidence of learning to break down and discuss.

Sara
Feb 23, 2019 10:15 PM

If you have lots of weak readers, there is a good chance that you have lots of students who are actually poor decoders. Use reading fluency as a proxy for reading comprehension - 1 minute prompts regularly (maybe every 2 weeks) for weak students (while others are doing purposeful independent reading) - then use fluency admin and results to decide who actually needs decoding work and use assessment to guide decoding instruction (not comprehension). See Willingham re: relatively limited scope of comprehension skills that need to be taught https://www.aft.org/sites/default/files/periodicals/CogSci.pdf You will make much more progress deciding who needs help to decode better, who needs to build richer vocabulary and background knowledge and using quick assessments there to guide your instruction AND have your progress monitoring contribute to progress!

Aileen
Feb 24, 2019 05:29 PM

Dr. Shanahan,
What assessments, looking for specifics here, would you benchmark with 2-3 times a year? Since you mention what level texts they can read, would the Fountas and Pinnell Benchmark Assessment fit what you are suggesting? If not, could you please explain what type you are advocating?
My concern is that our district does use the F&P Benchmark 3 times a year and it is given to the whole school. This testing takes about 2.5 - 3 weeks to get through the whole building. During this time, our intervention reading support stops as it is us reading teachers and teacher assistants who administer the test. It used to be done by classroom teachers, but unfortunately, we found that results varied greatly with certain teachers always having their students moving way up - later finding that the results weren't accurate. So our superintendent decided that all reading teachers and their TAs would administer it for better validity. Sorry, got off track here. But in essence, this assessment takes time.
But is the best assessment we can give 2-3 times a year that would be useful? Look forward to hearing what you suggest and what others think or are doing.

Tim Shanahan
Feb 24, 2019 07:16 PM

Dr. Gwen— you might think testing weekly is wise,but no one has ever raised avgiement that way.

Tim

Tim Shanahan
Feb 24, 2019 07:19 PM

Theresa—

Testing those things frequently in those ways is meaningless too. You are getting no real information from that so wasting the kids’ instructional time is not worth it.

Tim

Tim Shanahan
Feb 24, 2019 07:22 PM

Sara— 1 minute oral reads are too unreliable to provide any useful information every two weeks. DIBELS requires two one minute reads and it’s reliability is so low that it can only provide useful info when given at 3 month intervals and the research suggests that 3 minutes is necessary. You are testing too much. Use that time to teach weak readers.

Tim

Tim Shanahan
Feb 24, 2019 07:26 PM

Aileen—
First, any testing regime that interferes with teaching to that amount is a really bad idea and should be discontinued. with reading comprehension you should be able to test whole classes of kids asking the, to read a text or two and answer questions. Those should give you a pretty good idea who is comprehending particular text levels. Of course with younger kids you’ll want to know about decoding and fluency, too.

Tim

Sati
Feb 24, 2019 10:11 PM

The comment about reading teachers and TAs undertaking F&P BAS instead of classroom teachers because of a concern around reliability has got me thinking. I wonder if focusing on establishing shared understandings and norms amongst classroom teachers in administering and using the assessment would be a more promising next step. When classroom teachers undertake running records with their students, they come away with an understanding of each student's reading behavior(s) that is more nuanced than just a level. This would allow the assessments to be both for progress monitoring and also as a source of information for informing instruction.

Michelle
Feb 25, 2019 02:14 AM

More time reading widely, across many genres and levels and discussing texts with peers and teachers.

Kathe Ball
Feb 25, 2019 01:33 PM

You made a comment about districts helping teachers with teaching difficult vocabulary, increasing fluency, increasing reading stamina, and ways to deal with difficult or complex sentences. Do you have an article or several articles that you would suggest reading that would support teachers working on just those skills?

Izabela Bardy
Feb 26, 2019 04:35 PM

Agreed!

Petra Schatz
Feb 26, 2019 06:51 PM

We have been working to discourage the practice of weekly assessment but one of the problems we are encoutering is teachers are using their McGraw Hill Wonders materials that have weekly assessments. I wonder if as an author on the program you could help them to add some langauge into their professional development that also discourages this practice. Thank you!
Petra

Lynn Strang
Feb 27, 2019 03:43 PM

Our District purchased the 2017 Wonders Reading Program for grades K-5. One of our administrators said that Wonders Reading Program is not a curriculum that it is a scope and sequence. I understand that there are several definitions of curriculum. It is my understanding that the Wonders Reading Program we purchased which has all the components of foundational reading skills, comprehension, grammar and writing which we use to teach students; is in fact considered an English Language Arts curriculum. Can you please provide clarity around curriculum and a purchased reading program? Thank you!

Gerardo Loera
Mar 02, 2019 08:31 PM

What are effective ways of determining the complexity of texts can understand?

Rochelle
Mar 03, 2019 03:30 PM

"A watched pot never boils" comes to mind.

Thank you for writing this post.

Angi Hartman
Mar 09, 2019 02:49 PM

I’d like to repeat Kathe Ball’s question. I am needing guidance as to what to do THIS WEEK to improve my reading instruction. I am so confused about everything I’m doing due to recent study of articles on what’s effective.

What Are your thoughts?

Leave me a comment and I would like to have a discussion with you!

Comment *
Name*
Email*
Website
Comments

Should We Administer Weekly Tests Linked to Standards?

22 comments

One of the world’s premier literacy educators.

He studies reading and writing across all ages and abilities. Feel free to contact him.