Monitoring the Reading Comprehension of Older Students

Fair point, Tiffany. Most of those problems are inherent in the evaluation of comprehension. Growth rates are definitely much faster and easier to measure with younger students, but it is still impossible to measure all of those individual standards or skills that many schools aim at even with younger children.

tim

Chase-

There isn't a whole lot, but these research studies should provide some direction.

Vaughn, S., Martinez, L. R., Williams, K. J., Miciak, J., Fall, A-M. & Roberts, G. (2018). Efficacy of a high school extensive reading intervention for English learners with reading difficulties. Journal of Educational Psychology, Advanced online pub. doi:10.1037/edu0000289.

Vaughn, S., Roberts, G., Wexler, J., Vaugn, M., Fall, A.-M. & Schnakenberg, J. (2015). High school students with reading comprehension difficulties: Results of a randomized control trial of a two-year reading intervention. Journal of Learning Disabilities, 48(5), 546–558. doi:10.1177/0022219413515511.

Tim

I work in a UK primary school. What would you say were the issues with the Reading SATs? Similar to what you wrote above? What would you suggest as an alternative method of schools assessing comprehension? Should teachers instead be assessing how well students apply strategies, retention of vocabulary taught etc?

Dr. Shanahan,

Thank you so much for this excellent post. It was eye-opening for me on a question that has bothered me for years. Like the teacher who sent you the original question, I used to give weekly multiple-choice quizzes based on a one-page reading story. (I’m a fifth grade teacher.) These were part of our adopted curriculum. But I always struggled to know what to do with this information! Okay, so one of my students only answered 2 questions right out of 5. I know this person is a struggling reader, but there’s no useful information here. (I gave up these tests last year. Thankfully I work in a school that gives teachers lots of flexibility.)

So your point about the complexity of measuring reading comprehension is very helpful. Now I see that rather than trying to measure one massive and complicated thing called “comprehension” I should focus instead on giving assessments the results of which will actually be able to guide my teaching. If I’m teaching them how to make complex sentences using subordinating conjunctions, I can measure whether they’re getting it. If not, I will be able to clearly see in their answers what the issues are and I will be able to re-teach the concept or help the students who need it individually. Or if they’re learning about pronouns, I can ask specific questions about pronoun antecedents to see if they really know what “it” refers to in a given sentence.

In other words, there’s no point in assessments that don’t give you information that can change your teaching and student learning.

Thank you!

And thanks to your many readers, who post excellent comments. I have been reading this blog for about six months now, and I have found it to be tremendously helpful. The discussions are stimulating, and I love your focus on things that actually have evidence to back them up.

The question above sounds very much like the program Beyond Textbooks, out of Vail school district(and many others in and out of AZ use it as well). They assess each of the essential standards either every 5 or 10 days, test, then depending on student score, they go through a reteach/enrich process.

I appreciate the alternative ideas for monitoring growth, for example in the area of vocabulary acquisition. You mentioned the same kind of monitoring can be done with cohesion and text structure. Can you say more about better ways to monitor those two areas and give an example?

+1 in regards to Joelle Nappi's post! I'm trying to wrap my head around a form of reading comprehension assessment that would be truly actionable for teachers.

One of the most effective ways to monitor subtle changes in students’ comprehension is through discussion. However, the text matters. Test passages tend to be stilted and poorly written—inconsiderate (Vacca & Vacca). Use of questions in discussion that mirror those on state assessments as formative assessment offers teachers a window into students’ thinking processes and leads to, as Pearson suggests, responsive teaching. Our obsession with weighing the lamb versus feeding makes me wonder if Durkin would find much different results today than she did decades ago. Not much changes without modeling and demonstrating what comprehension looks like and sounds like.

Dr. Shanahan, I found your response interesting.

I am curious what you would say about progress monitoring with Middle School, sixth, seventh, and eighth grade as well as what tools you would recommend.


Thanks for your insight.

Jonathon--

The problem isn't with the tests. They give a reasonable idea as to who is comprehending what they are reading for the most part. The problem is that comprehension is complicated and we are misunderstanding what these tests tell us. They reveal how well students comprehend, they do not tell why -- that would take a much more extensive testing regime that would go far beyond what screeners and monitors are needed to do.

tim

Joelle--

Cohesion can be measured by asking questions that tap into whether a student is following the chain of reasoning -- such as asking who he refers to in the third sentence or checking to see if the student grasps the connection between synonyms -- or checking the meaning of pronouns like any or one... etc. In terms of text structure, with narrative it might be having student summarizing a story on a story grammar chart or with expository it might be asking the student to identify the structure (comparison, problem-solution, enumeration, cause and effect) or, again completing such a chart for a selection.

tim

April--

The point is that since you cannot efficiently and in any meaningful way track student progress in reading comprehension, evaluate how well the student is learning the things that you are teaching. Can they actually apply these skills and abilities in challenging text independently. For example, if you are teaching 6-8 vocabulary words per week, are they learning these words? Are they retaining them over time? Can they comprehend their meanings in text? We know vocabulary growth improves comprehension over time -- so monitoring their success with what you are teaching should tell whether there is likely progress for the student.

tim

Judy--

Actually what the research is telling us is that questions that mirror the skills noted in standards are not giving us a window into children's thinking processes. That's just what they cannot do. They can reveal whether the child understood what he/she read, but not how they came to that answer. -- But your point, engaging students in discussion of what they read does play an important role in children's comprehension development.

tim

I also need clarification as Tiffany does for younger readers. What are your thoughts on how to progress monitor grades 3-5?

Jen-
What I said for high school would be true for middle school, too. I would likely start the year with a reading test like this:
https://www.serpinstitute.org/reading-assessment
That would allow me to group students appropriately, particularly with regard to Tier 2 interventions. Regular classroom instruction would emphasize fluency, comprehension (strategies, written language), and writing with both narrative and expository text. We'd monitor student progress with what we were teaching.

tim

Thank you for this post. I found it very informative- along with the comments section. Dr. S - thank you for taking so much time to answer all the questions.

My experience with struggling readers suggests a need for parental concern. The parent should be advised to seek tutoring for this student, who may have missed early reading instruction, for whatever reason. Classroom teachers can't address comprehension problems at this level, mostly because bureaucracy is not addressing the problem earlier. A summer tutorial, offered by private or public, could bring this student's reading level up to grade placement and beyond. This should be the student's and/or parent's choice, once they are informed. Teacher accountability requires truth in education. Don't take the system's problems and make them your own. One on one instruction is needed here.

Kathy and Tiffany--

The comprehension problems that I noted for older students, are also problems in the elementary grades (they are an issue of comprehension not development). However, in the elementary grades it is very reasonable to monitor decoding ability, oral reading fluency, spelling -- and those skills can be profitably examined 3-4 times per year.

tim

I am wondering about your thoughts on assessing reading comprehension or mastery of a standard solely shown through writing. Each quarter we administer a midterm, final, and 2 formative assessments based on the standards taught during those weeks. We are now adding a weekly writing assessment,based on a prompt that assesses the end of the learning continuum. The students cannt receive support during the first write, but 1-2 weeks later there is a reteach and students would write based on a similar prompt to assess growth. Do you have thoughts on assessing reading comprehension this way?

J-

I like assessing reading comprehension through writing, but it is important to remember that it will under estimate comprehension ability a bit, because performance is affected by both comprehension and writing ability. A student who understands a text may not be able to make it clear that he/she understood the text through writing (but perhaps could orally). This is especially true for second language learners. I'm willing to risk that however because it encourages teachers to engage kids in written response to text -- which is a skill worth developing and that contributes to student learning.

tim

I just reread your response to a teacher who was looking for a comprehension assessment tool for middle and high school students. I teach reading to middle and high school students, and I am looking for a "substantial, validated reading test" - your words in the posting - to replace the LexiaRAPID online reading assessment, which is terminating in June 2023. The LexiaRAPID assesses word recognition, vocabulary knowledge, syntactic knowledge, and reading comprehension, and provides Lexiles and text Lexile levels. I have been looking for a replacement for months but have not found anything comparable for older students that is useful as a screener and progress monitoring tool. I am interested to know if you, or teachers who follow your blog, are familiar with any such reading assessment for older students. Thank you. - Kathryn

What are some examples of reading comp test I can use with a student in grade 11 in an ID program?

Rin-

I would use ETS's ReadAuthentix if you want to know how well a high schooler can comprehend what they read. But that is not a monitoring test per se. You wouldn't give this kind of test multiple times in a year (there is no such test for reading comprehension at high school). If you want something more diagnostic then I'd go for CAPTI/ETS ReadBasix.

tim