What Phonological Awareness Skill Should We Be Screening?

Teacher question: I read a research study (Kilpatrick, 2014) that questions the value of segmentation tests for measuring phonemic awareness, because such tests did not correlate well with first- and second-grade reading achievement. At our school we have used DIBELS in Kindergarten and Grade 1 to identify children at risk for reading difficulties. Is this really useful or are we identifying kids as needing help when they do not? Should we be using measures of blending and manipulation instead?

Shanahan's response:  This question seems so straightforward, but it actually has a lot of moving parts. The two tests being compared, DIBELS and CTOPP, have different purposes, there are things you need to know about phonological awareness (PA)  development, and there are problems with the analysis in the study that you read.

          Let’s take one of these at a time.

Difference between CTOPP and DIBELS

          Perhaps the easiest one to deal with has to do with the purposes of the two tests. CTOPP sets out to provide a thorough analysis of phonological awareness in a way appropriate to a wide range of ages, including low literacy adults. DIBELS, instead, is just a screening test; it doesn’t purport to provide a thorough inventory of skills, only a reasonable quick prediction of who may need more help.

          The CTOPP would certainly provide your district with a more detailed analysis; sort of a everything you ever wanted to know about your students sound perception skills. It not only will tell you who is having trouble, but it will provide specific diagnostic information about what exactly might need to be taught. And, for that, it will take you about 30 minutes of testing per child; a big investment that no one in his/her right mind would be willing to make several times a school year with classes of real kids.

          In contrast, the DIBELS assessment can be given in a couple of minutes, and all it will tell you is who has an adequate level of PA to be able to learn to read. As I say, they have different purposes. If you are just trying to figure out who may need some extra attention in PA, DIBELS is the way to go, while if you are trying to make a detailed instructional plan, like you might want to do in a Special Education program, then CTOPP is the way to go. It’s up to you to decide what you want.

Sequence of development

          Your letter and the study that you cite seem to conceive of a collection of different phonological awareness skills. However, most experts on the matter seem to believe that phonological awareness is actually a single line of development, with particular issues or skills emerging at different points of development (Anthony & Francis, 2005; Anthony, Williams, Duran, Gillam & Liang, 2011).

           Analyses of many different test instruments, thousands of kids, and multiple analytical methods suggests that PA is a single continuum with the following characteristics: phonological sensitivity progresses from large language units to small language units, and from syllables to onset-rimes to individual phonemes within words; detection ability precedes manipulation of sounds, and blending precedes segmentation development; and children do not move through this progression in distinct stages, but may be consolidating one level of learning while starting to progress in the next.

           The point of all that is that the highest level of development in phonological awareness is segmentation of words. Basically, kids have a lot to learn about phonology, but once they can easily fully segment words they have sufficient phonological awareness to learn to read. If they only can blend, then they are not likely to have sufficient sensitivity for the demands of learning to read.

           Given this, it should not be surprising that a predictive screener is going to evaluate whether kids can segment and other more complete measures might consider blending, onset-rime proficiency, and the ability to separate syllables. One progression of learning, but two different purposes for testing.

Problem with study

          The Kilpatrick study that you cite is an interesting one, but it has some formidable problems. First, this it asks a normative question and yet focuses on a very small and narrow sample of students. The data could be absolutely correct for that sample, but tell us nothing about how this test works with a normal population of kids. One thing I noticed was that the standard deviation — that is, the amount of variance — for segmenting was different than for the other measures, and that it also seemed to differ from that reported with other populations who have used that test.

           If there is not equivalent variation in one of the subtests, then a comparison of the subtests will not come out right. Why there was less variability in this measure with this small group of kids I can’t tell you, but it does suggest the possibility of a different result with a more representative sample. (Some explanations, this was a small group that would not be adequate to representing a population; the test administration could have been flawed; perhaps there was something about local teaching that was messing with the variability in particular skills; the children were tested over a long period of time — 2 to 3 months, a significant amount of time in PA development — which could have influenced variability in odd ways).

            Given that segmentation is the highest level of phonemic awareness (and the one that experimental studies aim at accomplishing through their instructional interventions), usually I would expect that to have a better or equal correlation with reading, at least in Kindergarten and Grade 1 (with second graders, CTOPP should be reserved for struggling readers).

Conclusion

          If you are looking for a detailed plan of instruction for each kid tested, then use something like the CTOPP rather than DIBELS. If all that you are trying to do is identify early young kids who might struggle in learning to read, then use DIBELS along with a measure of alphabet knowledge and you should get a good picture of who might require some extra help. In such a case, the point is to determine if a student can fully segment with ease. If he/she can, then he will likely do well learning to decode; if he/she can’t appropriate PA instruction could focus on anything from syllable separations to onset-rimes to blending to segmentation.

References 

Anthony, J.L, & Francis, D.J. (2005). Development of phonological awareness. Current Directions in Psychological Science, 14: 255-259.

Kilpatrick, D.A. (2014). Phonological Segmentation Assessment Is Not Enough: A Comparison of Three Phonological Awareness Tests With First and Second Graders, Canadian Journal of School Psychology, 27(2): 150–165. 

Hintze, J., Ryan, A.L., & Stoner, G. (No date). Concurrent validity and diagnostic accuracy of the Dynamic Indicators of Basic Early Literacy Skills and the Comprehensive Test of Phonological Processing.