Scoring Methods for Progress Monitoring Using CBM in Early Writing

Authors

  • Abigail A. Allen PhD CCC-SLP Clemson University
  • Kyle Wagner NCSP PhD Findley University
  • R. Alex Smith PhD University of Nevada, Las Vegas
  • Kristina N. Randall PhD Clemson University

DOI:

https://doi.org/10.18666/LDMJ-2023-V28-I2-12058

Keywords:

curriculum-based measurement, early writing, assessment, trait-based rubric

Abstract

This initial exploratory study assessed the evidence of reliability, criterion validity, and growth of scores obtained from total correct word sequences (CWS), correct word sequences per response (CWSR), and a trait-based rubric used with a curriculum-based measure of writing (CBM-W) for progress monitoring. Participants were 26 writers with or at risk for a learning disability (LD) in grades 2-3. Participants were given the CBM-W task twice weekly over 6 weeks and were given the Spelling and Writing Samples subtests of the Woodcock-Johnson Tests of Achievement-III at pre- and posttest. Results indicated large inter-rater reliability coefficients but mixed alternate form reliability and validity evidence. CWS was the only metric to demonstrate significant growth (r = .60). CWS has the strongest evidence for use as a progress monitoring scoring metric for students at risk for LD, although information from CWSR and rubric scores could be instructionally useful.

Author Biographies

Abigail A. Allen PhD CCC-SLP, Clemson University

Abigail A. Allen PhD CCC-SLP is an associate professor of Special Education at Clemson University.

Kyle Wagner NCSP PhD, Findley University

is an assistant professor of Education at Findley University.

R. Alex Smith PhD, University of Nevada, Las Vegas

is an assistant professor of Special Education at University of Nevada Las Vegas.

Kristina N. Randall PhD, Clemson University

is a clinical assistant professor of Special Education at Clemson University.

References

References

Allen, A. A., & Lembke, E. S. (2022). The effect of a morphological awareness intervention on

early writing outcomes. Learning Disability Quarterly, 45(2), 72-84. https://doi.org/10.1177%2F0731948720912414

Allen, A. A., Poch, A. L., Lembke, E. S. (2018). An exploration of alternative scoring indices

using curriculum-based measurement in early writing. Learning Disability Quarterly, 41(2), 85-99. https://doi.org/10.1177/0731948717725490

Coker, D. L., & Ritchey, K. D. (2010). Curriculum-based measurement of writing in

kindergarten and first grade: An investigation of production and qualitative scores. Exceptional Children, 76(2), 175–193. https://doi.org/10.1177/001440291007600203

Deno, S. L. (2003). Developments in curriculum-based measurement. The Journal of Special

Education, 37(3), 184-192. https://doi.org/10.1177%2F00224669030370030801

Deno, S., Marston, D., & Mirkin, P. (1982). Valid measurement procedures for

continuous evaluation of written expression. Exceptional Children, 48(4), 368–371. https://doi.org/10.1177/001440298204800417

Gansle, K. A., VanDerHeyden, A. M., Noell, G. H., Resetar, J. L., & Williams, K. L.

(2006). The technical adequacy of curriculum-based and rating-based measures of written expression for elementary school students. School Psychology Review, 35(3), 435–450. https://doi.org/10.1080/02796015.2006.12087977

Graham, S., Harris, K. R., & McKeown, D. (2013). The writing of students with learning disabilities, meta-analysis of self-regulated strategy development writing intervention studies, and future directions: Redux. In L. Swanson, K. R. Harris, & S. Graham (Eds.), Handbook of learning disabilities (pp. 405–438). Guilford Press.

Graham, S., Harris, K., and Hebert, M.A. (2011). Informing writing: The benefits of formative assessment. A Carnegie Corporation Time to Act Report. Alliance for Excellent Education. https://media.carnegie.org/filer_public/37/b8/37b87202-7138-4ff9-90c0-cd6c6f2335bf/ccny_report_2011_informing.pdf

Hampton, D. D., & Lembke, E. S. (2016). Examining the technical adequacy of progress

monitoring using early writing curriculum-based measures. Reading & Writing Quarterly, 32(4), 336-352. https://psycnet.apa.org/doi/10.1080/10573569.2014.973984

Hier, B. O., January, S. A. A., & Van Norman, E. R. (2020). A comparison of CBM-WE scoring

metrics and progress monitoring frequency among second-grade students. School Psychology Review, 49(3), 306-320. https://doi.org/10.1080/2372966X.2020.1763758

January, S. A. A., Van Norman, E. R., Christ, T. J., Ardoin, S. P., Eckert, T. L., & White, M. J.

(2019). Evaluation of schedule frequency and density when monitoring progress with curriculum-based measurement. School Psychology, 34(1), 119. https://doi.org/10.1037/spq0000274

Jung, P. G., McMaster, K. L., & delMas, R. C. (2017). Effects of early writing intervention

delivered within a data-based instruction framework. Exceptional Children, 83(3), 281-297. https://doi.org/10.1177/0014402916667586

Lembke, E., Deno, S. L., & Hall, K. (2003). Identifying an indicator of growth in early writing

proficiency for elementary school students. Assessment for Effective Intervention, 28(3–4), 23–35. https://psycnet.apa.org/doi/10.1177/073724770302800304

Marston, D. B. (1989). A curriculum-based measurement approach to assessing academic

performance: What it is and why do it. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp. 18–78). Guilford Press. https://psycnet.apa.org/record/1989-97717-002

McMaster, K. L., Ritchey, K. D., & Lembke, E. (2011a). Curriculum-based measurement for

beginning writers: Recent developments and future directions. In T. E. Scrugg, & M. A. Mastropieri (Eds.), Advances in learning and behavioral disabilities: Assessment and intervention (Vol. 24, pp. 111-148). Emerald. https://doi.org/10.1108/S0735-004X(2011)0000024008

McMaster, K. L., Du, X., Yeo, S., Deno, S. L., Parker, D., & Ellis, T. (2011b).

Curriculum-based measures of beginning writing: Technical features of the slope. Exceptional Children, 77, 185–206. https://doi.org/10.1177%2F001440291107700203

Parker, D. C., McMaster, K. L., Medhanie, A., & Silberglitt, B. (2011). Modeling early

writing growth with curriculum-based measures. School Psychology Quarterly, 26(4), 290–304. https://doi.org/10.1037/a0026833

Romig, J. E., Therrien, W. J., & Lloyd, J. W. (2017). Meta-analysis of criterion validity for

curriculum-based measurement in written language. The Journal of Special Education, 51(2), 72-82. https://doi.org/10.1177/0022466916670637

Schrank, F. A., McGrew, K. S., & Woodcock, R. W. (2001). Technical abstract

(Woodcock-Johnson III Assessment Service Bulletin No. 2). Riverside Publishing.

Smith, R. A., & Lembke, E. S. (2021). Aspects of technical adequacy of an early-writing

measure for English language learners in grades 1 to 3. Assessment for Effective Intervention, 47(1), 59-63. https://doi.org/10.1177/1534508420947157

Stecker, P. M., Fuchs, L. S., & Fuchs, D. (2005). Using curriculum‐based measurement to

improve student achievement: Review of research. Psychology in the Schools, 42(8), 795-819. https://doi.org/10.1002/pits.20113

Videen, J., Deno, S., & Marston, D. (1982). Correct word sequences: A valid indicator of

proficiency in written expression. (No. IRLDRR-84). University of Minnesota, Institute for Research on Learning Disabilities. https://files.eric.ed.gov/fulltext/ED225112.pdf

Wagner, K., Smith, A., Allen, A., McMaster, K., Poch, A., & Lembke, E. (2019). Exploration of

new complexity metrics for curriculum-based measures of writing. Assessment for Effective Intervention, 44(4), 256-266. https://doi.org/10.1177%2F1534508418773448

Woodcock, R. W., McGrew, F. A., Mather, N. & Schrank, F. A. (2007). Woodcock-

Johnson tests of achievement (3rd ed.). Riverside Publishing.

Published

2023-12-05

Issue

Section

Articles