Chicago Annenberg classroom assignment collection and scoring system¶

Element · All elements
Evidence · 7 claims (7 for) · 2 studies (2 associational), q2 · 0 of 2 report an effect size · 7 claims rest on one study

Description¶

Teachers in grades three, six, and eight submitted typical and challenging writing and mathematics assignments; teacher raters scored them against rubrics for construction of knowledge, written communication, and connection to students’ lives, with many-facet Rasch adjustment onto a ten-point scale. In 1997 the analysis examined "a total of 349 assignments from 74 teachers," rising to 953 assignments in 1998 and 715 in 1999.

Design Implications¶

Context¶

Requirements¶

• Teachers were asked to provide four typical assignments per year and two challenging assignments per year, scored by raters working one standard at a time with random double-scoring to control for rater bias

Constraints¶

• Because of changes in the sample of schools over time and the variable amounts of data provided by different teachers, a statistical model was needed to estimate time trends

Target Learners¶

• elementary school students in grades three, six, and eight in Chicago public schools

Target Learning Goals¶

• intellectual quality of writing and mathematics assignments
• authentic intellectual work in classroom tasks

Claims¶

• The intellectual quality of classroom assignments in Annenberg Challenge schools improved between 1997 and 1999, though some 1999 averages were lower than 1998 [+W]
• A 1998 baseline study found typical Chicago assignments were mostly rudimentary basic-skills exercises, but most students completed the more demanding tasks they were given [+W]
• Assignments teachers designated as challenging tended to score higher on intellectual quality than typical assignments [+W]
• More than 80 percent of sixth and eighth grade mathematics assignments in 1999 provided only minimal or no intellectual challenge [+W]
• Writing assignments in 1999 showed more intellectual challenge than mathematics, with 48 to 64 percent reaching moderate or extensive challenge depending on grade [+W]
• A majority of writing and mathematics assignments in 12 Chicago Annenberg schools fell into the lowest two challenge categories [+W]
• Writing assignments made more challenging demands than mathematics assignments across all three grades, with grade-level patterns differing by subject [+W]

Related Elements¶

• Authentic intellectual work scoring rubrics with double-scoring design for inter-scorer reliability
• Assignment-scoring standards and rubrics for authentic intellectual work in writing and mathematics
• Scoring standards and rubrics for assignments and student work in writing and mathematics

Examples¶

-

Key Sources¶

• Anthony S. Bryk, Jenny K. Nagaoka, and Fred M. Newmann. (2000). Chicago Classroom Demands for Authentic Intellectual Work: Trends from 1997-1999. Consortium on Chicago School Research. https://consortium.uchicago.edu/publications/chicago-classroom-demands-authentic-intellectual-work-trends-1997-1999