Bridget Dalton
berdwyer
Blaine Smith
danagrisham
jillcastek
TDWolseyWhat the NAEP2011 Writing Assessment Means for Technology Use in Schools
On September 14, I attended a Webinar on the outcomes of the Writing 2011 National Assessment of Educational Progress for Grades 8 and 12. The full report can be accessed at http://nces.ed.gov/nationsreportcard/writing/.
As we know, NAEP assessments have occurred since 1969 and provide a pretty reliable snapshot of educational progress in the U.S. The 2011 writing assessment is the first that has used technology as part of the assessment.
All 8th and 12th graders who participated in the assessment did so using a word processing program on the computer, rather than using pencil and paper. Because of this, new scales and achievement levels were established and webinar presenters (including Arthur Applebee and Elaine Chin) were careful to stress that the findings could not be directly compared to past results for that reason, although results indicate students have the same strengths and weaknesses as revealed in the past by pencil and paper tests.
Also in the 2011 assessment, the types of writing required of students are in alignment with the Common Core State Standards (CCSS, 2010) stressing the reinforcement of three writing capacities. These three types of writing prompts were used with different proportions of use for the two grade levels:
| % Assessment 8th | % Assessment 12th | |
| Persuasion | 35 | 40 |
| Explanation | 35 | 40 |
| Convey an Experience | 30 | 20 |
Scores were based on six performance ratings and scored as “first drafts” rather than polished writing samples. The ratings may be found in the report.
Writing prompts were provided with interesting technology. On a computer screen divided vertically like the pages of a book, the left half contained the prompt with specific types of multimedia, including an audio prompt at 8th grade and a video prompt at the 12th grade. In the webinar, a prompt for conveying information asked students to describe a desert island and the multimedia added the sound effects for the prompt.
There were a number of program tools that students could elect to use as they wrote (prompts were based on Universal Design) and students’ tool use was tracked and generally associated with higher scores…but more on that later.
Findings
The major assessment findings were that at both 8th and 12th grades, only about 24% of those assessed scored at the “proficient” level, as is seen in the screen shot from the website (below). About half the students assessed scored at the Basic achievement level.
Gaps continue to exist among ethnic groups and there is also significant variance for student performance by gender and school location. For example, at the 8th grade level, the scaled scores showed Asian students scoring the highest at 165, while the lowest scoring ethnic group were Black students at 132.
There is also a rather large gender difference with the scaled score for 8th grade males at 140 and for 8th grade females at 160. Scores for 12th graders were closer, but the gap is still sizeable.
The following screen capture illustrates this point:
Finally, the school type and location appears to matter. While public schools comprised 92% of the sample, scores for students in private schools scored higher overall with students in Catholic schools scoring the highest. Suburban school districts had higher scaled scores than did those in cities, towns, and rural locations. These scores can be seen in Table A:
At 12th grade, the highest scoring ethnic group was White at 159, while the lowest was Black at 130. The gender gap was somewhat less stark but still extant, as was the benefit of attending school in a suburb.
Poverty continues to make a substantial difference in students’ scores. Those eligible for free lunch had a scaled score of 134, while those who were not eligible had a scaled score of 161.
Technology Use
Beyond overall scores, there are many implications for educators. One question appears to have a positive answer: the use of word processing tools for the assessment appear to be more engaging.
Of interest to those who enjoy the intersections of literacy and technology are the findings that emanate from the word processing actions used by students. Questionnaires were given to teachers of 8th grade students completing the assessment. Teachers reported on how frequently they had students using computers to write and revise drafts. The data showed that 44% of students had teachers who reported using technology in writing; these students scored higher than those who had teachers who did not use technology as much in their writing.
As noted above, students who used the tools available (cut/paste, text-to-speech, spell-check, thesaurus) scored higher than students who did not use these tools.
For 12th graders the technology was a little more sophisticated, as the prompt included video as well as audio. In this sample prompt, students were asked to speak to an audience of college admission committee members to persuade them how technology is important and valuable to them. The video provided statistics of technology use.
Both 8th and 12th grade students who said the use a computer more frequently to edit their writing scored higher than students who did not.
Implications
For all educators, there is an urgent need to embrace technological tools for communication and composition in our homes and schools. There are examples everywhere of sound technology use in schools (see my blog on Literacy Beat dated August 22, 2012 on the use of Strip Designer for writing at the Kindergarten level).
This summer I taught a class on reading fluency and the students and I discussed writing as a part of reading fluency. The experienced classroom teachers in my class had a spirited discussion about the use of spell-check in composition, with half arguing that over-reliance on this would negatively impact students’ writing at the early elementary grades. While I am unaware of research that directly addresses this question, I believe that tool use is a hallmark of human beings and should be encouraged.
While the 2011 Writing Assessment did not include 4th graders, there was a separate study conducted to assure NCES that 4th graders could use the tools. While findings have not yet been released, they appear to be positive for tool use. Right now, NCES is recommending that Keyboarding skills begin at third grade. Interestingly, early childhood education groups are recommending children learn to keyboard much earlier. I recall, from my own teaching in the early 1980s, teaching keyboarding skills to my first graders in a California public school. I also offer this photograph of my son-in-law and my twin granddaughters (aged 3 and 1/2) on a “Digital Morning” with the understanding that teachers will increasingly see tech-savvy students like these in their classrooms.
There are an increasing number of resources for learning about and using technological tools in the classroom. Most organizations are now providing online assistance to teachers in using technology. For example, ASCD now has a website with tools for teachers to learn to use the CCSS (http://educore.ascd.org/) and the International Reading Association is now merging their Engage feature with Reading Today Online to form a new blog. Such assistance is becoming more common. Of course, there are books out there, too, such as the one DeVere and I published with Guilford this year (see Bridget Dalton’s blog on Literacy Beat dated August 7, 2012).
Thus, I have ventured to suggest some recommendations for educators, as shown below.
Teachers need to:
1) Find ways to incorporate technology into their classrooms with the tools (however limited) that they already have.
2) Argue on behalf of technology, using the research evidence at hand—such as the 2011 NAEP Writing Assessment outcomes.
3) Seek workshops and professional development opportunities to develop their own expertise in technology use.
Administrators need to:
1) Support teachers’ use of technology in the classroom.
2) Argue at the district level on behalf of technology use.
3) Seek workshops and professional development opportunities for themselves and their teaching staff.
Teacher Educators need to:
1) Work collaboratively within the university to distribute technological use across the teacher preparation programs instead of relying on “Ed Tech” courses.
2) Seek workshops on technology use for themselves.
3) Where possible, seek student teaching placements for teacher candidates where technology is being used productively.
If you have other ideas, I welcome your responses!
References:
Wolsey, T.D. & Grisham, D.L. (2012). Transforming writing instruction in the Digital Age: Techniques for Grades 5-12. New York: Guilford.
Common Core State Standards Initiative (2010). Preparing America’s students for college and career. Washington, DC: National Governors’ Association and CCSSO. Retrieved from: http://www.corestandards.org/the-standards.
Share this:
Filed under: writing | 1 Comment »
Literacy Beat 