What the NAEP2011 Writing Assessment Means for Technology Use in Schools

What the NAEP2011 Writing Assessment Means for Technology Use in Schools

On September 14, I attended a Webinar on the outcomes of the Writing 2011 National Assessment of Educational Progress for Grades 8 and 12. The full report can be accessed at http://nces.ed.gov/nationsreportcard/writing/.

As we know, NAEP assessments have occurred since 1969 and provide a pretty reliable snapshot of educational progress in the U.S. The 2011 writing assessment is the first that has used technology as part of the assessment.

All 8th and 12th graders who participated in the assessment did so using a word processing program on the computer, rather than using pencil and paper. Because of this, new scales and achievement levels were established and webinar presenters (including Arthur Applebee and Elaine Chin) were careful to stress that the findings could not be directly compared to past results for that reason, although results indicate students have the same strengths and weaknesses as revealed in the past by pencil and paper tests.

Also in the 2011 assessment, the types of writing required of students are in alignment with the Common Core State Standards (CCSS, 2010) stressing the reinforcement of three writing capacities. These three types of writing prompts were used with different proportions of use for the two grade levels:

%  Assessment 8th %  Assessment 12th
Persuasion 35 40
Explanation 35 40
Convey an Experience 30 20

Scores were based on six performance ratings and scored as “first drafts” rather than polished writing samples. The ratings may be found in the report.

Writing prompts were provided with interesting technology. On a computer screen divided vertically like the pages of a book, the left half contained the prompt with specific types of multimedia, including an audio prompt at 8th grade and a video prompt at the 12th grade. In the webinar, a prompt for conveying information asked students to describe a desert island and the multimedia added the sound effects for the prompt.

There were a number of program tools that students could elect to use as they wrote (prompts were based on Universal Design) and students’ tool use was tracked and generally associated with higher scores…but more on that later.


The major assessment findings were that at both 8th and 12th grades, only about 24% of those assessed scored at the “proficient” level, as is seen in the screen shot from the website (below). About half the students assessed scored at the Basic achievement level.

Gaps continue to exist among ethnic groups and there is also significant variance for student performance by gender and school location.  For example, at the 8th grade level, the scaled scores showed Asian students scoring the highest at 165, while the lowest scoring ethnic group were Black students at 132.

There is also a rather large gender difference with the scaled score for 8th grade males at 140 and for 8th grade females at 160. Scores for 12th graders were closer, but the gap is still sizeable.

The following screen capture illustrates this point:

Finally, the school type and location appears to matter. While public schools comprised 92% of the sample, scores for students in private schools scored higher overall with students in Catholic schools scoring the highest.  Suburban school districts had higher scaled scores than did those in cities, towns, and rural locations. These scores can be seen in Table A:

At 12th grade, the highest scoring ethnic group was White at 159, while the lowest was Black at 130. The gender gap was somewhat less stark but still extant, as was the benefit of attending school in a suburb.

Poverty continues to make a substantial difference in students’ scores. Those eligible for free lunch had a scaled score of 134, while those who were not eligible had a scaled score of 161.

Technology Use

Beyond overall scores, there are many implications for educators. One question appears to have a positive answer: the use of word processing tools for the assessment appear to be more engaging.

Of interest to those who enjoy the intersections of literacy and technology are the findings that emanate from the word processing actions used by students. Questionnaires were given to teachers of 8th grade students completing the assessment. Teachers reported on how frequently they had students using computers to write and revise drafts. The data showed that 44% of students had teachers who reported using technology in writing; these students scored higher than those who had teachers who did not use technology as much in their writing.

As noted above, students who used the tools available (cut/paste, text-to-speech, spell-check, thesaurus) scored higher than students who did not use these tools.

For 12th graders the technology was a little more sophisticated, as the prompt included video as well as audio. In this sample prompt, students were asked to speak to an audience of college admission committee members to persuade them how technology is important and valuable to them. The video provided statistics of technology use.

Both 8th and 12th grade students who said the use a computer more frequently to edit their writing scored higher than students who did not.


For all educators, there is an urgent need to embrace technological tools for communication and composition in our homes and schools. There are examples everywhere of sound technology use in schools (see my blog on Literacy Beat dated August 22, 2012 on the use of Strip Designer for writing at the Kindergarten level).

This summer I taught a class on reading fluency and the students and I discussed writing as a part of reading fluency. The experienced classroom teachers in my class had a spirited discussion about the use of spell-check in composition, with half arguing that over-reliance on this would negatively impact students’ writing at the early elementary grades. While I am unaware of research that directly addresses this question, I believe that tool use is a hallmark of human beings and should be encouraged.

While the 2011 Writing Assessment did not include 4th graders, there was a separate study conducted to assure NCES that 4th graders could use the tools. While findings have not yet been released, they appear to be positive for tool use.  Right now, NCES is recommending that Keyboarding skills begin at third grade. Interestingly, early childhood education groups are recommending children learn to keyboard much earlier. I recall, from my own teaching in the early 1980s, teaching keyboarding skills to my first graders in a California public school.  I also offer this photograph of my son-in-law and my twin granddaughters (aged 3 and 1/2) on a “Digital Morning” with the understanding that teachers will increasingly see  tech-savvy students like these in their classrooms.

There are an increasing number of resources for learning about and using technological tools in the classroom. Most organizations are now providing online assistance to teachers in using technology. For example, ASCD now has a website with tools for teachers to learn to use the CCSS (http://educore.ascd.org/) and the International Reading Association is now merging their Engage feature with Reading Today Online to form a new blog. Such assistance is becoming more common. Of course, there are books out there, too, such as the one DeVere and I published with Guilford this year (see Bridget Dalton’s blog on Literacy Beat dated August 7, 2012).

Thus, I have ventured to suggest some recommendations for educators, as shown below.

Teachers need to:

1)   Find ways to incorporate technology into their classrooms with the tools (however limited) that they already have.

2)   Argue on behalf of technology, using the research evidence at hand—such as the 2011 NAEP Writing Assessment outcomes.

3)   Seek workshops and professional development opportunities to develop their own expertise in technology use.

Administrators need to:

1)   Support teachers’ use of technology in the classroom.

2)   Argue at the district level on behalf of technology use.

3)   Seek workshops and professional development opportunities for themselves and their teaching staff.

Teacher Educators need to:

1)   Work collaboratively within the university to distribute technological use across the teacher preparation programs instead of relying on  “Ed Tech” courses.

2)   Seek workshops on technology use for themselves.

3)   Where possible, seek student teaching placements for teacher candidates where technology is being used productively.

If you have other ideas, I welcome your responses!


Wolsey, T.D. & Grisham, D.L. (2012). Transforming writing instruction in the Digital Age: Techniques for Grades 5-12. New York: Guilford.

Common Core State Standards Initiative (2010). Preparing America’s students for college and career. Washington, DC:  National Governors’ Association and CCSSO. Retrieved from: http://www.corestandards.org/the-standards.


Finding the needle in the haystack

A post from Bernadette

Successful online readers access information speedily, effectively and efficiently (Leu, Kinzer, Coiro & Cammack, 2004). However, given the sheer volume of information available online, finding relevant information for a task focus can be laborious, rather daunting and somewhat overwhelming and akin to finding a ‘needle in a haystack’.

Two issues warrant attention. The first issue relates to judging the relevance of a search result blurb for your inquiry focus. As Eileen (a struggling reader in 6th grader) noted to me, “The blurb might tell you something but when you go inside it, it’s something different. Don’t get your hopes up if it says what you want to find ’cause it mightn’t always be inside it”.

The second issue relates to remembering ‘aha’ websites. Given the amount of time we all spend online finding your way back to particularly relevant websites, which you have previously visited, can be rather taxing. In this post I will explore two digital tools (Yolink and Diigo) which may help to enhance search functionality.

Yolink (http://www.yolinkeducation.com/education/) is an add on browser extension tool that scans web pages, search engine results and digitized books to find your inputted search terms and deliver information that is relevant to your inquiry. Yolink is a supportive digital tool in two ways. Firstly, it enables the reader to dig deeper behind the links without painstakingly navigating to and opening each website link in turn. It does this by previewing and filtering the search results and highlighting snippets of information from relevant sections of search results for the reader. Secondly, Yolink can also search within bodies of texts (e.g. digitized versions of books) for key words or phrases to find information relevant for a particular research focus. Yolink helps you move, as the developers say, ‘from search to find’.

Sample lesson plans and resources are available on the Yolink education site (http://www.yolinkeducation.com/education/teachers.jsp).

For example, the lesson plan for ‘Polar Bears in a Changing Climate’, prepared by Julene Reed, is an example of a Challenge Based Learning unit using Yolink, and is based on the Apple Learning Interchange. Students are assessed on abilities in areas, such as creativity and innovation, communication and collaboration, research and information fluency and critical thinking, problem solving and decision making.

Bookmarking relevant sites by adding them to your favorites tab is one way to collect and retrieve websites that you visit. Creating subfolders with meaningful names is helpful when you want to revisit a particular website. However, if like me, you engage in squirreling behaviors where you tab multiple web sites on an hourly/daily basis you can end up with hundreds of websites in multiple subfolders. Just why you wanted to bookmark a particular website can be lost in the moment of tabbing! It is also difficult to backtrack quickly to a specific website when you want to locate information. So the proverbial haystack looms again!

A digital tool for organizing research online is Diigo or Digest of Internet Information, Groups and Other Stuff. (http://www.diigo.com/ ). Diigo is a cloud-based information management tool that enables users to collect, highlight, bookmark, tag, clip, share and annotate websites.

Teachers can create an educator account with Diigo. This will enable you to generate student accounts and establish collaborative research groups within your classroom. Diigo is helpful when conducting research, creating personal learning environments and collaborating with others. Some of the features of this tool which are useful include:

  • Annotating and highlighting snippets of information on websites with sticky notes.
  • Saving a screen shot of a web site on a particular day and revisiting to review changes over time or simply to archive the website.
  • Categorizing relevant information through the use of tags and lists on websites for quick retrieval of information.
  • Creating collaborative groups where teams of students can research information and post their findings and annotations for others in the group to review. Members can then interpret, critique and synthesize information from a variety of online sources.
  • Accessing your own digital library, as part of a personal learning environment, from any computer or through apps on Ipads and Android Tablets or smart phones.
  • Developing professional learning opportunities for teachers through Diigo created educator groups.

Leu, D. J., Kinzer, C. K., Coiro, J., & Cammack, D. (2004). Towards a theory of new literacies emerging from the Internet and other Information and Communication Technologies. In R. B. Ruddell & N. Unrau (Eds.), Theoretical models and processes of reading (5th ed., pp. 1570-1613). DE: International Reading Association.

My Pop Studio: Develop critical thinking and media skills with this free online game from Renee Hobbs and the Media Education Lab

post by Bridget Dalton, 9//13/12

Usually I blog about digital tools and instructional strategies, but today I want to introduce you to someone whose work I’ve followed for a number of years – Renee Hobbs. Renee is Professor and Founding Director of the Harrington School of Communication and Media at the University of Rhode Island. Renee is quite unusual in that she combines ‘making stuff’ in the Media Education Lab, with conducting research on media literacy and consulting on copyright and fair use policy. You can get a sense of the breadth and depth of her work by accessing slide shows of her many presentations available at http://www.slideshare.net/reneehobbs.

If you would like to hear directly from Renee about her leadership role in media literacy, view this video of an interview with her at the 10th Anniversary of the National Association for Media Literacy Education:

photo of Renee Hobbs

My Pop Studio

Today, I want to feature My Pop Studio, a free online ‘creative play experience’ developed by Renee and colleagues at the Media Education Lab. The goal of My Pop Studio is to engage young adolescents and teens in creating, manipulating, critiquing, and reflecting on mass media that is directed at girls. It includes a Magazine Studio, a TV studio, a Music Studio, and a Digital Studio. My Pop Studio is designed for use at home and at school (teachers can download a curriculum guide at http://mypopstudio.com/for_parents.php

screen shot of My Pop Studio

If your students and/or children try out My Pop Studio, please consider posting a comment about your experience.

Insights From A Service Learning Project: Creating Digital Projects with iPads to Encourage Safe Driving

A new post by Jill Castek

Melanie Swandby, a 7th grade teacher at Lighthouse Community Charter School in Oakland, CA was conducting a service learning project geared toward promoting safe driving habits.  Melanie was happy to explore digital content creation with her students, extending her original vision for the project with the goal of producing clear and coherent writing in which the development, organization, and style were appropriate to task, purpose, and audience (CCSS Initiative, 2010). She invited Heather Cotanch and I to explore the use of iPads to create digital products that would resonate with teens and the wider community. We were excited to witness the content creation process which included elements of collaboration, experimentation, and flexible grouping to support peer facilited tech-help.

Why Digital Content Creation?

Digital tools are transforming what it means to be literate in today’s world. In the past, it may have been that decoding words on a page was enough to consider a student literate. Today, we live in a world with ever increasing importance on digital tools and technologies as a means of accessing and sharing ideas.  Students need to become facile with the full range of communicative tools, modes (oral and written), and media. Having the ability to comprehend, critically respond to, and collaboratively compose multimodal texts will play a central role in our students’ success in a digital information age (Partnership for 21st Century Skills, 2007; IRA, 2009).

Setting the Context for Digital Content Creation

Melanie’s class  worked to actively create projects that resonated with their intended audience without needing elaborate direction with the use of iPad apps. First, we provided a basic overview of the affordances of three digital composition apps (ShowMe www.showme.com, VoiceThread www.voicethread.com, and iMovie for the iPad www.apple.com/ipad/from-the-app-store/apps-by-apple/imovie.html – these three content creation apps were chosen because they allow users to integrate still images, include a drawing tool, and have the capacity to include voice and sound effects).  Then, we shared an example product from each app and students were off and running. They soon discovered many features of the apps themselves as they worked.  This new knowledge was distributed throughout the classroom as peer support and flexible grouping was implemented.

Students completed digital products can be viewed from their student-created website Hitting The Road Safe http://hittingtheroadsafe.webs.com and at Safe Driving VoiceThreads https://sites.google.com/site/swandby/safe-driving-voicethreads and also ShowMe http://www.showme.com under the username jill.castek@gmail.com and password safedriving.

iMovie. While using iMovie, students worked in groups and took on different roles such as creators, actors, and editors. Collaboration came in many forms, for example, some students did not want to appear on camera, but were willing to write a script and film a partner.

 Other groups took turns incorporating found pictures and discussing sequencing to communicate a strong, clear message. Because of the ease of use and multiple options within the iMovie app, the editing process can become never ending.  To support a more skilled use of the app, we pointed students toward a YouTube editing tutorial. Students who found themselves with extra time added captions or experimented with the background music offered within the tool. These “extras” gave the movies a professional feel while extending the students’ knowledge of the technology and supportive the processes of reflection and revision.  While the iMovie app proved easy for students to navigate, explore, and edit, teachers would be well advised to guide students through ample planning of their project during their first few interactions with this tool.

ShowMe.  Possibly the greatest successes were achieved with students use of the ShowMe app. Like iMovie, it produces a video, but its affordances allowed students to deliver the most complete, succinct messages of all three tools (student work is available at showme.com under the username jill.castek@gmail.com and password safedriving). During the showcase at the end of the project, the student audience commented on the ability for students to appropriate humor about a serious topic to be showcased. This was achieved through the use of voice, drawing, and integration of selected images. This app has limitations in the amount of media that can be uploaded and may have prompted the students to choose wisely from their options, making the message clear rather than being lost in elaborate visuals.

From the first introduction of this app, the students demonstrated an eagerness to peruse the tools and begin incorporating images, drawing and voice together rather than compiling images for a later use (a pattern we noticed with other tools). Even after several projects were lost due to glitches with the system, students simply started over learning from their mistakes, making strategic use of the drafting process, and integrating their new knowledge into final products.

VoiceThread.  This tool offered the most structured means of conveying ideas and the students took to the tool readily.  Once slides containing images were created, they could be moved around as the message was drafted and revised. Once sequenced, students could voice over the visuals to communicate their message.  Completed VoiceThreads can be viewed at https://sites.google.com/site/swandby/safe-driving-voicethreads.

Students created multiple drafts of their VoiceThread project and practiced their voiceover several times to ensure the tone and quality of the message was spot on. Unfortunately, the VoiceThread interface selectively saved some of voiceovers, which required students to re-create their projects more than once.  However, this redrafting wasn’t something students balked at and the message conveyed in each subsequent draft was more extensive, and richer in vocabulary and details.  The limits of the technology were not discouraging, but rather a valuable introduction to the process of creating technology-based multimodal products.

What Did We Learn?

Students completed projects included a logical sequence but also incorporate personal touches through the use of music, voice, sound effects, and pictures remixed and used in creative ways.  By including a specific focus on intended audience, Melanie’s students were readily able to form and frame a persuasive message. For example, students who chose parents of teen drivers as the target audience drew on experiences from their out-of-school lives and combined them with statistics from a school-based text. This resulted in charts and graphs representing percentages, an articulated message free from teenage jargon and pictures free from gore (as opposed to an increased shock value to presentations geared toward teen drivers).

Collaboration is key. Collaboration was widely fostered by encouraging students to turn to each other as resources and to help each other figure out how to accomplish their goals. For example, one group of students was using the ShowMe app and wanted include text in their presentation (there is no feature in which students can type using a keyboard). Students offered each other a workaround demonstrating the use the notepad feature and taking a screenshot to import it into the project. Other students offered another option and hand-wrote text on a piece of paper in bold marker and took a picture to import into the project.  Still others shared how to use their finger to write the message manually. As was the case here, students often knew what feature that they wanted and found innovative ways to use the app to meet their goals. These observations reinforce the idea that step-by-step instruction by a teacher is not necessary before students use new apps.  We discovered taking the time was not worthwhile and may, in fact, detracted from the collaborative and discovery nature of the work and curtail digital competence.

Time for experimentation is vital.  We recognized at the outset of the project thatstudents were eager to learn how to use the apps offered to them in the act of content creation.  While our instincts told us to model for students, it became increasing clear to us that experimentation with the apps supported student learning much more efficiently.  It became evidence that when technology is being used, a new role for the teacher is created.  She is no longer the “sage on the stage” and must be more comfortable circulating to support implementation by being the “guide on the side.”

Creativity and humor were strategically to convey ideas. As students created their projects, they infused persuasiveness through their use of creativity and humor.  Creativity extended well beyond being able to draw well.  When asked to reflect on the project, students reported being more engaged in the digital creation process, than the paper and pencil task (even though they needed to develop digital skills quickly to use the tools).  They also enjoyed viewing the projects created by other classmates (even though they were very familiar with the content contained within them).  Students created multiple digital drafts of their project (and were glad to do so).  They appeared to use the multiple drafts to improve the project iteratively.  If a student wanted to revise or rethink a portion of the digital creation, the opportunity to do this was manageable as opposed to the static poster version from which the students began. As pairs worked collaboratively, new ideas for improvement were shared amongst partners, which led to subsequent (improved) drafts. Even though students might have stumbled through the first couple of tries, they got better at it each time. Persistence was key!

Student Insights

Through the implementation of this project, we aimed to test a process by which students could create digital products (including drawings, images, and voice)  that could be shared with a school and community audience.  At the end of the project, students were asked to share what was different about digital content creation. One student remarked, “It’s more creative and more fun to play around with. It’s more exciting. You can put your voice into it and you can make it more fun.” This student aptly points out that digital projects are flexible.  If a student wants to revise a portion of the digital creation, this is manageable. In contrast, changes on a static page can be messy or difficult and offer little room for rethinking of an idea. Another student shared, “You can use funny pictures but you can still have a serious message.”  This learner points out that students could develop and incorporate their own multifaceted literacies. Although humor was never mentioned as a component of the project, students freely infused their personalities through media to reach their intended audiences on a level that demonstrated a high degree of literacy skill. A third student pointed out, “It’s a lot faster than when we usually do projects, you can write in different ways like voicing your message.”

Communicating with a Real Audience

In viewing the final projects,  the audience (made up of members of the school and community) found the addition of suspenseful music, images, and the story-lines conveyed through multiple modes generated a tangible impact that was memorable. Witnessing the audience’s reaction interaction was one way that the students owned their success. It was clear that all students felt accomplished and through the act of digital content creation, they became more skilled in the digital literacies that are a vital  part of our 21st century world.


Common Core State Standards Initiative. 2010. Common Core State Standards for English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects. Available at http://www.corestandards.org/assets/CCSSI_ELA%20Standards.pdf

International Reading Association. (2009). Integrating literacy and technology in the curriculum: A position statement.  Newark, DE: International Reading Association.

Partnership for 21st Century Skills. (2007). Learning for the 21st century. Available at http://www.21stcenturyskills.org/reports/learning.asp

%d bloggers like this: