Assessment Literacy: An Educator’s Guide to Understanding Assessment, K-12

By Thomas DeVere Wolsey & Dana L. Grisham with Susan Lenski

We are excited to announce that Assessment Literacy: An Educator’s Guide to Understanding Assessment, K-12 by Literacy Beat bloggers Thomas DeVere Wolsey and Dana L. Grisham with guest blogger Susan Lenski is now available.

Literacy Beat readers are invited to take 20% off the list price. Just point your browser to Guilford.com or use the promo code 2E on the Guilford website.

Assessment Literacy Cover

Assessment Literacy: An Educator’s Guide to Understanding Assessment, K-12

Overview:

This clear, no-nonsense book guides current and future teachers through the concepts, tools, methods, and goals of classroom literacy assessment. The expert authors examine the roles of formative, summative, and benchmark assessments; demystify state and national tests and standards; and show how assessment can seamlessly inform instruction. Strategies for evaluating, choosing, and interpreting assessments are discussed, as are ways to communicate data to parents and administrators. User-friendly resources include boxed vignettes from teachers and researchers, practical assessment tips (and traps to avoid), and 12 reproducible planning forms and handouts. Purchasers get access to a Web page where they can download and print the reproducible materials in a convenient 8½” x 11″ size.

Thanks to our reviewers: Missy Provost, Troy Hicks, Judith Dunkerly-Bean, Paula Dreyfuss, and Linda Smetana and to Diane Lapp, author of the foreword.

Exploring Digital Assessments: How Teachers Improve Learning Outcomes

By Thomas DeVere Wolsey

EduForum 2018 at The American University in Cairo, New Cairo, Egypt

November 3, 2018

Assessment is the fuel of learning. Confronted with mountains of data, teachers can feel overwhelmed.  Particularly frustrating, some assessments interrupt instruction and distract students. In this session, we explore how to make the climb over the data mountain manageable. We will investigate effective practices to align assessments and learning outcomes through technology-driven formative assessment. Embedding assessment directly into learning activities can help teachers adjust instruction and engage students with their learning at the same time.  Learn how to create electronic exit tickets. Make a game your students will enjoy while making real-time use of feedback based on their progress through the game environment. Assessment need not be a tedious chore. Put learning assessment in service of engaging learning environments.

As part of today’s presentation on digital tools for formative assessment, we used Survey Monkey to conduct a pre-assessment of what we know about assessments.

Formative Assessment

Next, we used Edmodo to create a short portfolio with images. Next, we explored project-based learning using an online portfolio at Dreamdo. Here is an example:  https://edu.dream.do/en/dreams/sciences/the-5th-grade-geology-unit

Games are great learning tools, but they can be very useful for assessment, as well. Three online games and assessment tools we tried out were OLogy, Sim Scientist, and Kahoot!

The power of the teacher’s human voice coupled with images of a student’s own work were the focus of our exploration of screencasting.  Cambridge English videos showed us how to use screencapture and a free screencapture program, Jing,  were demonstrated.

Online tools make generating rubrics a snap. Check these out:

*Teachnology offers rubrics at: http://www.teach-nology.com/web_tools/rubrics/ but you’ll need to register.
*Rubistar, a project of ALTEC, has a useful rubric generator available at: http://rubistar.4teachers.org/index.php
*Google add-in by dostuffgood.org Create and send customizable rubrics and scores to students by email for any assignment. https://chrome.google.com/webstore/detail/online-rubric/fiiglmgmcodoglllnbfebbhkfidikfbo?utm_source=permalink

Finally, we examined the role and possibilities of Big Data and educational assessment. Read more here and here.

800px-DARPA_Big_Data

Want to join our classroom? Navigate to classroom.google.com and enter the code in the image below. Some of the tasks have expired links, but you can see how the demonstration was set up.

Class Code

Learn more about EduForum 2018:

Link: http://eduforum-eg.com/index.php/sessions/exploring-assessments-how-teachers-improve-learning-outcomes/

Link: https://events.aucegypt.edu/?event=14151038

Meet the Influencer: Don Leu

Influencers Banner

Influencers

Don Leu is a colleague, mentor, and friend to the Literacy Beat bloggers, and he has consistently influenced our research since we met him. Don and the New Literacies Research Lab always have something innovative in the pipeline to lead our thinking. In this post, we are very pleased to introduce Don to you.  We asked Don to tell us about the ORCA project, Online Research and Comprehension Assessment. ORCA addresses the need for assessments and resources for online inquiry and research in our schools.  Read Don’s response to learn more about ORCAs and find the professional development resources that support it, all provided as a public service. 

Don Leu

Don Leu

What is Orca? 

Central to our students’ success in life will be the ability to conduct inquiry online in order to learn (Organisation for Economic Co-operation and Development [OECD], 2011; Rideout, Foehr, & Roberts, 2010) What does this process look like and how might we determine our students’ ability in this area so we can prepare appropriate instruction?  The ORCA Project (http://www.orca.uconn.edu) recently developed eight authentic assessments to measure online inquiry skills in science (human body systems).  The assessments are now freely available online.   A video describing these assessments is also available (see below).


The assessments appear in two formats: ORCA-Multiple Choice  (or ORCA-Closed) and ORCA-Simulation.  In each, students conduct online research about an important question in science and responses are largely auto-scored. Both formats have demonstrated acceptably high levels of reliability and validity, though the ORCA-Simulation has demonstrated a 10% higher level of reliability, compared to ORCA-Multiple Choice (See Leu, et al., 2014).

Our research with representative state samples of 1,300 students in Maine and Connecticut shows that, on average, 7th graders only perform successfully on about half of the skills required in online research, suggesting that they are not fully prepared in this area.  It also shows students are especially weak in critical evaluation skills and communication skills.  (See Leu, et al., 2015)

You are welcome to use these assessments for instruction, assessment, or professional development.  They may be accessed online without cost. A professional development module is also available.

ORCA

ORCA

References

Leu, D. J., Forzani, E., Rhoads, C., Maykel, C., Kennedy, C., & Timbrell, N. (2015).  The new literacies of online research and comprehension: Rethinking the reading achievement gap. Reading Research Quarterly, 50(1). 1-23. Newark, DE: International Literacy Association. doi: 10.1002/rrq.85. Available at: http://www.edweek.org/media/leu%20online%20reading%20study.pdf

Leu, D. J., Kulikowich, J., Sedransk, N., Coiro, J. Forzani, E., Maykel, C., Kennedy, C. (April 4, 2014). The ORCA Project: Designing Technology-based Assessments for Online Research, Comprehension, And Communication, American Educational Research Association. Philadelphia, PA.

Organisation for Economic Co-operation and Development [OECD.](2011). Students on line: reading and using digital information. Paris: OECD. Available at  http://dx.doi.org/10.1787/9789264112995-en

Meet Don:

Donald J. Leu is the John and Maria Neag Endowed Chair in Literacy and Technology at the University of Connecticut. He holds a joint appointment in Curriculum and Instruction and Educational Psychology in the Neag School of Education. A graduate of Michigan State, Harvard, and Berkeley, he is an international authority on literacy education, especially the new skills and strategies required to read, write, and learn with Internet technologies and the best instructional practices that prepare students for these new literacies. Don directs the New Literacies Research Lab in the Neag School of Education. He is a member of the Reading Hall of Fame, Past President of the Literacy Research Association, and a former member of the Board of Directors of the International Literacy Association.

Find Don at the University of Connecticut and the New Literacies Research Lab.

Student Oral Language Observation Matrix: Spreadsheet Style

By Thomas DeVere Wolsey

A time-tested standby to help teachers understand English learners’ oral language proficiency is the SOLOM or Student Oral Language Observation Matrix. The instrument is not a test, but it is an informative assessment that teachers use to inform instruction. There are many versions in html, Word, and PDF, but an interactive version in Excel (.xls) may prove useful.

SOLOM and Excel

SOLOM is in the public domain, so you may find some variations in the various published versions of the Matrix.   Teachers and teacher educators use the Matrix, developed by the San Jose (California) Bilingual Consortium, for a variety of purposes:

  • It fixes teachers’ attention on language-development goals;

  • It keeps them aware of how their students are progressing in relation to  those goals; and

  • It reminds them to set up oral-language-use situations that allow them to observe the student, as well as provide the students with language-development activities.

    Source: Center for Applied Linguistics

Download SOLOM (Excel)

Download SOLOM (Excel) here.

The Box.net file opens in preview mode; to download, find the upper right ↓ download arrow. Figure 1 shows where to locate the download icon.

SOLOM (Excel)

Figure 1: Download from Preview Mode – SOLOM Excel

What are the advantages of the Excel version of SOLOM?

  1. You can replicate this SOLOM digitally without killing any trees (no paper needed).
  2. You can add sheets for each student to keep all your results in one file.
    • Each sheet is accessed by the tabs at the bottom, left, of the spreadsheet. See figure 2. This template includes three sheets, but you can add more if you need them. Start here to learn how.

      Excel Tabs

      Figure 2: Excel Tabs

    • If you choose to do so, you can calculate results across sheets – a topic for a future post.
  3. The Excel spreadsheet does the calculations for you, an important feature if you have many students’ results to enter.

Want to review some common Excel terms? Navigate here. Maybe you want to dive into Excel vocabulary a bit more deeply? Point your browser here.

SOLOM Practice on YouTube

Several good YouTube videos allow you and your colleagues to practice using SOLOM. Try this one. This Playlist may also be useful.

Excel Geek?

If you happen to be an Excel geek, you can read this paragraph. Otherwise, just skip to the “Sources,” below. Excel is a powerful spreadsheet that harnesses the calculating abilities of the processor on your computer (or in the cloud). This version of SOLOM employs the COUNTA function to actually count the number of entries for each column, the SUM function to add up the column totals, and the VLOOKUP function to assign overall scores to a proficiency level. Shout out to gebobs for helping me find the function I should use instead of the one I was unsuccessfully trying to use!

SOLOM Sources

The best original source for SOLOM I can find is found at http://www.cal.org/ and opens as a PDF.

I adapted SOLOM for Excel from Arch Ford Educational Service Cooperative;  in Word format at SOLOM. (note: I removed the word “even” from cell B8).

Please share your variations and adaptations of SOLOM (Excel) in the comments section. What might you do to improve this tool? How have you used apps other than Excel to improve SOLOM or similar assessments

Pecha Kucha, a Presentation Format with Many Possibilities

By guest posters W. Ian O’Byrne & Sue Ringler Pet, & regular blogger Thomas DeVere Wolsey

The nature of literacy is rapidly evolving and these changes demand an expanded view of “text” to include visual, digital and other multimodal formats (Rose & Meyer, 2002; New London Group, 2000; Alvermann, 2002). A richer and more complex definition of literacy requires a complex theoretical framing of the “multiple realities” that exist between educational research and practice (Labbo & Reinking, 1999).  Several colleagues* decided to experiment with the pecha kucha presentation style at a session of the Literacy Research Association, December 5th, 2013. What they learned from the session and their ideas for PK-12 classrooms and teacher preparation coursework is summarized in this post of Literacy Beat. Our pecha kucha session used multiple methods united by similar perspectives to investigate shifts in the space and stuff (Lankshear & Knobel, 2006) of learning.

Evolving pedagogical models for new literacies and emerging technologies hold “explosive possibilities” (Barab & Kirshner, 2001) for reading and writing spaces. Specifically, these studies examine literacies as cutting across chronotopes of time and space (Bakhtin, 1937) and evolving into “communities of inquiry” in which participants require new knowledge and identities (Gee, 2005).

Since the technological advances documented in these studies drove much of the change that we see in information and communication, researchers and educators attempted to answer the important question:  How can the use of new and digital literacies in instruction enable “explosive possibilities” for meaning-making and identity construction? These studies examined literacies and digital texts while documenting perceived changes in social practice through the lens of teachers and students as agents of change.

What is Pecha Kucha?

Pecha kucha, Japanese for the sound of conversation, is a presentation method in which 20 slides are shown for 20 seconds each (6 minutes and 40 seconds in total). The format utilizes images more than words, keeps presentations concise and fast-paced, powers multiple-speaker events, keeps the interest level up, and gives more people the chance to show. Would you like to hear several Japanese speakers pronounce the term? Click here.

Teachers and Students use Pecha Kucha

Pecha kucha is well-suited for the age of the Common Core and other rigorous standards.  The Common Core calls for students to evaluate information from diverse sources, present information in an appropriate style, and make strategic use of digital media. Further, the pecha kucha style requires student presenters to be concise and choose their words and images wisely and well. Students might present pecha kucha via webcast or video (think, YouTube or Vimeo) so that parents and other community members can participate. They may work in small groups around selected topics. Who says every presentation has to be made to the entire class, anyway?

Teacher Educators and Teacher Candidates use Pecha Kucha

The IRA standards for Literacy Professionals call for teacher candidates to employ traditional print, digital, and online resources to “meet the needs of diverse students” and “prepare learners for literacy tasks of the 21st century.” Arguably positioned in one of the most influential roles with regard to the explosive possibilities of digital literacies in PK-12 education, teacher educators must continually model well-considered integration of digital tools in university classrooms. Within the context of a disciplinary literacy course, for instance, professors may choose the pecha kucha platform for in-class presentations in lieu of the tired Powerpoint® platform, especially in cases where visuals are preferable to print text, to effectively encapsulate and express important concepts, terms, or ideas. In this setting, pecha kucha presentations can be posted and revisited on Blackboard or similar course platforms for review. Professors may also invite undergraduate and graduate students to learn and employ pecha kucha to explore and represent basic literacy concepts with digital images and metaphors — and teach them to classmates. Teaching and/learning such “basic” literacy terms (e.g., phonemic awareness, syntax, semantics) through a multimodal digital platform (pecha kucha) may lead to enriched understandings of the ways in which reading involves the coordination of multiple systems including traditional “components” theory of teaching reading instruction as well as sociocultural theories of literacy acquisition.

How to Create Pecha Kucha: Resources and More

What are the steps to creating a pecha kucha presentation?

  • This website lists presentation steps in pecha kucha format and a template is available there, as well.
  • A few tips for beginners might be helpful to teachers who want to coach their students and minimize frustration.
  • Richard Edwards suggests that pecha kucha can be easily adapted to two-person teams; that is, a 20 slide X 20 second presentation by one student can become a 10 slide X 20 second presentation by two students. He also staggers presentations over class sessions such that no one class session is devoted to a long series of pecha kucha presentations, which, like traditional presentations, can be quite tiring for the audience.
  • Because pecha kucha is image intensive, it is very important that students learn the basic principles of Fair Use and apply them. This post from an earlier LiteracyBeat column may be a good start.  Learn more about Creative Commons and how it works to give students and other users the tools to share and use the creative work of others.

Similar to pecha kucha, Ignite presentations include 20 slides but they advance at the rate of 15 seconds each (total of five minutes). Some fairly good information about both ignite and pecha kucha are available from Trinity Valley Schools (opens as a PDF).

Assessing Pecha Kucha

Of course, any presentation in a classroom is an opportunity to learn and a chance to demonstrate what has been learned.  Assessment includes the possibility of feedback about content knowledge, processes leading to learning, and presentation, speaking, and listening proficiency appropriate to the grade level. Mr. Holliday designed this rubric as a means of assessing and providing feedback on the pecha kucha format. This university rubric from iRubric takes into account content knowledge  and this  one, by Danny, is designed with the junior high or middle school audience in mind. Educator Jeff Utecht suggests that participants rate the pecha kucha presentation using a form in Google Docs for quick analysis and feedback. Also on the blog post are additional ideas and a rationale for using pecha kucha.

Typical assessments measure and provide feedback as to how the presenter met the pecha kucha criteria (including 20 slides X 20 seconds each, 6 minutes 40 seconds total), concision, design, and cohesion, as well as content. Choo (2010) suggests that makers and composers of digital texts consider the following:

•           How do words function to “relay” or contribute to the meaning of an image?

•           Where will the image be placed in relation to the words and why?

•           How much of the frame-space will the image occupy, compared to the words?

•           Is the focal point of the text on the image or on its words, and why? (p. 172)

Here is one attempt at pecha kucha by DeVere recreated from the December 2013 presentation at Literacy Research Association. It is not quite perfect (you will notice it is longer than the allotted time!), I am sure you’ll agree, but do play the video and let us know what you see.

What have you done in your PK-12 or university classroom with pecha kucha?

*Presenters at the Literacy Research Association, Dallas, TX: Kelly Chandler-Olcott (Chair), Stergios Botzakis (Discussant), Sue Ringler Pet, Greg McVerry, Junko Yukota with William Teale, Joan A. Rhodes, Katina Zammit, William Ian O’Byrne, Thomas DeVere Wolsey

Guest posters:

W. Ian O’Byrne is an assistant professor of educational technologies at the University of New Haven. Read his blog post on the topic of pecha kucha here.

Sue Ringler-Pet works at Iona College, and you can read more about her here.

References:

Alvermann, D.E. (2002). Adolescents and literacies in a digital world. New York: Peter Lang.

Barab, S.A., & Kirshner, D. (2001). Guest editors’ introduction: Rethinking methodology in the learning sciences. The Journal of the Learning Sciences,10(1-2), 5-15.

Bolter, J.D. (1991). Writing space: The computer, hypertext, and the history of writing. Mahwah, NJ: Lawrence Erlbaum Associates.

Choo, S.S. (2010). Writing through visual acts of reading: Incorporating visual aesthetics in integrated writing and reading tasks. High School Journal, 93(4), 166-176.

Gee, J. (2005). Semiotic social spaces and affinity spaces. In D. Barton & K. Tusting (Eds.). Beyond communities of practice. Cambridge: Cambridge University Press.

Labbo, L. & Reinking, D. (1999). Negotiating the multiple realities of technology in literacy research and instruction. Reading Research Quarterly, 34(4), 478-492. doi:    10.1598/RRQ.34.4.5

Lankshear, C. and Knobel, M. (2006). New Literacies: Everyday Practices and Classroom Learning. 2nd ed. Maidenhead & New York: Open University Press.

The New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92.

Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for learning. Alexandria, VA: ASCD. Retrieved from http://www.cast.org/teachingeverystudent/ideas/tes/

Essential Reading

A post from Bernadette

ira_essentials_150

Articles on the International Reading Association (IRA) websiteIRA E-ssentials, provide a range of  “actionable teaching ideas”  on a growing range of literacy topics. These articles are provided free with your IRA membership on the members only section of the website. They are  also available to non–members for a cost of $ 4.99 per article once you create an account on reading.org. You can download these pdf articles to your computer or any portable reading platform for on-the-go reading access. What is really appealing about the E-ssential topic range is that they are written by well-respected authors in the in the field of literacy (including our own Literacy Beat blogger, DeVere Wolsey). These concise articles include further suggested readings on the topic and incorporate links to multimedia content including websites, blogs and videos. All are strongly situated in real classrooms with strong classroom exemplars. Connections to the Common Core State Standards in the US are also included. Topics  are wide ranging and so far include critical literacy, vocabulary development, visual literacy, assessment, text complexity, writing workshop, motivation and engagement, graphic novels, and adolescent  literacy. Here are some of my current favourites to whet your appetite:

Digital discussions: Using Web 2.0 tools to communicate, collaborate, and create -Brian Kissel, Karen Wood, Katie Stover, & Kim Heintschel.

In this article the authors explore how students can communicate through social media like Facebook and Twitter; how students can collaborate  with others in a global classroom through blogs and wikis; and how students can become creators and composers through VoiceThread and Audioboo.

I hadn’t thought of that: Guidelines for providing online feedback that motivates students to learn– Diane Lapp, with Thomas DeVere Wolsey & Patrick Ganz

Interactions in the classroom are no longer confined to face-to-face (FtF) discussions. In this article the authors provide insights into providing formative instructional feedback  using a range of digital tools that applies the strengths of FtF feedback, in terms of intent, tone, and format, in an online environment.

Critical Literacy With New Communication Technologies -Vivian Vasquez & Carol Felderman

In this article the authors explore components of critical literacy in the classroom including the relationship between language and power and the importance of inquiry-based questions stemming from the interests of children. With the introduction of digital technologies Freire’s notion of ‘reading the word and the world’ takes on new meaning in a  flattened world of global communities. The authors explore the  transformative power of digital technologies to develop critical literacies in the classroom.

What do the PIAAC results suggest?

A post by Jill Castek

In light of the PIAAC data being released last month (PIAAC stands for Program for the International Assessment of Adult Competencies) I’ve been thinking a lot about opportunities for school-based and life-long learning.  This post focuses on what PIAAC is and reasons why might be interested in further exploring these data, and what they might suggest about the integration of technology into teaching and learning opportunities.

What is PIAAC? 

PIAAC is a survey coordinated internationally by the Organization for Economic Cooperation and Development (OECD). It assesses key cognitive and workplace skills and measures competencies needed by adults in the 21st century, including literacy, numeracy and problem solving in technology-rich environments.

PIAAC was designed to better understand the skills of the adult working-age population (ages 16-65) both nationally and internationally. It provides  international comparison of the adult workforce that will enable the United States to better understand its global competitiveness and benchmark how well education and training systems are meeting emerging skill demands. With these data, researchers can examine and analyze what conditions and factors impact skills growth, maintenance, or loss over a working-age life cycle.

Twenty-four participating countries and regions, including the United States, assessed adults in 2011–2012.  Data from this survey were released in October 2013. Nine countries will administer an additional round of PIAAC in 2014.

What do the PIAAC data show? 

There are a number of interesting and possibly surprising results brought to light by the PIAAC data.  To examine some of these patterns, check out the publications put together by the OECD available at http://www.oecd.org/site/piaac/publications.htm

In perusing these data, I learned that only between 2.9% and 8.8% of adults demonstrate the highest level of proficiency on the problem‑solving in technology‑rich environments.  Given the prevalence of technology in our world, and the proliferation of technology in our lives, I would have expected a much higher level of proficiency for the wider population.  This suggests to me that not only do we need to integrate technology more systematically into K-12 education, but that we also need to offer multiple opportunities for skill development across the lifespan.  Not doing so puts our learners at a disadvantage for college and career readiness and limits their participating in our digitally-centered world.

Education and Skills Online 

The developers of the PIAAC assessment have designed a suite of assessment tools that can be used by researchers within their own studies for a fee.  This assessment is called Education and Skills Online (E&S Online).  It is designed to provide individual level results that are linked to the PIAAC measures of literacy, numeracy, and problem solving in technology rich environments.  These valid and reliable assessment tools are a computerized measure that assesses a set of cognitive and non-cognitive skills that individuals need for full participation in modern societies. The suite of tools incorporates flexibility and adaptability to provide reliable and valid measures of critical skills associated with work, home, and community. The skills and knowledge measured include being able to understand and use printed and electronic texts, reason with numbers, and solve problems in technology environments.  If you’re a researcher working with technology, using such a measure of learning to determine the skill level of your learners (and benchmarking them to national norms) may offer you new and valuable insights.  It might also inspire you to provide more opportunities to guide learners in their use of technology.

There is a great deal to explore with the PIAAC data in terms of national and international trends.  A quick Google search for PIAAC will offer you a variety of resources to explore.  I look forward to your reflections and ideas.  Comments are encouraged and welcomed!

%d bloggers like this: