Stop Worrying About Shanghai, What PISA Test Really Tells Us About American Students

Sputnik replica
Sputnik replica

The latest results from the Program for International Student Assessment (PISA) are public, and already some pundits are declaring it “a Sputnik wake-up.” Others shout back that international comparisons aren’t valid. Rather than wade into that debate, I’d rather look more closely at the questions in the PISA test and what student responses tell us about American education. You can put international comparisons aside for that analysis. 

Are American students able to analyze, reason and communicate their ideas effectively? [Think Common Core standards] Do they have the capacity to continue learning throughout life? Have schools been forced to sacrifice creative problem solving for “adequate yearly progress” on state tests? For more on that last question see my post “As NCLB Narrows the Curriculum, Creativity Declines.” 

PISA provides some answers to those questions and offers an insight into the type of problem solving that rarely turns up American state testing. FYI: PISA is an assessment (begun in 2000) that focuses on 15-year-olds’ capabilities in reading literacy, mathematics literacy, and science literacy. PISA assesses how well prepared students are for life beyond the classroom by focusing on the application of knowledge and skills to problems with a real-life context. For more examples of PISA questions and data click here. 

Do American students learn how to sequence or simply memorize sequences

Here’s one insight into what American students can (and cannot) do that can be gleaned from the 2003 PISA test results. We spend a lot of time in school getting students to learn sequential information – timelines, progressions, life cycle of a moth, steps for how to. Typically the teacher teaches the student the sequence and the student correctly identifies the sequence for teacher on the test. Thus we treat a sequence as a ordered collection of facts to be learned, not as a thinking process for students to use.  This memorization reduces the student’s “mastery” of the chronology to lower order thinking. I was guilty of this when I first started teaching history “Can someone give me two causes and three results of WWII?” 

Sample sequencing problem from PISA

The Hobson High School library has a simple system for lending books: for staff members the loan period is 28 days, and for students the loan period is 7 days. The following is a decision tree diagram showing this simple system:

Pisa-1

The Greenwood High School has a similar, but more complex library lending system:
All publications classified as “Reserved” have a loan period of 2 days.
For books (not including magazines) that are not on the reserved list, the loan period is 28 days for staff, and 14 days for students. For magazines that are not on the reserved list, the loan period is 7 days for everyone.
Persons with any overdue items are not allowed to borrow anything. 

Task

Develop a decision tree diagram for the Greenwood High School Library system so that an automated checking system can be designed to deal with book and magazine loans at the library.  Your checking system should be as efficient as possible (i.e. it should have the least number of checking steps). Note that each checking step should have only two outcomes and the outcomes should be labeled appropriately (e.g. “Yes” and “No”).

Student Results

Only 13.5% of US students were able correctly answered the question. Does it really matter if students in Shanghai did any better? (The student results were rated on a rubric scale.) 

When students are asked to observe a process and develop a sequence they have an opportunity to use a full spectrum of higher-order thinking skills – they must recognize patterns (analyze), determine causality (evaluate) and then decide how they would communicate what they’ve learned to others (create). Sequencing can be taught across the curriculum at a variety of grade levels – we simply have to ask the students to observe and do the thinking.

In case you’re wondering,  correct response should look like this.
Click image to enlarge.

pisa answer
pisa answer 
 

Image credit/ NASA
 

Education for Innovation or More Test Prep?

Intel is hosting an education digital town hall at the Newseum that will explore new ways to “cultivate tomorrow’s thinkers and entrepreneurs to sustain economic and educational success.” (December 7 at 8:45 a.m. – 11:45 EST) Participants include Education Secretary Arne Duncan; Angel Gurria, the Secretary General of the Organization for Economic Co-operation and Development; Rob Atkinson with ITIF; and Tom Friedman of the New York Times.

Let’s see how the Duncan sidesteps the issue of testing and innovation – while US students spend endless hours honing their test taking skills, the demand for routine skills has disappeared from the workplace. Anyone know of a meaningful and rewarding career that looks like filling out a worksheet? Maybe Friedman will be willing to tackle the stifling impact of testing on creativity thinking among our students. For my thoughts on the subject, see my post “As NCLB Narrows the Curriculum, Creativity Declines

“Education for Innovation” a live digital town hall 

Watch the video here.

You can submit questions you would like the moderators, PBS NewsHour’s Gwen Ifill and Hari Sreenivasan, to discuss with the speakers. Then, vote the questions you like best to the top. Click here

You can join the for the live, interactive webcast on Tuesday, December 7 at 8:45 a.m. – 11:45 EST or join the conversation at Twitter/InnovationEcon use the hashtag #Ed4Innovation
 

PISA-sample

 

More on the Program for International Student Assessment (PISA)

PISA is an assessment (begun in 2000) that focuses on 15-year-olds’ capabilities in reading literacy, mathematics literacy, and science literacy. PISA studied students in 41 countries and assessed how well prepared students are for life beyond the classroom by focusing on the application of knowledge and skills to problems with a real-life context. For a detailed example of how PISA assesses sequencing skills see my post “Why Don’t We Teach Sequencing Skills?

 

For more PISA questions in reading, math and science see my blog post “Are Students Well Prepared to Meet the Challenges of the Future?” You can find some great critical thinking questions to use with your students

 

Response to sample question
This short response question is situated in a daily life context. The student has to interpret and solve the problem which uses two different representation modes: language, including numbers, and graphical. This question also has redundant information (i.e., the depth is 400 cm) which can be confusing for students, but this is not unusual in real-world problem solving. The actual procedure needed is a simple division. As this is a basic operation with numbers (252 divided by 14) the question belongs to the reproduction competency cluster. All the required information is presented in a recognizable situation and the students can extract the relevant information from this. The question has a difficulty of 421 score points (Level 2 out of 6).

Please Vote for Elementary School Engineers

Inventor1 A colleague and creative friend, Brian C Smith is currently in the running for an H P EdTech Innovators Award with an innovative proposal for “The I.D.E.A Room.”  His project is based on his 4th-grade “Playful Inventors” workshop that Brian and his wife Wendy, (a STEM coach)  piloted in 2009. The project had great success integrating the arts, science, engineering, and technology in creative problem solving environment

 

Please take a moment to read their proposal
 and
cast your vote here.
  

More on the project from Brian …

Our faculty explored inquiry-based models of instruction and wanted to experiment with implementing a fully student-centered learning experience.  After analyzing data from NYS 4th grade test scores, problems under the physical science realm were identified as most troublesome.  Given their action research idea and the identified areas of weakness in science, a team of teachers designed the I.D.E.A Room program to provide students with opportunities to explore physical science concepts through the engineering design process while using technology as an integral component of their work. The Playful Inventors workshop, an after-school program implemented in the fall of 2009,  allowed students freedom of time to play, explore, design, test, and problem-solve.  Highlights of the success of the program include:

    •    Increased problem-solving strategies

    •    Reliance on cooperative learning

    •    Integration of the arts, science, engineering, and technology

    •    Creative uses of materials 

    •    Increased proficiency with technology, including computer programming

    •    Deeper understanding of key concepts of force and motion

Inventor2  Our most important initiative is to continually shift instructional practices to become constructivist in nature using inquiry-based methods. We have discovered that in the classrooms where this is the norm, students are more self-directed in their learning, willing to take risks, creative in their approaches to problem solving, and demonstrate stronger team approaches to learning.  In the I.D.E.A Room, projects are personal, yet learning is both iterative and social.  The work by the teachers on the I.D.E.A. Room project has built the foundation for this instructional shift.  

Our second initiative is to increase the use of technology to facilitate learning for both students and teachers. Students participating in the pilot program were able to use a wide variety of technology tools for learning, collaboration, and creation of content.  Both the teachers and students in this group will be instrumental in assisting others to learn how technology can be transformative.

At the end-of-the-year I.D.E.A. Room Community Workshop, students will collaborate to create their multi-media presentation and practice their presentation skills. The Jr. Engineers will facilitate the hands-on stations as community members, including invited engineers from local industry experts, business owners, parents, and others try their hand at creating, designing and programming using the I.D.E.A. Room materials.

Picturing the Story – An Interdisciplinary Approach to Culture, Environment, Language, and Learning

Fox and the Heron I’m pleased to served as an advisor to a new interactive resource for teachers and students. “Picturing the Story: Narrative Arts and the Stories They Tell” uses world art from the permanent collection of the Memorial Art Gallery dating from 1500 BCE to the 20th Century. Each work has a story to tell, either visually through imagery and symbol, or indirectly through custom and ritual. The stories reflect sacred beliefs, folk traditions, common human experiences, or unique cultural practices. 

Each work of art includes downloadable resources – the story that inspired it, the culture where it originated, the techniques used to produce it, as well as extensive lesson plans, activity suggestions, and recommendations for further reading. The lessons and stories are designed to be used at a variety of grade levels. 

Downloads resources include: 

  • Classroom Copy – Printable, condensed version of online materials, copy-ready for classroom use. 
  • Curriculum Connections – Organized by subject area:  Social Studies, ELA, Science, Art/Art History. Lesson extensions, children’s book recommendations, and activity suggestions, most with accompanying activity sheets.  
  • Learning Skill-based Activity Sheets:  Printable, copy-ready sheets that address specific learning skills, for classroom use with online materials or printed classroom copy. Includes skills such as constructing comparison, identifying context, recognizing sequence and many more.

Detail - Rama Sita and Lakshmana Return Every work of art has a story to tell. These stories can explain the unexplainable, teach a life lesson, or celebrate our common human experiences. Each work in this collection is approached from three different perspectives: 

1. Picturing the Story: Viewing a work of art while reading/hearing/seeing its associated story. The story is available as on-screen text, as an audio file voiced by a professional storyteller, in ASL video interpretation, or printable pdf version. In addition, an audio “guided-looking tour” of the work of art by a museum educator helps focus attention on important details and promote visual and verbal looking skills. 

2. Reading the Art:  Understanding the symbolism and references used in this work of narrative art. High-quality images of works of art, with zoom-able details, comparison or supporting images, and interpretive information connect elements of the work of art to its associated story.   

3. Connecting the Culture:  Exploring the cultural and functional context of this work of art. Historical and cultural context information, including maps, supporting photos, and other images, connect the work of art and the story to the cultural background, promoting document-based and inquiry-based learning.  Information addressing purpose or function of work, biographical information on artist (as available), geographical and environmental issues, and process and tools of creation allows the objects’ significance to extend into a variety of curriculum areas.

Details from: 
 “The Fox and the Heron”  Flemish, Frans Snyders ca. 1630-1640 
 “Rama, Sita and Lakshmana Return to Ayodhya”  Indian, Rajasthan, Rajput School ca. 1850-1900

Use Wiffiti to Engage Your Audience – Big Screen Live Presentation of Feeds from Twitter, Flickr and Text Messages

I’m always looking for ways to make my presentations more engaging and interactive. (A must if you’re advocating more student-centered instruction.) I’ve been using a TurningPoint ARS for years with great results and have tried live blogs at my larger workshops. As a convert to Twitter, I thought it was the logical next step. 

I’ve experimented with Twitter visualizers on my blog- StreamGraph,  TwitterCloudExplorer, and most recently, Wiffiti. When I saw how good Wiffiti looked on my blog, and I realized it would be a great way to capture the backchannel at workshops. Users can interact with Wiffiti from their mobile phones or the web. It looks great on the big screen – plus it can feed from Twitter, Flickr and text messages.  

I opened a free account and gave it a trial run at my recent workshop in Moriarty-Edgewood SD, New Mexico. It was easy to create a new Wiffiti screen with custom background. (I selected a local landmark neon sign from old Rt 66 in Moriarty.)  I set up the Wiffiti screen to capture Tweets tagged with my Twitter user name @edteck.

The evening before the presentation, I posted a Tweet asking for greetings – “Say good morning to my teachers’ workshop on old Rt 66 in NM. Where are you from? Why do you Twitter?”  As participants arrived in the workshop,  they were greeted on the big screen with encouraging words from all over the world. Pretty impressive when you’re talking about the impact of technology on teaching and learning!  Special thanks to all that sent greetings – it was an powerful demonstration of the new landscape of information and a display of the power of Twitter / social media!

I shot a bit of video to give you and idea what it looked like.  
 (Remember, the live version of this screen no longer has Tweets relevant to the workshop.)

New Wiffiti messages are instantly displayed center screen and are easily viewable from a distance. Older messages then fade back and move as an animated cloud. Updates from both mobile and web are displayed synchronously across all screens subscribing to the same tags, encouraging the creation of a wide, cross-channel audience.

 Using Wiffiti in Breakout Sessions

I also created a second Wiffiti screen to use during break outs. This one was designed to capture text messages from participants. For those that did not have cells, we set up computer stations where they could make comments directly from the Wiffiti website.

Here’s some sample comments – a nice mix of thoughtful observations, fun comments and a few critiques. (Note: I kept it real and I ran my system unmoderated, but it is possible to have someone monitor comments.) 

“School is where kids go to watch old people work really hard”

“My Brain Hurts!”

“Let’s get going!”

“disequilibrium, change, and freedom”

“same old stuff, different day!”

“the table in the back rocks!!”

“English teachers and librarians rule -all others drool!”

“having a blast!”

“science is over here.”

“This is a great workshop!”

“Enjoying the presentation Peter. Especially the film clips!”

“Rigor and Relevance for the English Department: Rigor: Apply knowledge and skills in complex ways to analyze and solve real problems…”

My bottom line? Wiffiti is a great way to harness back channel workshop comments. The free version works well and paid versions offer more opportunities to customize and monitor comments.