Would small data help accomplish what is called Good Data in education? Education systems around the world use big data such as standardized tests, school inspections and surveys to measure learning outcomes. Pasi Sahlberg believes that good education must be evaluated from a variety of evidence which includes both the quantitative and the qualitative. He argues that in order to understand how well schools are doing, we also need to collect “small data” using teachers’ and students’ “observations, assessments and reflections” of the teaching and learning processes in classrooms.
Small data is not a new idea of course. What is new is the technology that can now support this kind of analysis. Are school-based formative assessments better than standardized ones? What are some great examples of how small data is being used effectively in schools? Small data requires more trust in teachers’ and schools’ judgments, so how do we build that trust in education? Joining me in The Global Search for Education to talk about data is Finnish educator Pasi Sahlberg, author of Finnish Lessons: What Can the World Learn from Educational Change in Finland. Pasi received the 2012 Education Award in Finland, the 2013 Grawemeyer Award in the United States, and the First Class Knight of the White Rose of Finland in 2013.
Pasi, can you talk about the difference between big data and small data?
Big data is a commonly used term in daily discourse that often comes with a label that big data will transform the way we think, work, and live. For many of us, this is an optimistic promise, while for others it creates anxiety and concern regarding control and privacy. In general terms, big data means data of very large size to the extent that its manipulation and management present significant practical challenges.
The main difference between big and small data in education is, of course, the size of data and how these data are collected and used. Big data in education always requires dedicated devices for collecting massive amounts of noisy data, such as specific hardware and software to capture students’ facial expressions, movements in class, eye movements while on task, body postures, classroom talk, and interaction with others. Small data relies primarily on observations and recordings made by human beings. In education, these include students’ self-assessments, teachers’ participatory notes on learning process, external school surveys, and observations made of teaching and learning situations.
How do we use big data currently and what are its strengths and drawbacks?
We are entering the era of digital technology mediated learning environments in education. Promises that come with this revolution state that big data will lead to more powerful personalized learning, active student-centered pedagogies, responsive formative assessments, and increased overall efficiency in education. This said, big data in education can be seen as a purposeful or incidental recording of activity and interactions that provides variety and large amounts of analyzable data in digitally mediated educational environments. This requires new forms of ‘data skills’, especially syntheses and presentations that have led to the emergence of two new fields: educational data mining and learning analytics.
Educational data mining focuses on interpreting evidence from large amounts of noisy and unstructured data, for example, correlations between student behavior and learning. It answers questions like, “What sequence of study is the most effective for a specific student?” and “Which actions indicate student engagement and satisfaction?” Learning analytics is more concerned about applying tools and techniques of larger scale in educational settings, answering questions like “When is a student at risk of failing the course?” and “What grade is a student likely to receive without intervention?”
By definition, big data offers more versatile information about teaching and learning situations that can be used for tailoring more personalized learning solutions for students. Big data in education opens new doors also to researchers. Perhaps the greatest value of it, as Bill Cope and Mary Kalantzis conclude in the article, Big Data Comes to Schools, is “the possibility in any particular case to analyze a variety of data types using a variety of methods, cross-validating these against each other in a more powerfully holistic, evidence-based repertoire of research practices.” At the same time, however, we must be mindful of potential drawbacks of big data, especially those related to data privacy and research ethics. When big data is used to determine children’s educational pathways and thereby their future, or influence teachers’ careers in education, we have crossed the line of appropriate use of big data in education.
Another example of an emerging big data system in education is OECD’s Education GPS that hosts massive amount of data from education systems around the world. The OECD’s data, including those from PISA, is used by policy-makers to decide the directions in their own education systems based on what seems to have worked in other countries. Again, big data analysis alone without small data will probably lead to unintended outcomes in practice.
What are some concrete examples of the use of small data that you have seen in schools?
My favorite example is from my own time being a mathematics and science teacher in Helsinki. As so many of my colleagues, I soon realized that there are many students who don’t like these subjects. They thought that mathematics and sciences are too difficult to learn and often distant from their daily lives. One of the first things I learned in my classrooms was that many students, probably most of them, have self-created conceptions of science and images of mathematics. Students’ inner worlds are very difficult to capture in any other way than by helping them to think hard about their own minds and try to turn these thoughts then into tangible representations, such as stories and drawings.
I spent a lot of time then collecting small data, in and out of my classroom, to understand students’ own ideas about the mathematical and scientific worlds. Only after knowing what students imagine when they think about what mathematicians do (most of those are not positive images at all) I started to find better ways to make the world of mathematics accessible and understandable to students. Small data can help teachers to understand why some students don’t learn as well as they could in schools better than big data that most often show whether students learn or not what they are supposed to learn.
What kinds of data are we currently analyzing? What kinds of data should we be looking more closely at and why?
I have seen a notable shift during the past decade or so from teacher-collected evidence through classroom-based assessments toward digital technology mediated data. At the same time, standardized tests have become a central tool in collecting these data. Schools around the world now post their students’ test results in school. Teachers spend more time analyzing these data and trying to make sense of what the results mean in their own work with students. I am not saying that this is necessarily a bad thing. But if teachers are steered by external data from standardized knowledge tests and inspection reports, it undermines the complexities of teaching and learning in schools.
There are school systems where much broader sets of information are analyzed and used to guide the work of the school. In Finland, a central element of each school is a “student welfare team” that is made of teachers, wellbeing specialists, counselors, and school leaders. Through these first-hand small data, early interventions to help students at risk are then decided and made in schools. This team meets regularly, in most schools weekly, to discuss and process information brought in by teachers concerning their students’ wellbeing, behavior, and learning in schools.
Do you believe that school-based formative assessments are better than standardized ones?
School-based formative assessments and census-based standardized assessments serve different purposes. I think that we need them both. However, in many countries, system-level standardized tests that are used to measure all the students all the time could be gradually replaced by higher quality sample-based assessments. Teacher-led formative assessments should enrich the evidence that schools use for improvement purposes. In a way, this is a sensible combination of big data and small data to accomplish what is sometimes called Good Data in education.
Can students’ reflection on their own learning be incorporated into our standards of achievement?
Good learning includes students’ ability to understand and regulate their own learning process. This requires, however, that students have rather developed metacognitive habits and skills to understand how they learn. Students’ reflection on their own learning can be incorporated into judgment of student achievement as soon as these skills exist at an appropriate level. Again, I believe we are talking about the conditions of using small data for student evaluation here.
Small data requires more trust in teachers’ and schools’ judgments about what they do. How could trust be built in education?
Trust is perhaps the most needed single ingredient missing in many education systems today. Politicians don’t trust educators. Ministers can’t rely on school principals. And, as a consequence, there is a lack of trust in teachers who, in turn, can’t trust students. Experience suggests that trust in people and in institutions can only be built by genuine deeds that give them more responsibility and agency.
In Finland, trust in schools and education in general began to flourish soon after the government allowed schools to design their own curricula (steered by national framework curriculum), to evaluate the effectiveness of their own work, to control their budgets, and to professionalize the middle level leadership. Ceremonial speeches and political appeals have had diminishing impact on changing the culture of education in this respect. When schools have real ownership and responsibility for small data, I believe professionalism in schools will be enhanced. That is one of the conditions before trust in our teachers will start to grow.
Join me and globally renowned thought leaders including Sir Michael Barber (UK), Dr. Michael Block (U.S.), Dr. Leon Botstein (U.S.), Professor Clay Christensen (U.S.), Dr. Linda Darling-Hammond (U.S.), Dr. MadhavChavan (India), Professor Michael Fullan (Canada), Professor Howard Gardner (U.S.), Professor Andy Hargreaves (U.S.), Professor Yvonne Hellman (The Netherlands), Professor Kristin Helstad (Norway), Jean Hendrickson (U.S.), Professor Rose Hipkins (New Zealand), Professor Cornelia Hoogland (Canada), Honourable Jeff Johnson (Canada), Mme. Chantal Kaufmann (Belgium), Dr. EijaKauppinen (Finland), State Secretary TapioKosunen (Finland), Professor Dominique Lafontaine (Belgium), Professor Hugh Lauder (UK), Lord Ken Macdonald (UK), Professor Geoff Masters (Australia), Professor Barry McGaw (Australia), Shiv Nadar (India), Professor R. Natarajan (India), Dr. Pak Tee Ng (Singapore), Dr. Denise Pope (US), Sridhar Rajagopalan (India), Dr. Diane Ravitch (U.S.), Richard Wilson Riley (U.S.), Sir Ken Robinson (UK), Professor Pasi Sahlberg (Finland), Professor Manabu Sato (Japan), Andreas Schleicher (PISA, OECD), Dr. Anthony Seldon (UK), Dr. David Shaffer (U.S.), Dr. Kirsten Sivesind (Norway), Chancellor Stephen Spahn (U.S.), Yves Theze (LyceeFrancais U.S.), Professor Charles Ungerleider (Canada), Professor Tony Wagner (U.S.), Sir David Watson (UK), Professor Dylan Wiliam (UK), Dr. Mark Wormald (UK), Professor Theo Wubbels (The Netherlands), Professor Michael Young (UK), and Professor Minxuan Zhang (China) as they explore the big picture education questions that all nations face today.
The Global Search for Education Community Page
C. M. Rubin is the author of two widely read online series for which she received a 2011 Upton Sinclair award, “The Global Search for Education” and “How Will We Read?” She is also the author of three bestselling books, including The Real Alice in Wonderland, is the publisher of CMRubinWorld, and is a Disruptor Foundation Fellow.
Follow C. M. Rubin on Twitter: www.twitter.com/@cmrubinworld
Recent Comments