I usually don’t participate in social media, but since I worked with Grant, and since my and others’ research on authentic intellectual work since then launched further studies and professional development to implement the framework in US schools and abroad I’d like to respond to Grant Wiggin's blog on authentic assessment.
Origins of Concern for Authentic Assessment
Education reformers in the1980’s voiced many criticisms of common testing procedures (multiple choice, short answer, true-false). Most of the critiques boiled down to the objection that, because they are designed mainly for convenience of scoring, and in the case of standardized tests, in order to achieve a statistically normal distribution (where half the population must score below the mean) such tests measured only meaningless, trivial, contrived forms of mastery. Such tests failed to indicate students’ ability to master complex intellectual tasks, or to explain solutions to important problems, both within and beyond academic disciplines.
Beyond the nature of tests, other procedures were criticized. Grading was as unfair and invalid. As secret enterprise in which the teacher’s criteria were usually not available to the student or the public, a grade gave no indication of what kind of mastery had been demonstrated. Grading on the curve, rather than by demonstration of mastery, like standardized testing, denied half of the students the opportunity to succeed. Teachers making only “final” judgments about the quality of student work, without prior feedback and an opportunity for students to improve, had no instructional value.
Doug Archbald and I (Beyond standardized testing: Assessing authentic academic achievement in secondary schools. National Association of Secondary Principals, Reston, VA, 1988) summarized these concerns and presented examples of assessments that addressed many of them. We began by noting that traditional assessments, especially standardized tests, communicate very little about the quality or substance of students’ intellectual accomplishments. As mentioned above, the type of accomplishment measured is usually considered trivial, meaningless and contrived by students and adult authorities as well.
We realized before suggesting new assessments that addressed this criticism, we first needed to identify criteria that would identify worthwhile, significant, and meaningful intellectual accomplishment. But instead of beginning with abstract notions of intellectual quality, we considered examples of intellectual accomplishment by successful adults in diverse endeavors (e.g. scientists, historians, jurists, literary and artistic critics, journalists, physicians, designers, skilled technicians and trades people). From these examples, we tried to deduce characteristics that differentiated their intellectual work from most of the work students do in school. We proposed three main criteria: disciplined inquiry, integration of knowledge, and value beyond evaluation (production of discourse, things and performances that have utilitarian or aesthetic value apart from determining the competence of the learner). We considered achievements that met these three criteria “authentic” in the sense that they represented the kind of intellectual work done by adults that would be considered worthwhile, significant, and meaningful.
From this perspective there is no particular technique or set of techniques that constitute an authentic assessment. Instead, an authentic assessment can be any task or activity that requires, or makes demands, for authentic intellectual work.
Research and Development on AIW Since the 1980’s
Since the 1980’s I, along with many colleagues at the University of Wisconsin and beyond, published many research reports on instruction and assessment that promotes authentic intellectual work (while the research in Chicago that Grant mentioned was among the most significant, several other studies also offer important evidence on the power of AIW), and on professional development to help teachers teach toward AIW. Grant Wiggins cited one of the most recent publications. It is Newmann, King and Carmichael (2007), Authentic Instruction and Assessment: Common Standards for Rigor and Relevance in Teaching Academic Subjects. Des Moines, IA: Iowa Department of Education, available at http://centerforaiw.com/sites/centerforaiw.com/files/Authentic-Instruction-Assessment-BlueBook.pdf.
This book
a) refines the criteria for authentic intellectual work into the following categories: Construction of Knowledge (organizing, interpreting, evaluating, or synthesizing prior knowledge, rather than retrieving or reproducing it); Disciplined Inquiry (having a prior knowledge base, in-depth conceptual understanding, elaborated communication), and Value Beyond School (solving problems that have utilitarian, aesthetic, or personal value beyond demonstrating success with school tasks);
b) summarizes research showing strong positive relationships between teachers who promote AIW and student achievement on measures of authentic intellectual work and on standardized tests as well (the most recent and comprehensive review of literature related to the AIW framework can be found in Saye, J. and Social Studies Inquiry Research Collaborative, 2013. Authentic pedagogy: Its presence in social studies classrooms and relationship to student performance on state-mandated tests. Theory and Research in Social Education, 41:1, 89-132);
c) provides examples of student work, teachers’ lessons and assessments that meet the criteria to varying degrees;
d) outlines a professional development program to implement the AIW framework.
Since 2007 the Center (www.centerforaiw.com) has worked with the Iowa Department of Education and districts elsewhere in more than 100 schools and 3000 teachers to implement the AIW framework. We are in the process of revising the 2007 publication to refine the presentation of the AIW framework and report in more detail on professional development to sustain such work. Contact the Center for more information.
Comparing Our Work to Wiggins
Grant Wiggins’ comments here that ‘authentic tests are representative challenges within a discipline” are consistent with our criterion of disciplined inquiry, but this represents only one of our three criteria. To be sure, it could be argued that representative challenges within a discipline must necessarily involve construction of knowledge. In this sense, it seems that Wiggins embraces at least two of our three criteria. Construction of knowledge, however, may not always represent a challenge within a discipline, as when teachers ask students to organize information in new ways not relevant to a specific discipline (“Write a story that illustrates possible problems in always getting what you wish for”).
We agree with Wiggins that “hands-on” or “real world” activities do not necessarily demand authentic intellectual work. Our third criterion “value beyond school” requires only that the intellectual challenge has utilitarian or aesthetic meaning beyond demonstration of competence demanded in school. Students in a classroom could be given dimensions of a bedroom, along with information about paint such as square feet covered per gallon and price, then asked to calculate the cost of painting the room. The activity occurred in school and did not involve “hands-on” activity, but the problem occurs in the world beyond school and solving it has utilitarian meaning, beyond showing the teacher that the student knows certain mathematical definitions and applications.
The main difference that I notice between our work and Grant’s is that our framework is more concise and, we think more useful to teachers. We focus exclusively on the nature of authentic intellectual work by adults wherever it occurs, and articulate just three main criteria for recognizing it. We deliberately do not suggest techniques for assessing it –only that whatever techniques are used make intellectual demands that call for construction of knowledge, through the use of disciplined inquiry, to produce discourse, products, or performances that have value beyond school. In contrast, Grant proposed an “unwieldy” (his words) set of about 27 criteria, dealing with multiple issues, that “might bear on authentic assessment.” Still, many educators have valued Grant’s work.
Though we chose different paths to authentic assessment, I respect his contribution and believe the conceptual compatibility between our perspectives is on solid ground.