For decades, test developers have attempted to create a generalized electronic scoring system for texts. AI-based solutions have made some inroads, but they require huge amounts of text to produce results and aren't yet reliable enough for use in individual assessment. Moreover, like the electronic scoring systems now being produced by educational test developers, they measure easily quantified aspects of texts (like vocabulary and punctuation) rather than what we really want to know—whether or not we are providing optimal support for the development of the agile optimally developed minds that are required to thrive in our rapidly changing, crisis-ridden world.
CLAS is the only computerized developmental scoring system that measures mental development reliably enough to justify its use in individual assessment contexts. It's a truly general system that can readily be adapted for use in diverse knowledge domains. Rather than measuring whatever aspects of texts are most easily quantified, CLAS measures growth along a well-validated developmental scale called the Lectical Scale. Its algorithms are based on a deep study of learning and development involving human expertise and analytics. When CLAS produces a score, it tells us where an assessment performance lands on the Lectical Scale.
As of this writing, CLAS algorithms are based on human and computer analysis of more than 50,000 assessments (and growing). At its core is the Lectical™ Dictionary—the world's first developmental dictionary—a constantly evolving curated taxonomy of the development of meanings. (More about the Dictionary below.)
Like the LAS (Lectical Assessment System), our human scoring system), CLAS can be used:
CLAS has several advantages over the LAS:
Since its initial introduction in 2014, CLAS has been used to score thousands of assessments for several research and evaluation projects.It now does alll of Lectica's scoring (with human oversight).
The Lectical Dictionary is at the core of everything we do here at Lectica. It not only makes CLAS possible, but also helps us describe learning sequences and develop learning resources like skill maps and formative report feedback.
The Lectical Dictionary is composed of units of meaning called "Lectical Items"—words or phrases like "evidence" and "reliable evidence" that carry meaning. Each Lectical Item is assigned to a Lectical Phase (1/4 of a Lectical Level), based on a combination of empirical evidence, the judgment of our analysts, and a variety of helper algorithms. The goal is to assign items to the lowest level at which the simplest meaning they carry is likely to be useful.
For example, an examination of items containing the word evidence reveals easily observed progressions in the development of its meaning, such as the following:
Lectical Dictionary entries begin with first speech and cover the full span of development. In addition to being assigned to a Lectical Phase, many of the items in the Lectical Dictionary have been assigned to thematic strands such as deliberation, conflict resolution, the physics of energy, or evidence. For example, at this writing, over 35,000 terms in the Lectical Dictionary relate to evidence. Longitudinal and cross-sectional analyses of sequences like the evidence sequence have demonstrated that each successive conception builds upon previous conceptions (Dawson & Gabrielian, 2003; Dawson- Tunik, 2004). These findings are entirely consistent with the developmental theory upon which the dictionary is based (Fischer, 1980; Piaget, 1985), and suggest that Lectical Items assigned to a particular phase can be said not only to represent the understandings of that phase but also play the role of building blocks for future conceptions. Because of this, when we look at the distribution of Lectical Items across phases within a given performance, we are to some extent, looking at the historical pattern of an individual’s development.
The rate of Dictionary development has increased as what we learn about patterns in the acquisition of concepts is gradually integrated into our methods and technology. For example, we have learned that there are regularities in the progression of verb conjugation within particular developmental levels and that, in some cases, new single-word Lectical Items typically don’t appear next to some conjunctions until the phase following the phase to which they have been assigned. When adequately regular, patterns like these allow a degree of automation in the curation process. (We call this process lexicating™.) The Lectical Dictionary is continuously monitored, refined, and added to by trained analysts. As we build new assessments in new subject areas, the Lectical Dictionary is becoming an increasingly comprehensive curated taxonomy of meanings.
Unlike the purely computational scoring algorithms produced through the analysis of "big data" analysis or "meta-analytics," CLAS is not a form of AI. Even now, its ongoing maintenance and development requires a great deal of expert human judgment. In fact, humans make all judgments about the placement of Lectical Items, which requires the analysis of meaning (something AI can't do).
If you'd like to know more about CLAS, please contact us.
IES (US Department of Education)
The Spencer Foundation
NIH
Dr. Sharon Solloway
The Simpson Foundation
The Leopold Foundation
Glastonbury School District, CT
The Ross School
Rainbow Community School
The Study School
Long Trail School
The US Naval Academy
The City of Edmonton, Alberta
The US Federal Government
Antonio Battro, MD, Ph.D., One Laptop Per Child
Marc Schwartz, Ph.D. and former high school teacher, University of Texas at Arlington
Mary Helen Immordino-Yang, Ed.D., University of Southern California
Willis Overton, Ph.D., Temple University, Emeritus