Why This Study?
Why Now?
The 2022-2023 academic year was the first full session in the post-Covid era – an era that created significant stress on the educational system. While the effects of Covid most certainly affected middle-school math teaching, broader trends in student math performance had already been underway. Academic debates about the ‘best’ way to teach math and emerging capabilities of AI in education have further complexified the landscape.
The Goal
The Learning Engineering Virtual Institute (LEVI) aims to reverse the decline in performance by achieving a five-fold increase. The LEVI vision includes improving our understanding of middle-school math education. To achieve this, we used Cognitive Task Analysis (CTA), an approach proven effective in other domains for understanding skilled professionals’ cognitive performance. While this is not the first research to investigate cognition in teaching, this is the first study that extensively applies cognitive task analysis to math education.
Cognitive Task Analysis...
an approach proven effective in other domains for understanding skilled professionals’ cognitive performance
Naturalistic Decision making is
a field of study focused on how people actually make decisions in real-world settings —not in the limited confines of a laboratory.
Using this approach, NDM researchers have discovered and specified a number of models of cognitive performance collective known as macrocognition.
NDM researchers have revealed how expertise develops, which is fundamental to helping people achieve it.
Participants
The participants in the study are not a representative sample of middle school math teachers in the United States, but they are diverse in terms of experience in teaching middle school math, which is the key demographic for the study.
Report Of Findings
The CTA findings provide several complimentary views of the cognitive work of teaching middle school math.
From an NDM perspective, the context of work refers to the pressures and tensions that performers face.
The CTA findings revealed five major cognitive tasks, each comprising several subtasks.
A Mastery Model shows the progression of proficiency as people acquire experience in a given domain, and documents performance progress across levels of mastery.
A macrocognitive analysis consolidates findings about experienced domain practitioners along the known macrocognitive dimensions of performance.
The cognitive task analysis findings provide a set of difficulties faced for each of the cognitive tasks, as experienced at the different levels of proficiency.
Recommendations for accelerating proficiency achievement and for designing and evaluating technology enablers, specifically computing technologies
Expertise in Middle School Math
Read the story below for a summary of what expertise in teaching looks like.
The Context
From an NDM perspective, the context of work refers to the pressures and tensions that performers face.
The CTA findings revealed 12 aspects and three emergent features of context within which middle school math is taught.
The Cognitive Tasks
The practice of teaching middle school math comprises an extensive set of cognitive tasks. Any attempt to catalogue them is necessarily reductive, but framing them as cognitive activities brings into high relief how teachers think through the work they need to achieve.
The CTA findings revealed four major cognitive tasks, each comprising several subtasks.
The Mastery
The Mastery Model below shows how such developments manifest across levels of proficiency.
Advanced Beginner / Novice
Overwhelmed, Lost
Views students as mysterious or imbued with perceived individual traits – e.g., bad behavior + low confidence = struggle with math
- Views students as bucket model, recipients of content
- Experiences curriculum as overwhelming set of materials to be delivered to students
- Requires practice
- Struggles to adapt to complexity in the moment
- Views the learning unit as individual students
- Views the learning unit as needing to be ordered
- Assumes progress as mostly linear for all students
- Overwhelmed by data
- Assumes curriculum is given, current, immutable, and unit focused
- Paces according to predetermined structure
- Experiences many surprises from envision plan and execution misalignment
- Engagement with others viewed as demanding and primarily focused on corrective feedback on own performance
- Sense of self as awkward and unprepared – in content knowledge and/or pedagogy
- Desire to please, connect, and improve
Competent
Getting their ‘sea legs’, Able
- Begins to consider contextual information about learners and their performance
- Begins to recognize learning gaps and common misconceptions
- Begins to differentiate between High/Low abilities
- Includes social and system issues in diagnostic thinking
- Views students primarily as calculators
- Begins to feel ownership of curriculum
- Able to implement without supervision and recognize professional presentation
- Recognizes potential for complexity
- Views the learning unit as divisible into subgroups, with focus to be primarily placed on low performers
- Views the learning unit as requiring engagement strategies
- Begins to recognize nonuniformity of progress across students
- Begins to experience constraints on curriculum as designed
- Begins to plan beyond curriculum, to environment
- Begins to see units as interconnected
- Lingering doubts about performance and requirements for interaction with others
- Understands limitations of preparation and appreciates structural challenges
- Begins to take ownership of self-development
- Emerging self-awareness about potentially successful approaches, including vulnerability
Proficient
Covers all the bases and knows complexity
- Able to place observed performance in broader learning contexts, including expected standards
- Actively seeks and analyzes data from multiple sources
- Able to routinely recognize misconceptions, learning gaps and patterns of performance based on past and expected experience
- Views students also as problems solvers
- Takes ownership of curriculum
- Recognizes the emergence of complexity and exceptions, and adapts accordingly
- Views the learning as divisible into subgroups, with focus to be placed also on high-performers and learners in roles
- Views the learning unit as developmentally age appropriate
- Anticipates varying rates of progress
- Understands relationship of curriculum to broader standards of learning
- Understands interconnectivity of units
- Appreciates variability of curriculum quality and systemic causes
- Actively pursues and manages relationships, especially challenging ones
- Appreciation for and pursuit of continuing professional development
- Comfort with uncertainty and trust in self
Expert
All this, and more
Views all learners as capable, with heightened sensitivity to structural challenges to success
- Views students as both learners and teachers
- Owns all aspects of implementation and helps others build ownership, including learners
- Anticipates complexity and uses principles to guide approaches
- Views the learning unit as empowered, and self-interested and perpetuating
- Views the learner as responsible for gauging their own progress
- Recognizes structural constraints to progress
- Assumes adaptive approach
- Understands anterior and posterior standards of learning across grade levels
- Sets expectations for others
- Places feedback and critiques in broader contexts, and responds accordingly
The Macrocognitive Processes
A macrocognitive analysis consolidates findings about experienced domain practitioners along the known macrocognitive dimensions of performance
Diagnose situations through recognizing typicality and discriminating key features; Mentally simulate workable course of action; Act quickly
Refined mental models about how things work help to select data that matters; Data helps select the best frame; Explain
Set and revise goals based on discoveries made during execution
Prepare for future events by anticipating trajectories
See antecedents and consequences; Understand the import of contexts
Manage co-responsible tasks across time and maintaining mutual beliefs and awareness across stakeholders
Actively direct apparati to monitor relevant information
Monitor and respond to ill-defined contexts
Develop sensitivity to emerging situations; See and represent problem at a deeper level; Spend ample time trying to understand the problem
Do more with available resources; Find novel solutions
Active recognition of performance and response to feedback, fostering motivation
| Macrocognitive Dimension | Task Exemplars |
|---|---|
| Naturalistic (Recognition-Primed) Decision Making | Experience informs how to implement lessons in ways that are most likely to be successful for the majority of learners to achieve conceptual and procedural success, assess mastery through a variety of data points available in the classroom environment, and call upon an extensive repertoire of interventions to help learners advance. |
| Sensemaking | Drawing on a large and diverse set of frames about how learners’ deal with particular content at particular stages of development, primes the abilities to see misconceptions at a glance, diagnose content gaps, categorize learners and their circumstances, select and evaluate specific data about student performance, and know which frames are appropriate to guide understanding, given the available or missing data about learners. |
| Flexecuting | Many repeated cycles of planning and execution under continuously changing contexts of administrative leadership, colleague rotation, standards evolution, interruptions, and curriculum revisions, enables adaptive planning, including workable pacing and sequencing, towards the never-changing goal of learner improvement. |
| Projecting & Anticipating | Engagement with hundreds of learners and their Responsible Others across the full-spectrum of academic calendars enables rapid assessment of learners’ starting points and their potential trajectories, which in turn informs which interventions may be necessary to ensure pivots toward desired trajectories. |
| Keeping the Big Picture | Deep understanding of the educational apparatus, in particular the expected standards of learning across all grade levels, provides the context of understanding whether learner progress is tracking appropriately and devising plans that set up learners for future success. |
| Common Ground | Routine and ad hoc collaborative activities, including robust information sharing across the network of stakeholders (i.e., learners, teachers, Responsible Others), provides continuous assessment of mental models about learner status, in turn supporting timely implementation of necessary interventions. |
| Managing Attention | Long-established and reinforced routines, targeted inquiries, seamless facilitation, and novel and distributed data collection help to maintain awareness across the large, complex unit that is a class, and in turn enable learners to keep focus on the tasks at hand. |
| Managing Uncertainty | Hard-earned trust in one’s teaching process and abilities, including skills at relationship building, engenders a patient approach to gaining familiarity where uncertainty exists, most notably in dealing with a new class of learners. |
| Detecting problems | Thorough, timely and collaborative dives into learner and teacher performance data provide early insights about which strategies may not be working and which learners need help. |
| Improvise | Having faced the full spectrum of constraints requires devising creative uses of resources, seeking leverage points to do more with less, and maximizing the learning value of projects. |
| Self-development | Years of reflection on practice and diversity of feedback supports sustaining the drive toward achieving and sustaining high-level performance. |
The Difficulties and Strategies
As performers gain experience, they encounter different challenges and develop strategies to overcome them. Therefore, teachers' perception of difficulty across domains varies depending on the performer's experience level.
Grade 5
Grade 6
Grade 7
Recommendations
While the primary goal of this CTA of Middle School Math Teachers is to describe cognitive performance, we also derived some recommendations for accelerating proficiency achievement and for designing and evaluating technology enablers, specifically computing technologies.
Creating the Conditions for Expertise Development
The following suggestions provide ways to ensure the development of expertise and to help it arrive faster.
School Conditions
Professional Development
Technology Applications
The final set of recommendations offer hypotheses about designing technology applications that enable proficient performance—and help it to develop—yet also do not disable the use of expertise.
