Why This Study?
Why Now?
The 2022-2023 academic year was the first full session in the post-Covid era. While the effects of Covid most certainly affected middle-school math teaching, broader trends in student math performance had already been underway. Performance on national assessments in the US show significant declines over the past decade. Debates about the ‘best’ way to teach and learn math, as well as the emerging capabilities of AI, have added complexity to the landscape but have not yet shown a pathway to righting the course.
The Goal
To jumpstart our understanding of middle school math teaching, we conducted Cognitive Task Analysis (CTA) with middle school math teachers.
While ours is certainly not the first research to investigate the cognitive processes in teaching, this is the first study that extensively applies cognitive task analysis to math education.
Cognitive Task Analysis is...
an approach proven effective in other domains for understanding skilled professionals’ cognitive performance.
CTA is rooted in…
Naturalistic Decision Making...
a field of study focused on how people actually make decisions in real-world settings —not the confines of a laboratory. NDM researchers reveal how expertise develops and how experts think.
Participants
The participants were not a representative sample of middle school math teachers in the United States. But they were diverse in their levels of experience in teaching middle school math, which allowed us to see the full process of proficiency development.
Report Of Findings
The CTA findings provide several complimentary views of the cognitive work of teaching middle school math.
From an NDM perspective, the context of work refers to the pressures and tensions that performers face.
The CTA findings revealed five major cognitive tasks, each comprising several subtasks.
A Mastery Model shows the progression of proficiency as people acquire experience in a given domain, and documents performance progress across levels of mastery.
A macrocognitive analysis consolidates findings about experienced domain practitioners along the known macrocognitive dimensions of performance.
The CTA findings provide a set of difficulties faced for each of the cognitive tasks, as experienced at the different levels of proficiency — and strategies teachers use to overcome them.
Our recommendations draw from the CTA findings and NDM to suggest approaches for accelerating proficiency achievement and for designing and evaluating technology enablers.
Leveling Up to Expertise
Read one teacher’s experience in reaching Expert-level proficiency, and the benefits that followed.
The Context
From an NDM perspective, the context of work refers to the pressures and tensions that performers face.
The CTA findings revealed 12 aspects and three emergent features of context within which middle school math is taught.
The Cognitive Tasks
The practice of teaching middle school math comprises an extensive set of interwoven cognitive tasks. Any attempt to catalogue them is necessarily reductive, but framing them as cognitive activities brings into high relief how teachers think through the work they need to achieve.
The Details
In attempting these tasks, teachers face difficulties, rely on information sources, and use strategies. The full set of CTA findings describes each of these for each of the levels of proficiency.
The Mastery Model
The Mastery Model details how proficiency development manifests across stages.
In addition to the stage profiles below, the full report provides a set of performance indicators that describe
how teachers go about their work at each stage.
Novice / Advanced Beginner
Overwhelmed and lost | Survives
Views students as mysterious or imbued with perceived individual traits – e.g., bad behavior + low confidence = struggle with math
- Views students as bucket model, recipients of content
- Experiences curriculum as overwhelming set of materials to be delivered to students
- Requires practice
- Struggles to adapt to complexity in the moment
- Views the learning unit as individual students
- Views the learning unit as needing to be ordered
- Assumes progress as mostly linear for all students
- Overwhelmed by data
- Assumes curriculum is given, current, immutable, and unit focused
- Paces according to predetermined structure
- Experiences many surprises from envision plan and execution misalignment
- Engagement with others viewed as demanding and primarily focused on corrective feedback on own performance
- Sense of self as awkward and unprepared – in content knowledge and/or pedagogy
- Desire to please, connect, and improve
Competent
Establishes footholds in the face of complexity | Delivers
- Begins to consider contextual information about learners and their performance
- Begins to recognize learning gaps and common misconceptions
- Begins to differentiate between High/Low abilities
- Includes social and system issues in diagnostic thinking
- Views students primarily as calculators
- Begins to feel ownership of curriculum
- Able to implement without supervision and recognize professional presentation
- Recognizes potential for complexity
- Views the learning unit as divisible into subgroups, with focus to be primarily placed on low performers
- Views the learning unit as requiring engagement strategies
- Begins to recognize nonuniformity of progress across students
- Begins to experience constraints on curriculum as designed
- Begins to plan beyond curriculum, to environment
- Begins to see units as interconnected
- Lingering doubts about performance and requirements for interaction with others
- Understands limitations of preparation and appreciates structural challenges
- Begins to take ownership of self-development
- Emerging self-awareness about potentially successful approaches, including vulnerability
Proficient
Covers all the bases with grace and efficiency | Adapts
- Able to place observed performance in broader learning contexts, including expected standards
- Actively seeks and analyzes data from multiple sources
- Able to routinely recognize misconceptions, learning gaps and patterns of performance based on past and expected experience
- Views students also as problems solvers
- Takes ownership of curriculum
- Recognizes the emergence of complexity and exceptions, and adapts accordingly
- Views the learning as divisible into subgroups, with focus to be placed also on high-performers and learners in roles
- Views the learning unit as developmentally age appropriate
- Anticipates varying rates of progress
- Understands relationship of curriculum to broader standards of learning
- Understands interconnectivity of units
- Appreciates variability of curriculum quality and systemic causes
- Actively pursues and manages relationships, especially challenging ones
- Appreciation for and pursuit of continuing professional development
- Comfort with uncertainty and trust in self
Expert
Goes beyond and just sees things differently | Innovates
Views all learners as capable, with heightened sensitivity to structural challenges to success
- Views students as both learners and teachers
- Owns all aspects of implementation and helps others build ownership, including learners
- Anticipates complexity and uses principles to guide approaches
- Views the learning unit as empowered, and self-interested and perpetuating
- Views the learner as responsible for gauging their own progress
- Recognizes structural constraints to progress
- Assumes adaptive approach
- Understands anterior and posterior standards of learning across grade levels
- Sets expectations for others
- Places feedback and critiques in broader contexts, and responds accordingly
The Macrocognitive Processes
A macrocognitive analysis consolidates findings about experienced domain practitioners along the known macrocognitive dimensions of performance
Diagnose situations through recognizing typicality and discriminating key features; Mentally simulate workable course of action; Act quickly
Refined mental models about how things work help to select data that matters; Data helps select the best frame; Explain
Set and revise goals based on discoveries made during execution
Prepare for future events by anticipating trajectories
See antecedents and consequences; Understand the import of contexts
Manage co-responsible tasks across time and maintaining mutual beliefs and awareness across stakeholders
Actively direct apparati to monitor relevant information
Monitor and respond to ill-defined contexts
Develop sensitivity to emerging situations; See and represent problem at a deeper level; Spend ample time trying to understand the problem
Do more with available resources; Find novel solutions
Active recognition of performance and response to feedback, fostering self-development
| Macrocognitive Dimension | Task Exemplars |
|---|---|
| Naturalistic (Recognition-Primed) Decision Making | Experience informs how to implement lessons in ways that are most likely to be successful for the majority of learners to achieve conceptual and procedural success, assess mastery through a variety of data points available in the classroom environment, and call upon an extensive repertoire of interventions to help learners advance. |
| Sensemaking | Drawing on a large and diverse set of frames about how learners’ deal with particular content at particular stages of development, primes the abilities to see misconceptions at a glance, diagnose content gaps, categorize learners and their circumstances, select and evaluate specific data about student performance, and know which frames are appropriate to guide understanding, given the available or missing data about learners. |
| Flexecuting | Many repeated cycles of planning and execution under continuously changing contexts of administrative leadership, colleague rotation, standards evolution, interruptions, and curriculum revisions, enables adaptive planning, including workable pacing and sequencing, towards the never-changing goal of learner improvement. |
| Projecting & Anticipating | Engagement with hundreds of learners and their Responsible Others across the full-spectrum of academic calendars enables rapid assessment of learners’ starting points and their potential trajectories, which in turn informs which interventions may be necessary to ensure pivots toward desired trajectories. |
| Keeping the Big Picture | Deep understanding of the educational apparatus, in particular the expected standards of learning across all grade levels, provides the context of understanding whether learner progress is tracking appropriately and devising plans that set up learners for future success. |
| Maintaining Common Ground | Routine and ad hoc collaborative activities, including robust information sharing across the network of stakeholders (i.e., learners, teachers, Responsible Others), provides continuous assessment of mental models about learner status, in turn supporting timely implementation of necessary interventions. |
| Managing Attention | Long-established and reinforced routines, targeted inquiries, seamless facilitation, and novel and distributed data collection help to maintain awareness across the large, complex unit that is a class, and in turn enable learners to keep focus on the tasks at hand. |
| Managing Uncertainty | Hard-earned trust in one’s teaching process and abilities, including skills at relationship building, engenders a patient approach to gaining familiarity where uncertainty exists, most notably in dealing with a new class of learners. |
| Detecting problems | Thorough, timely and collaborative dives into learner and teacher performance data provide early insights about which strategies may not be working and which learners need help. |
| Improvise | Having faced the full spectrum of constraints requires devising creative uses of resources, seeking leverage points to do more with less, and maximizing the learning value of projects. |
| Motivation | Years of reflection on practice and diversity of feedback supports sustaining the drive toward achieving and sustaining high-level performance. |
The Difficulties of Subdomains
Depending on their level of experience, teachers perceive subdomains of math at different grade levels as more or less difficult - both to teach and for learners to learn.
They provided their ranked order of difficulty, enabling a trend analysis across proficiency levels.
Commentary and Strategies
Teacher commentary, as well as strategies they use to encourage learning of the subdomains, is included in Appendix B of the full report.
Recommendations
While the primary goal was to describe cognitive performance, we also derived some recommendations for accelerating proficiency achievement and for designing and evaluating technology enablers, specifically computing technologies.
Creating the Conditions for Expertise Development
Ways to ensure the development of expertise and to help it arrive faster.
School Conditions
Professional Development
Technology Applications
Hypotheses about designing technology applications that enable proficient performance—and help it to develop—yet also do not disable the use of expertise.
Potential Augmentations
Potential Impediments
Details and Rationale
More details and the NDM-based rationales for these recommendations are provided in the full report.