The 2022-2023 academic year was the first full session in the post-Covid era. While the effects of Covid most certainly affected middle-school math teaching, broader trends in student math performance had already been underway. Performance on national assessments in the US show significant declines over the past decade. Debates about the ‘best’ way to teach and learn math, as well as the emerging capabilities of AI, have added complexity to the landscape but have not yet shown a pathway to righting the course.
To jumpstart our understanding of middle school math teaching, we conducted Cognitive Task Analysis (CTA) with middle school math teachers.
While ours is certainly not the first research to investigate the cognitive processes in teaching, this is the first study that extensively applies cognitive task analysis to math education.
an approach proven effective in other domains for understanding skilled professionals’ cognitive performance.
CTA is rooted in…
a field of study focused on how people actually make decisions in real-world settings —not the confines of a laboratory. NDM researchers reveal how expertise develops and how experts think.
The participants were not a representative sample of middle school math teachers in the United States. But they were diverse in their levels of experience in teaching middle school math, which allowed us to see the full process of proficiency development.
The CTA findings provide several complimentary views of the cognitive work of teaching middle school math.
From an NDM perspective, the context of work refers to the pressures and tensions that performers face.
The CTA findings revealed five major cognitive tasks, each comprising several subtasks.
A Mastery Model shows the progression of proficiency as people acquire experience in a given domain, and documents performance progress across levels of mastery.
A macrocognitive analysis consolidates findings about experienced domain practitioners along the known macrocognitive dimensions of performance.
The CTA findings provide a set of difficulties faced for each of the cognitive tasks, as experienced at the different levels of proficiency — and strategies teachers use to overcome them.
Our recommendations draw from the CTA findings and NDM to suggest approaches for accelerating proficiency achievement and for designing and evaluating technology enablers.
Read one teacher’s experience in reaching Expert-level proficiency, and the benefits that followed.
From an NDM perspective, the context of work refers to the pressures and tensions that performers face.
The CTA findings revealed 12 aspects and three emergent features of context within which middle school math is taught.
Understanding | Performance
Provided | Created
Comprehensive | Prioritized
New | Old
Volume | Insight
Self-determined | Contrained
Conceptual | Procedural
Planned | Executed
All | Individual
Process | Outcome
Students | System
Expertise | Opportunity/Misery
Simultaneity: Teachers engage in many tasks, all at once.
Disruptions: The tasks of teaching are continuously interrupted.
Variability: Teachers face extreme variance across all dimensions of their work.
The practice of teaching middle school math comprises an extensive set of interwoven cognitive tasks. Any attempt to catalogue them is necessarily reductive, but framing them as cognitive activities brings into high relief how teachers think through the work they need to achieve.
Cognitive process of determining where learners are in their development. Assessing primarily targets learners as people, their prior knowledge, and their mastery of math.
Transforming learning content, tools and resources into learning experiences, including lessons and interventions.
Determining whether learners are engaged and progressing. Whereas assessing is about the moment, monitoring is about the journey.
Considering courses of action and preparing to execute them in near- and long-terms.
Planning for learning, collaborating with colleagues, steering the development of one’s self, and dealing with other responsible parties.
In attempting these tasks, teachers face difficulties, rely on information sources, and use strategies. The full set of CTA findings describes each of these for each of the levels of proficiency.
The Mastery Model details how proficiency development manifests across stages.
In addition to the stage profiles below, the full report provides a set of performance indicators that describe
how teachers go about their work at each stage.
Overwhelmed and lost | Survives
Views students as mysterious or imbued with perceived individual traits – e.g., bad behavior + low confidence = struggle with math
Establishes footholds in the face of complexity | Delivers
Covers all the bases with grace and efficiency | Adapts
Goes beyond and just sees things differently | Innovates
Views all learners as capable, with heightened sensitivity to structural challenges to success
A macrocognitive analysis consolidates findings about experienced domain practitioners along the known macrocognitive dimensions of performance
Diagnose situations through recognizing typicality and discriminating key features; Mentally simulate workable course of action; Act quickly
Refined mental models about how things work help to select data that matters; Data helps select the best frame; Explain
Set and revise goals based on discoveries made during execution
Prepare for future events by anticipating trajectories
See antecedents and consequences; Understand the import of contexts
Manage co-responsible tasks across time and maintaining mutual beliefs and awareness across stakeholders
Actively direct apparati to monitor relevant information
Monitor and respond to ill-defined contexts
Develop sensitivity to emerging situations; See and represent problem at a deeper level; Spend ample time trying to understand the problem
Do more with available resources; Find novel solutions
Active recognition of performance and response to feedback, fostering self-development
Macrocognitive Dimension | Task Exemplars |
---|---|
Naturalistic (Recognition-Primed) Decision Making | Experience informs how to implement lessons in ways that are most likely to be successful for the majority of learners to achieve conceptual and procedural success, assess mastery through a variety of data points available in the classroom environment, and call upon an extensive repertoire of interventions to help learners advance. |
Sensemaking | Drawing on a large and diverse set of frames about how learners’ deal with particular content at particular stages of development, primes the abilities to see misconceptions at a glance, diagnose content gaps, categorize learners and their circumstances, select and evaluate specific data about student performance, and know which frames are appropriate to guide understanding, given the available or missing data about learners. |
Flexecuting | Many repeated cycles of planning and execution under continuously changing contexts of administrative leadership, colleague rotation, standards evolution, interruptions, and curriculum revisions, enables adaptive planning, including workable pacing and sequencing, towards the never-changing goal of learner improvement. |
Projecting & Anticipating | Engagement with hundreds of learners and their Responsible Others across the full-spectrum of academic calendars enables rapid assessment of learners’ starting points and their potential trajectories, which in turn informs which interventions may be necessary to ensure pivots toward desired trajectories. |
Keeping the Big Picture | Deep understanding of the educational apparatus, in particular the expected standards of learning across all grade levels, provides the context of understanding whether learner progress is tracking appropriately and devising plans that set up learners for future success. |
Maintaining Common Ground | Routine and ad hoc collaborative activities, including robust information sharing across the network of stakeholders (i.e., learners, teachers, Responsible Others), provides continuous assessment of mental models about learner status, in turn supporting timely implementation of necessary interventions. |
Managing Attention | Long-established and reinforced routines, targeted inquiries, seamless facilitation, and novel and distributed data collection help to maintain awareness across the large, complex unit that is a class, and in turn enable learners to keep focus on the tasks at hand. |
Managing Uncertainty | Hard-earned trust in one’s teaching process and abilities, including skills at relationship building, engenders a patient approach to gaining familiarity where uncertainty exists, most notably in dealing with a new class of learners. |
Detecting problems | Thorough, timely and collaborative dives into learner and teacher performance data provide early insights about which strategies may not be working and which learners need help. |
Improvise | Having faced the full spectrum of constraints requires devising creative uses of resources, seeking leverage points to do more with less, and maximizing the learning value of projects. |
Motivation | Years of reflection on practice and diversity of feedback supports sustaining the drive toward achieving and sustaining high-level performance. |
Depending on their level of experience, teachers perceive subdomains of math at different grade levels as more or less difficult - both to teach and for learners to learn.
They provided their ranked order of difficulty, enabling a trend analysis across proficiency levels.
Teacher commentary, as well as strategies they use to encourage learning of the subdomains, is included in Appendix B of the full report.
While the primary goal was to describe cognitive performance, we also derived some recommendations for accelerating proficiency achievement and for designing and evaluating technology enablers, specifically computing technologies.
Ways to ensure the development of expertise and to help it arrive faster.
The cognitive tasks of Assessing and Monitoring are best enabled through building deep familiarity with students over time. Schools must seek approaches for maximizing and preserve engagement between students and teachers.
Many students enter middle school with a variety of gaps from elementary school, and preparing students for high school mathematics requires knowledge of the path ahead. Assigning new teachers to teach grades 6 or 7 would enable them to gain experience while being buffered by more experience teachers.
Two major themes from the CTA findings concerned the variety of attempts to design systems that meet all needs, for all learners, all the time – and the variety of ways in which such systems were confronted with surprise. Middle school math teaching is a bastion of a complex adaptive system – this recommendation promotes building a resilience engineering perspective into administrative roles.
While experience is necessary for expertise development, not all experience is the same for helping to advance toward mastery. Professional development in the form of CBLE offers the best opportunity to rapidly acquire a broader experience base with complex problems, to get the right kinds of feedback, to practice, and to evolve one’s mental models.
While CBLE offers the potential to cover a lot of ground, developing some skills required for proficient teaching of math requires practice through repetition, which is difficult to come by in environments already strained for time. These include building content knowledge and identifying misconceptions.
Given that the Mastery Model describes proficiency levels, it is well suited to serve as a rubric for evaluating where teachers are along the pathway toward proficiency.
The cognitive processes of Collaborating and Developing Self emphasize the importance of the mentoring relationship. Mentorship is a learned skill that requires instruction, practice, feedback, self-reflection, and intention.
Hypotheses about designing technology applications that enable proficient performance—and help it to develop—yet also do not disable the use of expertise.
The CTA findings revealed many examples of teachers tracking and interrogating data, particularly through ‘homegrown’ tools like notebooks, spreadsheets and shared drives. Tools that enable malleable data tracking and flexible interrogation might be introduced to aid teachers in tracking, among other things.
Middle school math teachers face a dizzying array of curriculum representations, portrayed across a variety of tools, and managed by an assortment of personnel with varying perspectives.
Process data are an essential component to the cognitive task of Assessing. Technologies that mask student work stand to disable the ability of teachers to assess students in the process of learning.
As information technologies become increasingly complex, it becomes equally as important for them to provide users with understanding of how they work and why they produce the results they do. Technologies that offer only an opaque view into their inner workings stand to disable the ability of teachers to make effective use of their potential.
The CTA findings revealed many instances of technology vendors offering little support to their intended end-users, enterprise software purchases left unused, and repeated critiques of widely-sold tools that were a poor match to actual need. Technology implementations that do not follow a learning engineering approach stand to disable teachers’ ability to adapt and turning learning teachers into passive tool operators.
More details and the NDM-based rationales for these recommendations are provided in the full report.