Cognitive Task Analysis of

Middle School Math Teaching

Why This Study?

Why Now?

The 2022-2023 academic year was the first full session in the post-Covid era. While the effects of Covid most certainly affected middle-school math teaching, broader trends in student math performance had already been underway. Performance on national assessments in the US show significant declines over the past decade. Debates about the ‘best’ way to teach and learn math, as well as the emerging capabilities of AI, have added complexity to the landscape but have not yet shown a pathway to righting the course.

The Goal

To jumpstart our understanding of middle school math teaching, we conducted Cognitive Task Analysis (CTA) with middle school math teachers.

 

While ours is certainly not the first research to investigate the cognitive processes in teaching, this is the first study that extensively applies cognitive task analysis to math education.

Cognitive Task Analysis is...

an approach proven effective in other domains for understanding skilled professionals’ cognitive performance. 

CTA is rooted in…

Naturalistic Decision Making...

a field of study focused on how people actually make decisions in real-world settings —not the confines of a laboratory. NDM researchers reveal how expertise develops and how experts think. 

Participants

The participants were not a representative sample of middle school math teachers in the United States. But they were diverse in their levels of  experience in teaching middle school math, which allowed us to see the full process of proficiency development.

0 +
Education Professionals
> 0
Years of Experience
0
Different States

Report Of Findings

The CTA findings provide several complimentary views of the cognitive work of teaching middle school math.

The Context

From an NDM perspective, the context of work refers to the pressures and tensions that performers face.

Learn more about the context of work.

The Cognitive Tasks

The CTA findings revealed five major cognitive tasks, each comprising several subtasks.

Learn more about the cognitive tasks.

The Mastery

A Mastery Model shows the progression of proficiency as people acquire experience in a given domain, and documents performance progress across levels of mastery.

Learn more about the Mastery Model.

The Macrocognitive Processes

A macrocognitive analysis consolidates findings about experienced domain practitioners along the known macrocognitive dimensions of performance.

Learn more about the macrocognitive analysis.

The Difficulties and Strategies

The CTA findings provide a set of difficulties faced for each of the cognitive tasks, as experienced at the different levels of proficiency — and strategies teachers use to overcome them.

Learn more about the difficulties and strategies.

Recommendations

Our recommendations draw from the CTA findings and NDM to suggest approaches for accelerating proficiency achievement and for designing and evaluating technology enablers.

Learn more about recommendations.

Leveling Up to Expertise

Read one teacher’s experience in reaching Expert-level proficiency, and the benefits that followed

The Context

From an NDM perspective, the context of work refers to the pressures and tensions that performers face.

The CTA findings revealed 12 aspects and three emergent features of context within which middle school math is taught.

Goal

Understanding | Performance

Middle school math teachers perpetually consider what does and what should define student achievement. This tension is between understanding and performance, between ensuring that students gain an appreciation for the complexity of mathematics and ensuring that they can meet the moment when called to action. This aspect shapes decisions about where to focus and suggests where emphasis should be placed and in which moments.

Curricula

Provided | Created

Curricula instantiate the goals of instruction and advise on what counts as achievement. This tension is between what is provided and what is created to meet the goals and achieve. It is ratcheted up by decisions made outside of one’s control (e.g., vendor selection), frequent revision and overhauls, and mandates imposed across several levels of oversight (e.g., in sequencing). This aspect shapes planning for what to do and how to do it.

Coverage

Comprehensive | Prioritized

Various standards determine the content to be covered in the course of any given school year. This tension is between ensuring that all content is covered and prioritizing which content should take priority in the context of time pressure. The aspect shapes planning and decision making for how long to try and do.

Tools

New | Old

While differences in resource levels exist, all middle school math classrooms are infused with tools. This tension is about the value introduced by new and old technologies, where value is considered according to judgments about potential efficiencies, perceived learning gains or other student-centered benefits, and the degree to which use is mandated. The aspect shapes planning and decision making for what to try and do, when, and for how long.

Data

Volume | Insight

The outputs of evaluations are data about student performance, and at any given moment, the teaching process faces a potentially overwhelming deluge of data. This tension is between getting more data or getting insight. More data can overwhelm professionals, especially if they are not trained in how to interrogate it, and supposed insights can often be countered with the right data. This aspect shapes the planning process for determining what to do about how the work is going.

Autonomy

Self-determined | Contrained

At any given point in their careers, teachers are endowed with different levels of autonomy. This tension is between determining one’s courses of action and constraints imposed by others. Both circumstances offer advantages for teachers and students, yet both can introduce challenges, which are felt most urgently when circumstances flip.  This aspect shapes decisions and planning about what to do, when, and for how long.

Endgame

Conceptual | Procedural

Hand-in-hand with the goal is the consideration of the ultimate requirement for learning math. This tension is between the conceptual and the procedural. While this tension is not unique to math, it is ever present in the policy discussions and practical concerns that surround math instruction—see for example, the math war that has been raging for decades. The aspect shapes planning for what to try and how to try it.

Pacing

Planned | Executed

Teaching is generally governed by seasonal ebbs and flows but managed by pacing schedules. This tension is between the plan as stated and the plan as executed, the delta of which may be tracked with more or less precision. The aspect shapes planning and decision making for when to try and do, and for how long.

Target

All | Individual

Middle school math teachers must continuously consider their target of focus. The tension is between all—which can mean the entire class or subsets—and the individual student. Attention toward one or the other runs the risk of denying attention to the other. This aspect shapes the decision making and planning process for determining what to try and do, when, and for how long.

Evaluation

Process | Outcome

Teachers have only indirect access to highly variable data with which to monitor and evaluate their own performance, including data about the performance of their students. This tension is between whether process or outcome data should be used to craft the stories that characterize their own performance. Each may be more or less appropriate for different purposes and audiences. This aspect shapes the sensemaking process for determining how the work is going.

Responsibility

Students | System

Teachers mediate the relationship between students and system. This tension is between the needs of students and the demands of the educational system. While every student is different and under continuous evolution, the system can be just as fickle—and both can misalign with frequency. The aspect shapes decisions about which entity receives emphasis in any given moment.

Development

Expertise | Opportunity/Misery

The vagaries of educational administration place middle school math teachers in a precarious position. This tension is between building the experience base necessary to become an expert teacher and pursuing career opportunities and/or avoiding professional misery. Movement can require gaining new skills, which takes time, and stunts gains in familiarity—i.e., with students, colleagues, technologies. This aspect shapes decisions about how to manage one’s long-term development.

Three emergent features

Simultaneity: Teachers engage in many tasks, all at once.

Disruptions: The tasks of teaching are continuously interrupted.

Variability: Teachers face extreme variance across all dimensions of their work.

The Cognitive Tasks

The practice of teaching middle school math comprises an extensive set of interwoven cognitive tasks. Any attempt to catalogue them is necessarily reductive, but framing them as cognitive activities brings into high relief how teachers think through the work they need to achieve.

  • Assessing

    Cognitive process of determining where learners are in their development. Assessing primarily targets learners as people, their prior knowledge, and their mastery of math.

  • Implementing

    Transforming learning content, tools and resources into learning experiences, including lessons and interventions.

  • Monitoring

    Determining whether learners are engaged and progressing. Whereas assessing is about the moment, monitoring is about the journey.

  • Planning

    Considering courses of action and preparing to execute them in near- and long-terms.

  • Managing

    Planning for learning, collaborating with colleagues, steering the development of one’s self, and dealing with other responsible parties.

The Details

In attempting these tasks, teachers face difficulties, rely on information sources, and use strategies. The full set of CTA findings describes each of these for each of the levels of proficiency.

The Mastery Model

The Mastery Model details how proficiency development manifests across stages.

 

In addition to the stage profiles below, the full report provides a set of performance indicators that describe

how teachers go about their work at each stage. 

Novice / Advanced Beginner

Overwhelmed and lost | Survives

  • Views students as mysterious or imbued with perceived individual traits – e.g., bad behavior + low confidence = struggle with math

  • Views students as bucket model, recipients of content
  • Experiences curriculum as overwhelming set of materials to be delivered to students
  • Requires practice
  • Struggles to adapt to complexity in the moment
  • Views the learning unit as individual students
  • Views the learning unit as needing to be ordered
  • Assumes progress as mostly linear for all students
  • Overwhelmed by data
  • Assumes curriculum is given, current, immutable, and unit focused
  • Paces according to predetermined structure
  • Experiences many surprises from envision plan and execution misalignment
  • Engagement with others viewed as demanding and primarily focused on corrective feedback on own performance
  • Sense of self as awkward and unprepared – in content knowledge and/or pedagogy
  • Desire to please, connect, and improve

Competent

Establishes footholds in the face of complexity | Delivers

  • Begins to consider contextual information about learners and their performance
  • Begins to recognize learning gaps and common misconceptions
  • Begins to differentiate between High/Low abilities
  • Includes social and system issues in diagnostic thinking
  • Views students primarily as calculators
  • Begins to feel ownership of curriculum
  • Able to implement without supervision and recognize professional presentation
  • Recognizes potential for complexity
  • Views the learning unit as divisible into subgroups, with focus to be primarily placed on low performers
  • Views the learning unit as requiring engagement strategies
  • Begins to recognize nonuniformity of progress across students
  • Begins to experience constraints on curriculum as designed
  • Begins to plan beyond curriculum, to environment
  • Begins to see units as interconnected
  • Lingering doubts about performance and requirements for interaction with others
  • Understands limitations of preparation and appreciates structural challenges
  • Begins to take ownership of self-development
  • Emerging self-awareness about potentially successful approaches, including vulnerability

Proficient

Covers all the bases with grace and efficiency | Adapts

  • Able to place observed performance in broader learning contexts, including expected standards
  • Actively seeks and analyzes data from multiple sources
  • Able to routinely recognize misconceptions, learning gaps and patterns of performance based on past and expected experience
  • Views students also as problems solvers
  • Takes ownership of curriculum
  • Recognizes the emergence of complexity and exceptions, and adapts accordingly
  • Views the learning as divisible into subgroups, with focus to be placed also on high-performers and learners in roles
  • Views the learning unit as developmentally age appropriate
  • Anticipates varying rates of progress
  • Understands relationship of curriculum to broader standards of learning
  • Understands interconnectivity of units
  • Appreciates variability of curriculum quality and systemic causes
  • Actively pursues and manages relationships, especially challenging ones
  • Appreciation for and pursuit of continuing professional development
  • Comfort with uncertainty and trust in self

Expert

Goes beyond and just sees things differently | Innovates

  • Views all learners as capable, with heightened sensitivity to structural challenges to success

  • Views students as both learners and teachers
  • Owns all aspects of implementation and helps others build ownership, including learners
  • Anticipates complexity and uses principles to guide approaches
  • Views the learning unit as empowered, and self-interested and perpetuating
  • Views the learner as responsible for gauging their own progress
  • Recognizes structural constraints to progress
  • Assumes adaptive approach
  • Understands anterior and posterior standards of learning across grade levels
  • Sets expectations for others
  • Places feedback and critiques in broader contexts, and responds accordingly

The Macrocognitive Processes

A macrocognitive analysis consolidates findings about experienced domain practitioners along the known macrocognitive dimensions of performance

Diagnose situations through recognizing typicality and discriminating key features; Mentally simulate workable course of action; Act quickly

Refined mental models about how things work help to select data that matters; Data helps select the best frame; Explain

Set and revise goals based on discoveries made during execution

Prepare for future events by anticipating trajectories

See antecedents and consequences; Understand the import of contexts

Manage co-responsible tasks across time and maintaining mutual beliefs and awareness across stakeholders

Actively direct apparati to monitor relevant information

Monitor and respond to ill-defined contexts

Develop sensitivity to emerging situations; See and represent problem at a deeper level; Spend ample time trying to understand the problem

Do more with available resources; Find novel solutions

Active recognition of performance and response to feedback, fostering self-development

Macrocognitive DimensionTask Exemplars
Naturalistic (Recognition-Primed) Decision MakingExperience informs how to implement lessons in ways that are most likely to be successful for the majority of learners to achieve conceptual and procedural success, assess mastery through a variety of data points available in the classroom environment, and call upon an extensive repertoire of interventions to help learners advance.
SensemakingDrawing on a large and diverse set of frames about how learners’ deal with particular content at particular stages of development, primes the abilities to see misconceptions at a glance, diagnose content gaps, categorize learners and their circumstances, select and evaluate specific data about student performance, and know which frames are appropriate to guide understanding, given the available or missing data about learners.
FlexecutingMany repeated cycles of planning and execution under continuously changing contexts of administrative leadership, colleague rotation, standards evolution, interruptions, and curriculum revisions, enables adaptive planning, including workable pacing and sequencing, towards the never-changing goal of learner improvement.
Projecting & AnticipatingEngagement with hundreds of learners and their Responsible Others across the full-spectrum of academic calendars enables rapid assessment of learners’ starting points and their potential trajectories, which in turn informs which interventions may be necessary to ensure pivots toward desired trajectories.
Keeping the Big PictureDeep understanding of the educational apparatus, in particular the expected standards of learning across all grade levels, provides the context of understanding whether learner progress is tracking appropriately and devising plans that set up learners for future success.
Maintaining Common GroundRoutine and ad hoc collaborative activities, including robust information sharing across the network of stakeholders (i.e., learners, teachers, Responsible Others), provides continuous assessment of mental models about learner status, in turn supporting timely implementation of necessary interventions.
Managing AttentionLong-established and reinforced routines, targeted inquiries, seamless facilitation, and novel and distributed data collection help to maintain awareness across the large, complex unit that is a class, and in turn enable learners to keep focus on the tasks at hand.
Managing UncertaintyHard-earned trust in one’s teaching process and abilities, including skills at relationship building, engenders a patient approach to gaining familiarity where uncertainty exists, most notably in dealing with a new class of learners.
Detecting problemsThorough, timely and collaborative dives into learner and teacher performance data provide early insights about which strategies may not be working and which learners need help.
ImproviseHaving faced the full spectrum of constraints requires devising creative uses of resources, seeking leverage points to do more with less, and maximizing the learning value of projects.
MotivationYears of reflection on practice and diversity of feedback supports sustaining the drive toward achieving and sustaining high-level performance.

The Difficulties of Subdomains

Depending on their level of experience, teachers perceive subdomains of math at different grade levels as more or less difficult - both to teach and for learners to learn.

They provided their ranked order of difficulty, enabling a trend analysis across proficiency levels.

Grade 5

Grade 6

Grade 7

Grade 8

Commentary and Strategies

Teacher commentary, as well as strategies they use to encourage learning of the subdomains, is included in Appendix B of the full report.

Recommendations

While the primary goal was to describe cognitive performance, we also derived some recommendations for accelerating proficiency achievement and for designing and evaluating technology enablers, specifically computing technologies.

Creating the Conditions for Expertise Development

Ways to ensure the development of expertise and to help it arrive faster.

School Conditions

  • Maximize familiarity

    The cognitive tasks of Assessing and Monitoring are best enabled through building deep familiarity with students over time. Schools must seek approaches for maximizing and preserve engagement between students and teachers.

  • Assign new teachers to the middle grades

    Many students enter middle school with a variety of gaps from elementary school, and preparing students for high school mathematics requires knowledge of the path ahead. Assigning new teachers to teach grades 6 or 7 would enable them to gain experience while being buffered by more experience teachers.

  • Take a resiliency-engineering stance

    Two major themes from the CTA findings concerned the variety of attempts to design systems that meet all needs, for all learners, all the time – and the variety of ways in which such systems were confronted with surprise. Middle school math teaching is a bastion of a complex adaptive system – this recommendation promotes building a resilience engineering perspective into administrative roles.

Professional Development

  • Case-Based Learning Experiences (CBLE)

    While experience is necessary for expertise development, not all experience is the same for helping to advance toward mastery. Professional development in the form of CBLE offers the best opportunity to rapidly acquire a broader experience base with complex problems, to get the right kinds of feedback, to practice, and to evolve one’s mental models.

  • Practice

    While CBLE offers the potential to cover a lot of ground, developing some skills required for proficient teaching of math requires practice through repetition, which is difficult to come by in environments already strained for time. These include building content knowledge and identifying misconceptions.

  • Use Mastery Model as a rubric

    Given that the Mastery Model describes proficiency levels, it is well suited to serve as a rubric for evaluating where teachers are along the pathway toward proficiency.

  • Develop mentorship skills

    The cognitive processes of Collaborating and Developing Self emphasize the importance of the mentoring relationship. Mentorship is a learned skill that requires instruction, practice, feedback, self-reflection, and intention.

Technology Applications

Hypotheses about designing technology applications that enable proficient performance—and help it to develop—yet also do not disable the use of expertise.

Potential Augmentations

  • Data tracking and exploration

    The CTA findings revealed many examples of teachers tracking and interrogating data, particularly through ‘homegrown’ tools like notebooks, spreadsheets and shared drives. Tools that enable malleable data tracking and flexible interrogation might be introduced to aid teachers in tracking, among other things.

  • Curriculum management

    Middle school math teachers face a dizzying array of curriculum representations, portrayed across a variety of tools, and managed by an assortment of personnel with varying perspectives.

Potential Impediments

  • Masking student work

    Process data are an essential component to the cognitive task of Assessing. Technologies that mask student work stand to disable the ability of teachers to assess students in the process of learning.

  • Opaque inner workings

    As information technologies become increasingly complex, it becomes equally as important for them to provide users with understanding of how they work and why they produce the results they do. Technologies that offer only an opaque view into their inner workings stand to disable the ability of teachers to make effective use of their potential.

  • Bad implementation

    The CTA findings revealed many instances of technology vendors offering little support to their intended end-users, enterprise software purchases left unused, and repeated critiques of widely-sold tools that were a poor match to actual need. Technology implementations that do not follow a learning engineering approach stand to disable teachers’ ability to adapt and turning learning teachers into passive tool operators.

Details and Rationale

More details and the NDM-based rationales for these recommendations are provided in the full report.