Numeracy Enhancement Research & Development Learning Report Mama Math

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT TABLE OF CONTENTS

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT LIST OF TABLES Table 1: Project summary Table 2: : Intervention pathway Table 3: Evaluation Matrix Table 4: Learner assessment sample, by school and cohort Table 5: Average performance by schools Table 6: Learner mean score and lowest-band share, baseline to endline returners, by arm Table 7: Grade distribution by arm and cohort, returners (percent of pupils in each band) Table 8: Numeracy performance by strand, baseline to endline returners, by arm (percent correct) Table 9: Learner engagement and participation across all schools Table 10: Engagement levels comparable across school settings Table 11: Learner engagement scores out of 55 on comparable items, baseline to endline, by arm Table 12: Teacher practice scores out of 20, baseline to endline, by school and arm Table 13: Feasibility and delivery summary Table 14: School-level total mean percentage scores are descriptive. Table 15: Grade distribution by school (percent in band, baseline to endline) Table 16: Mean score and grade distribution by cohort and arm Table 17: Engagement by section and school, baseline to endline Table 18: Item-level scores by group (mean score out of 5) Table 19: Section B, intervention schools at endline Table 20 : Teacher practice by domain, baseline to endline Table 21: Uploads and AI feedback by teacher account Table 22 : Risk register

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT LIST OF FIGURES Figure 1: Learner mean score by arm, baseline to endline returners Figure 2: Mean score across the strands between the intervention and control schools Figure 3: Grade band distribution across intervention and control schools Figure 4: Learner engagement by school, out of 55 Figure 5: Teaching practice by teacher, out of 20 Figure 6: Theory of change

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT LIST OF ABBREVIATIONS AI Artificial Intelligence eBASE Effective Basic Services Africa FCT Federal Capital Territory NERD Numeracy Enhancement, Research and Development (the pilot) QEDA Quality Education Development Associates SSO School Support Officer UBEB Universal Basic Education Board (referred to as the FCT UBEB) UBEC Universal Basic Education Commission

1. EXECUTIVE SUMMARY The Numeracy Enhancement Research and Development (NERD) pilot tested whether an AI-enabled numeracy platform, Mama Math, could support Grade 2 teachers, through integrated teacher professional development (TPD), to identify learners’ errors for example in counting, comparing numbers, and place value, and give the teachers targeted remediation to reteach them in the same lesson. Over three days, teachers were trained in early-grade numeracy remediation using the concrete-pictorial-abstract (CPA) method and grouping learners by ability. The training covered identifying errors of learners’ work, grouping and differentiation, remediation with concrete materials, and the gradual release model of I do, you do, we do, delivered through the Math Lab session . Gender equality and social inclusion and the responsible use of AI ran through it. Mama Math was taught inside that pedagogy, as the tool for generating lessons, diagnosing errors, and guiding remediation. On the fourth day, School Support Officers and head teachers were trained separately, shifting their role from supervision to coaching and mentoring. The platform is operated by teachers, where they photograph students’ written work and upload the images, and the platform provides an analysis of student errors and suggests short remediation steps. The pilot ran from March to August 2026 in 6 government primary schools in the Federal Capital Territory (FCT): , 3 intervention schools using Mama Math and 3 control schools continuing with standard practice. The schools were paired across three settings, rural, semi-urban, and urban, one intervention and one control in each. The sample size of learners at baseline were 278 learners were assessed at baseline and 352 at endline. Baseline fell immediately after the teacher strike, when attendance was low, and the assessment covered all learners present. The evaluation combined a Grade 2 numeracy assessment, structured observation of teachers and learners, platform delivery records, teacher reflective diaries, and interviews with teachers, caregivers, and government stakeholders. The six sample schools were purposely selectedselected, and 3 teachers were observed per arm (one per school) and any change in learning is read at group level, by school and between intervention and control. NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT

The most notable change during the pilot is in teaching practice, where a twenty-item observation checklist was used to record whether the practices covered in the teacher training appeared in the lesson, across five domains: use of concrete materials and visual representation, targeted grouping and differentiation, re-teaching and remediation, feedback quality, and verification of learning. Intervention teachers moved from 5.7 to 18.3 practices per lesson while matched control teachers stayed flat, from 14.3 to 14.0. The difference between the results is 13.0 points, and all intervention teachers gained more in observed teaching practices than control teachers. The largest differences between the control and the intervention school teachers were in giving specific feedback and in checking that learners had understood; the two behaviors most prompted by the AI platform. 5.7 18.3 71.4% teaching practices per lesson (of 20) in intervention schools; control stayed flat at 14.3 → 14.0 intervention learners’ endline mean score, from 48.7% (control: 36.7% → 64.2%) 54% 11% of intervention pupils in the two lowest bands (control: 77% → 20%) 4 of 4 government respondents endorsed the programme; one has requested extension to 20+ schools Among learners, both groups improved between baseline and endline, from mean scores on the Grade 2 assessment, marked out of 46, rose from 48.7% to 71.4% in the intervention schools and from 36.7% to 64.2% in the control school. The movement was clearest among the lowest performers: the share of pupils in the two lowest bands, C and D combine], fell from average percentage score of 54% to 11% among intervention pupils and from 77% to 20% among control pupils. Across six teacher Mama Math accounts, the platform recorded 197 uploads and 175 rounds of AI feedback, concentrated among the more active teachers, with one case of a teacher sharing the tool with a colleague without formal training but through peer learning. Government stakeholders were also engaged through two-way feedback sessions. All four government respondents endorsed the program; one had already written for approval to extend it to more than 20 schools, and another cautioned that without continued monitoring such a program risks fading out. The main challenge during the pilot was caregiver engagement. Approximately 210 caregivers were expected at an orientation and only 36 attended, and of 15 caregivers targeted for an endline interview, only two could be reached. These two parents reported improvements in their children. The design assumed caregivers could be reached through their own phones or those of family members. In practice, most caregivers had no phone, and those who relied on someone else’s were reluctant to keep asking to borrow it for their child’s schoolwork. There was also a teacher strike at the beginning of the project which slowed down implementation. However, the most significant change in this pilot is in the observed teacher’s practice. NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Project summaryItems Details Intervention Mama Math, an AI-enabled early-grade numeracy platform, teacher-operated; in-app AI analysis provides targeted remediation for teachers to reteach. Location Six government primary schools, (all focused on Primary 2) in the Federal Capital Territory, Abuja, Nigeria. Design Three intervention and three control schools paired across rural, semi-urban, and urban settings. Period March to August 2026, approximately 8 weeks of classroom use. Learners assessed 278 at baseline, 352 at endline, of whom 228 were returners (those who did baseline and endline) Teachers observed Six, one per school, at baseline and endline. Platform delivery 197 uploads and 175 rounds of AI feedback across six teacher accounts. Result Teacher practice increased by 13 points difference against control, the share of intervention pupils in the two lowest grade bands C and D combined mean percentage score fell from 54% to 11% in the intervention and from 77% to 20% in the control. All four government respondents endorsed the program. Only three teachers were observed per arm, each in a single lesson, and the schools did not start from equal base- lines. The pilot shows early promise and feasibility, not proof of impact. That is the case for a larger, better-powered evaluation. Table 1: Project Summary

2. BACKGROUND & RATIONALE A growing body of evidence shows that children across sub-Saharan Africa are attending school but not acquiring the foundational numeracy skills they need to progress. Across low- and middle-income countries, approximately 70% of 10-year-olds cannot read or understand a simple text, an indication of severe learning poverty, and numeracy performance mirrors this pattern (World Bank & UNESCO, 2022). In Sub-Saharan Africa specifically, 86% of children fail to meet minimum proficiency benchmarks in literacy and numeracy (World Bank, UNESCO & UNICEF, 2022). These macro-level indicators signal systemic weaknesses in foundational learning systems, particularly in early-grade mathematics, where conceptual gaps emerge early and often remain unaddressed. In Nigeria, only about one in four children aged 7 to 14 demonstrate foundational learning skills, with sharp disparities between urban and rural areas and between northern and southern states, especially in early-grade mathematics (World Bank, 2024; UNICEF Nigeria, 2022). NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT ~ 70% 86% 1 in 4 of 10-year-olds in low- and middle-income countries cannot read or understand a simple text (World Bank & UNESCO, 2022) of children in sub- Saharan Africa fail minimum literacy and numeracy benchmarks (World Bank, UNESCO & UNICEF, 2022) Nigerian children aged 7–14 show foundational learning skills, with sharp urban–rural and north–south gaps (World Bank, 2024) However, despite significant government commitments to improve foundational learning, Nigeria’s National Foundational Learning and Numeracy (FLN) Policy (2022) implementation still faces challenges. The Global Education Evidence Advisory Panel ranks structured pedagogy and teaching by learning level among the most cost-effective ways to improve foundational learning (GEEAP, 2023), yet teachers have few practical tools to detect learning gaps early, target support, and track learner progress. The Mama Math tool has also undergone structured preparatory development and contextual design work in Cameroon, including alignment with early grade numeracy priorities and consultation with practitioners. Introductory demonstrations with sub-national education stakeholders have provided constructive feedback that has informed further refinement of the platform. This groundwork positions Mama Math for the first formal pilot focused on feasibility and classroom integration. The pilot is situated within the Federal Capital Territory’s government school system. It coordinates foundational learning with School Support Officers (SSOs) providing coaching and oversight at the school level. It also centers on working with the government to test a teacher- operated tool inside existing government school structures and co-create with government stakeholders from the start.

3. PROGRAM DESIGN & IMPLEMENTATION The pilot is called Numeracy Enhancement, Research and Development (NERD), and the innovation consists of two main components: one, teacher professional development (TPD) that equips teachers on evidence-based approaches to teach numeracy, and two an AI-enabled teacher support tool, Mama Math, that supports classroom delivery. NERD describes what it set out to do. Numeracy enhancement is the work in classrooms, where teachers used Mama Math, supported by training and coaching, to improve learners’ numeracy. Research is the evaluation of that work, measuring what changed for teachers and learners. Development is the use of those findings to refine Mama Math and the model around it for a larger rollout. Mama Math is therefore the tool at the center of the pilot, and NERD is the program that delivered and tested it. NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT 3.1. The innovation Mama Math Six Pillars The work in classrooms: teachers using Mama Math, supported by training and coaching, to improve learners’ numeracy. Photograph a learner’s work; the platform finds the errors and misconceptions, maps them to Grade 2 strands and returns step-by- step reteaching for the same day. An analysis of each uploaded piece of work that teachers use to check understanding and reteach in class. Curriculum-aligned lesson plans, illustrations for the pictorial stage, activity-based approaches and a resource library. Generate lesson plans, create and grade assignments, and ask a built-in chatbot how to teach a topic. Shows which learners struggle and where, so teachers can group and support pupils by level. Carries homework and announcements to caregivers so learning can continue at home. The evaluation of that work, measuring what changed for teachers and learners. Using the findings to refine Mama Math and the model around it for a larger rollout. N.E Numeracy Enhancement R Research D Development The TPD approach to remediation, leveraged the concrete-pictorial-abstract (CPA) approach and the gradual release model of ‘I do, You Do, We Do’, identifying each child’s actual level, remediating specific errors, and running a structured Math Lab session with the combination of Mama Math which supports teachers to put the method into practice quickly in their classrooms. It is a teacher support tool for early-grade numeracy that can be used on an ordinary smartphone, built around six pillars: Targeted Remediation Formative feedback Structured Pedagogy Lesson planning Differentiation Parental Engagement

To localize Mama Math to Grade 2 numeracy skills aligned with the national curriculum of Nigeria. To equip teachers to use it for rapid diagnostics within existing practice routines, reinforced by coaching. To look for detectable short-term gains in learners’ basic numeracy. To generate credible evidence on feasibility, acceptability, and early effectiveness to influence further study. NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT The platform manages learner accounts and tracks class progress on a dashboard. The AI analysis depends on internet access but the app runs even on very low bandwidth internet. Together, the training builds the method, and the tool keeps it in use, making an evidence-based numeracy pedagogy deliverable and sustainable in classrooms. The interface is in French, English, and the chatbot has English, French, English pidgin, Hausa and Yoruba for now. and Teachers use the platform in a weekly Math Lab session, a dedicated lesson in which they teach with concrete materials, collect learners’ written work, upload it to Mama Math, and reteach from the errors the platform identifies. The logic of the pilot is for teachers to use the platform to diagnose errors and receive concrete remediation guidance, so that their day-to-day practice should shift toward targeted feedback, re-teaching, and checking for understanding. If practice shifts in that way, learners should show early gains in numeracy. The pilot used various data sources to support evaluation of the tool. This included delivery records, in observed teaching practice, and in learner performance (See Annex I for more information). The objectives are as follows: 3.2. Theory of change Table 2: Interactive Pathway INPUTS Mama Math platform, teacher training, SSO coaching, weekly Math Lab sessions. ACTIVITIES Teachers run the Math Lab remediation routine, teaching through the concrete-pictorial-abstract approach and the gradual re- lease model (I do, you do, we do). Mama Math supports this with curriculum-aligned lesson plans, assignments to create and grade, error analysis and remediation of uploaded work, a built-in chatbot, teaching illustrations, a resource library, and homework and announcements for caregivers. TEACHERS PRACTICE OUTCOME A shift toward targeted feedback, reteaching, and checking for understanding. LEARNERS OUTCOME Early signs of gains in numeracy. SYSTEM OUTCOME Government-ready evidence and approval to extend the program.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT 3.3 Delivery partners and specific roles for the pilot Led field implementation and coordinated the School Support Officers. Built Mama Math; led monitor- ing, evaluation and learning; led data collection and authored this report. Government partners: co-creation, school selection, adapting the learner assessment, two-way feedback sessions. UBEC and FCT-UBEB Existing government coaches, trained to coach teachers and collect baseline and endline data. SSOs (LEA AMAC) 3.4. Government engagement The program was designed to be delivered through government structures, with its support focused on teachers in their classrooms Co-created the design and agreed priorities, the shape of coaching sessions and the Mama Math lesson-plan content; a government curriculum expert designed the lesson plans. Co-developed the Grade 2 assessment, aligned to the national curriculum so that what was measured matched what teachers are expected to teach. Led delivery: selected schools, issued approval letters and opened access through the Local Education Authorities. Ran through SSOs already supporting teachers; intervention-school SSOs were trained on the app and Math Lab, while control schools kept standard support. Teachers who are involved with the Mama Math have improved tremendously in their teaching skills… the children are now showing more zeal to learn mathematics. GOV-02, GOVERNMENT RESPONDENT FROM SCHOOL VISITS “

4. METHODOLOGY Six schools were chosen to specifically represent three settings: rural, semi-urban, and urban. Each setting contained one intervention and one control school, giving three matched pairs across the sample. Selection was made with relevant government stakeholders, who took part in identifying schools and facilitated access through the Local Education Authorities. Schools were identified with codes and settings (See Table 4). Intervention schools used Mama Math for 8 weeks; control schools continued as normal. The intervention group used the Mama Math platform with associated training and coaching, and the control group continued with standard classroom practice. Learners in all six schools were assessed at the start and end of the pilot through a pre and posttest, so that change in the intervention schools could be read against change in the comparison schools over the same period. NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Table 3: Evaluation Matrix Table 4: Learner assessment sample, by school and cohort Evaluation Question What was measured Data sources Does the Mama Math app show early signs of gains in the numeracy outcomes of Grade 2 learners in Nigeria? Learner numeracy scores and engagement Grade 2 numeracy assessment, learner engagement observation How acceptable is the Mama Math app to teachers, policymakers, caregivers, and learners? Use of and views on the app Interviews with teachers, caregivers, and government; SSO and head teacher survey; learner engagement observation How does training support teachers', policymakers', and caregivers' capacity to use the Mama Math app for early-grade numeracy in Nigeria? Knowledge after training and change in teaching practice Pre- and post-training survey; teacher observation checklist; teacher reflective diaries Is the Mama Math intervention feasible to deliver in schools? Platform use and delivery conditions Platform records; all surveys, observations, interviews, and diaries School (setting) Baseline Returners Endline only Endline total SCH-I-01, intervention (rural) 55 50 20 70 SCH-I-02, intervention (semi-urban) 39 33 15 48 SCH-I-03, intervention (urban) 50 44 21 65

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT SCH-C-01, control (rural) 50 43 36 79 SCH-C-02, control (urban) 50 29 28 57 SCH-C-03, control (semi-urban) 34 29 4 33 Total 278 228 124 352 Returners are pupils assessed at both waves; the 124 endline-only pupils are reported descriptively. Learner numeracy outcomes were measured with a Grade 2 assessment marked out of 46 across six strands, at baseline and endline, and marked against the answer key. Alongside it, the team used a twenty-item teacher observation checklist across five domains (one observed lesson per teacher at base- line and endline), a learner engagement observation. Qualitative data came from teacher reflective diaries kept at baseline, Week 4, and Week 8, endline interviews with teachers in both intervention and control schools, and interviews with caregivers and government stakeholders. Transcripts were coded in MAXQDA. What each tool measured is described with its results in Section 5 and details in the annex. 4. LIMITATIONS OF THE STUDY First, only three teachers were observed in the intervention schools and three in the control schools. With so few teachers, the pilot has low statistical power which means the results cannot be generalized to teachers beyond this pilot. Each teacher was also observed during a single lesson at baseline and at endline, and one lesson may not reflect how a teacher usually teaches. A larger study should observe more teachers, over several lessons each. Second, the intervention and control schools started at different levels at baseline. The six schools were chosen with government to cover rural, semi-ur- ban, and urban settings, not to match learners’ starting scores. For this reason, differences between the two groups are described. A larger study should select schools that start from comparable levels. Third, few caregivers took part. Most did not own a phone, and those who relied on someone else’s were reluctant to keep borrowing it for their child’s schoolwork. Only two caregivers were interviewed at endline, so the pilot says little about learning at home. A future study needs a way of reaching care- givers that does not depend on owning a phone. Fourth, the pilot ran for a single term, and a teacher strike at the start shortened the time for implementation further.

5. RESULTS & FINDINGS The Grade 2 numeracy assessment is graded on a scale of 46 under 6 strands: number recognition, counting fluency, number discrimination, place value, simple operations and word problems. At baseline 278 assessed learners and at endline 352 learners were assessed,. of these, 228 were present at both points (baseline and endline) and form the returner cohort, 124 were assessed at endline only. 50 learners assessed at baseline were not reassessed at endline. Learners are placed in one of four grades from their total: A (35 to 46), B (24 to 34), C (12 to 23) and D (0 to 11). Both arms improved between baseline and endline. Among returners, intervention schools average score from baseline to endline 48.7% to 71.4% and control schools from 36.7% to 64.2% (figure 1 NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT 5.1. Learner outcomes Intervention Baseline mean Returner mean 37% 64% Control 49% 71% School type Figure 1: Learner mean score by arm, baseline to endline returners Baseline vs Returner learners: Mean score Lowest Bands (C+D) and Mean Score C+D baseline Intervention Control C+D endline Mean endline 54% 11% 71.4% 77% 20% 64.2% Baseline and endline mean scores: intervention 48.7% → 71.4%, control 36.7% → 64.2%. The control group started lower, so its gain in points was larger. Read the two arms side by side, not as a league table.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Table 5: Average performance by schools Table 6: Learner mean score and lowest-band share, baseline to endline returners, by arm Table 7: Grade distribution by arm and cohort, returners (percent of pupils in each band) School codes SCH-I-01 SCH-I-02 SCH-I-03 SCH-C-01 SCH-C-02 SCH-C-03 Baseline mean 61.0% 38.9% 42.9% 39.5% 22.0% 54.3% Endline mean 66.0% 73.6% 73.5% 61.7% 57.8% 65.9% Group Baseline mean Endline mean In two lowest bands, baseline In two lowest bands, endline Baseline mean 61.0% 38.9% 42.9% 39.5% Endline mean 66.0% 73.6% 73.5% 61.7% Group and cohort Grade A Grade B Grade C Grade D Intervention, baseline 12% 34% 42% 12% Intervention, endline 46% 42% 9% 2% Control, baseline 4% 19% 46% 31% Control, endline 24% 56% 17% 3% The grade bands make the same movement visible across the whole distribution. Grade bands on the 46-mark assessment: A = 35 and above, B = 24 to 34, C = 12 to 23, D = 0 to 11. Figures are for returners and are descriptive; the school did not start from equal baselines. The details for each school’s grade distribution are found in annex C (Table 15) and the visual representation is found on figure 2 annex C Both groups improved on all six strands (Table 8). At endline, intervention learners scored higher than control learners on five of the six strands, and the two groups finished at a similar level on number recognition. The largest gains in both groups were in simple operations, number discrimination, and word problems, while place value remained the lowest-scoring strand, however the control improved more on place value than the control school. Other factors during the term, such as regular classroom teaching and learners’ normal development, may have contributed to these gains, and the pilot cannot separate them from the effect of the intervention.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Table 8: Numeracy performance by strand, baseline to endline returners, by arm (percent correct) School codes SCH-I-01 SCH-I-02 SCH-I-03 SCH-C-01 Number recognition 82.6 86.3 72.7 87.9 Counting fluency 46.0 72.3 33.7 63.3 Number discrimination 40.6 68.4 29.3 64.5 Place value 18.5 40.2 2.4 11.9 Simple operations 45.0 77.8 32.6 71.1 Word problems 26.7 54.3 14.2 45.0 The engagement observation showed a 10-point increase in learner engagement in intervention classes, which rose from 25.0 to 35.0, while control classes remained unchanged at 32.0. The learner engagement checklist measured how actively learners took part in mathematics lessons. An observer watched one Grade 2 lesson per school at baseline and at endline and, for each behavior on the checklist, counted how many learners showed it. The first part of the check- list looked at active engagement, such as learners attempting the task, answering the teacher’s questions, staying on task, and working with classmates. The second part looked at deeper participation, such as learners doing tasks with- out being asked, acting on the teacher’s feedback, and understanding what they were expected to do. Together these two parts give a total engagement score of 55, which is used to compare baseline with endline and intervention with control schools. A third part, used only in intervention schools at endline, recorded how learners behaved while the teacher was using Mama Math. The gain was visible in active participation and in deeper forms of engagement, such as explaining and demonstrating. One teacher wrote in her diary that pupils were “enjoying it (maths) better now than any other subject” (teacher reflective diary). 5.2. Learner engagement “ enjoying it (maths) better now than any other subject Mathematics Teacher

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Table 9: Learner engagement and participation across all schools Read by setting, the intervention school outgained its control pair in all three, largest in the semi-urban pair. School Group Setting Baseline /55 Endline /55 Change SCH-I-01 Intervention Rural 32 39 +7 SCH-I-02 Intervention Semi-urban 21 38 +17 SCH-I-03 Intervention Urban 22 28 +6 SCH-C-01 Control Rural 35 34 −1 SCH-C-02 Control Urban 35 34 −1 SCH-C-03 Control Semi-urban 26 28 +2 Setting Intervention change Control change Difference Rural (SCH-I-01 vs SCH-C-01) +7 −1 +8 Semi-urban (SCH-I-02 vs SCH-C-03) +17 +2 +15 Urban (SCH-I-03 vs SCH-C-02) +6 −1 +7 Table 10: Engagement levels comparable across school settings Group Baseline Endline Change Intervention 25.0 35.0 +10.0 Control 32.0 32.0 0.0 Table 11: : Learner engagement scores out of 55 on comparable items, baseline to endline, by arm Learner engagement increased in the intervention schools and remained constant in the control schools, across both sections of the comparable score, largest in the semi-urban pair.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT 5.3. Teacher outcomes The teachers practices focus was on differentiation and remediation, delivered through the concrete, pictorial, and abstract approach. Teachers worked through place value and through addition and subtraction using bundles, sticks, bottle tops, stones and play money, moving from physical materials to pictures to symbols, and practiced the sequence of modelling, guided practice, and independent work. Activity-based remediation, practical word problems, and positive rein- forcement completed the pedagogy, all organized around the structured Math Lab session through which the model is implemented. Gender Equality and Social Inclusion (GESI) addressed direct relevance to numeracy instruction; strategies for creating inclusive class- room environments that promote equitable participation; and approaches to identifying and actively challenging gender stereotypes that limit learners’ particularly girls’ engagement with mathematics. The responsible use of AI was the second theme, taught as a human-in-the-loop approach including checking the tool’s output for errors, and built in as cross cutting themes. The platform was introduced and practiced within the same pedagogy, enabling teachers to enroll learners, generate level-based lessons and practical word problems, upload pupils’ work for analysis, and use the resulting insights to identify learning gaps and provide targeted remediation. SSOs and head teachers were trained to move from a supervisory role to coaching and mentoring, to monitor teachers, to run post-observation feedback, to support a WhatsApp community of practice and on how to collect data using all the evaluation tools. The classroom observation checklist was used to measure teacher practices, whether they appeared in teaching at baseline and endline. The observation checklist score reflects the number of the 20 target practices demonstrated during the lesson, with a higher score indicating that a wider range of the intended practices was applied. Each group’s score represents the average across three teachers, which is why the results may include decimals. For example, a score of 5.7 indicates that, on av- erage, about 6 of the 20 target practices were ob- served in a typical lesson, while a score of 18.3 indicates that about 18 of the 20 practices were observed. The strongest result in the pilot was the improvement in teaching practice, as reflected in the twenty-item observation checklist, interven- tion teachers moved from 5.7 to 18.3 practices per lesson, while matched control teachers held level, from 14.3 to 14.0. The difference between those movements is 13.0 points, equivalent to 65% points. Every intervention teacher gained more than every control teacher, a complete separation between the two groups. The Difference: 13.0 points, equal to 65 % points Every intervention teacher gained more than every control teacher. Scores are averages of three teachers on a twenty-item checklist across five domains of four practices each; a score of 18.3 means about 18 of 20 practices were seen in a typical lesson. 5.7 18.3 14.3 14.0 of 20 target practices observed per lesson held level across the same period Intervention Teachers Control Teachers

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Table 12: Teacher practice scores out of 20, baseline to endline, by school and arm Group Baseline Endline Change SCH-I-01 13 18 +5 SCH-I-02 3 18 +15 SCH-I-03 1 19 +18 Intervention average 5.7 18.3 +12.7 SCH-C-01 16 15 -1 SCH-C-02 17 17 0 SCH-C-03 10 10 0 Control average 14.3 14.0 -0.3 Intervention teachers improved in every domain and by endline were using all four feedback and verification practices. Control teachers stayed level in most domains and used fewer concrete materials at endline than at baseline. The largest gaps were in specific feedback and checking that learners understood, the two behaviours most prompted by the platform. 01 use of concrete materials and visual representation 02 targeted grouping 03 differentiation 04 re-teaching and remediation, feedback quality, and verification of learning. 5.4. Feasibility and delivery The pilot examined whether the model could be delivered in government classrooms, and whether teachers could use the mama math in their lessons, learners, caregivers, and government could take it up[PB50.1]. The table below shows the feasibility of from[PB51.1] the pilot, both quantitative and qualitative. This section draws on all the pilot’s data sources: the pre- and post-training survey (11 participants) and the SSO and headteacher impression survey; the platform records of uploads, feedback, and lesson plans; the classroom observation of teachers and learners at baseline and endline in both arms; teacher interviews and reflective diaries; caregiver interviews; and government interviews.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Category Intended (design and target) Pilot evidence Outcomes Teacher professional development Training before implementation to build teacher capacity in the concrete-pictorial-abstract approach, covering targeted remediation, differentiation, formative assess- ment, lesson planning, and platform use, with SSOs and headteachers prepared to move from a supervisory to a coaching and mentoring role. 11 participants trained from a near-zero baseline: 91% had never heard of the platform, only 27% had prior remedi- ation training. Post-training platform knowledge reached 78%; 82% were very confident on the critical functions; about 80% were ready to implement; training quality was endorsed by 100%, and all SSOs and headteachers were confident in the coaching role. Observed practice then rose from 5.7 to 18.3 of 20 in the intervention arm while the control held level, from 14.3 to 14.0, a difference of 13.0 points, with every intervention teacher outgaining every control teacher. The training built functional capacity quickly and it supported teachers with instruction in the classroom, the strongest and most consistent result in the pilot. “the training reminded me of some missing understanding method of teaching” (TCH-I-01). App usage (the six pillars) Teachers use the platform across its six pillars: targeted remediation, structured pedagogy ( concrete-picto- rial-abstract, the Math Lab routine), differentiation, formative feedback, lesson planning, and parental engagement. Targeted remediation and formative feedback: 197 uploads and 175 AI feed- back rounds in total. Structured peda- gogy: 8 Math Lab sessions. Differenti- ation: the grouping and differentiation domain rose from 0.33 to 3 out of 4 in the intervention arm while the control held level, a difference-in-differences of +2.67. Lesson planning: 41 plans by the teachers. Parental engagement: of about 210 caregivers invited, 36 attend- ed orientation and 2 of 15 approached were interviewed. Five of the six pillars were used as designed, the exception was paren- tal engagement, where the home loop did not close and the differen- tiation gap over the control is the clearest sign the app changed prac- tice.. Teachers grouped learners by ability for small-group work, which one teacher said “Grouping helps those that are shy to come out” (TCH-I-02b). Parental engagement is the exception: the home loop did not close (“it's only 2 parents responding” (TCH-I-02). Table 13: Feasibility and delivery summary

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Category Intended (design and target) Pilot evidence Outcomes Dosage and classroom integration Regular use across the term, teachers uploading learner work and acting on the returned guidance as part of the lesson cycle, delivered under coaching and in ordinary classroom conditions. Teachers were expected to run one Math Lab session a week, uploading learners' work to Mama Math in each session. Over the eight weeks, the six teacher accounts recorded 197 up- loads, , concentrated in the more active teachers (the heaviest accounting for 76 of 197 uploads and 20 of 41 plans). Delivery was shaped by conditions outside the tool: the AI loop needs internet and cannot run offline; device access and connectivity were named across teacher and SSO responses; one account stopped when the teacher lost his phone. Delivery worked and feasible under supported conditions and limited by connectivity and devices. “there is no way you can access… gener- ate your lesson notes… or upload your work” without a network (TCH-I-01). Acceptability The model and tool are accepted in use by teachers, learners, SSOs, headteachers, and government, and teachers intend to continue. Every intervention teacher interviewed used the platform and intended to con- tinue, valuing lesson preparation and error analysis most. Learner engage- ment rose in the intervention arm from 25.0 to 35.0 of 55 while the control held level at 32.0, a difference of 10 points, a behavioral sign of acceptance. All SSOs and headteachers endorsed the model, and the government endorsed it and situated it in the national agenda. Caregivers are the exception, with most not engaging. Acceptance was high across the groups that used the model, and the learner-engagement gap over the flat control shows it in behavior. “For me everything is correct and easier, so it is perfect” (TCH-I-01); a request that it be “sustained if possible by the government” (TCH-I-01). The group it did not reach was caregivers.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Category Intended (design and target) Pilot evidence Outcomes Government engagement The government co-creates the pro- gram, adapts it to the national cur- riculum, and engages across the full cycle through UBEC, the FCT-UBEB, and the SSO structure. The government co-developed the design and the learner assessment, aligned it to the national curriculum, issued the approvals, and joined feed- back sessions across the pilot. All SSOs and headteachers endorsed the model, and the government requested exten- sion to further schools. The government engaged from design through delivery and moved from interest to intent. “Mama Math has been structured for schools using the current curriculum… so many tools that will definitely push the quality of education higher” (SSO, GOV-01). Delivery worked, but it was shaped throughout by conditions outside the tool. The teacher strike at the start of the pilot delayed implementation and connectivity was the main barrier: the AI analysis needed the internet, so where the network failed the mechanism stopped. Teachers were plain about it, one saying it is: “always easy when there is a network… and hard only when data or airtime ran out” (TCH-I-02), another naming his main need as “data… or charging my power bank” (TCH-I-03). Device access shaped how quickly some could start. Access to a phone also affected use. One teacher had no smartphone at the training and relied on a borrowed phone until she bought her own. Another uploaded learners’ work regularly from the training until late May, then stopped when he lost his phone. Government engagement provided the clearest indication of institutional acceptance of the program, moving beyond general approval to evidence of interest in wider adoption. All four government respondents endorsed the program, suggesting a consistent level of support among the government stakeholders interviewed. “ ““always easy when there is a network… and hard only when data or airtime ran out” (TCH-I-02), “data… or charging my power bank” (TCH-I-03) 5.5. Government outcomes

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT More importantly, this support was reflected in concrete steps toward institutionalization. One respondent reported that he had already “written for approval to extend the platform to over 20 more schools so that the government could adopt it as a numeracy strategy” (GOV-04). Another framed continued monitor- ing by SSOs as the condition for durability, warning that without it a program “goes into extinction” (GOV-02). A third raised a careful concern about verifica- tion, noting that teachers can report activity they have not carried out, and asked that the reports be shared so that reported use can be checked (GOV-01). These views emerged through the feedback sessions that ran across the pilot, in which the team shared interim findings with authorities and heard their questions and priorities in return. That format-built familiarity and ownership turned into a concrete request to expand the program, and it surfaced the verification concern early enough to design it. 4 of 4 20+ 2-WAY government respondents endorsed the programme more schools: one respondent has already written for approval to extend, so government can adopt it as a numeracy strategy feedback sessions shared interim findings and shaped next steps, build- ing ownership “ “written for approval to extend the platform to over 20 more schools so that the government could adopt it as a numeracy strategy” (GOV-02 5.6. Caregiver outcomes and the home loop The home component was where the pilot fell short, approximately 210 caregivers were expected at an orientation on how to support learning through the tool, and only 36 attended. Of 15 caregivers anticipated an endline interview; only two could be reached. The two who took part were positive: one, a father, described his “child moving from fear of mathematics to develops courage when the subject came up” (CG-01), the other reported her child naming shapes such as “square, rectangle, triangle” unprompted (CG-02). Most caregivers did not engage at all, so the home-school loop largely ran one way. There are three hypotheses for why caregiver engagement could not have worked. The first is phone access: the design assumed caregivers could be reached through their own phones or those of family members, but most caregivers do not own one, and those relying on someone else’s may have been reluctant to keep borrowing it for their child’s schoolwork. The second is the return path: caregivers who received messages may not have known how to send their children’s work back through the app. The third is time and awareness: a single orientation may not have been enough for caregivers to understand their role or how to use the platform.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT 5.7 Unintended effects Two unplanned effects emerged, both informative: the unprompted spread of the tool from one teacher to a colleague suggests the model can travel through peer networks, which matters for low-cost scale. The verification concern raised by a government respondent, though not a negative effect of the tool itself, surfaced with a real design question about how reported teacher activity is checked and is better treated now than after scale. 6. IMPLICATIONS OF THE FINDINGS Two unplanned effects emerged, both informative: the unprompted spread of the tool from one teacher to a colleague suggests the model can travel through peer networks, which matters for low-cost scale. The verification concern raised by a government respondent, though not a negative effect of the tool itself, surfaced with a real design question about how reported teacher activity is checked and is better treated now than after scale. Beyond the test scores, teachers and caregivers described a change in how children relate to mathematics. Several reported that pupils who had once avoided the subject began to attempt it more readily and showed more confidence in numbers and shapes. One intervention teacher noted that pupils “who are used to running away from mathematics are now engaged with it” (TCH-I-03). The interviews add how the people around the learners read the change, this perception is reported by teachers, caregivers, and government, and no learner was interviewed. The reported change is consistent, teachers most often described gains in the things the assessment covers, counting, number identifica- tion, and shapes, one describing a struggling learner who, once the counting materials came in, “started developing interest in math and now she can solve math to some extent”, and another a named pupil: “there is one pupil called… the boy doesn’t use to be active… but since we started this mama math teaching this boy began to take part” (TCH-I-02). Caregivers saw the same at home, one child naming “square, rectangle, triangle” unprompted. Government respondents reported it from their classroom visits as children showing “more zeal”. Several accounts were about attitude, describing children who “don’t even love math before now asking questions in class”, and a father whose child, once fearful of the subject, “develops courage to do math problems”. Respondents attributed this to the program, tying it to the concrete materials, the group work, and the reteaching of the TPD and intervention prompted. This perception corroborates the direction of the assessment and observation findings. Teaching practice had the biggest gains in the use of specific feedback and checking that learners understood, with learner engagement rising in the same classrooms, as noted above, while teachers’ diaries and interviews similarly described the same change and attributed it to the training and coaching. Gender equality and social inclusion were taught as equitable participation, and in practice teachers used the grouping and gradual-release routine to bring in the learners usually left behind, the shy and the struggling. One found that “the grouping helps those that are shy to come out… they express themselves through group work” (TCH-I-02).

The responsible use of AI was taught as a human-in-the-loop approach, the teacher checking the platform’s analysis against what they know of their pupils, and teachers did this in practice, going back to verify the tool’s feedback before reteaching. The training and the platform worked together, teachers were trained on the pedagogy and the app together, and several started from writing sums on the board and marking by hand. One had used a smartphone only “to call, texting, then just do WhatsApp” before the training put an app in her hands (TCH-I-02), and another was plain that “it’s through the training that make me know and understand something like that, if not I wouldn’t know about mama math” (TCH-I-01). By endline those teachers described a settled routine the training installed, and the platform sustained: generate the lesson plan, teach, upload the class work, and reteach from the analysis. One laid out the whole sequence, ending “what the mama math will tell me from the analysis, I will use it and reteach remediations” (TCH-I-03), and another had made a weekly remediation class part of her practice, grouping learners through “I do, we do, you do” (TCH-I-02). Capacity was reinforced by two coaching visits, at Week 4 and Week 8, in which SSOs observed lessons and worked through the platform’s guidance, and teachers attributed the change in their practice as much to this on-site coaching as to the tool itself. Teachers and government were clear that the on-site coaching made it hold, a government respondent described the sequence: after the initial training “it wasn’t easy,” but once field staff “came on board… they were on ground to give them more support and the implementation has been smooth and fast” (GOV-04). Teachers said the same, one describing the support as constant: “the team is doing a very good job by coming every day to come and monitor the activities of the teacher” (TCH-I-01). What teachers named most often was confidence, and they tied it to the training, one ended the pilot saying the program “build confidence in me to become a good teacher and also gave me more confidence to teach mathematics” (TCH-I-01), and another that it “build up confidence in the time of teaching” (TCH-I-03). A teacher who first found the method daunting described the arc: “when we went for the training, I was shock and wondering if I could be able to do this because it is a new method to me,” and then that the app was “now improving my teaching which I enjoy it more than before” (TCH-I-03). NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT There were also gaps, one teacher was candid that “the training is not enough,” and that it took “like six weeks with the help of colleagues” before she used the platform correctly (TCH-I-03). The recurring request was for more of the same, one arguing that “when you train and you retrain a teacher, that is when you get the best out of the teacher” (TCH-I-02). A government respondent framed continuous support and SSO monitoring as the condition for the whole thing to last, warning that without monitoring a program “goes into extinction” (GOV-02). Capacity also spread beyond the trained teachers: one said, “I started telling a primary three teacher about mama math” (TCH-I-01), and another registered a colleague “so that even if I am not around he can use it” (TCH-I-03), a diffusion that matches the platform record, where one account was taken up by a colleague who had not been part of the training. Teacher’s spoke of needing airtime as a daily problem, one naming his main issue as “data and smart phone or charging my power bank” (TCH-I-03). Device access shaped how quickly teachers could start: one had no smartphone at the training and could only “do one or two things” by borrowing before buying her own phone afterwards (TCH-I-02). The account that stopped in late May is explained in the teacher’s own diary, where he noted that he had uploaded learners’ work regularly “before I lost the phone” (TCH-I-03b), the device loss the platform pattern reflects (Table 17). The home side of the model is where feasibility fell short, the return path did not work for most caregivers, so the two-way loop ran one way. One teacher, uploading throughout, reported that “only two people have been answering all my assignments” (TCH-I-01), and described redirecting a parent who had tried to send work back by WhatsApp toward the app link. Government respondents extended the point to parents, many of whom lacked an Android phone at all (GOV-01). Government engagement was the most developed relationship in the study, going as far as a request to extend the program. Growing it, respondents were specific, meaning formal partnership and funding, curriculum integration, and extension to higher grades and more schools (GOV-04). Teachers made the same case from their side, one asking that the program “be sustained if possible, by the government to develop learners’ interest” (TCH-I-01). Taken together, the intervention is feasible to deliver in the classroom, where trained and supported teachers run it, and not yet feasible to sustain or scale without solving the data and device constraints, repairing the return path to caregivers, and putting in place the funding and monitoring that government respondents set as their own conditions. “when you train and you retrain a teacher, that is when you get the best out of the teacher” (TCH-I-02)

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT 7. LESSONS LEARNED Coaching makes the tool stick Initial training plus continuous on-site coaching by SSOs changed classroom practice. This is a pedagogy intervention carried by people; a coaching structure embedded in existing SSO roles gave teachers someone to turn to. AI strengthens TPD, not replaces it Training drove the practice change; Mama Math kept it alive by putting error diagnosis in the teacher’s hand each lesson, and made PD visible through uploads and error patterns coaches can act on. AI strengthens TPD, not replaces it The design assumed messages would reach phones. Most caregivers had none. A way forward: more sensitization, then a short (about two- day) training for the parents in the programme on opening the app, reading messages and sending work back. Measure delivery from the system Platform upload records, not teacher self-reports, gave a direct measure of fidelity. Retain this practice as government cautioned that reported and actual activity can diverge. The Contribution of the NERD pilot It shows that a teacher-operated model can produce a clear shift in teaching practice in government classrooms at low cost, and it identifies precisely where the home component breaks down and what a redesign must address. 01 02 03 04 04

8. SUSTAINABILITY AND NEXT STEPS The pilot ends with a clear pathway, and with more institutional momentum, government stakeholders accepted and began to act on it. All four respondents endorsed it, and one had already started the administrative steps to extend it to more than 20 further schools, so that the government could adopt it as a numeracy strategy. Durability was uniformly tied to ongoing monitoring by SSOs within the existing structures. Sustained implementation required what respondents have identified as continuous on-site coaching and monitoring via the SSO structure, delivery that can be verified through platform records, and integration into the educational program and professional-development calendar. All of these are conditions that a scale-up must be designed to meet. In addition, the platform has been rebuilt as a standalone app. Teachers and caregivers now receive notifications directly on their phones rather than working through links, which removes the point at which caregiver engagement broke down in the pilot. The next phase should test whether that change closes the loop, since the pilot could not. For funders, this pilot has already produced two things: a credible early signal on teacher practice, and genuine demand from the government that would ultimately own the program. Continued support now, to fund a properly powered evaluation and extended implementation time, is what would turn that promise into evidence solid enough to justify system-wide adoption, through structures that are already in place. NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT NERD EVIDENCE STEPS Commission a larger, better-powered evaluation that can convert promise into proof, with balanced baselines, a longer period of exposure than a single term, and continuous follow-up so that durability can be observed. Concentrate on delivery on the teacher-facing loop, where the pilot shows its strongest and most immediate effect, paired with sustained SSO coaching. Formalize the government’s partner- ship and pursue the funding to match it, moving from pilot endorsement toward a co-funded, government- integrated program through targeted applications for the next phase.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT REFERENCES World Bank, UNESCO, UNICEF, FCDO, USAID, & Bill & Melinda Gates Foundation. (2022). The State of Global Learning Poverty: 2022 Update. Washington, DC: World Bank. World Bank. (2024). Nigeria: HOPE for Quality Basic Education for All (Project Appraisal Document). Washington, DC: World Bank. UNICEF Nigeria. (2022). Foundational Literacy and Numeracy in Nigeria. Abuja: UNICEF Nigeria. Federal Republic of Nigeria. (2022). National Policy on Foundational Literacy and Numeracy (FLN Policy). Abuja: Federal Ministry of Education.

● Grade 2 Numeracy Assessment (marked out of 46 across six strands), baseline and endline, all six schools. ● Classroom Teacher Observation Checklist (twenty indicators, five domains), one observed lesson per teacher, baseline and endline, all six schools. ● Learner Engagement Observation Checklist, baseline and endline, all six schools. ● Teacher reflective diaries, kept at baseline, Week 4, and Week 8, intervention schools. ● Endline interviews with teachers in intervention and control schools; interviews with caregivers and government stakeholders. NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT ANNEXES Annex A. Data collection instruments Annex B. Learner performance by school, baseline to endline Table 14: School-level total mean percentage scores are descriptive. School (setting) Baseline % Returner endline % Change SCH-I-01, intervention (rural) 61.0 68.0 +7.0 SCH-I-02, intervention (semi-urban) 38.9 74.8 +35.9 SCH-I-03, intervention (urban) 42.9 72.7 +29.8 SCH-C-01, control (rural) 39.5 65.7 +26.2 SCH-C-02, control (urban) 22.0 61.5 +39.5 SCH-C-03, control (semi- urban) 54.3 64.8 +10.5

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Control Endline Control Baseline Intervention Endline Intervention Baseline Number Recognition Simple Operations Counting Fleuncy Number Discrimination Place Value Word problems 26.7 54.3 14.2 45.0 18.5 40.2 2.4 11.9 40.6 68.4 29.3 64.5 46.0 72.3 33.7 63.3 45.0 77.8 32.6 71.1 82.6 86.3 72.7 87.9 Figure 2: Mean score across the strands between the intervention and control schools

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Annex C. Learner grade distribution by school, baseline to endline Table 15: Grade distribution by school (percent in band, baseline to endline) School (setting) Grade A (baseline to endline) Grade B (baseline to endline) Grade C (baseline to endline) Grade D (baseline to endline) SCH-I-01 (rural) 22 to 30 53 to 62 20 to 6 5 to 2 SCH-I-02 (semi-urban) 5 to 61 13 to 30 62 to 9 21 to 0 SCH-I-03 (urban) 6 to 55 30 to 27 50 to 14 14 to 5 SCH-C-01 (rural) 0 to 19 20 to 70 60 to 9 20 to 2 SCH-C-02 (urban) 0 to 31 2 to 38 38 to 24 60 to 7 SCH-C-03 (semi-urban) 18 to 24 41 to 55 38 to 21 3 to 0 Each cell shows the percentage of pupils in that grade band, at baseline and at endline (returners). The movement toward the top bands is visible in every intervention school and, from a lower base, in the control schools. Grade bands on the 46-mark assessment: A = 35 and above, B = 24 to 34, C = 12 to 23, D = 0 to 11.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT School Cohort n Mean /46 Mean % A A % B B % C C % D D % SCH-I-01 Baseline 55 28.1 61.0% 12 22% 29 53% 11 20% 3 5% SCH-I-01 Returner 50 31.3 68.0% 15 30% 31 62% 3 6% 1 2% SCH-I-01 End- line-only 20 28.0 60.9% 5 25% 9 45% 5 25% 1 5% SCH-I-01 All end- line 70 30.4 66.0% 20 29% 40 57% 8 11% 2 3% SCH-I-02 Baseline 39 17.9 38.9% 2 5% 5 13% 24 62% 8 21% SCH-I-02 Returner 33 34.4 74.8% 20 61% 10 30% 3 9% 0 0% SCH-I-02 End- line-only 15 32.7 71.0% 7 47% 6 40% 2 13% 0 0% SCH-I-02 All end- line 48 33.9 73.6% 27 56% 16 33% 5 10% 0 0% SCH-I-03 Baseline 50 19.7 42.9% 3 6% 15 30% 25 50% 7 14% SCH-I-03 Returner 44 33.4 72.7% 24 55% 12 27% 6 14% 2 5% SCH-I-03 End- line-only 21 34.7 75.4% 11 52% 9 43% 1 5% 0 0% SCH-I-03 All end- line 65 33.8 73.5% 35 54% 21 32% 7 11% 2 3% SCH-C-01 Baseline 50 18.2 39.5% 0 0% 10 20% 30 60% 10 20% SCH-C-01 Returner 43 30.2 65.7% 8 19% 30 70% 4 9% 1 2% SCH-C-01 End- line-only 36 26.3 57.2% 6 17% 18 50% 10 28% 2 6% SCH-C-01 All end- line 79 28.4 61.% 14 18% 48 61% 14 18% 3 4% Table 16: Mean score and grade distribution by cohort and arm

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT SCH-C-02 Baseline 50 10.1 22.0% 0 0% 1 2% 19 38% 30 60% SCH-C-02 Returner 29 28.3 61.5% 9 31% 11 38% 7 24% 2 7% SCH-C-02 End- line-only 28 24.9 54.0% 2 7% 14 50% 12 43% 0 0% SCH-C-02 All end- line 57 26.6 57.8% 11 19% 25 44% 19 33% 2 4% SCH-C-03 Baseline 34 25.0 54.3% 6 18% 14 41% 13 38% 1 3% SCH-C-03 Returner 29 29.8 64.8% 7 24% 16 55% 6 21% 0 0% SCH-C-03 End- line-only 4 34.0 73.9% 2 50% 2 50% 0 0% 0 0% SCH-C-03 All end- line 33 30.3 65.9% 9 27% 18 55% 6 18% 0 0% Annex D. Learners engagement The learner engagement checklist has three sections. For each item the observer counted how many learners showed the behavior and scored. Sections A and C were used in all six schools and together give the comparable total of 55. Section B applies only to intervention schools, at endline, when the app was in use. School Group Section A /35 Section C /20 Total /55 Change SCH-I-01 Intervention 21 to 27 11 to 12 32 to 39 +7 SCH-I-02 Intervention 14 to 26 7 to 12 21 to 38 +17 SCH-I-03 Intervention 15 to 17.5 7 to 10.5 22 to 28 +6 SCH-C-01 Control 23 to 24 12 to 10 35 to 34 −1 SCH-C-02 Control 25 to 22 10 to 12 35 to 34 −1 SCH-C-03 Control 18 to 20 8 to 8 26 to 28 +2 Table 17: Engagement by section and school, baseline to endline

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT SCH-I-01 SCH-I-02 SCH-I-03 SCH-C-01 SCH-C-02 SCH-C-03 39 32 38 21 28 22 34 35 34 28 26 Figure 4: Learner engagement by school, out of 55 Item Intervention, baseline to endline Control, baseline to endline A1. Seated and oriented toward the teacher or task 4.3 to 3.8 4.3 to 3.7 A2. Respond when the teacher asks questions 2.3 to 3.0 3.3 to 3.3 A3. Visibly attempting the numeracy task 2.3 to 3.5 3.3 to 4.0 A4. Stay on task without needing redirection 1.7 to 3.0 2.3 to 3.0 A5. Peer interaction related to the task 2.0 to 3.3 2.7 to 2.7 A6. Struggling learners receive the teacher’s attention 1.7 to 2.3 2.3 to 2.0 A7. Actively engaged for most of the lesson 2.3 to 3.3 3.7 to 3.3 Table 18: Item-level scores by group (mean score out of 5)

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT C1. Initiate participation without being prompted 2.3 to 2.8 2.0 to 2.0 C2. Respond positively to the teacher’s support 2.3 to 3.0 3.0 to 2.7 C3. Understand what is expected of them 2.3 to 3.0 3.0 to 3.0 C4. Learners at different levels receive differenti- ated attention 1.3 to 2.5 2.0 to 2.3 Items A1 to A7 form Section A, active engagement. Items C1 to C4 form Section C, deeper participation. Table 19: Section B, intervention schools at endline School Section B /25 Items covered SCH-I-01 19 Learners continue their task while the teacher uses the app; stay calm when the teacher’s attention is on it; show curiosity about it; re- engage quickly afterwards; and receive direct attention following the app’s guidance. SCH-I-02 18 SCH-I-03 17 Annex E. Teacher practice by domain The twenty observation items grouped into five domains, each scored out of 4. The table shows the movement in each and the difference-in-differences against the control arm. The largest differences fall in feedback and in checking that learners have understood. Table 20 : Teacher practice by domain, baseline to endline Domain (out of 4) Int. base Int. end Ctrl. base Ctrl. end Difference-in- differences Materials and representation 2.67 3.67 2.33 1.00 +2.33

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Grouping and differentiation 0.33 3.00 2.33 2.33 +2.67 Re-teaching and remediation 1.33 3.67 2.67 3.33 +1.67 Feedback quality 0.67 4.00 4.00 4.00 +3.33 Checking for understanding 0.67 4.00 3.00 3.33 +3.00 Total (out of 20) 5.7 18.3 14.3 14.0 +13.0 Difference-in-differences are the intervention movement minus the control movement. With three teachers per arm, these figures describe a consistent pattern rather than a significance-tested effect.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Annex F. Platform activity by teacher account Annex G. Case pictures Platform use was concentrated on the more active teachers. Accounts are anonymized and ordered by the number of uploads. Teacher account Uploads Rounds of AI feedback SCH-I-01, account a 35 33 SCH-I-01, account b 28 28 SCH-I-02, account a 76 69 SCH-I-02, account b 10 10 SCH-I-03, account a 24 21 SCH-I-03, account b 24 14 Total 197 175 Table 21: Uploads and AI feedback by teacher account The platform was usable in real classrooms. Across six intervention teacher accounts it recorded 197 uploads and 175 rounds of AI feedback over the pilot. Use was uneven and concentrated in the more active teachers, with the heaviest user accounting for 76 uploads. The main constraint on delivery was con- nectivity. The platform’s AI analysis depends on internet access and cannot run offline, so uploads are clustered where and when networks allowed, with data cost, device access, and power as secondary barriers. Peer spread. At one intervention school a teaching assistant took up Mama Math alongside the lead teacher without separate training, the pilot’s single case of the tool spreading on its own between colleagues. It is a small sign that the model can move through ordinary peer networks. A caregiver approached for an endline interview had received the platform’s messages, understood them, and sat with his child on the work, but didn’t under- stand how to submit the completed task. He kept encouraging his child anyway and reported gains in confidence. His experience captures both the promise of the home component and the connectivity barrier that kept it from closing.

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Annex H. Risk register Several of the risks anticipated for the pilot materialized, and the evaluation reading of the findings reflects them. Likelihood and impact are pre-mitigation ratings. Table 22 : Risk register Risk Likelihood Impact Mitigation (designed in) Residual and implication for scale Teacher strike or industrial action Medium High Baseline scheduled ahead of training; coaching and platform use planned with slack; delivery measured from platform records Contact time is still lost in affected schools; scale needs schedule contingency and disruption tracking Connectivity as a hard dependency High High The AI loop requires internet and cannot run offline; schedule uploads for when connectivity is available, provision or subsidies data, and factor network reliability into school selection Connectivity remains a binding condition; a scaled program must secure reliable access rather than assume it Device access (teachers and caregivers) High High Training accommodated shared and borrowed devices; caregiver messages designed for basic phones with audio Some teachers were slow to start and many caregivers unreached; device assumptions must be tested and budgeted Non-equivalent groups at baseline High High Matched setting pairs; baseline measured on all instruments to characterize the gap Groups still differed at baseline, making the difference-in-differences unreliable; a powered, better-balanced design is needed for a causal test Short pilot window (single term) High Medium Findings scoped from the outset as evidence of promise, not proof of impact Too short to show durable change; a longer follow-up is needed to test persistence Teacher turnover or low digital confi- dence Low Medium Initial training plus continuous on-site coaching to build confidence and cover gaps Did not materially affect delivery; continuous training and peer support should be retained at scale

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Annex I. Theory of change N.E.R.D. Pilot - Mama Math AI-Enabled Numeracy Platform Figure 6: Theory of change PROBLEM Grade 2 pupils in Nigerian government schools are attending school but not acquiring foundational numeracy skills Teachers lack diagnostic tools to catch early errors, instruction stays undifferentiated, and professional development rarely translates into daily classroom practice INPUTS An AI-enabled, teacher-operated numeracy platform (Mama Math) and its implementing and government partners Teacher professional development, on-site coaching support, and partici- pating government primary schools ACTIVITIES Train teachers and school support staff in evidence-based numeracy pedagogy and platform use. Teachers use the platform to diagnose pupils’ errors and guide targeted remediation in the classroom. Provide ongoing coaching and engage government stakeholders and caregivers throughout delivery Support teachers through the community of practice on WhatsApp OUTPUTS A measurable volume of platform use: pupil work uploaded and AI feedback returned to teachers Teachers trained and coached; learn- ers assessed; government engaged through structured feedback sessions OUTCOMES Teachers shift toward targeted feed- back, re-teaching, and checking for understanding in daily practice Learners show early numeracy gains and stronger engagement; government stakeholders endorse and seek to extend the model IMPACT Evidence that a teacher-mediated AI diagnostic tool is feasible and accept- able to deliver in government primary schools A credible case, and early signs of promise, to justify a larger, bet- ter-powered evaluation of effectiveness

NERD | NUMERACY ENHANCEMENT RESEARCH & DEVELOPMENT Key Assumptions Indicators / How We’ll Measure Success Teachers and caregivers have sufficient smartphone, data, and network access to use the platform consistently Platform usage (uploads and feedback rounds); number of teachers trained and coached; number of learners assessed Ongoing coaching and monitoring continue alongside the platform, so that new practices are sustained and don’t fading after training Change in observed teaching practice; change in learner numeracy assessment scores, baseline to endline Caregivers can be reached and engaged to reinforce learning at home Government approval and uptake to extend the program; sustained use and coaching beyond the pilot period 01 01 02 02 03 03