ClassHero Case Study

Introduction

Over the last decade K12 institutions in the United States have spent millions of dollars on EdTech devices for their classrooms. The CoSN’s (the Consortium of School Networks) 2018-19 Annual Infrastructure Report found that last year 75% of the district’s surveyed have at least one device per student. The report also found that within 3 years 96% of the surveyed districts plan to be a 1:1 district. As a result, the EdTech industry has developed an ever-expanding list of educational software apps to be used on these devices. However given the relatively rapid and recent emergence of these tools in US classrooms, school districts have struggled to demonstrate how their EdTech investments in devices and software apps have impacted student achievement. 1 In fact, during this same decade of technology in the classroom expansion, Math achievement has remained flat for both fourth and eighth grade according to the NAEP Report Card. 2 In addition to flat NAEP scores, performance gaps by race, ethnicity, family income remain wide with the achievement gap being greatest for English Language Learners. 3 Not surprisingly, these achievement gaps for ELLs and other historically underserved populations persist and expand at the secondary school level resulting lower graduation and college matriculation rates. 4, 5

Download PDF Version

Albuquerque Public Schools is a school district based in Albuquerque, New Mexico.

APS is the largest of 89 public school districts in the state of New Mexico with some 95,000 students, making it one of the largest school districts in the United States.

The Promise of ClassroomTechnology

Independent peer reviewed research has indicated that using Math apps with EdTech devices in classrooms can accelerate learning and improve math achievement results if used frequently, and in alignment with the school and districts core curriculum.6

A 2018 study conducted by Faulder, Gulliford, Pitchford, and Outhwaite found that integrating a math app as part of a child-centered math program significantly  accelerated math learning.7 In the  study, students who used a math app as a part of their math instruction were 3-4 months ahead of their peers who did not use the math app as a part of their math instruction.8 The research by Faulder, Gulliford, Pitchford, and Outhwaite supports earlier research by the What Works Clearinghouse (WWC)  that indicated quick retrieval of basic arithmetic facts is critical for success in mathematics. 

Yet their research also  found that many students with difficulties in mathematics are not fluent in such facts.9 For that reason, the WWC recommends approximately 10 minutes of strategic skills practice at a students rate and level be incorporated into daily math instruction. Such strategic skill practice can improve mathematical ability and ultimately prevent subsequent failure.10

Background

Roadblocks to Realizing the Promise of Classroom Technology

Moving from research to implementation in the field of EdTech has its challenges. Ensuring that teachers can and do use EdTech tools as prescribed as well as ensuring that historically underserved students can access and engage in the content are the two primary roadblocks to realizing the promise of EdTech. Variation in App Usage is primarily determined by a teachers ability to integrate the app within their instructional program.11 Teachers must make instructional decisions to not only organize their classroom to use a math app based on resource constraints but also maintain pedagogical techniques they and/or school district leadership believe are effective.12 Many apps in today’s market require students to be on the computer for upwards of 90 minutes a day which makes implementation fidelity difficult if not impossible. Secondly, math skill practice provided by text book companies traditionally consists of worksheets full of math problem sets that all students are supposed to do with little differentiation or support for ELL students.

Teachers must make instructional decisions to not only organize their classroom to use a math app based on resource constraints but also maintain pedagogical techniques they and/or school district leadership believe are effective.

Equally problematic are online/computerized skill development software programs that differentiates practice but do not align to the district’s core curriculum or to the essential prerequisite blocks of that curriculum.13 As a result, students who struggle or do not enjoy solving math problems find skill practice tedious, boring or frustrating. This type of learning experience will not produce students who like math, feel confident in math or proficient in math especially for historically underserved students.

ClassHero: From Promise to Practice

ClassHero is a research based app that has been designed alongside real teachers in real classrooms to address the issues of EdTech implementation and engagement including for historically underserved populations. ClassHero addresses these issues by providing ALL students with differentiated, personalized, comprehensible evidence-based learning activities that are aligned to the district’s core curriculum and the classroom teachers daily learning goals/targets. With ClassHero, teachers are able to assign skills for daily practice via a web-based teacher interface (TI). Targeted skills practice can be assigned to either the entire class or personalized to individual students. Skills may be assigned either in alignment to a teacher’s daily curriculum, remedial intervention or extended learning for students who have already achieved mastery. ClassHero has three key design features described below:

1. Seamless District RTI and

ClassHero seamlessly integrates district RTI universal screening and progress monitoring data alongside teacher understanding and knowledge of their students to provide all students with differentiated and personalized evidence-based skills practice  that is aligned to Core Curriculum and informed by RTI assessment data. ClassHero takes  advantage of available district performance data – including, but not limited to student interim/benchmark assessment data, progress monitoring data, common assessment data, language proficiency data as well as prior skill practice data to help differentiate assignments to individual student needs.

2. Progress Monitoring Data

Providing teachers with progress monitoring data daily updates and summaries of the daily learning of all students allowing the teacher to plan instruction for the next day and monitor student progress in reaching individual learning goals. Additionally, ClassHero provides school level leadership weekly implementation data regarding teacher and student usage so administrators and instructional coaches know which teachers have been able to integrate strategic skills practice into their instructional program and which teachers need additional support.

3. Scaffolded Content and Language Supports:

ClassHero provides  students the necessary content and language scaffolded support so all students can access the curriculum. ClassHero includes built-in standards-aligned video instructional support for math topics and skills being covered during targeted skills practice. Video support may be assigned by teachers to students either as a form or remediation or content front-loading prior to a skill practice session.  ClassHero provides videos in English and Spanish and can be automatically assigned, as default based on english language proficiency level or may be configured by teacher to individual student needs. Additionally, ClassHero provides voice narration supports for student skill practice in multiple languages, including but not limited to: Spanish, Arabic, Chinese, Croatian, Czech, English, French, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Korean, Malay, Norwegian, Polish, Portuguese, Russian, Somali, Swahili, Slovak, Slovenian, Swedish, Tagalog, Tamil, Telugu, Thai, Turkish, Ukrainian and Vietnamese.

Purpose of Study

In October of 2019, Albuquerque Public Schools began working with an EdTech Company, to move research into action and implement a research based EdTech tool called ClassHero at Alamosa Elementary School. Based on the 2019-20 New Mexico Department of Education Report Card,  99% of students at Alamosa receive free or reduced lunch services and 96% of the students are students of color. 9% of the students receive English Language services and 19% receive Special Education services. The purpose of this case study is to examine if the implementation of ClassHero at a school with a high percentage of historically underserved students can enhance and improve the teacher’s ability to provide differentiated skill practice that is aligned to the schools core curriculum with the goal of accelerating the rate of math learning and improve student achievement.

Alamosa Elementary School

ClassHero Implementation in the Albuquerque School District at Alamosa Elementary School

Methodology

The methodology strategy of this case study was to use practical, cost effective and easy to replicate evaluation tools that are readily available. Accordingly,  we summarized student usage data by Common Core State Standard Domains that are aligned to the districts nationally recognized interim assessment tool. ClassHero recommends that students use ClassHero for a minimum of 3-5 days a week for 10 minutes a day with a minimum accuracy rate of 65%. Additionally, what skill practice students are engaged in should be aligned to the core instruction and assigned by the classroom teacher. 

As a result, we evaluated the  duration, frequency, accuracy and fluency of the teacher assigned skill practice for each student who attended Alamosa Elementary School  for the entire first semester of the 2019-20 school year. We then classified each student as 1) Not Used ClassHero, 2) Used ClassHero, or 3) Used ClassHero as prescribed above. Additionally, we collected the relative growth score compared to the expected growth on the district’s interim math assessment tool. We finally compared the average growth of these three groups of ClassHero usage and evaluated the differences in their average growth.

Summary of Key Findings

Teachers at Alamosa Elementary School did not begin using ClassHero as part of their instructional program until 55 days into the first term. This delayed implementation resulted in ClassHero only being used for 39% of the evaluation period. Even with the implementation delay the results of this case study provide great promise for the use of EdTech to accelerate math achievement for students and especially for students from historically underserved communities.

App Usage

In recent studies EdTech learning apps have had low usage rates and even lower usage rates at the recommended levels. One study found that only 2% of apps are used at the recommended level and 30% of apps are not used at all. During the implementation period most teachers (75%) were able to integrate ClassHero into their instructional program within a couple weeks From interviews with the schools administration and faculty, the staff felt that ClassHero’s easy to use teacher and student interfaces as well as the 10 min. engagement expectation allowed them to quickly incorporate ClassHero into their normal Classroom routine. This quick integration likely led to the high usage rate displayed in Figure 2. As a result,  during the evaluation period, 95% of students at Alamosa Elementary School used ClassHero and 36% of students used it at the recommended level. 

Student Achievement

During the seven week evaluation period running from the end of October 2019  through December of 2019 the Alamosa staff primarily focused their instruction in the Common Core State Standard Domains of Numbers and Operations. 69% of all ClassHero practice sessions were in the Number and Operation Domains which include Base Ten, Fractions as well as Counting and Cardinality. The remaining practice sections were spread amongst the domains of Algebraic Thinking (20%) and Geometry (6%). Since the focus instruction during the evaluation period was Number and Operations we analyzed the domain growth using the district’s interim assessment for that domain.  Our analysis showed statistically significant acceleration of learning for students who used ClassHero as prescribed versus students who did not use it at all in the domain’s . Even students who used ClassHero but not as prescribed showed accelerated learning rates. See Figure 3.

The use of ClassHero for Strategic Skill Practice as prescribed in the Number and Operations domains of Counting Cardinality, Base Ten and Fractions resulted in more than 3.6 weeks of additional learning for students who used ClassHero and more that 11.2 weeks of additional learning for students who used ClassHero as prescribed.

Conclusions

The summary of results is extremely encouraging and provides evidence that confirms the research that when EdTech tools that are designed smartly and with the user experience in mind can accelerate student learning with historically underserved students.

Works Cited

  1. The 2018 Technology & Learning Insights Report: Towards Understanding App Effectiveness and Cost.
  2. National Center for Education Statistics, “The Nation’s Report Card: Mathematics,” National Assessment of Educational Progress, 2017, nationsreportcard.gov/math_2017/#?grade=4.
  3. Michael Hansen et al., The 2018 Brown Center Report on American Education: How Well are American Students Learning? The Brookings Institution, June 2018, brookings.edu/research/2018-brown-center-report-on-american-education-trends-in-naepmath-reading-and-civics-scores/.
  4. Catherine Gewertz, “Math Scores Slide to a 20-Year Low on ACT,” Education Week, October 17, 2018, edweek.org/ew/articles/2018/10/17/math-scores-slide-to-a-20-year-low.html?r=1611432497&mkey=8026D2C4-0AD1-11E9-A985-EA9FC819EBCD.
  5. Nick Anderson, “College Admission Test Scores Raise Warning Signs about Math Achievement,” The Washington Post, October 25, 2018, washingtonpost.com/local/education/college-admission-test-scores-raise-warning-signs-about-math-achievement/ 2018/10/24/ab37ba0a-d7a5-11e8-83a2-d1c3da28d6b6_story.html?utm_term=.51a16adcad10.
  6. Integrating Computer-Based Curricula in the Classroom: Lessons from a Blended Learning Intervention by J. Cameron Anglum, Laura M. Desimone & Kirsten Lee Hill – 2020.
  1. J Educ Psychol. 2019 Feb; 111(2): 284–298; PMCID: PMC6366442; Published online 2018 Jun 25. doi: 10.1037/edu0000286; PMID: 30774149;; Laura A. Outhwaite,1 Marc Faulder,2 Anthea Gulliford,3 and Nicola J. Pitchford3,*
  1. Ibid.
  2. “WWC: Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools.” WWC | Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools, ies.ed.gov/ncee/wwc/practiceguide/2.
  3. Ibid.
  4. “DreamBox Learning Achievement Growth.” Center for Education Policy Research at Harvard University, 6 June 2017, cepr.harvard.edu/dreambox-learning-achievement-growth.
  5. Ibid.
  6. Ibid.