School closures and the suspension of in-person instruction in the wake of the pandemic have widely affected learning levels amongst students. It has also amplified the need to improve learning outcomes in large public education systems, backed by data that monitors the current system and provides targeted strategies to address learning gaps.
How does a public education system start to build back from here? A good starting point is establishing a rigorous baseline of how students are doing today. In early 2022, the state of Himachal Pradesh (HP) commissioned a large-scale third-party assessment to measure student learning outcomes. The assessment was designed and implemented by Convegenius Insights (CGI). The results complement existing data sets including the National Achievement Survey (NAS) and end-of-year Summative Assessments conducted at the school level. What sets the CGI assessment apart is the extensive use of technology, where tablets were used to test 20,000+ students completing grades three, five, and eight over a four-week period.
Three key insights stand out from this experience for educators and system leaders.
Faster and more reliable testing via technology
In what is perhaps a first for India, a state-wide assessment was conducted using tablets preloaded with testing software (vs. a pen and paper test). This is impressive given the difficult topography of the region (HP is a mountainous state) and many hard-to-reach schools. Extensive operations planning was undertaken by CGI to deliver the project including (1) procuring a large pool of tablets; (2) developing a hub and spoke model for charging tablets at district headquarters (hubs) and; (3) training 150+ assessors who transported tablets from hubs to spokes (schools) for actual testing.
Tech-enabled assessments are considered more reliable as there are fewer opportunities for students and assessors to tamper with data. They are also faster and could generate insights in just four to five weeks as compared to 10-12 weeks for paper-based assessments. After a quick orientation for both teachers and students, we received positive feedback on the digital interface from all stakeholders.
Data as a key input to academic planning
The intent of large-scale assessments is to provide a ‘system-level’ view of learning and identifying trends through historical or geographical comparisons. Such assessments are typically a sample and not census-based, undertaken by independent agencies and representatives at a district/ block level. They are not an indicator of individual student performance and provide reliable aggregated data instead. Read more about our work on large-scale assessments.
Informed by the results of this assessment, the department of school education in HP has put in place several interventions under their broader “Samarth” program. (Samarth is a systemic education transformation program that the Foundation supports in partnership with Samagra Governance.)
These interventions include an intensive catch-up program in the first few weeks (Shiksha Setu), workbooks for each student to practice what they are learning, and a repository of digital content tagged by learning areas. A follow-up assessment to establish their effectiveness is also in the pipeline.
The way ahead – meaningful analysis and visualization
Initial results of the third-party assessment indicate that more than 50% of students need support to catch up with grade-appropriate learning levels. Moreover, math performance was found to be worse than language, and students in grade eight appeared to be further behind than those in grades three and five (likely due to lower curricular load in early grades and a well-resourced primary school system in the state). The assessment dashboard also provides data at a district, block, and school level for more granular decision-making.
This year, the state of HP will integrate all its education data into a single visualization tool. This includes information on enrolment, teacher allocation, attendance, and learning. A data-driven approach has also been advocated by the World Bank, which recommends that all states set up a Command and Control Centre (CCC) for Schools.
This is an important step by HP to ensure teachers have what they need to help students catch up and bridge losses in learning levels. We hope these learnings can be replicated by other states to implement similar data-driven solutions and make better decisions. This will ensure that more students – ALL students – thrive in schools