Ednovate Pakistan: A Dialogue Fostering Evidence Driven Innovation in Education

Tuesday, July 11, 2017

LUMS School of Education (SOE) and Center for Economic Research Pakistan (CERP) collaborated to host Ednovate Pakistan: A Dialogue Fostering Evidence Driven Innovation in Education at the Lahore University of Management Sciences (LUMS) campus on July 11, 2017. The objective of the event was to engage in intersectional dialogue with key leaders and decision makers on the value of evidence-driven decision making and an innovation-based ecosystem for the public, private and non-profit spheres in the education sector. The goal of the engagement was to create a diverse coalition of partners including policy makers, implementers, researchers and donors, who will champion education research and innovation in Pakistan.
This was a by-invitation event, interactive in its nature, and invited participants who are looking for opportunities to put forward their ideas. The event consisted of a series of one-on-one conversations, panel discussions and question answers sessions with the audience.


Introduction


The opening remarks were delivered by Dr. Sohail Naqvi, Vice Chancellor of LUMS. He highlighted that LUMS School of Education is looking to bring a new perspective into the conversation on education and this conference will play a crucial role in this effort. He highlighted that in the institution’s 30 year history, LUMS has introduced several initiatives in innovation. The LUMS Center for Innovation is one such initiative, highlighting that innovation has great importance to LUMS.
Dr. Tahir Andrabi demonstrated the need for innovation in education by sharing some of his findings from his research projects. As part of the LEAPS project, a study across 120 villages, it was found that there are several low-cost private schools in these villages. This led to an examination of the decision making process used by an illiterate mother to choose a school for her children. There is room for substantial experimentation aimed at understanding the existing decision making process and identifying ways to make it more rational and structured.

 

 

 

 

 

 

 

 

 

 

 

Another project Dr. Andrabi worked on was to provide schools with loan products that specifically addressed the concerns of limited financial resources available to schools, particularly low cost private schools. However, these schools were completely disconnected from the education innovation market and so could not spend this money effectively to bring innovation to their classrooms. This points towards the wide space that exists for researchers to operate in the education landscape.
Dr. Andrabi stressed that it is not just researchers but also implementers who play an essential role in the education system. The findings that research bring forward need to be implemented which requires a completely different and specialized skill set. These differences in skill sets and requirements make collaboration even more important. The LUMS School of Education is ideally placed to fill this gap.
Over the past couple of years, education in Pakistan has come a long way. Enrollment for girls has increased at a higher rate than for boys. Every elected government has included education and enrollment in its agenda in an attempt to win elections. Therefore this could be a golden period for education in Pakistan, presenting researchers and policy makers with an opportunity to create impact.
The goal of this conference was to set the narrative by bringing together key leaders and decision makers from the education sphere so that they can find and engage with partners to explore innovation in education.  


Financial Innovations in Education
A Conversation with Nadeem Hussain, Founder of Planet N Group of Companies. Moderator: Prof. Asim Khwaja, Harvard University  

In this session findings of a joint project led by Dr. Andrabi and Dr. Khwaja with Tameer Bank were discussed. Financial institutions typically do not cater to the needs of small and medium schools. Schools in rural and urban areas are struggling due to financial scarcity. The reason for non-availability of a compatible loan product for schools is that the banks do not innovate and test new products. Commercial banks do not have an incentive to innovate since small and medium enterprises (SME) default rates are close to 30 percent. Therefore banks do not see commercial value in catering to the SME market segment. Over the years there has been a slow change in this approach, Tameer Bank being the first institution to test five different loan products. The sharia compliant financial models were designed by Dr. Andrabi and Dr. Khwaja. The models provided school owners with multiple options ranging from static financing, revenue share and risk based return amongst others. Most schools opted for the risk based return model and shared their profits in good months.

The idea behind giving out loans was not to limit the creativity or the ways in which the amount could be spent. School owners could use this money for whatever purpose or requirement they felt was needed by their school.  School owners want to make profits so they would naturally invest the money in the most optimal way.

 

 

 

 

 

 

 

 

 

 

The loan take up rate was close to 25 percent. This points towards a high number of owners willing to take up loans indicating a genuine demand for these products. The success of Tameer Bank’s model has encouraged bigger commercial banks to enter the market as well. Bank Alfalah has also realized the great potential that exists in this market, initiating the disbursal of loans to 150 schools. More recently this figure stands at more than 300. The reason for slow uptake and entry into this new market is due to the system commercial banks operate under. Commercial banks have to rigorously stress test their products, check profitability and sustainability of that market. This process made the implementation of this initiative slow.

The audience raised some concern about accountability in terms of service provided by the school after the uptake of the loan. The panel was of the opinion that this is something which concerns the parents and they can be the best judge of service. The schools make monthly repayments to the lending institution. Failing to do so results in the lending institution becoming much more involved in the running of that school creating greater accountability.


Bringing Affordable Quality Curriculum to Schools 
A Conversation with Ameena Saiyid, Managing Director, Oxford University Press. Moderator: Dr. Mariam Chughtai, Associate Director, LUMS School of Education

The session began with a discussion on the market entry approach taken by Oxford University Press (OUP) in Pakistan.  The initial approach was to engage people, particularly school owners and parents through melas. OUP would setup a mela and publish an advertisement that would cover multiple towns in Pakistan. The goal was to cover almost all of Pakistan using this approach and raise awareness of the value proposition on offer. To encourage sales, books and other materials were offered at a discount. At a later stage, these mela points and book fairs were converted into permanent distribution centers. OUP is currently operating 28 financially sustainable bookshops. What started as a marketing effort has now turned into successful sustainable operations.

The second phase of the approach was to send OUP representatives on school visits. By using face-to-face interaction, school owners were made to see the benefits of using OUP books and teacher training material. These visits provided an opportunity to build a relationship with these schools and answer concerns directly. They also provided an opportunity to distribute free promotional material.  The schools which adopted OUP material experienced higher enrollment which encouraged other schools to also use OUP material.

Another secret to OUP’s successful market entry was utilization of local support and collaborations. Local players such as CERP and others have helped OUP in developing books which were relevant to the area’s context and culture. Today there is not a single private school which is not using at least one OUP book, said the speaker. This is mainly because the material meets their requirements. It is through this strategy of localization of content and understanding the needs of schools that OUP found so much success in Pakistan.

It has not all been smooth sailing as OUP faces many challenges in maintaining its success. Piracy of books is a major problem. There were two issues with piracy, some schools sold the books which they had received for free while others made pirated copies of OUP books. The latter had a significant impact on OUP revenues. Local authorities were ineffective so OUP had to devise a different strategy. The idea was to win the hearts and minds of people which was done by improving the printing quality of books so that there was a marked difference in colors as compared to the pirated version. Children are attached to better colors and so preferred OUP books. Artists such as Taimur Rehman, Farooq Faisal and Shahzad Roy were also engaged to conduct concerts and puppet shows as part of anti-piracy campaign

Another major problem was resistance and threats from the government sector. OUP successfully worked with only one government school in training their teaching staff. This engagement was followed by veiled threats from stakeholders in the government sector who wanted OUP to remain in the private sector. This was mainly because existing publishers know government contracts are highly lucrative and do not want any other publisher to enter the market. Others have faced similar resistance from government sector.

There are many other areas for OUP to focus on in order to grow further. Currently, the focus is on self-learning and blended learning. The process of generating online content for schools has already started. The content includes animations and exercises that will enrich the learning experience of children. To make the content more appealing actors, singers and academics have been approached to recite and record books. Zia Mohyeddin and Pervez Hoodbhoy are some of the people OUP is working closely with.

Moving forward, OUP believes blended learning is the future and has already started working with two tech start-ups. There is a long way to go and many challenges to overcome, making the seeding of linkages and partnerships with stakeholders from across the education landscape imperative.


Evidence-driven Education Management
A Conversation with Kasim Kasuri, CEO, Beaconhouse School System and Prof. Salima Hashmi. Moderator: Prof. Tahir Andrabi, Professor at Pomona College and Dean LUMS School of Education

The conversation started with emphasis on the central role of teachers in the education ecosystem, highlighting that there is a lot of variation in types and backgrounds of teachers, which impacts their role. When it comes to low cost private schools, teachers are generally young and not career oriented. In the public sector, teachers and their roles are defined by their
seniority instead of capabilities. To be able to answer questions relevant to education management, a school system that employs better quality of teachers must be examined. Beaconhouse School System (BSS) provides an ideal platform to conduct such studies.

Kasim Kasuri briefed the audience on some of the steps BSS has taken to develop its management processes and how technology was used to digitize its systems. The need to initiate change was felt when BSS faced difficulties in school evaluation since the evaluation mechanism of BSS was very subjective. The outcome of this evaluation exercise was completely inaccurate. Thus it was decided to engage an external entity to compare learning outcomes amongst schools. For this purpose, the School Evaluation Unit (SEU) was established to evaluate every BSS school. In this system, school evaluations are based on evaluations of teachers. Teachers are evaluated on the basis of classroom observations, student results and parent feedback. In view of these changes BSS has now moved away from a prescriptive curriculum to an open ended curriculum. The SEU has been operational for six years and every school is evaluated after a three year cycle. If the school performance evaluation is below par, the SEU management together with senior school management plan and implement improvements. For these schools, the evaluations become a yearly procedure until results improve.

Another initiative is focused on better understanding customers. For that, BSS has introduced Beaconnect. The data from this platform has provided insight into parents’ perception of their child’s school. Under Beaconnect, input from parents is sought through text message by asking them questions against 22 identified indicators. The indicators include a mix of learning and management related questions. This feedback is rich in information and it should help BSS improve further.

The two systems, Beaconnect and SEU provide support to one another. The information from the two platforms can be cross checked. It is also possible to see why gaps between the two scientific evaluations exist and this information can be used to improve them both. The role of Beaconnect is set to grow even further as school principals will soon be evaluated using this platform. This change will enable BSS to improve schools at the management level.

With so much data, possibilities are almost endless and this is where BSS has engaged CERP. The goal is to identify the best performance management systems. This involves two types of analysis. The first is to use existing data to answer questions such as what are the potential factors effecting a teacher’s decision to leave the organization. About 50 percent of teachers leave within the first 2 years. Are these better teachers or worse teachers? The second is a more difficult and time consuming component which is to pilot and conduct experiments. The goal here is to track student performance on a student-to-student basis. This will provide an insight into whether there is a causal relationship between student performance and type of teacher. The approach of BSS has now changed from being subjective to data driven. The hope is to use evidence based decision making as much as possible. The limitation here is that education is delivered through individuals who are different from one another. Therefore, it is essential to work on the professional development of these teachers while also improving systems at an organizational level.

During the session Professor Salima Hashmi underscored the importance of the arts in the learning process. The sad reality is visual learning is not given the importance that it deserves. In Pakistan, this concept has been reduced by teachers and students to simply drawing cartoons, Mickey Mouse in most cases, in classrooms.  A large number of art teachers do not know the basics of being an art educator. Art teachers need to integrate art into all streams of education without any limitations.

When art is integrated with education, children learn better as children are able to visualize and understand better. Art educators can teach ethical values through empathy, leading children to observe and reflect. This is an interdisciplinary approach which leads to of diversity, dialogue, and thinking through the material. This makes the overall learning experience richer.

Prof. Hashmi shared two anecdotes from her teaching experience to demonstrate the importance of the arts in education. A five-year old boy who was a poor performer created havoc for two months in her class. He would paint an apple with black paint every time, until one day, the child dipped his brush in the red to paint a red apple. When asked why he decided to use the red paint, he replied that his father finally returned after leaving his mother. The second anecdote was about a girl from the West Indies who was thought to be academically challenged and had a speech impairment. Prof. Hashmi found that the girl slowly started to understand and respond to what was being taught. Soon she started reading which proved she was not academically challenged and all she needed was time to be spent with her. 

Prof. Hashmi concluded the session by emphasizing that there has been a lot of progress in arts and education in Pakistan. This progress is a result of gathering evidence from the classroom and using it to improve the learning experience for children.

Catalyzing Innovation in the Public Sector 
A discussion with education experts working in and with the public sector. Moderator: Dr. Ali Cheema, Institute of Development and Economic Alternatives 
Panelist: Tahir Rauf, PMIU, Seema Aziz, Care Foundation, Helen Kamal, Ilm ideas II

The discussion was centered on identifying areas where the system has captured innovation and identify opportunities for further innovation.

There was a general agreement that considerable innovation has taken place in education over the past few years. PMIU was established primarily to introduce and implement new initiatives in innovation. The broad range of initiatives include developing a school performance index to benchmark school performance. The index enables comparison across schools, which has assisted in better performance management. The provincial monitoring data has been made available to a greater audience, particularly donor agencies. These organizations cross check PMIU data with other sources which has enabled PMIU to further improve its databases. Recently, a mapping exercise was conducted to identify pockets where schools need to be established for out-of-school children. Another recent initiative undertaken was a survey on teacher motivation. This was conducted after it was discovered that increasing salary does not increase teacher productivity. This exercise provided insight on teacher psyche and the position they hold in school are important factors effecting teacher motivation. 

Care Foundation is running a large system of 900 schools with free education for 24 million students. In such a large system, introducing innovation to improve quality is difficult. Therefore, innovation can be made in the provision of water, electricity, and basic facilities to classrooms and schools. Another recent innovative initiative is the unprecedented handover of 5000 of the worst schools to the private sector. Simple innovations can also come a long way – it was Care Foundation which initiated double shift schools. When Care Foundation took over government schools under Public School Support Program (PSSP), government staff was retained and additional Care Foundation staff was recruited. Even with a marked difference in salaries the schools are performing efficiently. This is because Care Foundation brought about innovation in school management practices, eliminating the hierarchical structure and introducing a shared vision. This has motivated school staff to commit to the betterment of the schools.

The way forward for innovation in education should be to increase household incomes. A child spends more time at home, with an increase in household income there might be a much bigger impact on a child’s learning. Broadening the definition of innovation in the education space through the inclusion of the home environment should have a positive impact.

The panel dialog also highlighted some innovative initiatives which are drivers of improvement in learning outcomes. There is evidence that child nourishment comes a long way, particularly when dealing with rural areas where access to health facilities is limited. If children in the classroom are healthy, then learning would also improve considerably. In Punjab, the provision of the Non-Salaried Budget provides a great opportunity for schools to innovate at an individual level. This is a considerable amount of money that can be effectively used for innovation. About 40% of this budget is flexible and exercises are being conducted to simplify the spending process. 

There was general agreement that government policies which lead to centralization are disempowering teachers by discouraging their creativity and ultimately adversely affecting innovation. Government officials acknowledged that centralization and political hindrance create obstacles. Decentralization is a very difficult and time consuming process given the scale of public education.

However, there was general agreement that innovation can still exist at all levels. For this to happen, the concept of innovation needs to be introduced to teachers as well so that they actively participate in this process. This can have a significant impact since there are thousands of teachers working in the public sector.  If these teachers are given some assessable targets to find solutions, then the impact at the aggregate level could be huge. This can start a systematic process of success bringing success. When a teacher uses an innovative product well, the results will spread fast for others to follow.

Panel 2: Reconciling Education Narratives and Policies with Evidence 
A discussion with researchers, users of data and media representatives. Moderator: Dr. Faisal Bari, Institute of Development and Economic Alternatives.
Panel: Baela Raza Jamil, Idara Taleem-o-Aagahi, Ali Khizar, Ahmar Mahboob, the University of Sydney

The panel discussed the difference in the usage of data between research organizations and government departments. From the academic stand point, researchers are under great pressure to use more data. However, politics around data does not encourage its use. Government departments in particular distrust data. The public education system is vast so reliance on sample based data is most feasible. The government questions sample based results when they do not show results in the government’s favor, and want a holistic approach instead. To avoid these situations, researchers use selective data and give more room to narratives than is required. Researchers have to find a balance between data and narrative to meet government and academic requirements. These constraints drive them further away from getting to the right answer.

Finding a solution to low learning outcomes remains the most complicated issue. There are too many open questions. Should there be prescriptive learning? What motivates and compels a teacher to share ideas of knowledge with students? Can the systems absorb technological innovations? There is wide space for researchers to engage and conduct meaningful work.

When it comes to technology in education, currently one third of Pakistan’s population has digital access. The entire population will soon have access to education through technology as equipment and networks are becoming cheaper. Therefore emphasis should be on digitizing data collection mechanisms in schools. This will allow for benchmarking and tracking of students and schools. Better technology could also lower the level of distrust of data particularly in the public sector.

As long as data collection mechanisms are not digitized, researchers will continue to rely on government data. Due to the limitations of government data, researchers need to qualify all aspects of the data. The challenge to make improvements to government databases is that the government omits any form of criticism of its systems. The reason for the government’s attitude is misuse and misinterpretations of government data by the media. This has made the government very apprehensive to conduct social surveys to cross check or share its data. This attitude is slowly changing as the government has begun to engage in far more conversations than before. Hopefully, these conversations will get stronger. Pakistan has miles to go in order to improve the education landscape. It is essential to ensure alliances of public and private sector grow stronger.

 

What Comes Next?
Dr. Jishnu Das, World Bank

In this session, Dr. Jishnu Das shared his observations after 15 years of close involvement with Pakistan’s education sector. He believes the system in Pakistan has produced a large number of college graduates. This has been driven by focusing on making education available for all rather than a system where the system produces the most educated. The need is to think of ways in which interventions can be used to “divert the river” towards quality education.

Three changes are required to successfully divert the river. Firstly, a big discussion is required on data and its legitimacy by thinking about the level of legitimacy being given to data and then create systems for data collection accordingly. With data, there can be extreme inequality where averages would not represent anyone. Therefore, it’s essential to develop systems which provide data at a compatible level of accuracy. Secondly, stereotypes are self-fulfilling prophecies and should be avoided. Relationships based on trust are essential. For example, there are groups which claim that the government is corrupt or part of the problem, similarly low cost private schools hold parents responsible for their problems. By blaming and stereotyping groups, there is a greater inclination of them playing into those roles.  Finally, it is essential to discuss potential solutions with those who run the system. Their input would lead to greater ownership and a higher chance of success in implementing solutions.

Getting to a workable solution is not easy. There are unintended consequences even when solutions might seem simple. For example, it was decided that the 2017 census will potentially solve many problems with outdated data. However, 5th grade students in government schools suffered the consequences of this decision as 5th grade teachers were selected to conduct the census. 5th grade students only received four months of schooling in the entire year. To solve such problems, an authoritative government with a small group of people in charge with a lot of power can work. However, these people are often outsiders, with little understanding of the problem itself. On the other hand, if the model is changed to the government being run similar to a business or private entity, the risk factor increases substantially as most businesses usually fail.

Dr. Das suggested that it is through interventions that this system can improve. The statement that Pakistan is an education disaster is incorrect since the problems in education systems are common all over the world. The work being done in Pakistan has been published in international journals because these problems are faced by countries across the world.

Dr. Das concluded the event by emphasizing that working groups need to come together and deliberate what the next steps to take within the education ecosystem in Pakistan are. Ednovate can serve as an annual platform for stakeholders in public, private, non-profit and donor organizations to come together and deliberate the agenda of reform in the education space.