The Korean International School (KIS) in Ho Chi Minh CIty is the largest South Korean international school in the world. It follows the standardized curriculum set out by the South Korean government and serves students from kindergarten to high school.  KIS has a student population of 863 students, 530 of which attend the elementary school. Almost all of our students are of Korean descent with approximately 10% of students having been born in Vietnam. Many of our students speak other languages in addition to the Korean, English and Vietnamese they are required to speak at school. The  staff are mainly composed of staff from mainland South Korea. In addition, there are 17 native English speaking teachers who co-teach grades 3 - 6 and 11 Vietnamese teachers who co-teach kindergarten and grades 1 and 2. There is a high staff turnover simply due to the nature of these types of school. Contracts are usually short term and the staff are transient either moving back to their home country or moving onto the next one. It is unusual for some to break their employment contract and abstenteeism is low. 

​

KIS's academic calendar follows the Korean school year. The first semester starts in February (shortly after Chinese New Year) and ends in July. The students and faculty enjoy 6-7 weeks of holiday before the second semester starts at the end of August and lasts until the beginning of January. With another 6-7 weeks off. Ok, that last part was just to brag. 

​

Our elementary program is unique in the sense that homeroom classes are shared by a Korean and a native English speaking teacher. All subjects are taught in both Korean and English, though not simultaneously of course. A lot of collaboration and communication is involved to make sure pacing for both teachers is concurrent and that curriculum guidelines are met. English language acquisition is highly valued in Korea and this transfers over into the Korean international schools worldwide. In addition to providing a bilingual environment in which expat students can learn, after school, summer and winter programs are also offered in which the students can be immersed in English. These classes are not free. Parents pay a fee to the school and teachers are paid for their time. 

 

Each semester (or holiday break) students can choose from a wide variety of activities to suit their needs and interests. The classes are not mandatory and depend on participation by teachers, students, administration and parents. Teachers can offer a  class in anything they choose but they usually try to pick something that is going to appeal to both the student and the parents. Classes are only run once a certain number of students have signed up therefore it is in a teacher's interest to design an appealing class in order to earn extra income. This adds an interesting dimension to the  array and choice of classes offered. These classes are not free and parents pay extra for their children to attend. It's a bit of a balancing act to keep both parties happy. The school principals give autonomy to the teachers to create their own classes but request that they be of some educational or practical value.

​

​

 

 

 

 

 

 

 

 

 

 

 

 

 

Although KIS provides an abundance of special interest classes taught in English all year round, I will be focusing my attention to the After School Program offered to elementary level students for 18 weeks each semester.

​

The after school program has been running since the school opened in 1998. Administrative duties, kinds of classes offered and the number of teachers who sign up has varied over time due to staff turnover and relocating factors. Some classes (particularly sports centered ones) have remained popular over time, while other classes have wavered with interest depending on the subject or teacher. Harry Potter English clubs were wildly popular a few years ago but not anymore and some teachers naturally command a large number of sign ups because of their popularity with students. Though the program itself has remained consisted over time, it's elements fluctuate constantly in terms of staff, options and student enrollment.  

​

It works like this: 

​

• At the beginning of each semester, teachers decide what kind of class they would like to teach in the after school program. This can range from academic lessons, sports, arts, music, games, drama, etc. 

• Classes are taught 4 days a week (Mon - Thurs) and last 40 minutes. 

• The school day ends at 2:50 so there are 2 periods in which students can sign up for classes (3:05-3:45 and 3:50-4:30). The students can then share the school buses with high school students when they leave the campus at 4:30. 

• The teachers advertise their classes online through the school website and sign up sheets are distributed to the students to bring home.

• Classes are only delivered if a minimum of 8 students sign up.

• The current cost per (40 min) class is $80usd per semester. 

• Teachers earn $35usd per hour for every after school program class they teach.  

​

The aims of the after school program are to help students demonstrate confidence in their English skills, foster a good rapport between students and teachers, and to help cultivate an immersive English speaking environment in the school.     

​

 

2. Identify purpose for evaluation and specify evaluation questions

​

 

 

 

 

 

 

 

 

 

 

With a drop in enrollment this year, now is a good time to evaluate if the after school program is effectively meeting its goals and what can be done to improve it. This would be evidenced through student enrollment and levels of satisfaction. The primary stakeholder are the school, the parents, students and teachers. Each have a different vested interest in what the after school program offers and each party must be satisfied in order for the conditions for effectuating the after school program to be met. The purpose of an evaluation would be to determine the effectiveness of the after school program and to provide practical recommendations for its improvement. 

​

Evidence that the program is succeeding could be demonstrated by asking:

 

• Have enrollment rates remain consistent since its implementation?

• Have students' confidence and abilities in English increased? 

• Is overall satisfaction about the program being reported by stakeholders?

​

 

 

 

 

 

 

 

 

 

 

 

 

 

This evidence will be collected through data and input from stakeholders. Quantitative data can be compared to show how enrollment rates have changed over time. To investigate further, the types of classes offered in the past and at present, should be considered carefully. This would be so that themes can be identify regarding most popular classes (are they sports, computer classes, art?). Interest in the topic or content of the program may be an important factor to consider regarding enrollment. 

 

The program topics or content may have little effect however if there are other factors at play. Has a new language center opened up nearby, are private tutors lowering their fees, are there after school care and transportation issues to be considered? Qualitative data in the form of surveys, interviews, observations and perceptions  may provide a deeper understanding and insight into the after school program. Keeping in mind their young ages, a blend of dichotomous (yes/no), ranking (1-5),  questionnaire with few open ended questions will be administered. Questions about their English levels, confidence, ability and frequency of use will be asked. Students will also be asked what classes they've taken in the past and which ones they would like to see offered in the future. A questionnaire will be sent to parents and will focus on the perceived value (academic and financial) of the after school program, accessibility, satisfaction with course selection, alternative English programs offered in the community and other competing priorities (extra curricular or academic). Teachers will also be asked to complete more open-ended questionnaires about the after school program and participate in interviews. Questions to be asked pertain to what teachers have previously taught, perceived popularity and success of the program, changes in students' English skills, personal desire to keep participating and suggestions for improvements. 

 

 

 

​

​

 

 

The program evaluation will be more credible if all stakeholders are consulted and involved in the process. This will be done by following The Joint Committee on Standards for Educational Evaluation's guidelines. The purpose of this evaluation is to assess the success of the after school program as per shown by enrollment rates, increased English abilities and overall satisfaction of stakeholders. This endeavor is of value since the program itself aims to build a better community at KIS for students, parents and teachers. 

​

​

The Rainbow Framework for Evaluation

 

 

​

 

 

 

 

3. Construct a program theory

 

Program Theory: Theory of Action and Theory of Change

​

I'll be using the Logic Model to organize and develop my program theory. This format best communicates the goals and orientation of the program being evaluated. It will provide a conceptual framework for evaluation and help explain how the after school program is understood to contribute to producing desired outcomes . 

 

​

"The logic model is a systematic and visual way to present and share your understanding

of the relationships among the resources you have to operate your program, the activities you plan,

and the changes or results you hope to achieve.” (W.K. Kellogg Foundation 2004)   

​

The logic models is based on the theory of action/change. It creates a simple image that aims to describe the activities and inputs that bring about change  and how these actions are linked to the results the program is expected to achieve.  They are useful when trying to implement change because they create actionable plans and strategies with clear outcomes and steps for solving program problems.        

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

THEORY OF ACTION/THEORY OF CHANGE

​

• Change theory: this identifies one or more causal mechanisms by which change comes about for individuals, groups and/or communities. 

• Action theory: this explains how interventions are constructed to activate their theory of change in terms of the activities that will be undertaken and what level of success will be needed for each result to produce the final intended impact

 

The theories of action and change will provide a framework to observe and explain how the activities and inputs undertaken by the KIS after school program make particular contributions to the observed and intended impacts and results. The theories of action and change will be used for monitoring and evaluation in the following ways:

 

• Data will be collected to investigate whether the program works particularly well with certain demographics of students or types of classes (how many students enroll for which kinds of classes)

• Inspection of the quality and quantity of inputs and activities that support effective delivery and outcomes of the program

• Examination of progress or lack of progress in achieving results (are desired outcomes being reached? why or why not?)

 

The theories of action and change will cover: 

​

• How  other projects and programs contribute to producing impacts. This will include considering how  other after school programs compete with the existing one at KIS and how well the KIS program markets itself to parents and students.

• How  the contexts in which the intervention is implemented affect activities and results.

• Potential unintended results, both positive and negative. This could be accessibility, transportation, schedules and other after school commitments and extra curricular activities.

• Examining any assumptions that are made in relation to the success of the program and considering other contributing factors

• Analyzing how participants become engaged in the KIS project and how policies may affect this

• Making suggestions about how the short and long term objectives of the KIS after school program can be sustained.

 

The process for developing the theories of action and change will include: 

​

• An assessment of the needs and problems the after school program is trying to address in relation to English language acquisition and use.

• The resources and opportunities that are needed to do this. 

• A review of existing documentation which explains how and why the after school program was initially developed.

• Research and evaluation of the after school programs at other KIS worldwide. 

• Research and evaluation of other similar after school programs in different schools.

• Talking with stakeholders about how they understand how the after school program works and what it is intended to do. 

• Getting stakeholder feedback about how they think the desired outcomes may best be achieved. 

​

The outcomes and impacts will be assessed through:

​

• Measuring increased skills or knowledge or changes in behavior of participating individuals

• Other individuals affected by participants – for example, students taught by teachers whose skills have been improved by the project or program. 

• Feedback and consensus about the intended impacts. 

• Feedback about different intended impacts and if there is any dissension?

 ​

Sources:

​

http://toolkit.pellinstitute.org/evaluation-guide/plan-budget/using-a-logic-model/

​

Huey-Tsyh, C. (2005). Practical program evaluation: Assessing and improving planning, implementation, and effectiveness. Thousand Oaks, CA: Sage Publications

​

The Rainbow Framework for Evaluation accessed at: http://www.betterevaluation.org/

​

Logic Model Development Guide, W.K. Kellogg Foundation accessed at : https://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide

​

​

 

4. Identify, describe, and rationalize your evaluation approach

 

 

 

 

 

REVISED EVALUATION APPROACH: 

Collaborative Inquiry

​

The KIS after school program could likely be evaluated using several approaches, but I believe a collaborative approach would work best. 

 

What does a using collaborative approach mean? 

 

It would mean partnering with stakeholders and coming together to ask questions, looking at research and exploring different perspectives to uncover the best way to go about the evaluation process. The working relationship between participants would emphasize active and meaningful problem solving in all phases of the planning and implementation stages of the program evaluation. Since the ultimate success of the program evaluation process depends upon the willingness of teachers, parents and students to commit to the initiative, the principal of KIS will be sought to generate a sense of support and ownership among the various stakeholders.

​​

 

 

 

 

 

 

 

 

 

 

 

 

 

The resource recommended to base the collaborative approach on would be the Facilitator’s Guide for collaborative inquiry by Learning Forward Ontario. Although chock-full of information, the document identifies 4 main stages of collaborative inquiry:

 

​Framing the Problem

​

• Determining a meaningful focus

• Spheres of concern, spheres of influence

• Writing a purpose statement

• Developing an inquiry question

• Formulating a theory of action

• Identifying underlying assumptions

​

Collecting evidence

​

• Identifying types of data

• Where to collect evidence

• How to collect evidence

• When to collect evidence

 

Analyzing the Data

​

• Organize the data

• Reading the data

• Describing the data

• Classifying the data

• Interpreting the data

​

Collaborating and sharing the findings

​

• Preparing to write

• Writing the report

• Referencing

​

I believe a collaborative inquiry approach makes the most sense for the evaluation of the KIS English after school program because at this time there isn’t a clear, predefined, agreed-upon set of program goals.  It is just mandatory that all Korean International Schools provide one for their students. Using a collaborative approach would be helpful to establish the foundation of specific goals and evaluate whether the program is meeting them or not. 

​

Working collaboratively with different stakeholders will increase the contribution of information, opinions and perspectives. We can obtain richer knowledge which will help identify what is working and what is not, to help make the program more effective. In line with the theories of action and change, a collaborative inquiry approach will also help determine incentives for participating in the program, changing social norms, to identify any barriers as well as any opportunities for improvement.    

​

My approach would be hands-on because I work at the school and am currently teaching in the after school program 2 afternoons per week. I have a vested interest in the program which is why I want it to succeed. Collaborating on the program evaluation would ease any bias or prejudices that would arise from having a lone evaluator.

  

The high staff and student turnover throughout the years is a bit of a challenge in assessing, realizing and aligning the program to achieve the long term outcomes. Each year students experience a different version of the program. Therefore, it makes the most sense to approach the evaluation collaboratively to ensure that our process is looking at the many ways this program is diversified, how it changes over the years and how this affects its success. This can only be achieved by having a wide range of contributors sharing their knowledge, experience and advice.

 

Donohoo and Velasco (2016) identified other problems with a collaborative approach. Those being time, motivation and / or commitment to participate, supports for the collaborative inquiry process and an overall lack of collaborative inquiry culture at the school.

 

Mitigating these problems would include sharing information with stakeholders about the collaborative inquiry process so they understand the process and are more likely to cooperate. Participation would be voluntary and tax stakeholders’ time as little as possible. Identifying why the program is important to stakeholders would help with motivation and participation levels.

 

Sources:

​

Donohoo & Velasco, (2016). The Transformative Power of Collaborative Inquiry: Realizing Change in Schools and Classrooms. Crown, Thousand Oaks, CA

 

Learning Forward Ontario (2011). “Collaborative Inquiry: A Facilitator’s Guide.” Retrieved from file:///C:/Users/HP/Desktop/Module%201/Collaborative_Inquiry_Guide_2011.pdf

​

​

​

​

Connecting with the Evaluation Community 

​

​

                                         

​

​

http://www.artofhosting.org/what-is-aoh/methods/

​

​

 

 

 

 

 

"The Importance of Measuring Participants’ Reasons for Being in the Program"

by Tiffany Berry and Dana Wanzer

​

You can find the article at http://aea365.org/blog/ed-eval-tig-week-tiffany-berry-and-dana-wanzer-on-the-importance-of-measuring-participants-reasons-for-being-in-the-program/, which was posted on December 1st 2016.

​

This article discussed why you should measure participants’ motivation for joining or continuing to attend a program. The authors examined students’ intrinsic and extrinsic motivations to measure their reasons for participating in a program and found that students with highly internal motivation were more successful than those who were participating due to external factors. 

​

​

​

​

​

​

​

​

​

​

 

 

This is something important to keep in mind when analyzing outcomes in an evaluation. I am currently developing an outcome evaluation for the English After School Program at my school and believe that measuring students' reasons for participating can help avoid personal bias and explain some differences in program outcomes. Measuring motivation would shed some light onto why the program is or isn't as successful for some in terms of English language acquisition, use and confidence, as well as explain low or high reports of recommendations and satisfaction.   

What do you think?

 

• Do you think it’s important to consider participants’ reasons and levels of motivation when measuring a program outcomes? Should this be considered as a factor in a program’s overall success or lack of success?

​​

• In comparison to other factors such as cost and accessibility, how much of an impact do you think motivation has in terms of program outcomes?

​​

• Does measuring motivation help us determine how much of a program’s short comings are process related or due to external factors?

​​

Personal motivation is a huge factor in any personal goal that is undertaken. Be it educational, social, health or financial services, the programs undertaken in organizations endeavor to support people’s individual and collective goals. But intrinsic motivating factors might be important in determining a program’s success and sustainment.

 

A final note…

The authors say that students who begin as extrinsically motivated but become intrinsically motivated throughout the program have similar positive outcomes to students who were intrinsically motivated from the start.

 

 

 

 

 

 

 

 

 

More from AEA365...

​

"Reflecting and Learning for Better Development Results" - Kristin Tindell

http://aea365.org/blog/icce-tig-week-reflecting-and-learning-for-better-development-results-by-kristin-lindell/

​

This post appealed to me because it reminds us of the importance of reflection throughout the evaluation process. I am conducting an outcome evaluation on the after school program at my school and my goal is to assess and improve the current program's development outcomes. Reflection enables us to devise better strategies and to adjust. This leads to better, more comprehensive evaluations. 

 

Under the USAID Learning and Knowledge Management branch, the author introduces the MERL workstream as a part of every evaluation. MERL stands for: 

 

• Monitoring

• Evaluating

• Research

• Management

​

She emphasizes that at the cornerstone of their adaptive management technique are four key questions: 

 

• what does the data mean?

• what should we keep doing that's going well?

• What should we stop doing? 

• What should we change?

 

Keeping these reflective questions in mind throughout an evaluation can be helpful in understanding progress and inform future work. Another benefit is that reflection can lead to program adaptations.

 

When designing a MERL cycle ask what do you want to learn. The author is mindful that by being specific you can avoid unnecessary collection and analysis of data.  Sometimes there can be too much information. She suggests to focus on learning and decision making rather than accountability. Furthermore, consider the organization's culture and be open, agile and creative. Challenge assumptions, be adaptive and take risks! 

​

"District Approvals for Data Collection with Students" - Siobhan Cooney

http://aea365.org/blog/siobhan-cooney-on-school-and-district-approvals-for-data-collection-with-students/

(Shared by Heather Enzie in Module 2, PME 802; Adapted)

Tips

​

• Plan in advance for data collection that you plan to undertake

• Get approvals from administrators early and ongoing

• Assume that you will need many hours to do this!

• Don’t assume that all proposed collection will be approved

• Offer the school district something in return for the support of your data collection initiative.

 

Thoughts and questions:

​

• What is the most efficient way to collect data from teachers, parents and elementary school children? On line, in class, electronically, paper, in personal..?

• What types of questions should be asked? 

• How would I go about choosing a focus group for data collection to inform an English after school program?

 

Dilemmas in Evaluation Use

​

​

​

   

​

​

  

​

"Utilized Focused Evaluations for Equity Focused and Gender-Responsive Evaluations" - Michael Quinn Patton

https://www.youtube.com/watch?v=jQP1FGhxloY

​

​

​

​

​

​

​

​

​

​

​

​

​

​

Emphasizes use of evaluation results and gives dramatic examples of the consequences of ignoring or dismissing data with dramatic examples of Rwanda and Minneapolis. Recommends a research-based approach to evaluations that promote data informed decision making within context of: 

​

• political philosophy

• time

• money

• knowledge about what works and why

• regulatory environment

• hitorical momentum

• organizational environment

• citizen needs

​

Stressed importance of timelines, relevance and use of the evaluation. It is of no use to go through with an evaluation if the results aren't going to be considered and used for positive change.  There has to be systematic responses to and follow up on recommendations, in addition to havving a schedule (timeline) to make sure feedback is consistent and effective. No one has time to waste.

​

Evaluation Standards: 

There are not enough resources or time to do everything so focus on what's most important.

​

• Utility - ensures relevance and use

• Feasibility - realistic, prudent, diplomatic, frugal

• Propriety - ethical, legal, respectful

• Accuracy - technically, adequate to determine merit or worth (are results valid?)

​

     Design Evaluations to have Meaningful Use. 

​

Different Evaluation Purposes: 

​

• Making Judgments - Summative

• Improving - Formative

• Ongoing Development - Developmemtal

• Knowledge Building - Meta-evaluation

​

Premise:

​

• No evaluation unless primary intended users will use the info provided

• Primary intended users were involved in the process

• Evaluation is part of initial program design and the primary intended users want the information

• Make implications part of every discussion throughout process - make it useful

​

Challenges: 

​

• Inappropriate use

• Misuse

• Compliance Use

​

"The impact on our program came not so much from the findings but from going through the

thinking process that the evaluation required." 

​

​

​

 

 

 

 

 

 

 

"Evaluation Use: Theory, Research and Practice since 1986" - Shula and Cousins, 1997

​

• Discusses the limitations of meta-analytical evaluations - evaluating just for the sake of producing data. Of what good is it if it is not acted upon? 

• Should evaluations just be used for accountability? (No, a big issue is that they could be used as a weapon and  not take into account the larger context).

• Argues that all approaches conflict with one another in some way. As Patton previously pointed out, no one evaluation approach can 'do it all'.  So why not look at context? Customize.

• Seeing the emergence of collaborative modes of collaboration. 

​

Challenges: 

​

• Bias

• Consequences for being honest

• Time and resources

​

 

 

 

 

 

 

 

"Have We Learned Anything New About the Use of Evaluations?" - Carole Weiss, 1998

​

Challenges: 

​

• Conflicting beliefs within organizations, unable to agree on important issues

• Conflicting interest between program units, squabbles over resources and clout

• Changes in staff, new people new priorities

• Rigidity of organization, rules and standard operating practices that prevent adaptation of improved strategies

• Shifts in external conditions, budget cuts, changes in political climate

• Challenges can be used as a excuse for inaction or misuse 

​

Appreciates how complicated the phenomenon of "USE" is and how different situations and agendas can be from one another. What does Weiss mean by "use"? That they're useful for Decision Making. Evaluations are more likely to be "used" under 3 conditions:

​

• Findings are non-controversial, they create no rifts or conflicts of interest within the organization 

• Changes are small-scale and with the program's repertoire

• Stable environment, no big changes in leadership, budget, clients, or public support.

​

Constructivism

'All findings are conditional and contingent, and have meaning only in particular sociological conditions. Can be constructed differently by each person and each person's perspective is valid. Subject to alternative meaning. 

​

This idea warns against over-generalizations and synthesis!

 

Suggestions for improvement:

​

• Keeping in contact longer 

• Collaboration

• Sharing information

• Potential users consulted to define the problem

 

​

 

 

 

 

 

 

 

 

"Unbundling Evaluation Use" - Marvin C. Allen, 2002

​

How to evaluate if an evaluation was of "use":

​

• Did the evaluation generate a new understanding of certain aspects of the program?

• Were progressive changes made in the program as a a consequence of the evaluation? 

• Did program staff acquire new skills and insight during the evaluation (caused by and attributable to the evaluation)? 

​

Successful evaluations are dependent upon: 

​

• what particulars evaluation procedures are of concern? Is there a design, a comparison program, data collection analysis?

• How information is share information along the way

• The varying degrees of communication between evaluators and stakeholders

• The way knowledge is reported and presented

• Personal characteristics and actions of evaluators (commitment, willingness to involve users, political sensitivity, credibility, rapport, etc) 

 

Why unbundle? To become aware of the diverse nature of evaluations and aware of it's unique features. 

​

 

 

 

 

 

 

 

 

 

"The Use and Usability of Evaluation Outputs: A Social Practice Approach" - Murray Saunders, 2012 

​

Regarding the purposes of evaluations, the author sees 2 dimensions for a social practice approach: 

 

Context and Capacity + Usability 

​

Emphasis on Inquiry based evaluations that are: 

​

• Instrumental - findings are used to modify the project or evaluation

• Conceptual - helps to understand the program in new ways

• Enlightenment - adding  knowledge to the field, not just within the program evaluation

• Process use: cognitive, behavioral, program and organizational changes resulting from evaluation process

• Persuasive or symbolic - in order to legitimize a decision (could be problematic)

​

​

 

5. Identify data collection methods and analysis strategies

​

The outcomes to be assessed for the English After School Program represent changes in language fluency, use and confidence, stakeholder satisfaction. Therefore data needs to be collected to provide information about these changes. Because this is an outcome evaluation based on the theories of action and change, I believe it's best to include both qualitative and quantitative data collection methods . Sources for information include students, their families, teachers, administrative staff, and existing documentation (current and previous years' enrollment numbers and types of classes offered). 

​

The quantitative data would include looking at the number of students that have participated in the program and the type of classes offered on a graph or chart. For example:  

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

This data becomes more meaningful when considered in connection to the qualitative data accrued through surveys and questionnaires.

​

Parent and student questionnaires, along with informal teacher interviews are key data collection methods for this specific evaluation. In addition, a skills assessment could be incorporated into the students' survey or developed separately to assess students' proficiency with respect to the English language. To collect the latter information, staff and teachers would have to be consulted to receive individual students' language level placement history in order to compare results and determine of any progress has been made. 

 

The table below summarizes the data collection methods to be considered with respect the English After School Program evaluation: 

​

Parents - Questionnaire

​

• How long has your child been attending KIS?

• How many semesters have they been in after school classes?

• Which classes has he/she taken? 

• How satisfied are you with - course selection, teacher methodology/content, accessibility, time, value

• What classes would you like to see offered? 

• Why did you register your child in the program? academic, social, other reasons

• Are they taking any other English classes in the evenings or at the weekend?

• What level English are they currently in? 

• Have you noticed a difference in your child's English language use/confidence/abilities since taking the program or na?

• How likely are you to enroll your child again?

• Would you recommend the program? 

• What suggestions could we make to improve?

​

There would be 2 questionnaires for the students. One before participating in an after school class and one afterwards to measure any differences. 

​

#1 - Program Related 

 

• Why did you take this program? (motivational chart)

• Why this specific one (teacher, subject, friends, couldn't get what I wanted)

• Rate the class on a scale of 1 -10 

• What do you like best about the class? 

• What do you like least? 

• What classes would you like to see offered? 

• How confident are you at speaking English 1 -10 

• How much English do you speak at home? Never - Always

• What level English are they currently in? 

• Would you tell your friends to take the program? Yes - No Way

 

#2 - outcome related 

​

• How confident are you at speaking English in class/at home/in public ?

• How much do you speak English outside of class/outside of school? 

• Have your English grades improved? 

​

Teachers - Interviews

​

• Have you ever taught an after school class? Been "voluntold"?

• What did you teach? 

• What worked, what didn't?

• Any suggestions recommendations? 

• Any noticeable difference in students marks, behavior, confidence?

 

 

 

 

 

 

 

Data collection logistics 

​

Surveys and questionnaires are the preferred method for gathering information to compare against older statistics. This is the best way to support the evaluation rather than lengthy interviews. The parents surveys will be shared at home and the student surveys will be administered during the 20 minute morning period in their respective homerooms. It is recommended that parents receive the information at home online through the KIS school website so they can access the survey conveniently at any time. 

​

Administration, the principal and vice principals, and the teachers will be included in the evaluation process to allow them to articulate and establish their positions within the program. By including all stakeholders in the data collection and evaluation process will lead to many benefits including the increase in participation and symbolic perceptions of the program and its utility. 

​

Once the data has been collected it will be stored in the teacher's room in safe keeping, yet still accessible to all staff. The information will be shared via workshops with the teachers and administration staff. These stakeholders will gather to discuss the findings and consider together ways in which the program is working and ways in which it can be improved. New approaches to the English after school program can be examined at this time. This evaluation aims to bring the school community and stakeholders together while creating a safe environment in which to express and validate concerns and issues. The design of this evaluation is justified as it will help attain an accurate picture of the after school program and reinforce the value of it. 

 

 

 

 

 

 

 

Data anaylsis

​

Data can be analysed and presented in tabular or graphical formats. The data analysis will consider how quantitative and qualitative data interact. For example, is there a link between the level of satisfaction and type of classes being offered that may increase the level of enrollment? Analyzing responses to specific questions asked to parents and students about these things is a form of triangulation used to substantiate findings. Survey questions will assist in this way with data analysis and help identify areas for improvement and contribute to data reliability. 

​

 

 

 

​
 

 

​

​

​