Program Evaluation Design: Take 2

After communicating with Dr. Syer, I realized that my previous program evaluation design selection was inappropriate for the project we are to be conducting. As a result, I have decided to redo this and focus on the Talk around the Clock (hereby referred to as TAC) card program I mentioned during the Foundational Knowledge postings of Module 1.

What is the TAC program?

The TAC program was implemented during the 2015-2016 school year. It is designed to encourage the use of English among students during school hours. As an international British Columbian curriculum school in Guangxi, China, one of the most effective ways for us to develop our students' English capacity is by cultivating an immersion based environment. The program proved very effective, and so we have re-implemented it this year.

In its essence, the TAC program is quite simple:

  • Students are broken up into 10 groups that spread across grades
  • Each group is named after an animal, students remain in their groups for the course of the school year
  • At the start of each day, the group leaders will come to the principal’s office in order to collect three TAC cards for each group member
    • TAC cards have students' faces and names on them
  • As the day progresses, if students are caught not speaking English, they lose a TAC card
  • At the end of the day, group leaders collect all the remaining TAC cards and return them to the principal’s office
  • The principal (or other staff members) tally up the total number of TAC cards remaining for each team and add it to a score board
  • The team with the most points at the end of the two-week period is rewarded with a pizza party
  • The team with the least points at the end of the two-week period is responsible for performing an original song and dance in front of the remaining student body

Program Summary:

  • Focus of the Program: Encourage the use of English
  • Size of staff: Eight Canadian foreign teachers, one Canadian principal, Chinese staff
  • Goals: Inculcate an English-immersion environment during class hours
  • Resources: Three personalized TAC cards for each student, lanyards, a score board and supply closet
  • Community Demographics: 122 Chinese-native students with varying English proficiency

Past experiences:

Last year was the first time the TAC program was implemented. While we were uncertain about how successful it would be at first, it quickly proved effective.

For the first week or so, students were very quiet out of fear of 'being caught,' however they quickly became more comfortable, and resumed speaking with each other.

For the first month or so, we were collecting a lot of student cards over the course of a day. With time, students were losing less and less cards. In fact we started to have a problem, the program was proving to be too effective. Very few cards were lost, and when students did lose cards, it could result in conflicts between themselves and their group members.

Given these considerations, and confusion resulting from multiple winning teams (because nobody was losing any cards), we created a new collection of 'teacher TAC cards'. These would be given out to students who demonstrated exemplary use of English (or for other reasons, it was left up to each teacher's own discretion).

Students now felt motivated to demonstrate their English skills, and teams would often hand in more TAC cards at the end of each day than they had at the beginning.

At the start of this new school year, we had yet to take student photos and develop the TAC cards. As a result, the program took a few weeks to get underway. In those first few weeks, returning teachers clearly noticed the difference that the TAC program had made the previous year. Not only were students speaking other language in the hallways and classrooms, they had little motivation to change this habit. The old game of punishment and persuasion returned to the classroom.

Considerations for Evaluation/Purpose of Evaluation:

As this program has already proven to be very successful in encouraging an English-only environment during school hours, the focus of this inquiry is to look at student and teacher perceptions of it. With specific consideration paid to its efficacy, motivation and room for improvement.

Student Focused:

  • How does the program positively develop English skills?
  • What feelings does the program inspire in students? Do they value it? What motivates them to be successful?
  • How does the program work to develop lasting improvement in student's English skills?

Teacher Focused

  • Has the program contributed or changed classroom/school dynamics?
  • Can the program be viewed as a positive or negative contribution to the school environment?
  • How do teachers view their role with the TAC program?

The General Question:

  • How can this program be improved?

Guiding Questions:

  • How is the program helping to guide English acquisition?
  • What relationship do various stakeholders have with the program?

For the original project, please click here.