GPDI 802 - Program Evaluation Design

Having completed my evaluation design, I have decided to revisit this evaluation approach and articulate better, now that I hold a clearer picture of the full program.

This evaluation has been designed with a process based philosophy. The main intention of this, is to redefine the significance and perception of the TAC program for the various stakeholders involved.

Regarding the program, we considered it a success last year, based on declining numbers of TAC cards collected, as well as direct observation and experiences of the program being a 'success.' As mentioned by Laruene Johnson in the following post, schools often enact programs based on personal experiences and feelings of 'what works,' rather than conclusive data. With this in mind, we must consider the political fact that an evaluation which could potentially highlight the TAC program negatively (if data does not substantiate perceptions) would likely be met with resistance by the established administration, who proudly hold the TAC program as a school success story. As a result, the evaluation would not be able to continue.

Taking the preceding into consideration, a strong consideration when designing the evaluation was stakeholder perceptions and ownership with regards to the TAC program. Stakeholders can be broken down into four categories:

Administration:

• The original implementers of the TAC program
• Organizer of bi-weekly reward/punishment
• Counter/collector of TAC cards at the end of the day

Teachers:

• Collecting the most TAC cards throughout the day
• Directly observing student experiences

Students:

• Wearers of TAC cards
• Can be resentful of game

Chinese Administrative/Support Staff:

• Rarely engage with the program

Considering the disparate nature of all these groups and their relation to the TAC program, it is important to consider a design philosophy that will engage with all stakeholders in the program. The intention of the process is to reframe the program as being a program handed down from administration, with compliance not being optional, to a program that is seen as valuable to all stakeholder groups.

Two approaches stand out, participatory evaluation and appreciative inquiry.

By including all stakeholders in the evaluation process as participatory evaluation emphasizes, it will allow all participants to articulate and establish their position within the program. This is one of the main benefits of a process evaluation design, as the conducting of the evaluation leads to an increase in the symbolic perceptions of the evaluation.

As well, because the program can take on a negative aspect, due to reprimands for transgression (which is basically what the program is), and negative punishment, an appreciative inquiry approach is one that would emphasize highlighting the positive aspects, such as English language development.

Now, we can begin to see a design philosophy taking shape. One that emphasizes development of the participation of all stakeholders, originally as independent interactional work groups, in which they will articulate their perceptions of the program. Once perceptions from each of the four dominant stakeholder groups are recorded, the core evaluation team, with the assistance of volunteers will design information/data gathering processes, with the intention of developing a true picture of the program. The intention is that throughout the whole evaluation, volunteers from all four stakeholders groups are involved to help establish ownership and positive perception of the program within each stakeholder group.

Finally, once the information has been collected, a large interactional workshop involving all participants in the evaluation will convene to discuss the findings, and consider the TAC program together. During this time, work done of the evaluation will be considered and celebrated, and new approaches to the TAC program will be considered.

As previously mentioned, it is easy to define four distinct stakeholder groups with regards to the TAC program, all with different roles, responsibilities and relationships/perceptions towards the program. Therefore, this evaluation, with the intent to bring together the school community and create a greater sense of group ownership with regards to the program, has been designed to first allow stakeholder groups to express and validate their concerns in a safe environment of peer stakeholders. From there, the emphasis goes towards a collegial information gathering, and finally ends in a large workshop of peer evaluators, with representatives from each stakeholder group.

Therefore, the design is justified as it will help not only to produce an accurate and data substantiated picture of the TAC program, but also help to reinforce the perceived and symbolic values of the TAC program among the various stakeholder groups.

Please fine the original evaluation approach here.