4.1 Project Evaluation

QUICK LINKS
4.1.1 Conduct post-intervention end-line monitoring
4.1.2 Evaluate project effectiveness in achieving desired results and determine whether to exit project, and/or continue/adapt as needed
Tools and resources helpful to supporting this step

4.1.1 Conduct post-intervention end-line monitoring

Carry out final end-line monitoring on all indicators of indicators in accordance with the monitoring plan. The Change Analysis – before and now analysis (T11b) tool may be used to aid this process. Once data is analysed, compare end-line results with baseline assessment results to determine the extent to which project objectives, impacts, and outcomes were achieved. Identify successes as well as underperforming indicators for further follow-up discussion, reflection and learning with your team, representative members of the target audience, and any other relevant stakeholders. In cases where some values and behaviours are highly entrenched in the community norms/culture, the SOC project may be met with resistance or backlash. This however should not be construed as a failure, as behaviour change and cultural shifts are processes that take time to change and may require persistence, and continued efforts to achieve.

4.1.2 Evaluate project effectiveness in achieving desired results and determine whether to exit project, and/or continue/adapt as needed

Conduct an evaluation of the project using internal or external evaluators, with preference for using impartial third-party evaluators whenever feasible. These evaluations are particularly essential if desired behavioural change is not achieved as it will enable you to assess whether there is anything you could do to improve the project implementation strategy, or processes to improve outcomes. This can inform your decision as to whether you need to adapt the SOC approach using a different strategy or exit because the project is unlikely to achieve success.

Most often applying a mix of quantitative and qualitative methods of data collection gives a more in depth understanding to evaluate the project. The evaluation can be conducted ‘in –house’ by the team or using external actors. Evaluation should be participatory and involve stakeholders involved in the project as this will give an accurate overview of the project and how it was understood and accepted by the target community.

Evaluations should seek to assess the following in consultation with representative members of the target audience and other relevant stakeholders as appropriate (Adapted from [78]):

• The appropriateness and effectiveness of any processes employed by the project to:
  • improve and sustain animal welfare and the adoption of desired behaviour change
  • promote participation of communities throughout all stages of the project, and
  • promote gender equality and safeguarding of vulnerable groups.
• Stakeholder satisfaction with project. Consider assessing stakeholder perceptions related to the benefits and value of the project both in terms of animals and people, overall project accountability and transparency, satisfaction with the project/implementing organization, as well as recommendations for improvement.
• Was the process as participatory and empowering as it was hoped for the partners/stakeholders and target audience in particular?
• Check which of the strategies, modes of delivery as well as BCTs were effective in coining/framing the messages considering the APEASE criteria.

Ultimately, though it may be difficult to ascertain, try to see whether your communicated messages and the used modes of communication have resulted in tangible understanding of animal welfare and improvement in practicing/embracing animal welfare domains. As the SOC approach is about broader engagement and awareness, it may not be feasible to monitor how the population you have targeted are accurately doing in maintaining the action. If your project aimed to support changes in values and behaviour that have a strong cultural element or are embedded within the community, such change may be incremental and difficult to observe over a short time frame. This means that the evaluation may only be able to assess the basic indicators specifically identified as being important for this project, other benefits may only become apparent over a longer period, although, this is still sufficient to show that the project has been successful. Follow-up monitoring 1-2 years after the project can be useful for assessing the effectiveness of the project, as well as the sustainability of change, especially when conducted at repeated intervals.

Based on results of the assessment determine which of the following options is most appropriate and feasible:

1. Withdrawal of outreach/ campaign efforts and exit: this option is recommended when:
   • The desired behavioural change and related animal welfare improvements have been achieved and there is evidence that this change is sustainable. For example, if you have a representative sample of your target audience who are in the maintenance stage of behaviour change using them as champions will help them maintain their behaviour but will be useful in the dissemination of the target welfare practices to community members. Some of the way to incentivise them could be using competition as well as allowing them to host local and regional level contests in the animal welfare practices, they have learned. Or,
   • When the desired behavioural change and related animal welfare improvements have not been achieved/seem unlikely to be sustained and feasibility of achieving them is unlikely.

2. Continue outreach/ campaign efforts and do not exit: this option is recommended when desired behavioural change and related animal welfare improvements have not been achieved or are unlikely to be sustained due to issues with strategy, and the project has the necessary resources to continue supporting the project. In such cases, it is recommended you adapt the outreach/ campaign strategy and re-plan in collaboration with the community to continue working to achieve desired results.

Evaluation results and next steps in terms of gradual withdrawal of support or the project’s desire to continue support should be shared and discussed with the animal owning community and other relevant stakeholders for full transparency and accountability.

Schedule periodic communications with relevant community leaders or key stakeholders, social networks of target audiences, and/or local animal resource and service providers to discuss any challenges and provide support as needed to enable relevant parties to address them.

Consider including outreach activities to support continued maintenance of desired behaviours, provide reminders to action at relevant times (e.g. regular animal health checks), and/or continue fostering social norms to encourage the long-term sustainability of behavioural change as appropriate (e.g. radio or TV shows, posters, mobile messaging etc.).

Link to References Cited