Skip to main content

Resource

21. Example Questions for Evaluating Project Success

The resource is intended to support community facilitators design evaluation questions to track the implementation, progress and outcomes of the animal welfare improvement project/behaviour change processes. The resource can be used when developing community project participant’s individual interview or focus group discussion guide. The sample questions explore community vis-a-vis individual level household changes; group sustainability; and tracking behaviour change from the stages of change perspective. Note these are not exhaustive and need to be adapted to fit to your programme/project/context needs.

Keyword Search Tags

Project Phase:
Exit & Evaluation Phase

Approaches for Working With Communities: Community Development Approach, Community Engagement Approach, Societal Outreach and Campaigns Approach

Project Support: Facilitator Resources, Monitoring and Evaluation

Sample Questions for assessing animal welfare changes/impact at the community level

  • It could be helpful to start with questions which encourage the respondent to easily respond to e.g.
    • Why did you get involved in this project? What did you find the most interesting about this project?
    • What was the most difficult part for you? What could be done differently to make this project more successful in the future?
  • You can ask them to recall and explain what things were like before – and then ask them to explain how they are now. Eg) How were animals in your community before this project started? Probe: Were they happy? Why? Did they look healthy? Why? Did they have the things they need? What did/didn’t they have?
    • Then provide a follow on question, Eg) How are things for animals in your community now? Why? Probe: Do they have what they need? Do they look healthy? If yes, why? If no, why not?
  • Do people do anything differently with their animals now? Can you give some animal welfare related actions being carried by the community?
  • Are change agents (if being utilized) playing an active role in supporting your community? In what ways? What do you think they could do differently?
  • Is there a collective understanding amongst community members of the major animal welfare issues and how to address them?
  • What are the animal health service providers available in your area? Do these local service providers provide quality services to the satisfaction of community members? Probe: Are group members seeking the help of animal health services? Are local service providers responsive to community requests? Does the responsiveness differ based on people’s gender or socio-economic status? Can you provide examples?
  • Are community/group members seeking animal welfare related services proactively and in a timely manner? Can you give examples?
  • Are men, women and children in families who own animals all aware of their animals’ welfare issues and participate in activities that are important to improve and maintain good welfare?
  • What kind of actions have been taken in your community to ensure better animal welfare improvement is achieved? (Probing questions depending on your focus issue: What examples of community-led animal disease prevention measures are being carried out on appropriate timelines e.g., vaccines and boosters? Are appropriate measures to ensure clean and comfortable environments for animals in place? Are community members engaged in the project have a good knowledge of appropriate and welfare friendly gear? And use them on their animals? e.g., harnessing, saddle-fitting, cart balance and loading, are animals handled in ways that are safe? (Which doesn’t not prevent their development or creates risk of injury. What are the animal – owner/handler interactions? What are the wound prevention or/and management practices in your area? Have you seen changes on that?).
  • What changes in access to and availability of resources do communities perceive have resulted from their involvement in the project?
  • Have there been any benefits to the community from this work? Probe: Can you give specific examples?
  • Have there been any negative consequences on the community from this work? Probe: Can you give specific examples?

    Sample questions to ask when assessing group sustainability (if the project included Equine Welfare Group Formation)

    • Is your community different after forming a group? If yes, in what ways? If no, why do you think this is?
    • Is the group registered? Is your group functioning well with regular documented meetings? Have there been changes in leadership of your group? Are men and women part of the leadership and membership?
    • Is conflict resolution mechanism in place, has been used and the outcome recorded in village registers?
    • Are there examples of collective actions that have been undertaken by the group to meet their animals’ needs and are these recorded in the records of the community group e.g., collective feed purchases, vaccine campaigns, advocacy with government agencies to address their animal needs? 
    • What are the successful animal welfare improvements that were achieved through your collective/group actions? What were your group’s actions to encourage your members to change their behaviour to improve animal welfare? Were there mechanisms that your group established to change your members' behaviour to improve animal welfare? Can you give us examples? Can you tell us more about this? 

      Sample Questions for Assessing Changes/impact at the individual level 

      • Are all members of your family aware of your animal’s welfare issues? Probe: Your wife/husband? What about your children? Probe: What do they each do for your animal’s welfare?
      • Have you learnt anything new since being part of this group/work?
      • What are the changes you have seen in your animals, give examples/specify
      • What are the practices you have adopted at home that has are helping you improve the welfare of your animals? Give examples.
      • What kind of support have you received from Brooke/partners or change agents?
      • Do you do anything differently with your animals now, after being part of this project: Probe: Can you give some examples? Why didn’t you do the things that you do now with your animal, beforehand? What stopped you from doing things with your animal the way you do now, before this project started? What helped you to change your practice or what is a challenge for your not to change your practice? What do you want to know more or practice more to improve your animals’ welfare?
      • How would you rate your rate of motivation to continue to practice the new targeted behaviour?  Do you intend to keep caring for your animal in the ways you do now? If yes, why? What could make it difficult to do this? If no, why not?
      • What are the prompts and triggers to adopt the new behaviour you identified? [learning questions for reflection and for future programme/project intervention design]
      • Were you able to identify something that would help you to sustain the desired targeted behaviour? And what is it?
      • Have you faced unforeseen challenges? If so, what are the unforeseen challenges have you faced?
      • Is there any resource/time implication of adopting the desired new targeted behaviour when compared to the old practice? Probe: Do you think this will impact whether you continue to do this?
      • Vulnerable groups’ targeted question: Have you encountered any specific challenges to your gender/situation or position as a result practising this desired behaviour? e.g., making decisions to take the animal to be treated, deciding to pay to a service?
      • Has it been difficult to continue/maintain the new ways of treating and working with your animal? If yes, why? If no, why not? What or who made it easier for you to keep up these ways of working? What or who made it difficult to maintain the new changes? What kind of support do you still require to maintain/continue with practicing the desired targeted behaviour?
      • Have you started experiencing the benefits of animal welfare to your livelihood and the associated desired behaviour change?

        Tracking Behaviour change using Stages of Change Model

        As part of working towards behaviour change, it is important to track progress towards the realization of the desired behaviour. You can use the below questions to monitor progress or as part of overall project evaluation. The below points/questions can help you to check whether you were successful to introduce change within each stage of change. These suggested indicative questions (not prescriptive and exhaustive) can be used for FGD, KII, and within surveys/interviews, or can be included as a follow up of various participatory tools that can be used for evaluation phase. You can also ask these questions retrospectively once the groups/individual project participants progress to the next stage of change to understand the change progress as part of your learning.

        Pre-contemplation and contemplation (following Conception phase): 

        • Have community/individual’s knowledge and understanding about animal welfare improved? Do they identify the gap in practice on animal welfare now?  You can probe this by asking:- Can you explain what animal welfare means to you? Follow on: What was the standard of animal welfare in your community before this work started/your group formed? Probe: Why was this the case?
        • Have they become aware of what animal welfare concerns are there in their locality/individual homestead? Probe: Can you provide examples?
        • Are community/individual’s’ able to link their/o/u/c problem behaviour with the animal welfare gap.
        • Has the capability (physical and psychological) and motivation (Reflective motivation) of o/u/c changed after the intervention?
        • What is the belief system of the community/individual about the possibility of change? What is their change talks?
        • Did their value system change because of the intervention [during conception /planning phase] and/or your organization contribution or that of the CCA?
        • How would they rate (individual or collective depending on your availability of resource and relevance) their motivation for change? This will be asked retrospectively as part of overall evaluation, or as a question at the end of the first phase? If asked at the end, this could be clearer: “How would you have described your motivation for change at the beginning of this process?”
        • What kind of steps/actions they are willing to take or have taken so far?

          Preparation 

          • How would they rate their motivation to adopt the new desired targeted behaviour? 
          • What is your current understanding about the relationship between the problem behaviour and animal welfare practice gap; and how can you change this?
          • Did their value system change because of the intervention [during conception /planning phase interactions] and/or your organization contribution or that of the CCA?
          • What kind of steps/actions they are willing to take or have taken so far?
          • What kind of support has they received? e.g. Action plan development trainings, etc.
          • Have they (group and/or individual) developed action plan to act?
          • What were their barriers to develop action plans before the start of this programme/project? Have these barriers changed? How would they link your programme/project/CCA helped/contributed to reducing the barriers?

          What were their barriers to develop action plans before the start of this programme/project? Have these barriers changed? How would they link your programme/project/CCA helped/contributed to reducing the barriers?

            Action 

            • Did they find their action plans relevant to their reality when trying to implement them? (This shows doing good COM-B analysis helps in identifying potential barriers and adaptive programming with SI can further help the community/individual to get the relevant/appropriate intervention to help the shift to take concrete actions to perform the desired behaviour change).
            • What are the prompts and triggers to adopt the new behaviour you identified? [learning questions for reflection and for future programme/project intervention design]
            • Were you able to reinforce the supporting behaviours that will help you to sustain the desired targeted behaviour? Probe: Can you provide specific examples?
            • Have you faced unforeseen challenges? If so, what are the unforeseen challenges have you faced? 
            • Is there any resource/time implication of adopting the desired new targeted behaviour when compared to the old practice? 
            • How would you rate your rate of motivation to continue to practice the new targeted behaviour? [for survey or FGD or KII]
            • Vulnerable groups’ targeted question: what kind of actions were you to take? Have you encountered specific to your gender/situation or position? E.g. making decisions to take the animal to be treated, deciding to pay to a service? 
            • Have you faced any safeguarding issues when practicing this new targeted behaviour [This is particularly relevant when part of the socio-economic and COM-B analysis/gender analysis has shown that it is a hindrance for animal welfare or a prompt/favourable social norm for animal welfare practice without being exploitative of individuals/undermining their dignity and rights]
            • Who helped you to continue to continue overcome your challenges/distractions to go back to the old way? Did you have supporting relationships or mechanisms?

              Maintenance

              • How long were you able to hold of the maintenance of performing the desired targeted behaviour?
              • Have you seen the benefit in practicing the new desired targeted behaviour?
              • Who helped you to continue to overcome your challenges/distractions to go back to the old way? Did you have supporting relationships or mechanisms?
              • What kind of support do you still require maintaining the desired targeted behaviour?

              Termination

              • How long were you able to maintain/sustain performing the desired targeted behaviour?
              • Did you have supporting relationships or mechanisms or incentives to perform the desired behaviour and to improve your animal welfare practice? What or who helped you to continue to continue overcome your challenges/distractions not to fall back to the old ways of handling/managing/using your animal? 
              • Have you started communicating the benefits of animal welfare to your livelihood and the associated desired behaviour change to your friends, family, etc.?

                Facilitation Notes

                Facilitator needs to create an enabling safe space for people/groups who are marginalized to express their lived experiences regarding the different barriers they faced and how they have found the project to help them maintain human behaviour change that benefits their animals’ welfare improvement and also theirs and their family/community lives. Meeting invites to such meetings need to ensure both genders be able to meaningfully participate and for their views to be heard to determine the next steps.

                It is important to note that such discussions with communities should be voluntary and use ethical standards which includes informing the participants why they are selected, the purpose of the discussion/interview, that their views won’t affect their involvement in the project and how the interview/the discussion will be take and the purpose of the meeting should be clearly communicated to them. The following tools may also be used to support the evaluation of project success: