The MSC technique for evaluation

The highlight of the placement has been collecting the most significant impact stories. The reason I refer to it as most significant impact, rather than most significant change (MSC) is because I believe that change takes place over time. In this instance, I was keen to understand the immediate impact of the learning festivals in the lives of the event hosts, festival organizers and participants. I first identified the stakeholders who wanted to participate in this process, and asked them ‘During the last one week, in your opinion, what has been the most significant impact of the learning festival on you?’ and I probed further by asking why they thought this was the most significant impact. I wrote up the stories and kept them short and precise, and shared them with my placement supervisors, who selected the best stories for each festival. We then reached out to the wider organizing team and asked them to anonymously vote for the best stories for each festival. The stories that received the most votes were featured in the Global Learning Festival and Wyndham Learning festival final reports. All the other stories were included at the end of the reports. See the story below that was voted the best during the Wyndham Learning festival.


This image has an empty alt attribute; its file name is image.png
The Open Arms of The Kulin Nation – Tuti’s Story
Tuti learned about the WLF from a post that one of her classmates (Hellen) posted on the social media platform. Tuti has always been fascinated by the history of the Aboriginal Australians, and was keen to understand their way of life before and after the settlers came in. Also, as an environmental student and professional, Tuti was particularly keen to gain insights on how the Aborigines preserved and interacted with their environment. She also hoped that this session would help her understand the challenges and the conflict the aboriginal people have been facing.
Tuti not only gained more understanding about the history of the Aborigines, but she was also able to learn about how land and water (the environment in general) was at the heart of the Aborigine community and that they have always been passionate about conservation and living in harmony with nature. Tuti had recently completed an environmental unit and the insights from this session enabled her link theory and lived experiences of The First Nations People. Tuti felt that the facilitators were passionate about their identity and were so articulate in explaining the different language groups, important dates, and sacred sites through oral histories. The information gained in this session helped a great deal in filling the information gap that Tuti had about this community.
A significant insight gained from this session was that indigenous knowledge is key in protecting and preserving the environment, and that environmental practitioners and activists have so much to learn on this topic from the Aboriginal Australians as they lived in a thriving environment long before the settlers came in, and have continued to show their resilience in protecting their environment from bush fires and helping the flora and fauna thrive. 
The most significant impact on the festival on Tuti was the realization that the Aboriginal community are open, welcoming, approachable, and are happy to share their cultures and knowledge with other people. According to Tuti, the picture she had of the Aborigines was that they were a vulnerable and closed community and not everyone could interact with them. She says the mentions of the First Australians (Aboriginal and Torres Strait Islander people) on almost all platforms had played a big role in reinforcing this false notion.
Professionally, Tuti confirmed that the most significant impact of this session was that the indigenous knowledge and wisdom has the potential of addressing issues of climate change through environmental preservation. As a development practitioner, Tuti now more than even believes in the power of local knowledge and the fact that solutions to local issues is largely with the local communities.
WLF Most Significant Impact Story

Collecting these stories made me appreciate the use of MSC as an evaluation tool. In my past work experience, I only used quantitative methods for project evaluation, and this placement made me realise the potential of not only using the MSC techniques in evaluating the projects, but also in evaluating the effectiveness of partnerships in a project. For example, in one of the stories, an event host mentioned that the strong partnership with the Wyndham learning had provided them with not just a platform to share insights of their work with a broader audience but had also supported them with insights to on tailoring their events to suit a global and culturally diverse audience. The event host further cited that the feedback they received from experts from a different geographical location enabled them view their work and strategies from a renewed lens. In addition to appreciating the use of stories for evaluation, gathering these stories played a big role in the development of my reflective research question. I was able to pick up recurring themes (key being lifelong learning, learning cities, sustainable development, and dynamic global environment) which led to the formation of my research question: How can lifelong learning through learning cities contribute to sustainable development in a dynamic global environment?

MSC technique is easy to use, and it can be used by anyone. For me, once I understood the concept, I sharpened my listening skills, and the biggest lesson was to learn how to listen to learn, and not to listed to respond as I was used to. I find that the greatest advantage of this tool is the richness of the data collected. I found that when I let the people I interacted with talk, they not only shared stories about just the festivals, but also provided so much information around it. This made me realise that such is the reality of life, where things are interconnected and I find that sometimes, using quantitative methods of evaluation misses out on such details of other aspects of life that is interconnected to what we are evaluating. For example, as I collected the stories, people shared their experience of being part of the learning festival, and beyond the festival, they also shared the challenge that the COVID-19 pandemic had had on their lives and that of their loved ones. At the height of the pandemic, almost all the people I spoke with had in one way or another been affected with the virus. I found the collection of the stories enriching and in some of the sessions, I shared my experience of being separated from my family for almost two years because of the pandemic, and the sharing of the personal experiences made me realise the value of empathy in our work as development practitioners. The downside that I see with this method of evaluation is that the person writing the story might misinterpret some information, and to avoid this, I think it is important for the evaluator to review the stories with the respondents just to be sure that the correct information has been captured.

The MSC technique is one that I would like to use more often in my development practice, as I have come to realise that this tool can help me pick up subtle themes that I would have otherwise missed if I had used just the quantitative evaluation techniques. The main theme I picked was the need to reach out and make connection with other people online amid the lockdown, and the subtle theme that was resilience. In Australia, the community had just come from a season of bushfires before the pandemic and the subsequent lockdown, but they were forging ahead and were learning how to make their tomorrow not just bearable, but better. Other than using the MSC technique in my practice, I would want to use a mix of evaluation tools in my practice. For example, during the placement, I used surveys and stories to evaluate the festivals, and we ended up with rich data that will be used in the planning of subsequent festivals.

Leave a comment

Design a site like this with WordPress.com
Get started