Empowerment for the Right to Health: The Use of the “Most Significant Change” Methodology in Monitoring

Fanny Polet, Geraldine Malaise, Anuschka Mahieu, Eulalia Utrera, Jovita Montes, Rosalinda Tablang, Andrew Aytin, Erick Kambale, Sylvie Luzala, Daoud al-Ghoul, Ranin Ahed Darkhawaja, Roxana Maria Rodriguez, Margarita Posada, Wim de Ceukelaire, and Pol de Vos

Health and Human Rights 17/2
Published December 10, 2015

Abstract

Quantitative evaluations might be insufficient for measuring the impact of interventions promoting the right to health, particularly in their ability to contribute to a greater understanding of processes at the individual, community, and larger population level through which certain results are obtained. This paper discusses the application of a qualitative approach, the “most significant change” (MSC) methodology, in the Philippines, Palestine, the Democratic Republic of the Congo, and El Salvador between 2010 and 2013 by Third World Health Aid and its partner organizations. MSC is based on storytelling through which the central question—what changes occurred?—is developed in terms of, “who did what, when, why, and why was it important?” The approach focuses on personal stories that reflect on experiences of change for individuals over time. MSC implementation over several years allowed the organizations to observe significant change, as well as evolving types of change. Participants shifted their stories from “how the programs helped them” and “what they could do to help others benefit from the programs” to “what they could do to help their organizations.” The MSC technique is useful as a complement to quantitative methods, as it is a slow, participatory, and intensive endeavor that builds capacity while being applied. This makes MSC a useful monitoring tool for programs with participatory and empowering objectives.

Introduction

Everyone has the right to the highest attainable standard of physical and mental health. But this right is under increasing threat due to growing poverty, inequality, exploitation, and war. As poverty and malnutrition threaten the health of landless farmers in countries like the Philippines, they unite to defend their rights. In Kinshasa (Democratic Republic of the Congo, or DRC), health activists organize communities to tackle everyday health concerns. In Palestine, restrictions on the freedom of movement prevent Palestinians in the West Bank from getting to hospitals in East Jerusalem, where most hospitals are. Moreover, in the enclosed Gaza strip, the health care situation is even more distressing. In circumstances such as these, health and development must be struggled for from the grassroots level. A powerful movement for the right to health can make a difference, regardless of the form that social movements may take.1

In 2009, several of the authors of this article described our experiences using people’s empowerment as a strategy to improve population health in the Philippines, Palestine, the DRC, and others. We focused particularly on how our local partner organizations—people’s organizations structured from the grassroots to the national level—developed strategies for mobilizing and organizing their communities around strengthening primary prevention and curative care.2

These experiences with empowering processes at the grassroots level were enriched through support by Third World Health Aid (TWHA). This small Belgian nongovernmental organization works to further the right to health and sovereign development. As a Belgian partner of these people’s organizations for nearly 30 years, TWHA has ensured limited but stable financial support and has facilitated interactions and exchanges between initiatives on different continents.3 With its Congolese, Filipino, Palestinian, Latin American, and, recently, Laotian partners, TWHA focuses on addressing barriers that hinder the fulfillment of the right to health.4 Its goal is for local communities to increase their potential to organize and to collectively improve living conditions.

How can we measure the impact of interventions aimed at promoting the right to health? Quantitative evaluations and their measurable outputs can show which activities were undertaken, how many people were involved, and which changes were realized. But this quantitative information might be insufficient for understanding how these results were obtained in terms of behavioral processes at the individual, community, or larger population level. Our search for an additional type of evaluation process started with the ascertainment that qualitative information—essential for initiatives that focus mainly on the “software” (such as empowerment and organization) of community development—is based only on informal discussions during field visits. A broad participatory method that could ensure qualitative information gathering with(in) people’s movements along a structured participatory pathway, and also through which information gathering would become part of the learning process and of the empowering and organizational strengthening objectives, was needed.

In 2009, TWHA explored the possibility of adopting the “most significant change” (MSC)  methodology as part of its evaluation process.5 The main attraction of the method, which is based on stories of significant personal change, lies in the fact that it is fundamentally participatory in the data-collection phase and in the analytical process. Then, from 2010 to 2013, TWHA set up an MSC process to complement quantitative evaluation in an effort to better understand empowerment processes and their impact. This paper describes and discusses TWHA’s application of MSC in the Philippines, Palestine, the DRC, and El Salvador.

“Most significant change” methodology 

MSC is based on a storytelling approach through which the central question—what changes occurred?—is developed in terms of “who did what, when, why, and why was it important?” The approach focuses on personal stories that reflect on experiences of change for individuals, highlighting personal motivations, feelings, and interpretations in the life of the individual who speaks in her or his own name. These personal stories attest to changes in the self-representation or self-identification of the person involved (Did my self-perception change? Did my role change through the new capacities that I acquired?); in social recognition (Do others see me in a different way, and how?); and in personal identity, professional identity, or group (and class) consciousness (Did I become part of a group?). They also reveal the type of support that an individual received (Do I feel supported? Was my role facilitated by the group or organization?) and any broader change that occurred (Did this experience change my life, and how?).6

The method undertakes a series of steps: (1) raise interest of the people involved; (2) ensure a clear definition of the domains of change in which the participants as a group are interested; (3) define the time frame of analysis; (4) collect the “significant change” stories of all participants; (5) discuss and select the most significant stories, and express why these were selected; (6) give feedback on the results of this selection process to all participants involved; (7) verify the stories, perhaps including some important additional details; (8) when possible, combine with available quantitative data; (9) evaluate the overall process of recording and selecting the stories of “most” significant change; and (10) provide feedback to the whole people’s organization, from the grassroots to the national level.

Step five, the selection process of the stories at the community level, is essential. Stories are discussed and selected within the community groups, which reach consensus on the stories’ collective significance and explain why they selected a particular story. In this subjective process (“reaching a consensus”) of group discussion, the individual and collective impact of activities are at the center of the discussion, leading to insight on the most important changes at this local level. Steps nine and ten contextualize the stories of change in two ways. First, the participants must place some distance between themselves and the story to analyze how the process took place, how the most significant changes were selected, and whether some external influence was decisive in the selection process. This distance strengthens the value of the result because it ensures that the method is applied in comparable ways in different settings and countries (or makes clear the differences that might exist). Second, these steps ensure that all participants get an overall view of the process and its results as part of the strengthening process of the people’s organization and its ongoing initiatives.7

How have Third World Health Aid’s partners applied the most significant change technique?

After the method was presented at the 2009 TWHA partners’ meeting in the Philippines, partners agreed to adopt MSC as a methodology to complement results-based monitoring, help systematize experiences, and enable exchanges between partner organizations working in sometimes very different contexts. From the start, partner organizations shared the mutual aim of using MSC as a learning and capacity-building tool. In order to harmonize the process between partners, MSC application was proposed at the level of the “activist,” defined as an intermediary who links the staff of the partner organization and the communities—for example, the community health committees working with Etoile du Sud (Southern Star) in the DRC, youth leaders of the Health Work Committees in Jerusalem, and health workers at the Council for Health and Development, Gabriela, and Advocates for Community Health in the Philippines.

Five phases were proposed to operationalize the MSC process. First, a few days before the documentation of stories, a meeting was organized to explain and discuss the entire exercise without entering into technical details. Then came the “production phase” of stories by the participants: individuals could either write their stories down on paper or tell them to someone who would then document them, after which the storyteller would validate it. Next was the collective “objectivation”: in a group discussion, participants shared individual experiences and had the chance to enrich (or question) certain aspects of the stories. For example, a story might be rewritten to give it a more collective dimension. Certain aspects could be verified or detailed through a field visit or through consulting other sources. Then, the stories were discussed at all organizational levels, including the community groups, the national offices of TWHA partners, and the TWHA headquarters in Brussels. Each level identified and selected the most significant story and the reasons for its importance. During this process, the individual and collective impact of activities and organization-building were stratified, revealing insight on the most important changes at each organizational level. In the last phase, these results were fed back to and discussed in the local groups of participants. This restitution was important, as the information collected belonged to the storytellers. But even more essential was that the experience of each community was placed within the overall experience of its organization at the national and global levels.

While each story was given importance as a direct testimony of the process at the local level, the collective discussion, selection, and restitution process throughout the organization implied a raising of awareness at the local, national, and international levels on effective experiences in furthering the right to health.

The group discussions were led by an experienced local facilitator. The facilitator asked participants to tell stories about their personal involvement with activism, focusing on the changes they experienced in their personal role(s), as well as any new capacities they developed. Participants were also asked, in telling their stories, to emphasize the impact of any individual or organizational support they received and the role that it played in these changes. They were asked to relate their stories in a personal manner—in other words, “I” and “me,” as opposed to speaking on behalf of a group or making general assessments of changes that took place. It was hoped that these insights would offer a better understanding of unexpected outcomes or “veiled” effects in the personal educational and empowering processes, providing information about the “molecular” level of empowerment—that is, at the individual and community level—and revealing not only positive but also difficult or uncertain experiences. At the international level, the MSC process was expected to facilitate the exchange of information and experiences between partner organizations and to document actions and results useful for communication among countries.

In 2010, TWHA tested the method with a number of local partners, with encouraging results. Remaining questions and potential improvements were highlighted (for example, it became clear that partners’ understanding and concrete application of MSC varied and had to be streamlined). This preliminary test-drive was essential for building a broader perspective of the use of MSC, for it offered insight into how TWHA should accompany its local partners in this exercise. In addition, observations provided by external experts allowed TWHA to clarify its expectations and to optimize the whole process. Both the test-drive and these external observations allowed TWHA to draft guidelines on how to systematize and harmonize the MSC process among its local partners. From 2011 onward, it developed and applied a more systematic MSC monitoring strategy, and in 2012 and 2013, it conducted field exercises.

Results

TWHA and its partners used the MSC technique in the Philippines, Palestine, the DRC, and in El Salvador. In the Philippines, participating organizations included Gabriela, the Council for Health and Development, and Advocates for Community Health. In the DRC, they included Etoile du Sud. In Palestine, they included DRC Health Work Committees. And in El Salvador, participating organizations included the Citizens’ Alliance against the Privatization of Health and the National Health Forum. The examples discussed below do not represent a comprehensive evaluation of the mentioned interventions. (A comprehensive evaluation should include a selection of stories from all organizational levels and a discussion of their role in affecting the organization’s planning at each level.) Instead, in this paper, we try to illustrate the way MSC has been used and the type of results it offers. We present stories from different places, organizations, and periods to illustrate the method’s usefulness for evaluating processes and qualitative outcomes that cannot be put in figures.

The following presentation of the stories is organized along four domains: technical training, personal development, strengthening the link with the community, and organizational development. Most of the testimonies placed emphasis on one domain, while linking it to others. In the discussion after each testimony, we sometimes add contextual information to better illuminate the ongoing organizational process. This contextual information is based on our own personal experiences with these projects.

Technical training

Advocates for Community Health is active in remote regions of the Philippines, where people’s access to health care is limited. Malaria and other tropical diseases like schistosomiasis and filariasis are still highly prevalent. Fe, an MSC workshop participant, recounted the changes that she experienced:

I have learned many things from the seminars given by Advocates. We were taught how to determine different types of illnesses, especially malaria. Because of the training, I was able to help a child suffering from malaria. At first, the family did not know that the child had malaria. When I saw the child, I immediately advised his mother to go to the barangay microscopist because I was sure that all the symptoms of malaria were experienced by the child. After the blood examination, it was found out that indeed the child was suffering from malaria. The mother thanked me for that. I was also thankful because the trainings made me help others. Because of Advocates, I have experienced big changes in myself. How I wish that I learned all these things earlier, when my children were still little. In fact, all of us in the family suffered from different illnesses, but now because of the trainings, no one is suffering from any illness anymore. My trainings on herbal medicines have helped a lot.

The most important change in our organization is that all of our members are actively involved in health services in our barangay. In fact, when a medical mission was held in the barangay, we were asked by our barangay captain to facilitate the activity. All of our members were there to help. Some were tasked to get the blood pressures of the patients, others were assigned in the registration and distribution of foods. We were all very happy at that time because we felt that we were like brothers and sisters, ready to help each other and share whatever we can. After the medical mission, our barangay captain personally thanked us for the voluntary work that we did. This experience showed how far our organization has gone already. Our members show strong commitment to make our organization more stronger. I hope that in the coming months we can recruit more members.8

Fe’s story illustrates the first step of empowerment in a community. When staff from the Advocates for Community Health arrive in a village, they invite residents to participate in trainings on how to prevent and diagnose parasitic diseases. The trainees, mostly women, become health workers who practice in their community and sometimes even in neighboring areas. Their opinion is sought out and respected. When they are unable to treat a patient, they refer the patient to the nearest hospital, where they try to facilitate the patient’s entrance and treatment.

The second part of Fe’s story tells us more about the organization she is part of. Her involvement is high, and she is proud to be part of a community that helps fellow residents. The technical skills she acquired helped her gain confidence and find a place in her community.

Fe’s story was collected at the beginning of TWHA’s three-year program. Because MSC stories are collected every year in the same community, we could observe that the health workers in Fe’s village, Simpokan, are now united against injustices they face on a daily basis. They are not just substituting for the government’s negligence but also reinforcing the people’s organization of their community and using it as leverage in their discussions with local government officials. United, they have an impact on their community by negotiating more funding for their local clinic, securing medicines to treat malaria, managing a communal herbal garden (for herbal medicines), discussing the privatization of their local hospital with local authorities, and denouncing the harmful effects of large-scale mining projects in their regions.

Personal development

Formed in the late 1980s as a grassroots organization of health professionals providing health services to Palestinian people in need of care, Health Work Committees is active in Jerusalem and the West Bank. More recently, it has also begun to work with youth in response to the gradual demoralization among this population. Khaled is a university student in Jerusalem volunteering with Health Work Committees:

Before, I was very shy. I had only two friends at the university. During the training I was leaving each session with five to six new friends, we were finishing the training and going out as a group. We now have an active social life. The activities that influenced me most were those related to communication and group building. They helped me not to be shy anymore. The documentation, training, and field visits helped me to understand the real situation in which we live, and how to have critical thinking on what we are hearing on the news or elsewhere. Now I am not shy anymore, I can express my opinion freely anywhere. The training strengthened my personality, and I became more active in the university and joined the election of the student council. Now I’m the head of the student council.9

Khaled’s testimony shows the positive impact that the Health Work Committees had on his self-esteem, communication skills, and sense of responsibility. His story is representative of the organization’s efforts with youth. After Health Work Committees worked for three years with Khaled and his peers, the youth decided to create their own movement (which still enjoyed support from the organization’s youth coordinators). Their movement, known as Tawasul, is composed of 17 local groups in Jerusalem. Leaders from each of these groups meet every month to strategize. Tawasul organizes workshops and shares information with youngsters on different subjects, including Palestinian history, communication skills, life skills, violence against women, children in prison, and political prisoners. It also builds bridges between the Palestinian youth of Jerusalem, the West Bank, refugee camps, and Gaza. The Palestinian identity is indeed divided because of the walls and the difficulties in traveling between the highly fragmented Palestinian territories, but sharing the same goal in life helps them find the courage to fight for their rights.

In the Philippines, Gabriela is a women’s mass movement that has been active for 30 years. Many poor urban areas do not yet have people’s organizations. In these circumstances, Gabriela organizes free medical consultations for the community, which, in addition to attending to residents’ health needs, allow Gabriela to become familiar with the population and identify those residents who might be interested in becoming health workers. Gabriela then organizes training sessions to develop the skills of these future health workers, who frequently go on to form health committees. These committees often play an important role in people’s organizations that represent the community. Emma is one of these residents who was trained as a health worker:

The campaign that we are leading has trained me how to become a leader. What I needed to improve in my person is to have more self-confidence. Through the organization, I was able to deal with other powerful people (those in the government). I became courageous. As a woman leader, I was able to help in the formation of fisher folk organization in our community. I also learned to speak out in meetings to express my views and standpoint.10

Emma’s story underlines that empowerment is also a matter of courage. She is now able to debate with local politicians and to defend the interests of her community.

Strengthening the link with the community

Active in Kinshasa, Lubumbashi, and Goma in the DRC, Etoile du Sud seeks to organize local populations around the right to health. The MSC stories collected in these cities revealed a sense of fatalism among communities: “You never know whether water and electricity are coming or not, you never know.”11 The concept of rights is foreign to them, but Etoile du Sud is gradually transforming this attitude by working on issues that are essential to communities, like hygiene and water, and by placing these issues in an explicit human rights frame. This, in turn, serves as a basis for community organizing toward claiming these rights. Fidel is now an active member of Etoile du Sud in Kinshasa:

It was after training sessions for health brigadiers organized by Etoile du Sud in our neighborhood that I began to realize the work they were doing on the ground, when they went door to door to collect household garbage. It appealed to my conscience above all by the fact that these young people were doing this work as volunteers. It started to bother me not to clean my parcel and since then, it has become my everyday concern. 

In our people’s health committee (PHC), we have a lack of clean water, erosion is threatening the neighborhood, and there is a lot of insecurity. But after various sessions on self-management initiated by the PHC in our community, erosion control was clearly identified as the biggest concern. The population felt powerless in front of the threatening. But now my neighborhood, and especially members of the PHC, mobilized as one man to try to stop this scourge that threatens our neighborhood.

They organized themselves, some brought empty bags, others went for sand, while others provided labor, while mobilizing for a more permanent solution by the authorities. It is reassuring to see the population adopting a different behavior towards the passivity that characterized us.12

In Fidel’s story, the mobilization on cleanliness provided a good entry point for broader change in the community. His testimony shows the potential for change at both the personal and the community levels. His story evolves from one of indifference to one of sustained engagement. The volunteer work is a trigger for this change. As Fidel’s story reveals, the ideological struggle against “each for his own” is important and can be a guiding thread in the actions and training of activists, not only with regard to their personal development but also with regard to community organizing.

Fidel’s story also illustrates the different steps of empowerment: first, identify the problems; second, take action at the local level to solve some of the most urgent problems; and third, analyze, as a group, the root cause of the problems and address them as an organization.

Organizational strengthening

In the Philippines, Inday recounted:

Last year, I didn’t know yet how to help or facilitate women who are abused, but now I already know how and what is needed to be done to be able to fight for the rights of the people.

In the past year, I have especially grown in my outlook toward society. I have grown in the change of my personal attitudes and my views on my personal ambitions in relation to the situation as a whole. In particular, I have grown as an activist, more determined to organize communities and to support change. 

The growing strength of Gabriela has helped me grow and vice versa. From a part timer I have become a full timer, from an individualist I now serve the people. This is my most significant change. 

The most significant change that I saw is the organization of the people, and the better understanding of the people of their present situation and why we need to act on common interests. Gabriela is truthful with helping the people in need.13

As mentioned earlier, Gabriela is a women’s organization that defends the rights of women in poor urban and rural communities. Inday’s story illustrates what an organization is capable of achieving as a united actor. People in poor communities usually lack the opportunity to claim their rights before local authorities. Through an organization that represents their community, they have the chance to be heard by these authorities. Inday realized that people have common interests and that the best way to defend these interests and rights is to organize people.

The People’s Health Movement in Latin America, instead of using MSC directly as an evaluation method for its own work, promoted the method to strengthen the monitoring and evaluation capacity of the Foro Nacional de Salud (National Health Forum) in El Salvador. The National Health Forum is a social and community movement resulting from 12 years of struggle by various social and community organizations to ensure the right to health. Created in 2010 after the election victory of the Farabundo Martí National Liberation Front, it acts as an interlocutor between civil society and the Ministry of Health. With the assistance of an external expert, the National Health Forum organized an MSC training workshop in October 2011 to help its facilitators learn how to use the method in their communities. The workshop was set up as a practical exercise. Using MSC for collective reflection about health reform and the national mobilization campaign for a new medicines law, the organization came to the conclusion that while it had carried out many protests, their political and social impact had been limited. Although the organization’s mobilization skills were strong, its analyses were evaluated as too superficial. The MSC also revealed the National Health Forum’s lack of unified action proposals within an integrated global strategy.

The movement decided to concentrate on a campaign for the transparent and efficient use of public finances and the adequate financing of social needs. The demand for fiscal reform became central to nationwide mobilization in the communities. This unified action and organization-building eventually played a role in mobilization for the presidential elections of March 2014, in which the people’s movement campaigned for the popular Farabundo Martí National Liberation Front, more than 20 years after the peace agreements of 1992.14

The report that the National Health Forum submitted to TWHA contained information that had already been processed, and it presented the most significant changes by theme discussed. TWHA was unable to obtain copies of the written testimonies or information on the empowerment process within the organization.

Nevertheless, even though the illustration of the use of MSC by the National Health Forum did not correspond to what had been assigned, we consider this case worth mentioning because the method continued to be used to guide meetings in which the National Health Forum reflected internally on its work with community leaders. Our partner, the People’s Health Movement, reported that the MSC workshops had greatly benefitted the National Health Forum’s organizational strengthening.

Discussion

Use of the methodology in the different regions

Between 2010 and 2013, most of our partners thoroughly applied the MSC method, gradually increasing their appropriation of the technique. Nevertheless, implementation was diverse.

In the Philippines, the method was applied during three consecutive years in the same local branches of three partners (Gabriela, the Council for Health and Development, and Advocates for Community Health), always in a well-organized branch and in one facing difficulties with progressing. While concentrating on changes from the previous year, the consecutive exercises showed evolutions over a longer period. They also permitted facilitators and participants to fully appropriate the method. The partner organizations considered MSC to be a successful tool for their learning process, stimulating them to collectively analyze their commitments and roles. For the local staff, MSC became an exercise in listening, which revealed significant and diverse viewpoints and showed them concrete changes in the lives of community activists. In the Philippines, we see a clear evolution in the content of the MSCs over time, from changes at the personal level to evolutions in community organization to the strengthening of the people’s organizations.

In the DRC, the MSC technique was used mainly as a qualitative evaluation method. Notwithstanding Etoile du Sud’s increasing experience over time, the organization required continued external support to ensure further improvements. While Etoile du Sud remained doubtful of the usefulness of this method for its internal monitoring, the stories collected were nice illustrations of the results of the organization’s activities in Kinshasa’s neighborhoods.

In Palestine, the method monitored the Health Work Committees’ youth program in 2012 and 2013. Participating youth testified that the exercise was not easy for them. Their difficulties in expressing themselves might be linked to cultural elements: “We don’t like to talk about ourselves.” Nevertheless, participants were positive about having been triggered to explore their commitments and roles within the movement.

Applying MSC in Latin America was difficult. Since the People’s Health Movement is a network and not an organization, it was not possible to use MSC at the same intermediary level between the organization’s staff and the community. The movement’s partner organizations were motivated to invest time in evaluating mainly their own actions, not those of the network, as illustrated in our description of the National Health Forum in El Salvador. At the same time, however, the People’s Health Movement promoted the technique to strengthen its own monitoring of the local organizations.

Finally, for TWHA and its partners, MSC stories are useful not only for qualitative monitoring and evaluation but also for exchanging experiences among partners, for collective planning, for accountability reports to institutional and individual funders, and for dissemination to the general public. The built-in short feedback loops encourage self-evaluation and therefore contribute to capacity development.

Collection process

The process of collecting and selecting MSC stories is time-consuming and requires human and material resources. At the same time, it is, in itself, a process of group reflection and capacity-building. Before the storytelling starts, it is essential to discuss and identify the domains of change that should be documented.

Nevertheless, we did not succeed in achieving a uniform application of the methodology. In some cases, the qualitative monitoring received more attention, while in others capacity-building was at the center of the exercise. But if results of a qualitative evaluation are to be used mainly to serve the needs of the local organization in improving its functioning, we should accept that one or some objectives may receive more attention than others in a certain phase of organizational development.

Initial MSC exercises should be set up on a small scale and, if possible, with the help of an experienced facilitator. This helps ensure that the concepts and methods become known by different staff members, who can then facilitate the exercise within the organization. The training of facilitators aims to strengthen their listening skills during interviews and their ability to help storytellers better develop their stories by asking the right questions.

Facilitators can be local staff, volunteers, or leaders of the organization at the national level. They may also be hired externally. In Palestine, internal facilitation was used because it was seen as supporting capacity-building within the team. In Latin America, the first option was to invite an external facilitator, after which trained members of the organization took over. In the DRC, after the first try with an external facilitator was not convincing, a training of local facilitators was set up. In the Philippines, which had the best results, facilitation was conducted by the TWHA country coordinator, who was trusted by local groups but also sufficiently removed to be able to accompany the process as an outsider. These experiences show that sufficient flexibility should be embraced in order to find the most effective arrangement in terms of the facilitator’s skills and knowledge of the method, as well as in terms of participants’ trust in the facilitator.

The groups from which stories were collected varied: in the Philippines, 10 people were brought together for focus group discussions; in Palestine, smaller groups were used; and in the DRC, stories were collected through individual interviews. Proper instruction of participants is paramount to ensuring a smooth process. In newly organized communities in the Philippines, MSC participants tended to select the stories of their leaders as the “most significant.” In this initial stage of community empowerment, such a leader-oriented tendency is strong (especially with leaders who are articulate and confident). To help counteract this bias, “blind” sharing and discussion was organized for future sessions, in which numbers instead of names were used to identify the authors of each story. This proved a good adjustment to the MSC exercise.

During the presentation of the stories, there were times that a listener would ask questions or add an anecdote to confirm the story while the teller was talking. While the listener might seem to interrupt the teller, it never disrupted the process; instead, it made the sharing more intimate and interesting. Such sharing was helpful in gathering more information about the strengths and weaknesses of programs, which do not always surface in formal assessments. Nevertheless, flexibility is needed to adapt to different socio-cultural circumstances. In the DRC, for example, some women had difficulties relating their experiences in front of a group, particularly if men were present. Therefore, Etoile du Sud decided to conduct individual interviews in which a woman was the facilitator. In Palestine, youth were initially reluctant to talk about themselves. However, during the exercise, their attitude evolved, leading to growing openness. At the end of the process, some of the participants were even willing to participate in a small video compilation on their significant change stories.

Documentation is a complex part of the MSC process. In the different approaches that were tried, participants revealed much more than they put on paper, even if well prepared. Writing is a difficult exercise. “The ball-pen weighs heavier than the plough,” as most peasants would say. Even when someone helps write down the story, the problem is not completely solved. At the time of sharing, much more is told; participants inspire one another. Therefore, it is important to help the storyteller to complete her or his story based on new elements that were expressed during the sharing. An alternative method—not used in our experience—could be to record this moment of sharing to fully document the stories (with the storyteller’s direct participation and agreement).

Moreover, certain stories required additional exploration in order to better understand how change occurred and how the change evolved.

By applying the MSC method each year in the same groups (with, if possible, the same participants), we could analyze the evolution of those groups through the information highlighted in their stories. This allowed changes over a longer period of time to become clear.

Evolution in the stories

The stories give an indication, particularly in the Philippines, of how these programs have effected changes over time among the members of people’s organizations. Remarkably, we have observed an increased level of awareness and commitment among the participants, which we had not observed or measured using the standard evaluations.

In the first two years, participants testified mainly on how these programs had become relevant in their individual lives—in other words, how the programs affected their sense of place within the community. By 2013, participants had increasingly shifted their stories from “how the programs helped them” and “what they could do to help others benefit from the programs” to “what they could do to help their organizations.” Different local activists explicitly expressed the process they went through and summarized in their stories the gains they achieved within themselves, the community, and the organization.

Comparison to other experiences 

We found one other MSC-evaluation document, produced by the Belgian organization COTA.15 The document’s analysis confirms the advantages of MSC as a flexible and adaptable method that ensures the full participation of stakeholders. The technique can be used for different purposes: monitoring, internal and external evaluation, communication, and so on. The document also notes that MSC allows for the identification of personal (and group) evolutions that are considered to be important by the actors themselves. Therefore, MSC is relevant for describing changes and evolutions in perception and behavior. In terms of the advantages of MSC, the main difference between our analysis and COTA’s document seems to be that COTA did not observe the important shift in the type of reflections over the years, from personal significant changes to community and organizational changes.

As COTA’s document notes, one of the biggest limitations of MSC is that it can be time-consuming. From a purely technical evaluation perspective, this might be true. But MSC is not merely an evaluation method. Our experience stresses that MSC, as a recurrent method of self-evaluation, can become an empowering instrument for participatory planning and for intermittent evaluation of the empowering process in an organization. In this sense, this time is well-spent. Further, while COTA’s observed risk of subjectivity (selecting only the positive stories) is real, this risk can be alleviated through adequate facilitation. The document also mentions that negative stories might lead to tensions, which in a participatory evaluation might be difficult to handle. In the experience of TWHA and its partners, this issue has never been mentioned. Nevertheless, the Salvadoran story shows how a negative experience, collectively analyzed with this methodology, can give rise to new opportunities for capacity-building.

COTA’s last consideration is that the small number of stories might be exceptions and not representative of the general picture. This underscores the need to combine MSC with quantitative elements and to organize collective discussions of the individual stories, analyzing whether experiences are shared or exceptional. In both cases, lessons can be drawn to understand circumstances and dynamics of success.

Conclusion

This MSC exercise in four different regions of the world enabled TWHA to strengthen its ability to set up an extensive and flexible evaluation process. While the collection and analytical process was adapted to each cultural environment, it followed the same pathway. Also, partner organizations found important points of comparison in their work toward strengthening empowerment around the right to health. The process of MSC implementation over several years, with the participation of the same storytellers, allowed TWHA to observe significant change—and evolving types of change—over time.

As a methodology, MSC is far from perfect and does not intend to be. Constant self-questioning is part of the method. The technique is useful largely as a complement to more traditional methods. MSC is a slow, participatory, and intensive endeavor that builds capacity while being applied. This makes MSC a useful monitoring tool for programs with participatory and empowering objectives. A more extensive literature review comparing MSC with other qualitative methods is necessary to place this experience within a broader research context. This can surely be a topic for further research. Nevertheless, documented experiences of MSC—beyond those referred to in this manuscript—are hard to find.

Acknowledgments

We are grateful to the community members and the staff of local organizations in all countries who involved themselves unwearyingly in this MSC process. The application of this MSC methodology was part of a program funded by the Belgian Development Cooperation.

*Daoud al-Ghoul, director of youth programs for the Health Work Committees in Jerusalem and the Kanaan Network of Palestinian civil society organizations, was arrested by Israeli occupation military forces on June 25, 2015. At the time this issue went to press in mid-November, he had not been formally charged and was still being detained. Al-Ghoul’s unlawful detention violates the Fourth Geneva Convention, the International Covenant on Civil and Political Rights, and the UN Declaration on Human Rights Defenders. In solidarity with our co-author, the authors of this paper demand his immediate release and an end to the targeting of this Palestinian human rights defender.

Fanny Polet, Geraldine Malaise, and Anuschka Mahieu are Project Officers at Third World Health Aid in Brussels, Belgium. 

Eulalia Utrera, MD, is Coordinator of Advocates for Community Health in Manila City, Philippines.

Jovita Montes is a member of the national secretariat of Gabriela in Quezon City, Philippines.

Rosalinda Tablang is Deputy Executive Director of the Council for Health and Development in Quezon City, Philippines.

Andrew Aytin is Policy Officer for Third World Health Aid in Quezon City, Philippines.

Erick Kambale is President of Etoile du Sud in Kinshasa, Democratic Republic of Congo.

Sylvie Luzala is an active member of Etoile du Sud in Kinshasa, Democratic Republic of Congo.

Daoud Al-Ghoul* is Director of Youth Programs for the Health Work Committees in Jerusalem and the Kanaan Network of Palestinian civil society organizations.

Ranin Ahed Darkhawaja is a member of youth programs for the Health Work Committees in Jerusalem.

Roxana María Rodríguez is planning, monitoring, and evaluation consultant in San Salvador, El Salvador. 

Margarita Posada is Coordinator of the National Health Forum and at the Citizens’ Alliance against the Privatization of Health, El Salvador. 

Wim De Ceukelaire, MD, is Director of Third World Health Aid in Brussels, Belgium. 

Pol De Vos, MD, PhD, is Researcher and Post-Graduate Course Coordinator at the Public Health Department of the Institute of Tropical Medicine in Antwerp, Belgium.

Please address correspondence to Pol De Vos. Email: PDVos@itg.be.

Copyright © 2015 Polet et al. This is an open access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited.

References

  1. P. De Vos, W. De Ceukelaire, G. Malaise, et al., “Health through people’s empowerment: A rights-based approach to participation,” Health and Human Rights Journal 11/1 (2013), pp. 23–35.
  2. Third World Health Aid, Empowerment: United for health (Brussels: Third World Health Aid, 2013).
  3. Third World Health Aid, Synthèse du programme 2011-2013 de Fonds de Soutien Tiers Monde, “Empowerment pour le droit à la santé” (Brussels: Third World Health Aid, 2011).
  4. Third World HealthAid, La santé, une question de lutte contre l’injustice (Brussels: Third World Health Aid, 2011).
  5. Third World Health Aid, La technique du changement le plus significatif, suivi de, La responsabilisation pour le droit à la santé (Brussels: Third World Health Aid and the General Direction of Developement Cooperation, 2010).
  6. J. Lennie, The most significant change technique: A manual for M&E staff and others at Equal Access (February 2011).

    Available at http://betterevaluation.org/sites/default/files/EA_PM%26E_toolkit_MSC_manual_for_publication.pdf.
  7. R. Davies and J. Dart, The “most significant change” (MSC) technique: A guide to its use (April 2005). Available at www.mande.co.uk/docs/MSCGuide.pdf.
  8. Fe T., MSC session in Palawan, Philippines, December 2011.
  9. Khaled E., MSC session in Jerusalem, Palestine, October 2012.
  10. Emma P., MSC session in Roxas, Philippines, November 2012.
  11. Fidel T., MSC session in Kinshasa, Democratic Republic of Congo, January 2013.
  12. Ibid.
  13. Inday G., MSC session in General Santos, Philippines, November 2012.
  14. J. Partlow, “Former guerrilla commander wins El Salvador presidential election,” Washington Post, (March 14, 2014).
  15. N. Wilbeaux, Technique du changement le plus significatif (Brussels: COTA, 2007). Available at http://www.cota.be/download/fo-apc/fiche9_gcp_mostsignificantchange.pdf. See also Third World Health Aid, Technique du most significant change dans le programme 2011–2013 (September 2011); J. Vanden Balck, “Récit d’appropriation de la technique du Changement le Plus Significatif. L’expérience de FSTM,” L’écho du COTA (2012), pp. 3–7. Available at www.cota.be/exaequo/?p=122.