Looking for Evidence of the Impact of Introducing a Human Rights-Based Approach in Health: The SaluDerecho Experience

María-Luisa Escobar, Leonardo Cubillos, and Roberto Iunes

Health and Human Rights 17/2
Published December 10, 2015

Abstract

This paper summarizes the background, methodology, results, and lessons learned from SaluDerecho, the Initiative on Priority Setting, Equity and Constitutional Mandates in Health. Originally facilitated by the capacity-building arm of the World Bank in 2010, it was implemented in Latin American countries and later expanded to other regions of the world. Segmentation, decentralization, and lack of coordination in health systems; weak information systems; stratified societies; and hierarchical power relations in participating countries are some of the characteristics that inhibit a human rights-based approach to health. Hence, deliberate interventions like SaluDerecho are vital. Facilitating the participation of multiple stakeholders in a more informed and transparent dialogue creates a “safe” working environment to co-create policy solutions to improve transparency and accountability. The proposed evaluation methodology involves several steps that begin with an assessment of behavioral changes in actors (including policy makers, citizens, payers, and health care providers) that reshape relationships and, over time, change the functioning of health systems. Despite certain limitations, SaluDerecho has provided evidence of positive change among participating countries.

Introduction

The experience discussed in this paper was born of the identification of latent forces in eight Latin American countries in their effort to achieve universal health coverage. When operationalizing the right to health, countries face technical, political, and socio-economic challenges in establishing legitimate priorities in health. Multiple stakeholders with conflicting positions on how to fulfill the right to health, as well as limited resources and rapid technological and demographic changes, make solutions complex. The ongoing process of realigning these forces to enhance their transformational power became today’s Initiative on Priority Setting, Equity and Constitutional Mandates in Health, also known as SaluDerecho.

International human rights law recognizes that the right to health extends beyond health care to include the underlying determinants of health.¹ Health status is not only the result of health care services that are available, accessible (including affordable), culturally acceptable, and of high quality but also the result of other determinants not necessarily under the realm of health system policies—like education, clean air, water and sanitation, and income.² SaluDerecho deals only with health care. For the purpose of this paper, we define a human rights-based approach (HRBA) as being based on seven “rights-based functional principles”: participation, accountability, non-discrimination, transparency, human dignity, (citizen) empowerment, and rule of law. These principles guide the behavior of health system actors and influence the processes and structures of health systems. Transparent, evidence-based policies and programs resulting from more inclusive processes and greater citizen participation are at the core of an acceptable and effective social contract.

Many Latin American health systems are characterized by segmentation, decentralization, weak information systems, and poor coordination.³ Additionally, powerful interest groups within these often hierarchical societies reinforce mistrust and a lack of communication among stakeholders. Often, systems are organized and ruled by processes that tend to perpetuate the absence of participation, accountability, and transparency.⁴ Inefficiencies in health care delivery, budget restrictions, and some citizens demanding high-cost and non-prioritized health care services are commonly found.⁵ In such an environment, an HRBA will not occur spontaneously, even when national constitutions recognize and protect the right to health.⁶ A deliberate effort is necessary to implement an HRBA.

Carrying out an HRBA in practice implies that health system actors learn to use rights-based principles in the design and implementation of health policy. Sustained and targeted interventions are required to change processes, the behavior of stakeholders, and the relationships among them. Members of government, doctors, insurers, patients and citizens need to learn to work together to develop ways of achieving more participation, transparency, and accountability.

SaluDerecho originated in Latin America as a multi-stakeholder, highly participatory process around the judicialization of health—a complex problem where human rights, health system financing and policies, markets, and politics interconnect. SaluDerecho started by identifying the conflict areas among key stakeholders, understanding the causes of judicialization, and developing mutual trust. Consequently, a “safe space” for multi-stakeholders to collaborate, learn, and co-create and implement potential solutions was built. This represented a long and complex process of not only sharing knowledge and experiences but creating national and regional coalitions among a range of stakeholders seeking organizational and institutional reforms. Two years after SaluDerecho’s creation, countries in Africa and Eastern Europe joined the discussion around inefficiencies in service delivery, prioritization and health benefits plans, participation, and transparency and accountability in policy design and implementation.

Although the ultimate goal of an HRBA is to achieve improved health outcomes, this impact is not direct (see Figure 1). In reality, human rights-based policies materialize in health care that is more available, accessible (including affordable), culturally acceptable, and of high quality, which, in turn, improves population health (assuming that other health determinants do not change). This subtlety provides guidance on how to evaluate the impact of carrying out an HRBA. In this paper, we discuss the method used thus far to assess the impact of SaluDerecho and argue that an evaluation of the impact of an HRBA should be pursued in phases.

This paper is divided into five sections. Following this introduction, we summarize SaluDerecho and its context. We then discuss methods for evaluating impact. The subsequent section discusses SaluDerecho’s results, and the final section provides conclusions and lessons learned. The health systems of participating countries and their legal frameworks are beyond the scope of this paper and are discussed elsewhere.⁷

The SaluDerecho initiative: Regional context and background

Argentina, Brazil, Chile, Colombia, Costa Rica, Peru, and Uruguay (and, later, Mexico) were the first countries to become involved in SaluDerecho when it began in 2010. These countries had seen an increase in litigation around access to health care services and supplies. Although health reforms introduced in the 1990s helped expand coverage and reduced out-of-pocket payments, social demands and health needs had grown faster than advances in public policy.⁸ Deficiencies in the flow of information exacerbated a lack of transparency and accountability.⁹ Furthermore, the design of service delivery policies and benefit plans did not follow participatory and transparent processes, and weak technical criteria inhibited the legitimacy of chosen priorities.¹⁰

The countries in question faced the challenge of prioritizing the provision of services in a legitimate way while simultaneously respecting and protecting constitutional rights. The fact that citizens resorted to the courts also indicated inefficiencies in the delivery of promised care. However, the technical complexities of a fair prioritization process were further compounded by the additional pressure on health spending and service delivery capacity generated by the phenomenon of judicialization.¹¹

In these Latin American systems, a fluid multi-stakeholder dialogue was almost nonexistent. Differences in stakeholders’ expectations, interests, and needs were at the core of the problem. For example, while patients demanded interventions not yet prioritized by ministries of health (and therefore not in the official treatment protocols and not funded with public resources), physicians wanted total autonomy regarding medical decisions. Furthermore, the pharmaceutical and medical inputs industries, interested in expanding the market share for their products, lobbied governments to introduce those products as mandatory benefits. Meanwhile, judges had to defend patients’ constitutional right to health, and finally, ministries of finance argued that public resources had to be spent efficiently. The tension generated among these actors when establishing priorities in health kept these stakeholders apart and stubbornly attached to their respective positions—which, while legitimate, often contradicted one another. As a result, not only did stakeholders not communicate with one another, but the tension among them evolved into a polarized political debate in some countries.¹²

SaluDerecho concept and strategy

As an external and neutral party, the World Bank began convening a series of informal discussions with mid-level officials from ministries of health and academics from the region to expand mutual knowledge and understanding of the issues. These dialogues created a needed “safe and neutral space” for multi-stakeholder dialogue, unleashing the potential for collaborative solutions.¹³ This process of progressively transforming the tension among stakeholders into strong relationships is part of SaluDerecho’s theory of change.¹⁴ To do this, SaluDerecho designed and managed a strategy building on synergies among global, regional, country, and online activities. Team-building tools were used to maintain interest and cohesion among the different stakeholders, countries, and fields of knowledge, and to strengthen the role of a regional debate in changing the discourse of the more polarized national debates happening in some countries.

The initiative’s regional strategy was based on the knowledge that Latin American countries faced similar challenges in addressing the right to health and that the incorporation of multiple perspectives would create new opportunities for finding solutions to complex problems; it helped ease the tensions within countries, develop national consensuses, and scale up countries’ reforms. Global activities not only brought state-of-the-art knowledge but provided an international platform for regional and country activities to thrive.

At its core, SaluDerecho is an innovative multidisciplinary initiative that brings together different actors to build consensus and obtain both individual and collective commitment to learning and constructing solutions for action. Some of the most important issues addressed by SaluDerecho include patient waiting-list management, pharmaceutical policy, priority setting and evidence-based medicine, high-cost treatments, orphan diseases, training for judges, and transparency and accountability.

The transformation of a multi-stakeholder dialogue into a multi-stakeholder coalition for action was critical to the process and would be essential to the success of SaluDerecho. Therefore, identifying who would sit at the table was equally important and required stakeholder mapping. In the end, SaluDerecho evolved as a “social lab,” where new practices and knowledge grew and stakeholders collaborated in developing solutions.¹⁵

Supreme and constitutional court judges would be one of the most important parties for ensuring SaluDerecho’s success, both nationally and in the region. However, they would be among the most difficult to bring to the table, for judges do not typically convene with members of other branches of government, academics, or members of civil society to discuss contentious issues on which they might later have to rule.

In 2010, the World Bank’s facilitating team, with help from the Peruvian Constitutional Tribunal president, reached out to the Inter-American Court of Human Rights and won its support—ultimately leading to the convening of a number of court presidents, who met in Costa Rica.¹⁶ These judicial actors became a positive force in promoting high-level multi-stakeholder health policy dialogues at the regional level; a number of them invited their health ministers and other judges, advancing the discussions nationally as well. Some countries established strong multi-stakeholder coalitions that set the agenda, defined priorities, and identified host organizations to leverage resources, convene actors, and coordinate and institutionalize the dialogue process. Later, local and international academics, Ombuds offices, lawyers, superintendents of health, directors of insurance funds, and representatives of patients’ associations joined (see Table 1).

Table 1. SaluDerecho’s multi-level strategy¹⁷

To build even more support for the process, the facilitating team reached out to internationally recognized organizations that were influential in the region or had experience and knowledge in health systems. These organizations included the Pan American Health Organization, the World Health Organization, the Norwegian Agency for Development Cooperation, Reos Partners, the Salzburg Global Seminar, the Cochrane Collaboration, Dartmouth College, McGill University, Harvard University, Chr. Michelsen Institute, and others. Not only did these organizations participate, but they ultimately became change agents for SaluDerecho. Together, they participated in a series of regional, national, and global facilitated activities that resulted in a more informed and transparent multi-stakeholder dialogue, as well as specific country action items. As interest in SaluDerecho grew, so did the participation of other countries, including Albania, Kosovo, Macedonia, Egypt, Kenya, and Rwanda.

An online community was created to support knowledge sharing and systematization, collaborative learning, and coalition-building. The e-Community of Practice (e-CoP), available at www.SaluDerecho.net, offered a virtual space for debate, videos, live webcasts, and a knowledge repository.¹⁸ It facilitated policy dialogue and debate among more than 1,000 intellectual leaders and practitioners from more than 45 countries.

The synergies created by SaluDerecho’s parallel national, regional, global, and virtual activities explain the theory of change behind the initiative: a continuous process that informs, engages, and develops consensus among stakeholders is necessary to ground an HRBA to health policy.¹⁹ The process of change is a complex, dynamic, and non-linear phenomenon constructed and driven by many stakeholders. Figure 2 presents a simplified representation of the main areas critical to this effort.

While SaluDerecho has been a strong and positive force for shaping the way countries deal with the right to health, it faces several limitations. SaluDerecho’s rapid growth has introduced significant pressures on resources available to the facilitating team. Also, its very nature means that SaluDerecho is a process of constant change, making it difficult from the onset to develop and implement a systematic method for collecting and recording data to facilitate the assessment of impact. Since SaluDerecho does not set a specific outcome to be achieved but rather evolves with the country coalition process, the identification of indicators to measure progress is challenging. Finally, country coalitions are exposed to internal political pressures and to volatility resulting from changes in government administrations.

Evidence of impact: Methods

This section discusses methodological issues for assessing the impact of an HRBA and summarizes SaluDerecho’s experience using the Capacity Development Results Framework and the Outcome Harvesting methodology and tools.²⁰

The impact of an HRBA to population health is not direct. Assessing such impact involves conceptual, methodological, and practical challenges that can jeopardize the robustness of any result. The impact of human rights-based policies should be measured by changes in the availability, accessibility (including affordability), acceptability, and quality (AAAQ) of health care services, which—all other things being equal—might improve health outcomes.

We believe that evaluations of the impact of HRBAs should be pursued in phases. The first of these phases should assess the existence of enabling conditions for participation, empowerment, transparency, and accountability in a given health system. This requires identifying conditions that are specific to the type of health system and country characteristics—for example, the existence of information systems and stakeholder knowledge to manage and interpret data, and the existence of cultural barriers preventing stakeholders from convening with agents of different hierarchical rank.

The second phase should assess whether those rights-based principles are actually being used in practice and, if so, how they are being used. For example, it should verify whether and to what extent participatory processes for policy design are truly participatory; whether, in contexts where good information systems and trained personnel are in place, information is actually flowing transparently; and whether existing accountability mechanisms are being effectively used.

The third phase should assess whether resulting policies introduce changes to AAAQ—for example, whether a policy introducing mandatory health insurance for all improves access to (and the quality of) care. Finally, the last phase should evaluate whether the change in AAAQ (all other things being equal) results in better health outcomes—for example, whether more accessible and better quality health services translate into better health status. Each phase may require a different evaluation approach (methods and indicators), and, in all cases, indicators should be tailored accordingly. Methodologically, the most challenging are these last two phases.

SaluDerecho is at a stage between the first and second phases. It uses a results framework developed at the World Bank to monitor and evaluate complex processes of social change, such as the introduction of human rights-based principles into health system functioning.²¹ The Capacity Development Results Framework indicates whether a process of change brings intermediate outcomes, including increased awareness and knowledge, as well as organizational and institutional changes resulting in policies aimed at achieving the ultimate goal. Within this framework, SaluDerehcho used the Outcome Harvesting tool to assess evidence of its impact.²² The tool helps understand the process of change and how outcomes contributed to such change, but it does not look for attribution. Its application is highly participatory and uses a mix of approaches (including stakeholder surveys, online focus groups, country data, opinion polls, monitoring by nongovernmental organizations, and institutional analysis) to gather and analyze data. It identifies flexible indicators and measures local stakeholder behaviors, collaborative actions, and innovations that advance institutional and organizational change. Ricardo Wilson-Grau and Heather Britt explain how the tool differs from other methods:

Outcome Harvesting does not measure progress towards predetermined outcomes or objectives, but rather collects evidence of what has been achieved, and works backwards to determine whether and how the project or intervention contributed to the change.²³

Under this framework, SaluDerecho, as part of a long process of complex change necessary to ground an HRBA, achieves intermediate outcomes supporting the introduction of rights-based policies toward the ultimate goal of improved health. To assess evidence of its impact, it is necessary to have clarity about what SaluDerecho seeks to achieve:

(1) increase stakeholders’ awareness and knowledge about health systems challenges;

(2) increase participation and empowerment so that policies are the result of consensus among all parties involved;

(3) help countries improve transparency so that knowledge and information flows freely and is used for policy design and correction; and

(4) improve accountability while supporting stakeholders’ commitment to change. Table 2 identifies the key rights-based principles associated with each SaluDerecho results area.

Identifying outcome indicators

An outcome is what an individual, group, community, organization, or institution did or is doing that reflects a significant change in their behavior, relationships, actions, or policies.²⁴ Outcomes need to be defined through a process of identifying who did what, when it happened, and how this influenced change. Outcome descriptions are verified by stakeholders and then substantiated by independent individuals to validate the findings.²⁵ Using a customized outcome mapping tool, SaluDerecho mapped its outcomes at two levels for analysis and interpretation: institutional and organizational changes related to policy, and learning and capacity changes related to awareness, knowledge, skills, collaborative action, and innovative solutions.²⁶ Achieving effective multi-stakeholder coalitions, co-creating solutions to improve information flows, and spurring new organizational arrangements are then identified as outcomes.

An example of improved stakeholder awareness can be seen in Costa Rica, where, after further research, patient waiting lists were found to be an important cause of litigation. A series of knowledge-exchange activities on the topic resulted in improved awareness, better knowledge, and joint learning. This then allowed Costa Rica to develop solutions to manage patient waiting lists. The outcome is associated with better transparency and participation.

In addition, several of these countries had institutions (superintendencies, Ombuds offices, and so forth) and legal frameworks that would be more effective if they could benefit from one another’s knowledge and information, and if policies could be a result of a multi-institutional debate.²⁷ However, there were no effective multi-institutional structures for debating policy, and the institutions did not work together to reach consensus for achieving a common goal. The creation of inter-institutional bodies like the mesas nacionales (national working tables) to agree on policy were organizational and institutional changes that resulted in improved transparency, participation, empowerment, and accountability. The Mesa Nacional de Diálogo in Uruguay and the Mesa Nacional de Trabajo in Costa Rica are examples of these.

Moreover, access to specific technical knowledge to inform judicial rulings became more evident to judges through the inter-institutional work resulting from SaluDerecho. This led judges to request an information system that would grant them opportune access to evidence-based medicine. This example of enabling improvements in judicial transparency can be seen in Brazil, Costa Rica, and Uruguay.

Table 2. Rights-based principles associated with the achievement of outcomes in SaluDerecho

* P = participation; E = empowerment; A = accountability; T = transparency; N = non-discrimination; H = human dignity; R = rule of law

Limitations

Although Outcome Harvesting seems an appropriate approach for evaluating SaluDerecho’s impact, the methodology faced many challenges and limitations. The facilitating team had no previous experience with Outcome Harvesting and needed to be trained by specialized consultants who accompanied them throughout the process. The design of the harvest relies on a good definition of outcomes, which requires deep and detailed knowledge of what is happening at the national and regional levels in SaluDerecho. Moreover, determining the significance (to the change process) of an identified outcome requires a deep understanding of each country’s context, as well as how this context related to SaluDerecho’s regional activities.

Further, although outcome definition is validated with stakeholders, the definition of outcomes is not exhaustive, and outcomes can be missed. Validation required the collection of data from countries through various methods. Many independent agents who were contacted for substantiation responded promptly, but others were not readily available for interviews or surveys, making the evaluation process slower than expected. This resulted in a lengthy process that was also accentuated by the evaluation experts not being integrated into the facilitating team from the beginning, meaning that they had to rapidly become familiar with a complex multi-level process of change.

Evaluation phases not pursued by SaluDerecho and methodological issues

The two most challenging stages of assessing the impact of an HRBA are the evaluation of impacts on AAAQ resulting from policy changes, and the evaluation of impact on health status resulting from changes in AAAQ. Neither is appropriate for SaluDerecho. In both cases, it is necessary to isolate the impact of changes in AAAQ from the impact of all other non-health-system-related variables—since health services are only one of the contributors to improvements in health conditions.²⁸

A mix of qualitative and quantitative evaluation methods seems appropriate for determining the impact of policy decisions on AAAQ. Randomized control trials have been used to evaluate the impact of social policies.²⁹ The randomization process and ethical considerations are two challenges faced when implementing this technique. Alternatively, natural or social experiments (naturally occurring treatment and control groups resulting from the implementation of a policy) and “quasi-experiments” (using statistical and econometric techniques like difference-in-differences, propensity score matching, and regression discontinuity) designed to manage unobserved characteristics to isolate effects and control for endogeneity have also been used to evaluate the impact of social policies.³⁰ Besides the challenges of data availability and of the design of appropriate indicators, the evaluation of impacts on health resulting from changes in AAAQ might prove cumbersome, if not theoretically problematic, due to endogeneity.³¹

Endogeneity occurs when the correlation between dependent and independent variables can run both ways or when there are other external factors affecting the causality relationship between them— for example, higher income can be a result of better education, but better education can also result from higher income. Omitted external variables, such as labor opportunities and geography, can also affect income even if education stays the same, while geographic location can also affect the level of education.³² Similarly, the causal relationship between AAAQ and health is likely to run in both directions, making it difficult to determine whether a correlation between AAAQ and health status reflects the effect of AAAQ on health, the effect of health on AAAQ, or the effect of some other attribute, such as socioeconomic status, on both AAAQ and health status.³³

Results

The following discussion summarizes impact. Results observed at the national, regional, and global levels were identified using the Outcome Harvesting methodology. Because of space limitations, most of our examples refer to Costa Rica.³⁴

National

SaluDerecho has played a key role in integrating information from stakeholders and country practices to inform policy options. It expanded stakeholders’awareness and knowledge of the main health system challenges and developed trust among them, thus transforming tension and gridlock into action.³⁵ As a result of multi-stakeholder agreements for action reached in 2011, countries developed national coalitions, achieving greater transparency in information, new institutional arrangements for policy, and training on human rights.³⁶ An example of a previously nonexistent institutional arrangement achieved through SaluDerecho is the Uruguayan Mesa Nacional de Diálogo. It started in 2011 with the invitation by the president of the Supreme Court to the Ministry of Health to discuss health benefits and the right to health. Thereafter, the National Health Fund and academia joined, and the organizational arrangement was formalized. It is now a sustained platform aimed at anchoring inter-institutional policy debate and decision-making on health benefits.

Due to its success in forming an effective coalition, Uruguay became a model for other countries to follow. With Uruguay’s help, Costa Rica was also able to bring together a group of high-level judicial authorities, the minister of health, current and former presidents and directors of the Costa Rican Social Security Institute, and influential academics to form a powerful multi-stakeholder coalition. Costa Rica institutionalized the coalition by creating the previously nonexistent Mesa Nacional de Trabajo, a catalyst for change to address the challenges of fulfilling individuals’ right to health and of achieving universal health coverage.³⁷ Others, including the Ombuds Office, medical associations, pharmacists, lawyers, patients’ groups, and the University of Costa Rica, also became involved and together drove a national reform agenda.

Another example of previously nonexistent organizational forms can be seen in Colombia— home to an enormous volume of health-related litigation and perhaps the most polarized country on the issue—which created a national committee in 2014. The committee’s first task was to coordinate public policy on health benefits. It engaged the minister of health and social protection, the national health superintendent, and the Ombuds Office, and expects to progressively involve other stakeholders.³⁸

The collaboration among the judiciary, executive branch, and academics in Costa Rica gained momentum around the search for solutions to pressing issues. There was increasing discontent around the rising number of judicial actions, and many cases were demanding that the Costa Rican Social Security Institute pay for expensive pharmaceuticals for individual patients with less common diseases.³⁹ This situation, along with the rising costs of pharmaceuticals, was of great concern—indeed, even after the Social Security Institute updated its pharmaceutical formulary and policies, the volume of litigation did not diminish.

Because litigation is a symptom of underlying structural problems in the health system, through research and knowledge-exchange activities, the Mesa Nacional de Trabajo determined that patient waiting lists (inefficiencies in service delivery), and not pharmaceuticals, were the main cause of the continued litigation, and reached out to others in SaluDerecho in search of patient waiting-list management knowledge.⁴⁰ Models from Spain and Sweden were among those presented to the Costa Ricans and to the SaluDerecho community via www.saluderecho.net. All major Costa Rican hospitals joined the debate live via webcast. The working group then developed an action plan to address the most pressing tasks.⁴¹

Besides the management of patient waiting lists, the action plan included the identification of areas where the judiciary and the executive could collaborate to improve transparency in the system. The Costa Rican judiciary conducted a thorough quantitative assessment of health-related rulings, demonstrating to both the Ministry of Health and the judiciary the need to use evidence-based information. As a result, Costa Rican courts started to use evidence-based medicine in their rulings.⁴² Furthermore, Costa Rica joined the Cochrane Collaboration, which granted the judiciary and the Ministry of Health access to a network of 33,000 individuals who provide independent information and evidence on medicines.⁴³

Regional

SaluDerecho’s regional work allowed countries to be influenced by their neighbors, to advance reforms more quickly, and to scale up efforts. Countries now openly share their internal policy discussions, provide feedback to one another on proposed policies, and develop regional solutions to shared challenges.

Given the force of the multi-country collaboration and the similarity in the challenges faced by these countries’ health systems, SaluDerecho created regional platforms for multi-country dialogue, knowledge exchange, and learning.⁴⁴ The process emphasized participation, improved transparency, and created opportunities for countries to be accountable to one another. Countries formed a regional collaborative and identified lines of action, such as the systematization of claims data, capacity-building for judges and health officials, and the creation of a network of researchers.

Inspired by the regional dialogue and country-level discussions, each of the countries has begun to draw on national health data, the experiential knowledge of practitioners, evidence from research, citizen feedback, and global expertise to inform its reform options. Costa Rica became a leading changing force in the region when it adapted Brazil’s experience as reflected in the Cochrane Collaboration, reached a national high-level agreement of using evidence-based medicine to support judicial rulings, and spearheaded a regional debate on judicial accountability via www.saluderecho.net. The use of evidence-based medicine for decisions on benefit plans initially faced intense opposition from stakeholders in the different countries. However, the successful experience of Costa Rica and the series of regional dialogues it put forward promoted the adoption of a regional mechanism for using third-party evidence to support judicial rulings. Brazil, Colombia, Costa Rica, and Uruguay are now using this regional mechanism to improve transparency and accountability in settling health-related litigation.⁴⁵ The executive in Mexico is now using the information to guide priority-setting decisions.

At the regional level, SaluDerecho introduced new regional organizational arrangements around a shared purpose. A regional technical committee was formed in 2013 by officials from different countries to lead SaluDerecho’s agenda of activities for advancing reforms toward universal health coverage. Since 2010, four regional annual multi-stakeholder fora have been held. Each forum was hosted by a different country and focused on a specific topic.

Additionally, in 2014, two disease-specific country groups were formed, offering multi-country collaboration to advance policy solutions. One on mental health, hosted by Peru, offered the first multi-stakeholder dialogue on universal health coverage and mental health. Teams from Brazil, Chile, Colombia, and Peru attended. Patients’ associations were empowered to participate, identify challenges, and learn about new approaches to service delivery and their implications for mental health policy. The other group, a multi-country committee on orphan and rare diseases, influenced the content of Chile’s recently approved congressional bill for sustainable health care for these patients.

Online

Online activity included using social media and SaluDerecho’s website to create a permanent and specialized open forum to sustain an e-CoP, a tool for knowledge sharing, mobilizing stakeholders nationally and internationally, and facilitating their participation in debates and decision making processes. Countries use www.saluderecho.net to organize in-depth discussions on policy issues, write blogs, share public statements, and upload videos and documents. Priority topics include the right to health and judicialization, health systems, prioritization in health, equity in health, and governance and health. For example, Costa Rica used the website to connect with international experts and to mobilize all hospital directors in the country to learn about other countries’ systems for managing patient waiting lists.

The e-CoP is a powerful tool for improving transparency when used to live broadcast, across countries, previously closed policy processes for feedback from health system actors. Examples include the 2012 Colombian public hearing on compliance with sentence T-760 of 2008 (ordering structural changes to the system) and the Uruguayan national dialogue on drugs and high-cost performance. Together, these events drew approximately 900 participants from around the world.

More importantly, this virtual activity is used for just-in-time health policy consultations and for supporting multi-stakeholder coalitions. For example, Colombia requested and received comments from the e-CoP on the draft of its pharmaceutical procurement policy. Brazil shared its clinical guidelines so that other countries could use them as they were formulating their own. Since 2011, the e-CoP has grown to more than 1,000 members from 45 countries, of whom 62% visit the site weekly for discussions and information.⁴⁶

Global

SaluDerecho’s global strategy focuses on building strategic alliances with multilateral organizations, renowned universities, and influential thinkers in order to develop a shared understanding of the use of human rights-based principles to improve health systems, raise awareness, and push the agenda to an international level. For example, stakeholders from Australia, Canada, the Netherlands, Norway, South Africa, the United Kingdom, and the United States, together with those from developing countries, developed a global dialogue around the challenges of operationalizing the right to health. This activity proved crucial in creating an enabling environment that allowed regional and national approaches to thrive. It mobilized needed resources, created political support, and brought state-of-the-art knowledge. Over time, SaluDerecho also engaged teams from Africa, East and South Asia, Eastern Europe, and the Middle East in a series of global dialogues, including symposia with the Salzburg Global Seminar and the Dartmouth Center for Health Care Delivery Science. These set the stage for a global dialogue among donors on operationalizing the right to health and moved the discussion beyond mere projects and programs.

Conclusions and lessons learned

SaluDerecho’s experience shows the power of facilitated dialogue among multiple stakeholders, as well as how regional and global activities can serve as an umbrella for individual countries’ efforts. This multi-level strategy helps countries speed up reforms and alleviates internal resistance to change. Creating a “safe space” for national and regional dialogue demonstrates the power of knowledge as a connector between multidisciplinary actors and as an essential ingredient that allows country and regional coalitions to thrive. SaluDerecho has increased participation, transparency, and accountability among stakeholders, influencing institutional and organizational changes and policy innovation at the national and regional levels. Nonetheless, much more remains to be done.

It is important to establish, from the outset, an evaluation framework and a systematic method for assessing results. Carrying out an HRBA is a long, complex process of change that requires a different approach from the one used to implement pre-designed programs and projects. The process develops its path as it evolves, and thus has implications for how to monitor and evaluate its impact. It is not like evaluating projects with predetermined inputs and expected outcomes.

The systematic evaluation of an HRBA faces information and methodological challenges, even when using a multidisciplinary approach and mixed evaluation methods. The Capacity Development Results Framework seems to be an appropriate framework for assessing SaluDerecho’s impact, as it is a process of change. Meanwhile, the Outcome Harvesting method allows the assessment of impact to be actively immersed in the country’s context and does not prescribe how the process should evolve and what its result should be. It is a participatory process where stakeholders, facilitators, and external parties collaborate to identify, formulate, verify, and make sense of outcomes by collecting evidence of what has been achieved.

The impact of an HRBA on health is not direct. Its assessment benefits from a phased evaluation process in which each phase uses different evaluation methods and indicators. Since countries are at different stages of carrying out HRBAs, the choice of evaluation approaches is context dependent. Evaluating the impact of AAAQ changes on health status needs to consider endogeneity, as methodological challenges can jeopardize the robustness of the results.

Even in middle-income countries in Latin America, multi-stakeholders need a facilitated process to “learn” how to use rights-based principles (participation, accountability, empowerment, transparency, and non-discrimination) in health policy. This process represents the first step toward operationalizing the right to health. In the end, the need for dedicated efforts to “internalize” human rights-based principles should not be overlooked. International support is necessary for this process to evolve successfully in developing countries.

Acknowledgments

The authors wish to thank Jenny Gold for her valuable contributions to systematizing evidence and to all the members of the SaluDerecho community around the world for sharing their knowledge and experiences in creating a process to carry out a rights-based approach to health care.

María-Luisa Escobar is a health economist, independent consultant, and former Health System Practice Manager at the World Bank, Washington, DC, USA.

Leonardo Cubillos is a psychiatry resident at the Geisel School of Medicine at Dartmouth College, Lebanon, NH, and former Senior Health Specialist at the World Bank, Washington, DC, USA.

Roberto Iunes is Senior Health Economist at the World Bank, Washington, DC, USA.

Please address correspondence to María-Luisa Escobar. Email: escobar.marialuisa@gmail.com.

Copyright © 2015 Escobar, Cubillos, and Iunes. This is an open access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited.

Disclaimer: The findings, interpretations, and conclusions expressed in this paper are those of the authors alone and should not be attributed in any manner to the World Bank, its affiliated organizations, or members of its Board of Executive Directors or the countries they represent.

References

1.  L. Bonilla and L. Cubillos, Transparencia y rendición de cuentas: Para hacer público lo que es público, Memorias: CuartoEncuentroLatinoamericanosobreDerechoalaSalud y Sistemas de Salud (Washington, DC: World Bank, 2014),
2.  World Bank Institute, Initiative on Priority Setting, Equity, and Constitutional Mandates on Health, Report: Third Latin American meeting on the right to health and health care systems. June 3 to 5, 2013 Brasilia, Brazil (Washington, DC: World Bank, 2013), p. 9. Available at http:// api.ning.com/files/7Yks3h911A8pyWgDpsFZmts-b8j1ul- VdI E v YC r OW T F k u b X F g x I G 74 B WF uB o 1 O k v- r o V 7 Z L g e 5 e 2 u 1 h 7 3 j H Jz H 4 N 9J u c 7 J G F M h / ThirdLatinAmericanMeetingonRighttohealthBrasilia.pdf.
3. World Bank Institute, Primer encuentro regional (see note 16), pp. 5–6.
4. Ibid., p. 1.
5. Wilson-Grau and Britt (see note 20), p. 1.
6. World Bank Institute, Cases of mapping outcomes: Priority setting and constitutional mandates in health (November 11, 2013). http://api.ning.com/files/I0fvYV0V5N- 4blghFb4zAX7z7dMycMoQGaEGlCrksNh7BZeHpftKmaB- vQZSTeyXtfMH-xn36FaFOsPEMluiOrLVgDPlTGyWlg/ HarvestingOutcomesConstitutionalMandatesinHealth.pdf.
7. S. Otoo, N. Agapitova, and J. Behrens, The capacity development results framework: A strategic and results oriented approach to learning for capacity development (Washington, DC: World Bank, June 2009).
8. R. Wilson-Grau and H. Britt, Outcome harvesting (Cairo: Ford Foundation, 2013); World Bank, Outcome-based learning field guide: Tools to harvest and monitor outcomes and systematically learn from complex projects (Washington, DC: World Bank, 2014). Available at http:// ing-field-guide-tools-to-harvest-and-monitor-outcomesand-systematically-learn-from-complex-projects.
9. S. Funnell and P. Rogers, Purposeful program theory: Effective use of theories of change and logic models (San Francisco: Willey and Sons, 2011).
10. See https://www.youtube.com/user/Saluderecho.
11. Adapted from L. Cubillos, J. Gold, M. Escobar, et al., Multi-stakeholder actions for universal health coverage implementation in Latin American countries: The case of SaluDerecho, Health, Nutrition and Population Discussion Paper (Washington, DC: World Bank, forthcoming).
12. World Bank Institute, Iniciativa Regional Sobre Priorización, Equidad y Mandatos Constitucionales en Salud, Primer encuentro regional sobre derecho a la salud y sistemas de salud: Memorias (Washington, DC: World Bank, 2011). Available at fwrIpky1QtBWqHaQnJa3O*MKE4qqKamGES0wH*gNNs*Bzt K-IPrYpAEeh3b6VlRZW4Gq9tcyRQRsZP3mXsX/PrimerEncuentrosobreDerechoalaSaludySistemasdeSalud.pdf.
13. Z. Hassan, The social labs revolution: A new approach to solving most complex challenges (San Francisco: Berett-Koehler Publishers, 2014).
14. See W. Allen, Using a theory of change to better understand your program (February 28, 2013). Available at http:// www.learningforsustainability.net/sparksforchange/ using-a-theory-of-change-to-better-understand-your-program.
15. M. Bolívar, L. Cubillos, M. L. Escobar, et al., Diálogos construyendo futuro en derechos fundamentales: Una experiencia latinoamericana sobre procesos de cambio en derecho a la salud y políticas públicas (Villa de Leyva, Colombia: World Bank, 2014).
16. Yamin (see note 5), pp. 4–7.
17. Cubillos et al. (see note 5), pp. 393–398.
18. U. Giedion, R. Bitran, and I. Tristao (eds), Health benefit plans in Latin America: A regional comparison (Washington, DC: Inter-American Development Bank, 2014).
19.  Savedoff and Gottret (see note 4).
20.  M. L. Escobar, C. Griffin, and P. Shaw (eds), The impact of health insurance in lowand middle-income countries (Washington, DC: Brookings Institution, 2010), pp. 89–103, 106–121, 155–176; F. Knaul, R. Wong, and H. Arreola (eds), Financing health in Latin America, Volume 1: Household spending and impoverishment (Boston: Global Health Equity Initiative, 2012).
21. S. Pereira and L. Cubillos (eds), Evolución jurisprudencial del derecho a la salud en América Latina (Washington, DC: World Bank, forthcoming); World Bank, Universal Health Coverage Study Series, Argentina: Case No. 12, Brazil: Case No. 2, Chile: Case No. 21, Colombia: Case No. 15, Costa Rica: Case No. 14, Mexico: Case No. 1, Peru: Case No. 11 (Washington, DC: World Bank, 2014).
22. Cubillos et al. (see note 5), Table 1, p. 391.
23. L. Cubillos, M. L. Escobar, S. Pavlovic, et al., “Universal health coverage and litigation in Latin America,” Journal of Health Organization and Management 26/3 (2012), pp. 390–406; A. Yamin, “Promoting equity in health: What role for courts?” Health and Human Rights Journal 2/16 (2014), pp. 1–9.
24. B. Savedoff and P. Gottret (eds), Governing mandatory health insurance: Learning from experience (Washington, DC: World Bank, 2008).
25. T. Dmytraczenko and G. Almeida (eds), Toward universal health coverage and equity in Latin America and the Caribbean: Evidence from selected countries (Washington, DC: World Bank and Pan American Health Organization, 2015).
26. M. Marmot and R. Wilkinson (eds), Social determinants of health (London: Oxford University Press, 2005).
27.  Convention on the Rights of the Child, G.A. Res. 44/25 (1989), Art. 24.
28. Ibid.,pp.4–5.
29. World Bank, Outcome-based learning field guide (see note 20); World Bank Institute (see note 22).
30. Savedoff and Gottret (see note 4).
31. Marmot and Wilkinson (see note 2).
32. P. Gertler, J. Heckman, R. Pinto, et al. “Labor market returns to an early childhood stimulation intervention in Jamaica,” Science 344/6187 (2014), pp. 998–1001; R. Drake, G. Bond, and D. Becke, Individual placement and support: An evidence-based approach to supported employment (London: Oxford University Press, 2012).
33. U. Giedion, E. Alfonso, and Y. Diaz, The impact of universal coverage schemes in the developing world: A review of the existing evidence, Universal Health Coverage Study Series 25 (Washington, DC: World Bank, 2014). Available at http://siteresources.worldbank.org/ H E A L T H N U T R I T IO N A N DP O P U L A T IO N / Images/IMPACTof UHCSchemesinDevelopingCountries-AReviewofExistingEvidence.pdf.
34. H. Levy and D. Meltzer, “The impact of health insurance on health,” Annual Review of Public Health 29 (2008), pp. 399–409.
35. M. Schlotter, G. Schwerdt, and L. Woessmann, Econometric methods for causal evaluation of education policies and practices: A non-technical guide, Institute for the Study of Labor IZA Discussion Paper 4725 (Bonn: Institute for the Study of Labor, 2010).
36. Escobar et al. (see note 8), pp. 13–32; Giedion et al. (see note 30).
37. Cubillos et al. (see note 17).
38. World Bank Institute, Iniciativa Regional Sobre Priorización, Equidad y Mandatos Constitucionales en Salud, Segundo encuentro regional sobre derecho a la salud y sistemas de salud: Memorias (Washington, DC: World Bank, 2011), pp. 15–30. Available at http://api.ning.com/ f i les/GWqVMDLLY W9 pa l SDM 4 NG 2P1SyT0LsvF A k 9 Y 4 S w E T K u I w t i p Te g * P zF E P L 1 I I R l X y f l r C 2 R a Y M K 1 w * 8CF 9 v f T j N X Z b W 3 H d r Q
    
    /MemoriasdelSegundoEncuentroRegionalsobreDerechoalaSaludySistemasdeSalud.pdf.
39. 5.Availableathttp://www.api.ning.com/files/SVf VaVsl- 8W1k0c0drn*Zu6iH1gfE9zKhxQZkfslqn7p2iP3fev9mY9QD- NpbSj1P5mvZNReLg2URwkmKWMFmqpt4lIYyFLhvh/ MemoriasdelCuartoEncuentroCOLOMBIAFINALSD.pdf
40. M. Roberts and M. Reich, “Case study A: Defining an essential medicines list in Sudamerica,” in Roberts and Reich, Pharmaceutical reform: A guide to improving performance and equity (Washington, DC: World Bank, 2011), pp. 201–208.
41. See SaluDerecho, Avanza el diálogo intersectorial sobre las listas de espera en el sistema de salud costarricense. Available at go-intersectorial-sobre-las-listas-de-espera-en-el
42. See SaluDerecho, Listas de espera videoconferencias (Chile, Galicia (España), Uruguay y Suecia) con Costa Rica. Available at ferencias-con-costa-rica; SaluDerecho, III Encuentro costarricense sobre judicialización de la salud y sistemas de salud: San José—Diciembre 1 y 2, 2014. Available at http:// saluderecho.net/page/iii-encuentro-costarricense-sobre-judicializacion-de-la-salud-y-s.
43. Constitutional Chamber of the Supreme Court of Justice, Costa Rica, Exp. 14-002409-0007-00/Res. No. 2014003337 (March 11,2014).
44. See SaluDerecho, Medicina basada en la evidencia vs. medicina basada en la fe, en la autoridad, en la inercia, en la vehemencia, en los intereses. Available at http://www. saluderecho.net/page/medicina-basada-en-la-evidenciavs-medicina-basada-en-la-fe-en-la.
45. Bolívar et al. (see note13).
46. M. Tristan, T. Vreugdenhil, S. Morales, et al., “Improving judicial accountability: Providing EBM training to members of the Supreme Court of Justice of Costa Rica,” Newsletter of the International Society for Evidenced Based Healthcare 17 (2014).
47. Cubillos et al. (see note17).