
Sign it
Search
EXAME REVIEW
MIT comes to help Brazil evaluate the impact of public policies
Poverty Laboratory of the American university MIT begins its projects in Brazil, to disseminate the culture of evaluation of government programs
By Daniel Barros
Access_time 16 Mar 2017, 05:55 AM - Updated 16 Mar 2017, 16h37



 Chat_bubble_outline
Fisherman in the Amazon: The number of people bene ted by social programs does not attest to its success (FLAVIO CANALONGA / VEJA.com)
Cambridge, Mass. - Over the past few years, the government has launched a series of programs to tackle the country's social problems. One example is
Bolsa Familia , created in 2003 and considered the largest income transfer program in the world, with 14 million low-income families bene ted. From
time to time, the government usually publishes balance sheets on this and other programs, showing the number of people served and the amount of
resources applied.
More rare are the assessments of the real effects of public policies - not only for program bene ciaries but also for society as a whole. The Ministry of
Social Development, which is responsible for Bolsa Família, even has a secretariat that handles evaluations, but its focus has been on determining the
extent of programs and ensuring that families are receiving the bene t.
Inattention with a deeper appreciation of the money invested is not exclusive to the federal government. City Hall and state governments also confuse
impact assessment with measuring the coverage of a policy.
"Our politicians have mania for greatness. They think that saying how many millions of people are served by a program is enough to show the success of
Signof
it
the
of a culture
initiative, "says economist Cláudio Ferraz, a professor at the Ponti cal Catholic University of Rio de Janeiro and a critic of the absence
Search
public policy evaluation in Brazil. The beginning of the municipal administrations and the moment of transition by the federal government create an
opportunity to begin to change this culture. And there are new partners willing to help in this task.
In July 2015, the Massachusetts Institute of Technology (MIT), one of the most prestigious universities in the world, opened in Rio de Janeiro a unit of its
Laboratory of Poverty, a study center that disseminates rigorous assessments of the impact of public policies. The MIT laboratory was created in 2003
by France's Esther Du o and India's Abhijit Banerjee, both economists with a solid reputation in the academic world. For economists (and also a couple),
the measurement of the impact of social policies should follow the same rigor, for example, of a pharmaceutical industry that evaluates whether a drug
works and has side effects.
In 2005, the MIT center received a millionaire donations from a Saudi tycoon and changed the name to Abdul Latif Jameel Poverty Action Lab - or simply
J-Pal - and was able to quickly expand the performance. It has supported 800 studies of 143 associated researchers from 49 universities, including the
most prestigious universities in the world. "In Brazil, there have been a number of collaborative talks in the past, but in general it has been dif cult to get
government support for rigorous evaluations," says Ferraz, of PUC-Rio, who is one of the researchers associated with J-Pal.
It was only in the middle of last year that J-Pal was nally able to convince the federal government to play policy evaluation projects. The real work,
therefore, has just begun. Today, J-Pal has researchers involved in two fronts of work.
One of them is the evaluation of the impact of the Bolsa Verde, a program of the Ministry of the Environment that complements the Family Grant for
50,000 families in environmental conservation regions, at an annual cost of 100 million reais. Families involved in extractive activities receive 1 200 reais
a year to conserve natural resources, such as forests. "The purpose of the evaluation is to determine whether and what eligibility criteria for the program
should be changed," says Mauro Pires, director of extractivism in the ministry. "The results can be vital to the future of the Green Exchange."
J-Pal is also developing courses for public managers through the National School of Public Administration (Enap). The intention is to create a culture of
evaluation and use of these surveys to improve policies. "We want to not only evaluate the policies but also use the results to make decisions about
maintenance, deepening, restructuring or discontinuity of the programs," says Francisco Gaetani, president of Enap.
J-Pal follows a methodology called "social experiments". The idea is to measure the impact of a policy by separating the analyzed sample into groups
with similar socioeconomic characteristics, but with one essential difference: one receives the bene t and another does not. They are, in the jargon of the
researchers, the control and treatment groups, which then have their results compared.
In the case of Bolsa Verde, since the program only covers part of the potential public, there are families that do not receive the bene t and can then be
used as a basis for comparison with the ones they receive. Thus, researchers can establish - or not - causal relationship between the program and
improvements in social indicators. The methodology causes controversy. "Politicians and public managers fear that it is not easy to explain to the
population why they can not take a program at all," says Banerjee. "I feel that many policymakers are attached to the programs they design and do not
want to evaluate them, for fear the results will show that changes need to be made," says Esther.
Controversies aside, J-Pal research has had a signi cant effect on how to reduce poverty in the world. In India, anti- corruption policies have gained
momentum after evidence that a new control system could reduce 24% of the diversion of resources to public services, such as health, education and
sanitation. The survey measured interventions in the state of Bihar that reduced the cost without compromising the quality or coverage of services.
But politics was strenuously opposed by the local bureaucracy - so much so that it was extinguished before the results of the research were released. It
then had to be taken up again by popular pressure and copied by other Indian states. In Indonesia, the main food subsidy program had serious
implementation problems, detected in social experiments. Some adaptations, such as the creation of a card that the bene ciary could identify, have
increased the use of the bene t by more than 25%.
Much of this evidence was told in the book Economics of the Poor, published in 2011 by Abhijit Banerjee and Esther Du o and voted the book of the year
by the Financial Times. Esther became a star among development economists, especially after receiving the John Bates Clark Medal Award in 2010,
given to the brightest economists under the age of 40 - an honor that makes the academy consider it likely that both will share a Nobel Prize in
economics. Economy in the future.
Leaving late
The push of a renowned research center, such as J-Pal, can help create a culture of public policy evaluation in Brazil. But it will not work by itself. After all,
when the theme is to measure the impact of policies and adapt them based on the results of the surveys, Brazil is well behind schedule. Developed
countries, such as Canada and Belgium, ensure policy evaluations by law. Colombia has a body that helps other ministries to develop and use policy
impact studies - it's called Synergy.
But few examples are as inspiring as Mexico's. In 2005, the country created Coneval, a national council for assessing social development policies. This
body also conducts research and assists public managers in the task of thinking the evaluation from the initial design of the pilots of a program. The
practice began even in the 1990s, when the government headed by President Ernesto Zedillo initiated a program that would transform the way social
policy in Latin America is made.
It was the rst action of conditional income transfer that is known - a kind of Bolsa Familia of Mexico. At the time, it was called Progresa. But in addition
Sign it
to
the impact
innovation in policy design, Zedillo's team, led by Deputy Finance Minister Santiago Levy, was also courageous in thinking about assessing
Search
of PROGRESA since the pilot phase. During the rst years of the program, the Mexican government failed to take Progresa to all who needed it, given
budget constraints.
But Levy took the opportunity to come up with an assessment that tried to isolate the effect of income transfers. And he did it. He showed that in the 300
000 families who received the bene t initially, there was an increase in food acquisition and consumption, enrollment in schools and prenatal care. There
was also a reduction in the proportion of children with malnutrition and in the incidence of infectious diseases. These improvements followed, in most
cases, national trends. But when comparing the treatment and control groups, Levy showed that the advance was faster and more expressive among
those receiving the bene t.
Expecting goodwill from politicians to rigorously assess government actions seems somewhat utopian. But there is a particular stakeholder that has
in uence power: the nanciers. Development banks and civil organizations can demand or lobby governments for results assessments. This is what the
Inter-American Development Bank (IDB) has done. In 2008, only 9% of the projects nanced by the IDB had a projected impact assessment.
That year, the bank determined that it would pursue further evidence that the policies it supported effectively improved the lives of the people served.
Today, 43% of IDB- nanced initiatives in Latin America have impact assessments, including some in Brazil, such as two public safety projects in Espírito
Santo and one in Campo Grande, Mato Grosso do Sul.
International foundations, such as billionaire Bill Gates, also require rigorous assessments of the impacts of a policy to continue funding its expansion.
"The cost of impact assessment in relation to total spending on social programs is low," said Mexican Santiago Levy, now IDB vice president, in an
interview with EXAME in May 2016. "There is no methodological or technical impediment. It's a matter of wanting to do it. "
The adoption of the constitutional amendment limiting public spending makes policy evaluation even more urgent. New programs, such as Criança Feliz,
from the Ministry of Social Development, the reform of secondary education, new versions of Science Without Borders and Fies, among others, should be
evaluated from the outset. The municipal governments that have just assumed should also include evaluation mechanisms in their policy design. An
example: in the previous administration, the city of Embu das Artes, in São Paulo, was able to identify a serious problem in the policy of modernization of
basic health units after an evaluation made by the MGov consulting company, which uses cell phones to collect data.
The MGov team took advantage of the policy implementation schedule to compare waiting times and popular evaluation of units with and without
computerized systems. The result was surprising. Computerized units had lower levels of satisfaction. The MGov investigated the case and found that
employees were embroiled in the new system for lack of training. This allowed the city hall to adapt the implementation. A good evaluation can be the
difference between the success and failure of a public policy - and even an administration.
NEWS ABOUT
BOLSA
FAMILIA
CORRUPÇÃO MINISTRY OF THE ENVIRONMENT
POLICY
Recommended by


More_horiz
 Chat_bubble_outline
On the Web
SPONSOR Understand
what can happen to the dollar
now that Trump is the new
president.
Empiricus Research
For you
SPONSORED
See what will happen to the dollar
Empiricus Research
SPONSORED
7 tips to learn a language at
home alone
Babbel
SPONSOR
How to create from scratch a
second income of up to R $ 20
thousand per month
Holidays without End