The Impact of TOMS Shoes

TOMS Shoes is one of the most well-known “double bottom line” companies, where generally the two bottom lines include both profit and social impact.   For every pair of TOMS Shoes people like you and me purchase at Nordstrom or one of their other retail outlets, TOMS gives a pair of shoes to a needy child in a developing country, 20 million pairs since the founding of the company in 2006. (They also now do something very similar with their line of (not-cheap) sunglasses, providing corrective vision for a visually impaired person in a developing country for every pair purchased.)

In December 2010 I was contacted by the head of the TOMS Shoes giving program and asked if I would be interested in bringing together a team to submit a proposal for carrying out an impact study on their work.  Our proposal won a competition with a few other proposals, and our team (Beth Katz at USF, Felipe Gutierrez, an M.D. at the University of Arizona College of Medicine, two of our masters students: Flor Calvo now working at 3ie and Nicole Shevloff, now at the Environmental Defense Fund, and Brendan Janet, our fieldwork coordinator fresh off his masters from UC Davis) implemented a randomized controlled trial in El Salvador involving 1,578 children from 979 households in 18 rural communities.  We took baseline data, then gave kids aged 6-12 a pair of TOMS shoes in half the communities, waited a few months and surveyed again.

There were two main questions we wanted to address with the research, both of which have involved criticisms of the TOMS approach.  The first involved the question about whether giving away these shoes negatively impacts local markets by stealing customers from local shoe vendors.  We tested this by giving two discount coupons to every household in our study, one for a pair of cheap (≤ $10) shoes at a local store, the other for a pair of more expensive (>$10) shoes.  Then we would see if coupon redemption was lower in the treated communities.  We didn’t find statistically significant effects, but our estimates consistently indicated a small negative impact on local markets.  Specifically, local shoe vendors sell about one fewer pair of shoes for about every 20 pairs of shoes donated into a local community.  So there probably is a little bit of a negative impact, but it doesn’t appear to warrant mobilizing the national guard to prevent shoe donations.   This paper appeared in the 3ie publication the Journal of Development Effectiveness last fall.  It was also the subject of a story by The Economist, in which they kind of misinterpreted our results.

The second question was more important: what are the impacts from the shoe donations on the kids who receive them?   We collected data on a wide array of outcomes: Schooling attendance, general health, foot health, psychological impacts, and time allocation across an array of kid activities including schooling, homework, play time, domestic chores, watching TV, eating, and sleeping.  We also asked the kids whether they actually wore the shoes, and whether they liked them.  (Sometimes economists forget to ask the obvious stuff.)  The newer paper “Shoeing the Children: The Impact of TOMS Shoes in Rural El Salvador” is forthcoming in the World Bank Economic Review. and the results were reviewed in Vox, although we believe that Vox didn’t do a good job of giving TOMS credit for evaluating their work and the improvements they have made as a result.

So there was good news and bad news for TOMS in the results.  The good news is that 95% of the kids in El Salvador had a favorable impression of the shoes, and they wore them heavily: 77% of the children wore them at least 3 days per week, and the most common response by children was wearing them every day. So the notion that kids get these donated shoes and throw them in the dumpster simply isn’t borne out by the evidence.

The bad news is that there is no evidence that the shoes exhibit any kind of life-changing impact, except for potentially making them feel somewhat more reliant on external aid.  We did find that children receiving the shoes reduced their time watching television, but they also spent less time per day doing homework relative to the control group.  And in the course of this we find they seem to get a little more banged up, but not more than you would expect from kids who can be a little more aggressive now wearing shoes in outdoor activity.  We find some very small improvements in school attendance (about 0.16 of a day), but no overall impact on self-esteem.  The most negative finding was that the children receiving the shoes were significantly more likely to agree with the statement that “others should provide for my  family’s needs” and less likely to say that “my family should provide for its own needs.”

However, two caveats are important here, the first being that other types of development interventions may exhibit similar responses among recipients.  Indeed this is something that would be very interesting to test with other types of development programs.  The second was a point raised by the giving director at TOMS, which is that providing a child with shoes may warrant the negative side-effect of a modest increase in dependency.  We remain agnostic about this trade-off, but a clear lesson to us is that development organizations need to be more careful about their targeting. Shoes, for example, should be given away in a context where (a) kids don’t already have shoes; (b) kids need shoes; and (c) kids want shoes and will wear them. Same for any other type of in-kind donation: clothes, farm animals, health products, bednets, wheelchairs, clean-burning woodstoves, or what have you. Note that it is not common that these three conditions are met simultaneously.  But where they are, positive impacts are much more likely to be realized.

I have always been a greater fan of interventions that attack the root of poverty rather than give things away to the poor.  But in light of this study, what has made a great impression on each of us on the research team has been TOMS commitment to alter its program in response to shortcomings that have been manifest through our study as well as a number of other studies that the company has sponsored on its giving program.  TOMS is perhaps the most nimble organization any of us has ever worked with, an organization that truly cares about what it is doing, seeks evidence-based results on its program, and is committed to re-orienting the nature of its intervention in order to maximize results.   In response to children saying that the canvas loafer isn’t their first choice, they now often give away sports shoes.  In response to the appropriateness of their shoes in different contexts, in Mongolia they now give away these cute little kids’ snow boots.   In response to the dependency issue, they now want to pursue giving the shoes to kids as rewards for school attendance and performance.  Moreover, we are impressed with TOMS commitment to transparency.  Never once as researchers did we feel pressure to hide results that could shed an unfavorable light on the company.   By our agreement, they could have chosen to remain anonymous on the study; they didn’t.

This kind of approach does not describe every development organization, or indeed every one with which I have worked.  For every TOMS, there are many more, both secular and faith-based, who are reticent to have the impacts of the program scrutinized carefully by outside researchers.   Instead of demonstrating the effectiveness of their program on the poor to potential donors, many organizations today continue to avoid rigorous evaluation, relying on marketing clichés and feel-good giving to bring in donor cash.   TOMS is different, and we applaud them for their transparency and commitment to evidence-based action among the poor.  We also look forward to seeing results from their vision-correction program, a creative extension of their effort which we think is more likely to result in life-changing impacts.

Follow the Shrewd Samaritan on Twitter at @BruceWydick.

• ← Wheels of Fortune
• The Economics of Hope →