We are data nerds. We believe, as a matter of integrity and intellectual honesty, that organizations claiming to help others should support those claims with convincing quantitative evidence of impact. We rely on such evidence in law, public policy, medicine, engineering, and other diciplines that affect our daily lives. We should hold well-intentioned charities to the same standard.
The data show villages on divergent development paths. Both partner and control villages performed poorly during two bad agricultural years (2015 and 2016) and posted similar livestock scores over time. Yet, starting in 2015, partner villages outperformed control villages in terms of waterborne illness, education, lifestyle, and business. Do these descriptive statistics contain statistically significant impacts? Find out below.
In 2016, after two years of funding projects, we evaluated our impact. Our approach wasn't fancy or expensive (more on that below). Instead, it tested a foundational assumption that financing village-led projects without any sort of preconceived development agenda would allow local communities to illuminate development pathways obscured from the outside. In other words, we funded a bunch of projects, collected a bunch of development indicators, and tested whether anything had changed. Did communitites improve and, if so, how? Here's what we found.
Using community surveys spanning 2014 to 2016, across 21 projects and 32 villages, we tracked the following 13 development indicators pertaining to health, education, business, lifestyle, agriculture, and livestock, both in partner villages and villages that want to partner with Village X. We chose these indicators because they are easy to collect and indicative of village development trends. We then applied a difference-in-differences analysis to detect our model’s impacts, controlling for village population and number of households.
Each project profile on this website has graphs showing how development scores for a given village change over time, including an overall village development score and scores for each of the categories above. These scores are not precision instruments. Instead, they capture village development trends over time.
Want to take a deeper dive? Checkout our dataset here. It's part of our 100% tranparency guarantee.
We are not satisfied with our impact evaluation. It's a good start, but we can do better. In particular, we could track how projects we finance affect individual household spending within a village. How do families modify their spending in response to village-led projets? Do the projects make families wealthier?
Answering these questions requires overcoming two challenges: (1) our current portfolio of partner villages is not large enough (we need at least 300 per year); and (2) we do not possess a large monitoring and evaluation budget. We plan to scale our operations to overcome the first challenge and to partner with academic researchers to overcome the second one.