Beyond prizes, hackathons, and contests: Better ways to support innovation in international development

JOHN FEIGHERY @JohnFeighery, ANNIE FEIGHERY @AnnieFeighery

Co-founders at mWater

Today marks the deadline for the largest development prize yet, the MacArthur award offering the winner a $100 million grant “to fund a single proposal that will make measurable progress toward solving a significant problem.” MacArthur Foundation specifies: “Proposals focused on any critical issue are welcome.” The dream of making significant progress in a seemingly intractable problem did not begin with international development, but it is currently most alive in our sector. The problem is that, so far, prizes, hackathons, and contests have not worked well in any sector.

Large institutions that want to identify innovative approaches to reduce poverty often turn to startups and social entrepreneurs for fresh ideas in international development. They are motivated by the rapid advances seen in Silicon Valley that are credited to Lean and Agile, startup-styled management. Yet, many of the current approaches to innovate in aid do not work very well and often make it even harder for successful small enterprises to grow and reach scale. Our hope is that the prize culture can give way to investment culture, which is actually how the idea economy has proliferated in other markets.

In this essay, we explore these models, weaving in some of our own experiences as founders of a successful and widely adopted social enterprise in the water and sanitation sector. We discuss why so many of these prizes, contests, hackathons, and incubators fail to achieve the desired impact. Finally, we share some practical advice for the sector and highlight some promising alternative approaches to funding development innovation that may have more impact.

An explanation of startups and tech innovation

Startups are organizations or companies that are designed for rapid growth. They are typically less than five years old. They aim to follow a hockey stick pattern in their key metrics, which are often users (eyeballs) and revenue. They achieve this rapid growth by taking big risks that could lead to failure and the end of the startup, but also by rapid iteration.

Startups are run in a management strategy called Lean and Agile, which literally means what it says: very few employees, flat in terms of layers of bureaucracy, and nimble to pivot often with new knowledge. Lean and agile management organizes work into small sprints — as short as two weeks, for software startups — in which one iteration of a product is built and, if it fails, it is modified or killed off entirely in the next sprint. The flat team means any one member may be the lead of various sprints at any time.

The team comprises domain experts and technology experts. Both are critical. Our own startup is often misunderstood as a technology firm, when we are actually a platform for managing water, sanitation, and health programs. True technology experts are increasingly hard to find. Cloud-based workplaces like our own mean people can work from anywhere so, geographically, good technology developers are very hard to find no matter where you are. Due to the extreme demand for their talents, tech experts require lots of pay. Our experiences in running a startup taught us that A-level tech experts are critical to a team. Three A-level experts are superior to 20 B-level experts because more lackluster talent just slows down development. Students, volunteers, and anyone willing to work for less than market wages are usually lackluster talent.

Startups are often unfunded, but when they do have cash on hand they count it as runway: the number of months in front of them they can pay the bills while awaiting investment or acquisition. Runway is measured by burn rate, the monthly amount of cash needed to run the startup.

How the good idea of funding innovation is going wrong in aid

The role of hackathons, incubators, and startup competitions in Silicon Valley has been to provide quick cash to bootstrapped teams giving them more runway, but also to raise their profile to investors shopping for the next big idea or acquisition. The system only works if this next step is in place — large investments and acquisitions. Innovation initiatives that are inspired by these Silicon Valley practices are trendy in international development circles and widely publicized, but the evidence for their impact can be hard to find. Since they often fail to build products, software, and services that grow and attract new investment or acquisitions, they have a track record of wasting time. Here we describe some of these models and where they go wrong.

Innovation funds

In this model, the donor pools money into a ‘fund’ that is meant to mimic the Silicon Valley venture capital (VC) investment process. But what happens next bears almost no resemblance to the VC approach. These innovation funds typically put out a call for proposals across a broad set of disciplines, such as health, education, agriculture, water, and sanitation. They are always accompanied by a massive PR and social media campaign.

Large investment funds in aid have complex application processes that often involve site visits and audits. The donor has to hire consultants to sift through the thousands of responses, perform interviews, and conduct time-consuming and intrusive audits of the ‘lucky’ finalists. Sometimes they assign advisors to help strengthen the applicants’ proposals and they almost always ask for detailed project plans along with multi-year financial projections, which requires weeks of effort by the applicant. These processes are often designed to assess the applicants as if they were traditional aid organizations. We have been asked for HR manuals, annual procurement policies, MoUs with potential partners, and org charts; yet true startups are too new to have these bureaucratic policies and the org chart is a flat list of founders who do whatever needs to be done.

In the end, a few winners are selected for what is essentially a one-time grant, while the rest are given a couple of sentences of feedback from an anonymous panel of judges who come from completely different fields than the subject of the proposal. The competitors have lost weeks of time that should have been focused on their own growth, and they have wasted the most valuable commodity in startups: runway. The application process is designed to reduce the risk that the donor’s investment will be lost, but it actually functions to increase the likelihood the applicants will fail. These programs ostensibly dedicated to innovation get bogged down in outdated practices from grant models. The best role that investors in aid can fulfill is absorbing more risk, not minimizing their exposure to it.

Contests and prizes

Contests are an even worse version of the innovation fund model, because the rewards are small and there is an even smaller chance of success. One inspiration for this model may be the X-Prize competition to develop new low-cost vehicles for human space flight. Although the X-Prize could be thought of as a success since the prize was won, it failed at the larger goal of bringing about an era of routine and affordable spaceflight (SpaceX, which followed a very different path that we discuss later, seems much more likely to achieve this goal).

Furthermore, international development is not the same as launching rockets. The potential revenue from launching satellites is massive, so it was much easier to motivate private investors to back startups even though the market is high risk. Innovations that primarily benefit the poor produce small profit margins and are not as likely to attract traditional venture capital or angel investors, so who will supply the outside capital investment that these prizes are hoping to attract?

A recent trend in the innovation prize market is to offer a tiny amount of cash along with a package of free services and ‘mentoring’. What these organizations are effectively saying is: we don’t have enough innovative ideas within our own ranks so we reached out to you, but from now on our people will give you advice on how to execute your idea.

What startups really need are customers (or users) and runway, not well-meaning but uncommitted advisors or volunteers, free web hosting, or whatever else is of so little value that it is given away. And since the prize money is nowhere near the amount of money needed to start or grow a new company, they often fail to attract real startups and instead go to universities or joint ventures backed by larger, established NGOs. Most of the value of winning is in the media buzz generated by the announcement, and for a true startup this is a hollow victory if it doesn’t quickly lead to new revenue.

Hackathons and app challenges

A hackathon is a weekend-long party for developers, designers, and domain experts. The motivation is that these experts work so hard at their long term projects at their jobs all week that they need a break, and yet they love their subjects or tech talent so very much that on the weekend the best thing they can imagine doing is applying their abilities to something fast, sexy, and completely different. Hackathons are fueled by pizza and beer and typically run around the clock without breaking for anything longer than naps from Friday evening to Sunday afternoon.

This is a delicate subject because our own organization was actually born from a hackathon in 2011. However, we are more of the exception than the rule. After meeting each other and deciding to make a go of it, we bootstrapped mWater for 2 years before breaking even, working on mWater nights and weekends while we consulted to pay the bills. The hackathon brought us together, but that was all it did.

Thanks to the many free software-building tools that are the peace dividend of the social media revolution, it is very easy to build an app today. This can give the false impression that every hackathon has been wildly successful. After all, several teams built new apps. But these ‘apps’ are literally hacked together and will require months of re-coding and field testing before anyone could actually rely on them. After the hackathon prizes are awarded, the winners will have to do the hard work of validating the concept, coming up with a business or funding model, and learning to work together as a team. With no long-term funding commitment, most people just go back to their regular jobs. And contrary to the hype, no serious software engineers want to use code generated during a hackathon because it is generally poorly conceived and undocumented due to the rush to have a working demo.

Hackathons work best when they break some new ground by blending completely new teams, ideas, and talents to work on an old problem. They can inspire new ways of thinking, but they do not typically start new startups. On Monday morning after the hackathon, everyone goes back to their day jobs and the ideas created fall away unless a next step is clearly planned and financially supported.

Innovation marketplaces, open innovation platforms, and innovation challenges

These approaches stipulate that there are already so many people with great ideas out there, if we could just share them via a polished online forum, investors and customers will line up to fund them. This fad is currently at its peak, and our organization is asked almost monthly to write copy for an innovation marketplace or challenge, offering to feature our product or help us ‘refine’ our best ideas. We used to respond to these requests; now we ignore them because we realized that we had never received a single new customer or investment as a result of being featured on one of these forums. Now that there are so many of them, the visibility of any one platform has been diluted such that there is no positive return on investment (ROI) for investing the time required to join a new one.

The ‘innovation platform’ concept has already been tried in Silicon Valley and the results were equally disappointing. Quirky, a venture-backed startup, runs a slick “invention platform” for everyday people who have “a million ideas but only a few minutes to spare.” It turned out that the company’s approach of rushing new products to market without a commitment to quality and customer support did not scale well. After burning through nearly $150 million, the CEO was recently ousted and investors are looking to sell off the business.

Finally, ‘open innovation’ challenges seem to combine the invention forum with aspects of hackathons and contests to form a new business model (at least for the organizations running the contests) where ‘innovation challenges’ are carried out on behalf of large institutions. We can identify very few success stories where any of these ideas scaled beyond an initial concept and the impacts touted tend to focus on process rather than outcomes (“tackling climate change” or “reimagining higher education” as opposed to reduced pollution or higher graduation rates). New social enterprise startups beware: although these activities are fun and often personally fulfilling (much like speaking gigs), it is not a good way to be spending your most valuable resource, time.

What is wrong with these models?

1) They value ideas over teams and traction, making them nothing like the venture capital process they intend to mimic. In the Silicon Valley model, startups are expected to put their complete focus on developing a product that users or companies want to use and will somehow pay for, either directly, via monthly fees, per service fees, or large contracts; or indirectly, through advertising. Venture capitalists do not routinely hold open competitions where founders pitch ideas and then win money to go try them out. In today’s startup world, if you don’t have a working product, a strong team, and some early traction in the market, you won’t even get a meeting. There is a popular public misconception that good ideas are rare and have value, but in reality an idea alone has zero market value. It is the team that can execute a business around a promising idea that will succeed and change the world.

2) The selection panels don’t know enough about each specialized field of development practice, science, or technology to properly evaluate such a broad range of ideas. Real world VCs tend to be very specialized, allowing them to make better predictions about whether a company will succeed based on similar companies. Considering how broadly these innovation funds cast their nets, it is unlikely that any member of the selection panel will have enough specialized knowledge to evaluate the technical aspects of a proposal. Ironically, the large organizations who sponsor these models often have experts on staff with the right knowledge but they are not consulted. The end result is the selection of specious proposals that sound great in press releases but can’t deliver on their claims. This is counterproductive, because the disappointing results ultimately cause an erosion of support for innovation among the organization’s own staff.

3) The organization doesn’t buy-in. Donor organizations generally have a well-developed theory of change that they believe will lead to the greatest impact in their field. That’s their job. Outsourcing the generation of new models to startups, often for a fraction of what the organization pays its own staff, externalizes the challenge of creating organizational change. If and when a social entrepreneur comes up with a successful new model, they have to overcome organizational inertia to move the rank and file donor staff away from current thinking. It would be far better for the donor to work with entrepreneurs to find ways to integrate their innovations into current programs in a phased approach so that internal staff can evaluate and buy in to the new idea. This is the only hope we have to steer development funding in a new direction.

4) There are no next steps for integrating the innovation into the international development financial ecosystem. The problems we want to solve, like providing safe water and sanitation or building healthcare systems, will require massive capital investment in infrastructure and human resources. Any new approach will eventually need to tap into this stream of investment if it is going to work at the scale of the problems we face. Large organizations are uniquely situated to influence these investments, but instead almost all of their funding is tied up in established models with big contractors, leaving just a few scraps for the startups.

5) They waste time and duplicate effort. We have competed for and were short-listed for several high profile innovation funds over the past two years. Each time, we invested weeks of our precious time and real money traveling to host site visits or audits, lined up letters of support from local partners, and developed complex project plans and budgets. Each time, the donor funded at least one less experienced group trying to duplicate our platform or provide a new service that could have been built through a minor extension of our technology. To borrow from the USAID Digital Principles, we should always explore whether it is possible to reuse and improve what is already out there before trying to build something new. If the donor knows, for example, that a proposal they are funding includes building a mobile data collection system as part of their program plan, why not tell them to pair up with another group that already has an operational platform? They are uniquely able to encourage such collaboration since they have already vetted all of the short-listed proposals, but this rarely seems to happen due to the way the evaluation processes are set up.

Five better models for innovation funding

The idea of large organizations fostering innovation within their sector is fundamentally sound, we just aren’t doing it right in many cases. In the private sector, many large companies now have a venture capital investing arm, contributing their in-house skills at scaling and marketing that startups often lack in exchange for early access to new concepts. However, the end result of a successful investment is almost always an acquisition by themselves or a competitor. The corporate investor either gets a new business unit that will increase its future revenue, or they get cash from the sale that they can invest in more startups. If there was an equivalent model in the social impact space, how would it look?

Fortunately, there are a number of successful examples from other sectors to draw upon. In addition, a few large organizations in the international development arena are also starting to do innovation differently. Their approaches include staged investments, venture funds, and more authentic competitions. At first glance, they might sound similar to their less successful counterparts that we just covered. But the resemblance is superficial and in the final section we highlight what makes these models better and provide some recommendations for how you might implement them in your own organization.

1 — Staged investment models

USAID’s Development Innovation Ventures (DIV) was a pioneer in this space, providing relatively unrestricted funds to organizations as they progress through three stages: proof of concept, testing at scale, and transitioning proven solutions to scale-up (disclosure: we were a DIV Phase I grantee). What really sets DIV apart from the faulty innovation funds discussed previously is their commitment to rigorous evidence, their understanding and support for the challenges facing new startups (including the frequent need to pivot), and the quality of the review process, which brings in experts from relevant disciplines to evaluate every short-listed proposal. Also, through its phased approach, DIV recognizes that even the most successful startups need steady infusions of increasing capital investment to reach scale.

2 — Development alliances

Development alliances are a relatively new contractual vehicle that can be used to save some seats at the table for startups in traditional development activities. Instead of a traditional, winner take all, hundred million dollar request for proposals, the donor invites a diverse group of firms to submit information about their unique capabilities and working relationships in the target countries. The awards can be tailored to include a mix of large and small organizations. If done right this can inject innovative approaches into the established development ecosystem. This idea is starting to get some serious traction within USAID and is already showing positive results.

3 — Pooled investment models

A less formal, pooled investment model allowed our own organization to build out our platform and expand globally. Most of our initial seed money from USAID went toward field operations to prove out our mobile water quality app, but we also needed to perform some surveys for the project evaluation. We weren’t very pleased with the options available at the time, so we bootstrapped development of our own mobile survey system. Water.org was planning to consolidate their global monitoring and evaluation activities onto a single platform and liked our technology as well as our business model. They doubled down USAID’s investment, allowing us to build a web portal to design forms and manage data. Later that year, WaterAid conducted a broad survey of the market and decided that our unique approach to monitoring water points over time would best support their post-implementation monitoring programme. WaterAid contributed to the development of analysis and visualization tools as well as countless other improvements across the platform. Although these two large water NGOs might traditionally be seen as competitors, their openness to a pooled approach increased the value of their individual investments.

4 — Venture funds

Where the innovation funds we discussed earlier tend to be vague in their goals and lacking in next steps, better designed venture funds from government and philanthropic donors have proven very effective. These venture funds tend to be focused on specific needs of the organization, which helps ensure that there will be a market for the new company’s product if they are successful. This also means that the organization has expertise in the subject area and will have well defined metrics for assessing whether the new programs or technology can meet their needs.

One example of this approach working within the social enterprise sector is Acumen Fund, which makes debt and equity investments in organizations working to improve the lives of low-income consumers. Rather than running competitions or prizes, Acumen hires fund managers to evaluate new investment opportunities and manage their portfolio. This is much more effective than putting an application into the hands of a panel of judges who come from outside the organization, and it more closely resembles what happens every day in Silicon Valley.

This approach has also worked very well outside the social impact sector. For example, Palantir is a data analytics company created by some of the founders of Paypal, who wanted to use online fraud detection technology to help reduce terrorism. The new company struggled to find customers early on until it received an investment from In-Q-Tel, the venture arm of the CIA. From the 2005 investment through 2009, essentially all of Palantir’s customers were government agencies who found their software faster and more capable than the systems provided by the traditional large military contractors. Six years after their founding they turned their data skills toward private banks and insurance companies. Today Palantir is one of the most successful enterprise software startups in Silicon Valley, with a valuation of over $20 billion.

5 — Competitions instead of contests

What does it look like when an innovation transforms a large bureaucracy? We have an unprecedented example happening right now the space launch business, and it started out as a competitive program, not a contest like the X-Prize. When the time came for NASA to retire the Space Shuttle program, which had become too expensive to continue, there was a lot of political anxiety over the fact the United States would become totally dependent on Russia for launching astronauts and the majority of the cargo to the International Space Station. In a fascinating coincidence, the NASA Administrator in charge at the time happened to be Mike Griffin, whose previous job was to run the CIA In-Q-Tel program that spawned our previous example, Palantir. He wanted NASA to focus on space exploration and get out of the routine business of servicing the Space Station.

NASA provided roughly half of the initial capital investment required for SpaceX to build their Falcon 9 rocket, in a phased approach that required certain milestones to be met before payments would be made (two other less successful companies received a similar funding arrangement). NASA affirmed their role as the main customer for these new rockets, committing to spend $14 billion for Space Station launches over the next five years which will be split between SpaceX, traditional aerospace contractor Boeing, and a small group of other potential suppliers. If NASA had continued with business as usual, a consortium of the major aerospace contractors would have a monopoly and launches would still cost $200–300M, draining resources from NASA’s exploration programs. Instead, SpaceX has a waiting list of both government and private customers for their $60M launches and they are actively testing a system to land and reuse their rockets, which until now were discarded in the ocean after each launch.

This public competition model stands in sharp contrast to the contests and prizes being promoted in international development. Rather than offering a small financial incentive at the end of a massive financial risk taken by the startup, NASA put up half of the seed funding required in a series of guaranteed payments in exchange for meeting incremental milestones. Although it was a calculated risk at the time, NASA now recoups that initial investment every few launches. Imagine if the large donors in international development took a similar approach toward some of their routine operations.

Recommendations for better innovation funding

Now that we have identified a few truly promising models, here are some recommendations for how to succeed at implementing them.

Fund great teams and give them the chance to pivot to a better idea

While it’s always good to set aside a small amount of risk capital for potentially revolutionary new ideas, recognize that a lot of innovation is evolutionary. Part of the reason that the ICT4D field has gotten a bad rap lately for ‘pilotitis’ is that every idea has to appear revolutionary to get funding, and often the donor did not have the patience to stick with the teams as they learned and grew in their understanding of the problem. This is why startups in Silicon Valley typically raise at least 18 months of unrestricted runway capital in their first funding round and many will completely change their business model or pivot to an entirely new product during that period. It’s worth remembering that Instagram originated from a mobile check-in game, YouTube began as a video dating site, and the founders of Twitter initially set out to build a network for subscribing to podcasts. If we want to achieve similar success in scaling social impact startups, we need to shift the focus from flashy ideas to building great teams.

Direct venture funds at specific needs identified by your organization

You as a donor know the general direction that your sector needs to be heading toward, so make use of this knowledge. One approach might be to poll your staff experts about areas where existing models don’t seem to be working or there are big problems that don’t seem to be addressed with current approaches. Then go search for startups that have already starting to work on those problems. This is how most venture capitalists actually operate. They use their professional networks to seek out startups working on problems that they think are the most important and potentially valuable. They don’t look for new ideas, but rather look for strong teams that have already begun to work on important problems.

Stop supporting vaporware and fake innovation

Startups are uniquely able to take big risks since most will fail anyway, but there is a certain kind of startup that seems designed around making bold and unsubstantiated promises in order to win press attention and initial funding. We work at the intersection of water and technology, a corner that has been fertile ground for fake or misguided attempts at innovation. Some high-profile examples of these misguided efforts include the LifeStraw and the now infamous Play Pumps, both of which scaled a superficially attractive but entirely flawed model for rural water supply before testing it.

Other projects have been intentionally deceptive, such as the Water Canary, a cute yellow gadget whose creators promised in TED talks would become a network of real-time cholera detectors at a time when Haiti was in the throes of an outbreak. The device itself never materialized and nobody replied to our repeated offers to test it for free during our field work. Vaporware is a similar practice in the software world where a company announces a new product months or years before it is supposed to be released, often before a single line of code has been written, in order to get funding or ward off competitors. It is very easy these days to build real-looking mockups or even working prototypes that look impressive but are nowhere near ready for deployment under real-world conditions.

Require evidence

Fake innovations compete against real platforms for very limited funding and attention. They look very compelling to funders because they are telling them exactly what they want to hear: that there is a much cheaper or simpler solution to a hard problem (watch the Onion’s Compost-fueled cars for a hilarious take on this tactic). The late Carl Sagan provided a timeless toolkit for evaluating such claims, which begins with, “wherever possible there must be independent confirmation of the ‘facts.’”

The best protection is to follow the DIV approach and make sure that knowledgeable experts in your own organization or within their professional networks evaluate new ideas. If they say the startup is high risk but the fundamental concepts are sound, go for it. But always require evidence that a startup has what it takes to work on a problem. At minimum, this should include: (i) expertise in the critical subject areas within the core team; (ii) understanding of the competitive landscape and current approaches to the problem; and (iii) detailed information about how the proposed innovation is supposed to work and be an improvement over the status quo.

Engage with startups in your work

It is hard for a small startup with annual revenues of less than $1M get a seat at the table when most contracts are tens or hundreds of millions of dollars. Invite startups to the table to provide input when the big decisions are made, like joint sector reviews, cluster response meetings, and major new programs. Explore innovative financing arrangements, such as development alliances or outsourcing of routine operations that could be done more efficiently.

Identify a path toward integration into programs

The next step that is missing from the broken models discussed earlier is the direct financial and programmatic integration of the new venture into the donor’s funding model. From our perspective as a social enterprise startup, we see large organizations in our field spending hundreds of millions of dollars on established programs while providing very little evidence of their effectiveness to the public. Meanwhile, they offer to award us several hundred thousand dollars, with which we are supposed to develop a new technology or program model, complete a demonstration project, perform a rigorous impact evaluation, and come up with a financial model that doesn’t require any ongoing investment from them (this is perceived as ‘unsustainable,’ which is odd considering that they work in the aid sector). If a startup succeeds at coming up with a promising new model, shouldn’t the original donor be the first customer? As discussed earlier, this worked for Palantir and SpaceX. For both, financial sustainability came after many years of government agencies being their only customers.

Our innovation story

At mWater, we bootstrapped for nearly two years before we had enough revenue to support the team. In late 2013, we quit our day jobs and we have been entirely revenue-based since (although we still seek investments). We were able to do this because aid organizations stopped creating bespoke software and instead invested in our shared platform. We wrote every contract stipulating that what we built for their unique problem, we would build as a solution anyone could use; and that we would make it available for free to the world. This turns what would have been sunk, overhead costs for these organizations into an impact investment that increases the technical capacity of the entire sector.

Our first customers, Water.org and WaterAid, are still our biggest “investors” and they dogfood (eat their own dogfood, as in, use their own product they helped create) their own investments every day, which in itself helps us improve the platform long after any one feature was created. We have grown to reach over 8,000 NGOs, governments, communities, and researchers in 73 countries. 95% of these users are the free ones who never have a relationship with us, while about 5% represent investors who pay for more and more features as they innovate and advance their own technology use. They know their technology investments are valuable because they get used. In tech, unlike aid, if things don’t get used they go away.

Ironically, we have had competitors criticize us for our lean team (there are now 8 of us) but it keeps us agile and fast. This is the type of model innovation strategies should be advancing in every sector of aid. What we have done for water, sanitation, and health focused on SDG6, but we are moving to education, refugees, and nutrition. We recommend large organizations look at our success innovating and scaling and design their strategies to fund and fuel more startups like us.

Our experiences have led us to conclude that the current trend of innovation funds, prizes, and hackathons in the international development sector has run its course and it is time to move on. By carefully studying the venture capital model and other successful business venture funds and taking away the right lessons, we can build much stronger and more innovative social enterprises. If a new innovation for improving the lives of the poor works, it will require a lot of care and nurturing on the road to success. Startups in the social impact sector need less help getting started and more help getting to scale. Innovative approaches to the global problems we face in international development will truly succeed when their ideas make their way back into the financial and social ecosystem that inspired the entrepreneurs to try something new in the first place.