Building Markets

Back to all blogs
14

PDT’s Failure Report 1.0

We’re rolling out Failure Reports at PDT. I began drafting the guidelines for staff to draft their contributions, and then stopped and decided failure starts at the top. Therefore, before asking them to write something, I opted to get the ball rolling myself.  I made a list of what PDT’s doing wrong as viewed from the 30,000′ level and I shared it with Jennifer (our Deputy Director) and a few of the managers who added some more items. Then I sent the letter below to the PDT team.  Our next reports will be more focused on operational problems with a strong focus on proposed solutions. But, it’s a start.

Executive Director Scott Gilmore being held accountable by PDT staff for his thought crimes and other counter-revolutionary failures.

Failure Report 1.0

Dear Colleagues:

As I wrote earlier this month, PDT will be issuing an organization-wide failure report early next year along the lines of those provided by Engineers Without Borders.  We are still refining the guidelines.  Inspired by Cornell West, who asked “Yes it’s failure, but how good a failure?, we want a template that focuses on learning from these mistakes. So, in the spirit of failing well, I’ve chosen to draft our first report from the perspective of PDT’s management.

Please note this important disclaimer: The failures listed in this report don’t belong to the PDT staff. They belong to me and Jennifer and the rest of the leadership at HQ.  Our people in the field do amazing work under difficult and often dangerous conditions.  Each of the shortcomings listed is due to decisions or direction that came from the people at the top of the org chart.

1.     Our biggest accolades come for our Peace Dividend Marketplace projects that have redirected over $600m into the economies of Afghanistan, Haiti, and Timor. There are two problems with this. First, we really don’t know how much of that would have been spent in the local economies without our intervention. In some cases, we have not redirected, but accelerated international spending. For example, the donors would have spent several months trying to find a local vendor, but we helped them find it in a couple of weeks. Both outcomes are economically beneficial, but without being able to accurately distinguish the two it is very hard to measure the actual impact of these projects. This isn’t good enough.  Going forward, we need to implement a new component to our results tracking systems that also looks at the nature of the contracts being tendered.

2.     The second problem with our Marketplace projects is that we track the gross economic impact of foreign spending, instead of the net impact. As you all know, $1m of donor money spent buying Chinese desks from a Kabul vendor is not the same as $1m spent in Helmand buying vegetables from a local farmer. Frankly, the latter will probably have a bigger economic impact than the former. It will create more jobs, and possibly even greater tax revenue and GDP growth. But we don’t have a grip on how to accurately analyze the direct effects of increased local spending, and the gross dollar figure approach is too crude. It distorts how we measure the true economic benefit of our activities, and it leads to prioritizing large dollar value spending that may have very low local benefit, and it entirely ignores job creation. Furthermore, we definitely can’t predict impact in advance that would allow us to efficiently focus the project activities.  We need to be far better at this. With the expansion of our Economist team, Jennifer and I have asked that we develop a methodology for this analysis and that it be rolled out across our various country programs.

3.     Related to 1 & 2 above, we don’t do nearly enough impact measurement. It would be easy to blame the donors here, since for the most part they only care that we measure process: Budget performance, number of training sessions held, etc. But PDT believes passionately in enhancing impact through better data. We need to focus more of our resources on this. If we did, we could show that some of our projects are indeed producing remarkable results in terms of poverty reduction or job creation, but that others are simply not worth the effort (and then cut those projects to work elsewhere.)  For new projects, we will write into the contribution agreements impact measurement activities and clauses to ensure that the project will be primarily judged on those metrics.

4.     Perhaps one of our biggest failures is our fundraising strategy. We have grown far too dependent on government donor agencies. This has slowed us down and made us more process driven.  Our “unrestricted funding”, the money we can spend on finding and testing new ideas, is tiny. We should have been focusing on this far more, far earlier. With all the good press we’ve gotten, I should have been working far harder on turning that into financial support. In reality, we only started looking for private funders in the last 12 months! In the first quarter of 2011 we will be hiring a Development Director who will work directly with me to review and overhaul our funding strategy and activities.

5.     We’ve ignored the lessons learned by others. A lot of the “new ideas” that PDT develops aren’t actually new. Someone else in the aid industry thought of it before and tried it elsewhere. We need to put far more energy into ferreting this out. It will allow us to a) give credit where credit is due, b) focus our precious resources on genuinely new ideas, and c) build on the experience of others to fine tune or further innovate these concepts.  How can we fix this? Partly we need to be more curious and more energetic in our research.  Jennifer and I may also put up a big sign in the New York bullpen that reads “Great idea. Now find out who thought of it first.”

6. We have wasted far too much time (and readily available donor money), measuring the “economic impact of aid”. That work was important and useful and groundbreaking 5 years ago when the Economic Impact of Peacekeeping initiative showed that only a small fraction of international spending was going into the local economy. But now, it’s not new. It’s not surprising. And it’s very safe to extrapolate the data collected in Liberia in 2007 to estimate the donor footprint in country x in 2010. But the donors keep offering to pay for another study, so we keep doing it. We need to stop. Aid is rarely made more effective by the production of yet another study.  We’ve already cut some of these projects out of our pipeline. Going forward, we will ensure that new economic analysis projects clearly move the research forward in obvious ways.

7.     Our staffing systems are really lagging. To be fair, we’ve doubled in size almost every year over the last 6 years. This creates strains on our administrative processes. But still, if we want to keep our good people, and find more people like them, we need to be faster, smarter, and better on how we look after the PDT team. (This isn’t the fault of our HR team. They’ve been crying for more resources to do this, and I promise I’ll find them!) In the new year we will look at some new support structures for field teams and budget for additional resources in the HR unit.

8.     PDT’s communications efforts are terrible, I mean truly painfully awful. Have you seen our website? Or noticed that we have multiple websites? Our print material is even worse. We’ve spent so much time focused on the “doing” senior leadership (i.e. me) has totally forgotten about the “telling and advocating”.  Our project staff are understandably frustrated by this, and even more frustrated because we keep promising to fix this.  They need not only a plan, but a clear message, and assistance from HQ on how to deliver it.  The fix for this can’t be summarized in a sentence. Early in the new year our new Communications Director will be circulating a detailed plan for not just fixing this problem but for implementing an aggressively innovative new approach for comms.

9.     We are too quick to criticize others. For example, just yesterday I took a cheap shot at Bob Geldoff in a newspaper interview (sorry Bob, we love you). I agree that some folks in the aid industry deserve all the criticism we can shovel on them. But in more cases there are complex and intractable reasons why projects, NGOs, and donors behave the way they do. We need recognize this, and be far less eager to blame the entities, but rather we should blame the industry wide conditions and norms that not only tolerate failure, but in some cases encourage it.  To be frank, I’m not sure if we’re going to get better at this. But, perhaps, if we are quick to criticize ourselves it won’t seem quite so bad.

10. Unbelievably, we have almost no Knowledge Management system. We have a shared intranet, where important documents are stored, but nothing that actually functions well. While we rail about the importance of the aid industry and the UN capturing what Donald Rumsfeld called the “known knowns”, we have failed to systematically capture our own knowledge. This makes it especially hard on our project staff who often are forced to “figure it out” without the benefit of knowing that another PDT staffer did exactly that, in the same project, 2 years earlier. Jennifer and I will be talking to you, our IT team, and others to first determine the scope of our needs, and then to propose some solutions.

11. Related to the previous failure, PDT has been embarrassingly hypocritical in demanding reforms in the aid industry that we haven’t implemented ourselves. Here are just two examples. We’ve argued for years that the aid industry should bring in performance bonuses to reward excellence and encourage innovation. We also have tried to convince the UN to offer more staff recognition awards to highlight the best practices of their field staff. But in both cases, PDT has planned but failed to roll out similar programs internally. And refer back to items 1 through 3 above. We demand more data and analysis, but we don’t do enough of it ourselves.  How do we fix this? By being more self-aware. By reminding ourselves that as we grow, we will draw the same scrutiny we apply to others. And by using that scrutiny as an opportunity to not only advocate for innovation, but also demonstrate innovation.

We welcome your comments and look forward to working with you to fix these failures in the year ahead.

Sincerely,
Scott

Tags , , , , , , , , , , ,

14 Comments

  1. Ian says:

    Scott – I have to really commend you on doing this, and for sharing it publicly. It would go a long way to making aid more effective if more organizations would do this.

    I also like that you “started at the top” thus making it possible for those who work for you also to discuss failures without being afraid of negative reactions from management.

    I think it’s difficult for organizations to talk about failure, especially in this climate of funding cutbacks – but without admitting mistakes we cannot learn from them and will continue to repeat them. This makes a great start to your knowledge management system!

  2. Caroline says:

    Well done! I agree with Ian that it is a great initiative to create an environment that enables critical reflection, acceptance of failure in order to find solutions and overcome it, and hopefully will lead to improvements!

    The examples given are not unique to PDT — I can think of many organizations in the aid industry where they would apply just the same, which raises the question: how do we achieve a more systemic change? The debates about and initiatives to reform the UN and the humanitarian system are not showing demonstrating the much needed results. Are the solutions caught in the same thinking that has caused the problems in the first place? If so, how to step out of it and come up with new, workable ones?

    Good luck in your efforts! I do hope you succeed! And then share the lessons!

  3. Dennis says:

    I would also like to commend PDT for this honest assessment. Anyone who has tried to accomplish anything in Haiti knows that failure is easy. This is an unprecedented disaster and we all had to learn on the fly to deal with it. Honesty like this is how we improve.

    I’m just paranoid about the media coverage. For the last several weeks, journalists who filed stories after the quake have been coming back for their 1st anniversary coverage. They fly in from 1st world cities with pre-determined intent to find fault somewhere, then they tip-toe around a few tent cities and fly home to write scathing articles on how NGOs are failing in Haiti. The current mantra of the 1 year coverage seems to be all about failure.

    I am in the middle of such a media fiasco now. A reporter from the NY Times has been following one of our patients. She even went as far as taking part in an effort with a promotion-seeking doctor to evacuate the patient to NY for treatment last april even though we had been working with her for months and were about to fit her with a prosthetic leg. When the patient declined to go to NY because she didn’t like the way her medical care was being hijacked by people who she didn’t trust AND she was happy with our treatment plan we developed for her as registered providers, I got attacked for interfering with the patient’s best interest. A front page article in the NY Times created a fictitious “tug-of-war” scenario with me as the bad guy that nearly destroyed our efforts in Haiti. The writer was back in Port-au-Prince last week looking to find failure and issue a final coup-de-grace by manipulating our patient with pathetic questions like “dont you think you should be traveling the world by now?” and “dont you wish you went to NY?”

    I guess my point is that openly discussing failure in the current hyper-critical atmosphere is a bold and refreshing move. It certainly shows you are here to make substantive improvements in Haiti in an honest way. Be careful though!

  4. Re: #10 – I’ve been spending the last two and a half months developing a rather extensive proposal for a information/knowledge management project (named “Documentation Capacity”) for ACCORD in South Africa. Would be happy to share and discuss it with you, if you like.

    /david

  5. SynAffairs says:

    This is a really well-thought out list of organizational “new year resolutions”.

    But I’m wondering – how many will you actually accomplish? How? Which ones are the most important ones? You place many demands on your own time (#s 4,5 7, 10, all involve “Jennifer and I will work with you…” or some variation) – at what point should you just make the transition to more professional management your major goal in 2011 and task each director (IT, Development, Communications, HR) with all the planning and doing (in a team fashion; from what I hear about what’s hot and not in 2011, silos are already not). I really liked your line in the G&M article about surrounding yourself with great people not because you’re particularly lazy or stupid, but because you know a great team is the key to success (i.e., success = riding the rapid growth wave without getting smashed to pieces). In a similar vein, have you already thought about how field staff and other line staff can contribute towards these goals? But I guess that’s the next step right?

    Overall, great stuff! Keep up all the great work. As someone still very young in a (hopeful) international career, it’s great to know there are bosses like you out there (and with awesome writing skills too!)…

  6. Scott Gilmore says:

    SynAffairs. You hit the nail on the head: “how many will you actually accomplish?” We are wiring these issues into our 2011 Business Plan, which sets the priorities for all staff in the year ahead. And as you suggest, while Jen and I may take responsibility for action, we will turn to our managers to actually operationalize.

    As for being a great boss, you really should speak to some of the staff around here first. (poor things)

  7. […] year. This practice was also recently adopted by the Dividend Peace Trust who issued their first failure report this […]

  8. […] to some other recent “fail” celebrations such as Mobile Active’s FailFaire, Peace Dividend Trust’s failure report, and Engineers without Borders who followed suit. The Admitting Failure initiative and website, as […]

  9. […] Haiti is still a long way from full recovery. The earthquake brought a spotlight to a nation which had serious problems in need of attention long before January 12, 2010. It is all too easy to scream that things are not going well while providing half-truths, and it is always in the interest of an individual or NGO to shift the blame onto another without accepting personal failures. […]

  10. […] the early adopters, I was impressed with the leaders at Peace Dividend Trust – their failure blog is specific and the language does not mess around – i.e., not “we are working to strengthen our […]

  11. […] Failure. Several organizations now issue some form of failure report, including: Peace Dividend Trust, Engineers without Borders Canada, and Mobile Active’s FailFaire. Engineers without Borders […]

  12. I like this post….

    […] Even though this blog post is not breaking news, I enjoyed it and hyperlinked to it at my website. The only suggestion I have is to update it with a fresh video. […]…

  13. […] failure blog for Peace Dividend Trust, for example, is an impressive example. The information is specific; it’s clear they’re not […]


Rss Feed Tweeter button Facebook button Youtube button