Emergent Thoughts on Emergence

Just finished ‘Aid on the Edge of Chaos’ by Ben Ramalingam, which applies ideas about complex adaptive systems to aid and development. Though ‘finished’ wouldn’t be quite the right word, as I reckon I’ll need about four more readings to get close to understanding the content. As the title suggests, this will be more a provisional set of conclusions.

Conventional development thinking: 'whoops'
credit www.drsalonen.com

I’ve already written about how Ramalingam critiques the current mode of thinking in development but it’s worth repeating the basic point: we need to stop applying simplistic analyses to complex problems. Ideas derived from Newtonian physics aren’t able to fully describe the totality of our social, environmental, ecological, political and economic reality. Who knew? Actually, the observation that the world is complex is so obvious that I can’t believe we’ve managed to fool ourselves into thinking that you can reduce REALITY to a few squares on a log-frame all this time. 

Upshot: as Abraham Lincoln says, "we must disenthrall ourselves" (one of many, many brilliant quotes in the book).

One part of this disenthralling is to stop looking for panaceas in development. Owen Barder recounts them nicely for us: more capital, more savings, more aid, more technology, better policies, better institutions, better politics. In fact, the desire for panaceas is itself part of the problem. And no, thinking in terms of complex adaptive systems is not the panacea either. This might seem obvious: accepting that there isn’t one solution to the problem of achieving development sounds easy, but actually our brains have a sneaky tendency to slip back into that way of thinking. This came home to me listening to a podcast in preparation for this post (it’s sad but I actually do prepare); Humanosphere editor Tom Paulson, when interviewing Owen Barder, asked him whether complexity thinking means that we need to look at tax havens to achieve development. Barder: that’s exactly the type of panacea thinking we are trying to get away from. Yeah.

So what are complex adaptive systems/systems of organised complexity? Let me throw some words at you that together constitute a vague approximation to an answer. Complex adaptive systems have large numbers of mutually interacting parts, are open to the environment, are self-organising in their internal structure, have ‘emergent’ macroscopic properties over and above the properties of their constitutive parts (think human consciousness as more than putting together feet and white blood cells and nerves and lungs). Complex systems are interconnected and interdependent and dynamic, constantly evolving and adapting. From all that I’m sure it’s apparent that I don’t totally understand either. Looking at the box might be helpful at this point. In fact Ramalingam takes pains to emphasise that complexity thinking is itself a work in progress even for people who research it, so don’t feel bad.

Comparison of conventional aid thinking and complexity-influenced ideas about development

Okay so that’s all well and interesting that clever people have found out why everyone’s been wrong for the last seventy years but what does it mean for development? Ramalingam provides some broad-brush messages on this: complexity thinking can’t tell you what to do on Monday morning, but it can help us understand the world around us better, promote more open debate about the challenges facing us, and help us think up new approaches to problems. It can help us “see through new eyes”. In this sense, development is more of an emergent property of a society/economy rather than a outcome of a process to be engineered. Aid in an ideal world then becomes “an open innovation network,” an “internal catalyst...[to] identify, expand and sustain the space for change,” a “fluid, dynamic, emergent” process. We stop looking for ‘the answer’ because, as the NYU Development Research Institute strapline goes, “there are no answers for global poverty. There are only answer-finding systems.” If nothing else, you can’t say complexity thinking isn’t good at snappy one-liners.

The thrill from this enormous potential is kind of feeding into my disillusionment with NGO work at the moment; the organisations I have experienced thus far are not embedded in the relevant 'system', not looking to foster small adaptive changes whilst accepting the underlying complexity of the problem they are trying to address. The organisations I have worked for have their ‘product’ with its own intellectual framework which is superimposed onto a given situation. They have their own thinking, activities and feedback loops which are only tangentially aligned with the problem at hand with all its complexity, non-linear characteristics and emergent properties. And most dangerous of all, they are self-sustaining, insulated from their own inadequacies by the current aid system.

Ramalingam shows us complex systems thinking in action: it’s all over the place apparently, from emergent leadership in Obama’s election campaign, to holistic range management in Zimbabwe, to conflict resolution in Aceh after the Boxing Day tsunami, to the development of M-Pesa mobile money in Kenya. The book is filled with examples.

After the above whirlwind, I’m still left with a few questions though:

• How do you identify a system? What is the scale of a system?
• How do you know when something is not a problem of organized complexity?
• What would development institutions/donors/NGOs look like if you took them apart and rebuilt them to be equipped to face problems of complexity?
• What are the steps to changing aid in the right direction? Is it just that we read Ramalingam's book, realise we’ve been doing it wrong and change our approach? How do you change the system? (Ramalingam recognises the difficulties of this with the remark that “thousands of careers depend on sustaining certain ways of looking at the world”. That'd be politics again.)

Time to read it again I guess.

RCTs: Really Crap Things or Really Cool Things?

Credit: http://cplanicka.blogspot.com.es/

I’ve spent the last couple of months enrolled on two MOOCs (massive open online courses), ‘The Challenges of Global Poverty’, and ‘Evaluating Social Programs’, both effectively run by J-PAL, a research centre at MIT. I know what you’re thinking “Wow there Charlie, this year abroad’s taking you off the RAILS”. Well screw you. Anyway this has meant three things: 1) I’ve been exposed to a LOT of chat about Randomised Control Trials (RCTs); 2) I find the meme on the right much funnier than it actually is; and 3) I kind of wish I were an economist.

I want to talk about the first of those three things. Specifically: what are the downsides of RCTs?

First what they are (and I’m going to try to do this in my own layman words): a randomised control trial is a particular method of impact evaluation for an intervention (eg in development). It gets together a load of people, randomly assigns half of the group (the treatment group) to receive the particular intervention (e.g. access to microfinance) and half (the control group) to not receive the intervention, then at the end of the program measures the difference between treatment and control groups with respect to an appropriate indicator (e.g. income) to get the ‘impact’ of the intervention. Basically, the fact that the two groups were randomly chosen means that they were statistically the same, and so the cause of any difference in outcome between them at the end must be the intervention. Phew.

They’re argued to be pretty good at demonstrating the effectiveness (or not) of programs because they allow you to get to the heart of the causality, getting us closer to the ‘impact seeker’s Nirvana’.

Now after over two months of Esther Duflo and her ‘randomista’ (proponents of randomisation) babes enlightening me about all the cool things they’ve found out using their economist ‘gold standard’ tricks (like if you bribe them with dal, mothers are more likely to get their kids vaccinated. Or providing teenage girls with school uniforms can reduce rates of teen pregnancy more than sex-ed programs), I was ready to read some damning critique. But although RCTs to take a lot of flak, much of the arguments against seem weirdly bad. There’s a bit of a non-debate going on in many quarters.

1. Here’s one: it’s wrong to play God and experiment on people.
Reply: this one’s crap. As Howard White says, “all new policies are essentially experiments”. Now if we knew all the answers in development, then doing RCTs would be, at worst, expensive and useless. We don’t, so trialling and evaluating programmes as best we can to find out what does seems like the opposite of wrong.

2. And another, slightly better: RCTs are immoral because you’re rolling a dice to see who gets the bednet and who doesn’t.
Reply: you always have to choose who receives the intervention and who doesn’t; doing it explicitly and by chance its better than doing it implicitly and by virtue of the distance of the village from the nearest four-star hotel. RCTs never lead to “fewer people getting a service than they would if we haven’t been working on the evaluation” as Rachel Glennester, a randomista, puts it. Also, when you do an evaluation you don’t know whether the intervention works, so you might be better off not receiving it. Which is a bit of a backhanded benefit, but anyway.

Wait, so if the arguments against are rubbish, is this the answer for finding what works in development? Hah no, cause development is BORING, as thinkers ranging from Francisco Toro to my younger brother have argued (with slightly different meanings), and so there’s never one exciting answer, always lots of prosaic partial answers. As Lant Pritchett expresses it “RCTs are one hammer in the development toolkit and previously protruding nails were ignored for lack of a hammer, but not every development problem is a nail”. Some more thorny problems:

3. External validity: you can’t generalise from your RCT because the context matters. Owen Barder: “[in] the obvious example of the de-worming program, it clearly makes sense in communities that suffer from that kind of worm. But you clearly couldn’t generalize to communities where there aren’t worms”. ‘Solutions are slippery’ and you have to acknowledge the ‘hyper-dimensionalityof the design space’. Fuck. Sounds difficult and is. In fact so difficult that they do RCTs to see if the RCTs can be generalised.

4. Too much what not enough why: learning the impact of an intervention doesn’t tell you why it works or not. Also true.

5. Ironically, there is little rigorous evidence to suggest that rigorous evidence is used in policy. Lant Pritchett, chief development troller describes RCT-use as a ‘faith-based activity’. Meanwhile Philip Krause makes the obvious but valid point that “today’s rich countries didn’t get rich by using evidence systematically”.

6. Leaving out the big questions: you can’t run an RCT on whether aid works. You can’t use an RCT to determine what stimulates economic growth. You can’t use an RCT to test the effects of fixed vs. flexible exchange rates. Randomistas would accept this, and perhaps suggest that big ‘macro’ questions are made up of little ‘micro’ questions which they can answer. Debatable. But even WORSE, RCTs can’t answer many ‘micro’ questions either. Ben Ramalingam argues that many of the thorniest problems facing the world today are problems of ‘organised complexity’. With these, all the randomista assumptions about the intervention being the only difference between experiment and control group, linear cause and effect, etc. just don’t hold.

Basically, RCTs are good – so definitely read Poor Economics -... but not that good. According to the hype cycle, they are currently sunning themselves at the Peak of Inflated Expectations, soon to tumble down into the Trough of Disillusionment. Development is boring and there are no quick answers. Hooray!

The hype cycle, credit Gartner

DISCLAIMER: Having said all this, the above issues are slightly more nuanced than a short, deliberately-flippant blogpost would allow. Here are some extra resources (pick a couple) in the unlikely event you want to find out more.

A nice summary piece on RCTs in Slate

An in-depth document explaining RCTs by International Initiative for Impact Evaluation 'Big Cheese' Howard White

Some criticism of RCTs in the New York Times and a response from Jessica Goldberg

A development drums podcast on ‘Randomized Evaluations’

Some great blogposts questioning the relevance and impact of RCTs/raining on the parade

A more detailed look at the ethics of RCTs on the World Bank blog. Plus responses #1 #2 and #3

10 Reasons Why You Should Not Donate To Charlie Satow This Summer

I'm fundraising to work as a 'Coordinator' with Education Partnerships Africa this summer (more info in link). I'm also running the Madrid Half Marathon. My target is 1000 pounds. 

If either of these things move you please donate as little or as much as you want. However before you do, please read this list of reasons why you should DEFINITELY NOT donate.

1. Because I already went to Uganda like two times already. Uganda doesn’t need any more assistance from Charlie.
2. Because this is the third consecutive year I’ve asked you to give your (potentially hard-earned) money. FFS. Just get a job.
3. Because this is the second long distance run I’ve done for fundraising purposes in the last two years. I’m displaying a total lack of creativity, and on top of that I’m only running half as far as last time.
4. Because this money unashamedly goes on ‘administrative costs’. This year my job is to support Project Workers and to look for new schools to work with so essentially ALL your money goes on getting ME to Mbarara and keeping ME alive for a month (not on me going out for dinner/safari/getting fantastic ‘African print’ shirts tailored don’t worry). So if you think of this as a holiday for me or if you think ‘administrative costs’ are the devil’s work, it would be positively immoral for you to donate.
5. Because your money would do way more good if you gave it to any one of the following, rigorously-evaluated organisations (GiveWell, Schistosomiasis Control Initiative, Deworm the World Initiative). Or perhaps Oxfam, UNICEF or WFP, which generally all do excellent work. If you actually do this, send me a confirmation email or whatever because I will think of it as equal to or better than a donation to me. GO YOU.
6. Because my Facebook is just turning into a self-promotion channel (blog, fundraising etc) and you don’t want to reward that by donating. Though you could see it as giving the middle finger to the idea of Facebook, in which case you might want to reward that by donating.
7. Because you reject Peter Singer’s argument that we have a moral obligation to give to development organizations. That analogy about saving the child from the pond just isn’t valid.
8. Because you yourself are fundraising for a cause this summer. Good news! In that case let’s spread the love and mutually donate. Just let me know.
9. Because you’ve read a book by William Easterly or Dambisa Moyo and you think you’re all that and aid projects are BS. Run along you little scamp.
10. Because, just, like, me continually going on about development is plain irritating.

 If you aren't convinced by any of these arguments, I guess you should donate...

Involuntary guest post by Ben Ramalingam

I'm currently reading Aid on the Edge of Chaos by Ben Ramalingam, which attempts to apply complexity theory, i.e. the next big thing about five years ago (hate it when I get there late), to aid policy and practice. You should read it. Just finished Part 1, which dances lightly through the thinking and implicit theories behind the aid industry over the last sixty years and rips them apart, with wit, cartoons, punny chapter titles (Chapter 4 - 'The Goats in the Machine') and occasional classical references. The attack is directed at the linear, simplistic view of reality (cause and effect, rational actors, closed systems) derived from Newtonian physics, which underpins much of aid thinking and is totally inadequate for addressing the complex problems of development. There were some great passages which articulate, much more lucidly, elements of the frustration which underpinned my last post. Sharing is caring.

The problem with the aid industry:

"Despite the grander claims of some recent movements, development and humanitarian work is not a knowledge industry - except in the most idealistic interpretation. It is an export industry, and an exceptionally blunt, supply-oriented one at that. It gathers up poverty, vulnerability, and suffering from the South, packages them for sale in the West, and exports off-the-peg solutions back in relentless waves of best-practicitis"

And the traditional response:

"In the face of widespread institutional inertia, the resilient cookie cutters and travelling orthodoxies of foreign aid, the strategy most commonly found is to simply give up on trying to find a good solution. Just follow orders, do your job, and try not to get in trouble. Let the system do what it will."

The message:

"[A]id agencies are increasingly dealing with a world for which their learning, strategic, performance, and organizational frameworks were not designed"

If you fancy learning what more appropriate frameworks would be, take your pick from the following links on complexity in development, or, better, read the book. If I had to go for one (other than the book) I'd recommend the Owen Barder podcast (not that I've read all of these myself).

Papers/books (for the brave):

A (long) one by Ramalingam and others

A book on systems thinking by Donella Meadows

Podcasts (for the bus):

From Humanosphere with Owen Barder (short)

From Owen Barder himself (longer and better)

Blog posts (for the time-pressed):

From Ramalingam on his blog

Various from Duncan Green on his blog

From the Center for Global Development

Development double-think: how to (not) get out of bed in the morning

I'm currently in a very conceptual phase of my dissertation, with phrases like 'cognitive legitimacy,' 'socially constructed' and 'discourse' flying around. I'm trying to make them stop whirring using coffee and marks on bits of paper without success. In light of that I thought I'd continue the self-torture on this blog, by taking an assorted selection of ideas, articles and videos on a topic that occupies me a lot and attempt to construct a narrative. Let's see if pixels on a screen work better than ink on paper. Not promising coherence.

The problem is the co-existence of the following two facts:

1) Day-to-day, NGO/donor agency staff show up at the office and work hard and in good faith, to fill in the logframes/answer the emails/run the 'community consultation' session etc.

2) There is a consciousness among those who work in development that what NGOs/donors do is often extremely problematic conceptually and is fundamentally not what poor countries need.

Binyavanga Wainaina in a recent interview with the Guardian notes the guilty 'eye-rolling' of the aid worker betraying the fact she 'doesn't buy the bullshit' of aid work, just as he doesn't. Aid workers 'admit the irony, but the situation persists'. Neil McCulloch talks of the increasing popularity of the “political economy” approach to development assistance, attempting "to apply a more political approach to understanding development problems and, importantly, development “solutions”. In particular, a central tenet of the approach is that many development problems are fundamentally political rather than technical and that therefore solutions to these problems are most likely to come from inside a country’s polity than from outside." He finishes by remarking "Sadly, the political economy of donor incentives means that it [this approach] will probably remain a marginal pursuit [in development cooperation]." Banks and Hulme note in their 2012 paper on the role of NGOs in development the "increasingly professional and depoliticised nature [of NGOs] and their subsequent limitations in promoting long-term structural change."

I'm going to resist the temptation to make a massive judgement on whether aid is good or evil, not least because as I've remarked before, it's not the right question to ask. What I'm interested and perturbed by is the double-think that allows an development worker to carry out their job in good faith at the same time as knowing that it's inadequate as a solution to the task of helping countries to provide a better life for their people. This is something I feel myself in the work I do.

The paradox is easily explained in one sense: as Banks and Hulme put it, “organisational imperatives" prevail over the "development vision”. Much of the work that NGOs carry out responds to their desire to help not to 'satisfy any existing constituency' in Wainaina's words. There's no logic of the market or of politics to force NGOs to do what people actually need, so the development industry is free to define the task and the appropriate solution as they wish providing they can justify the whole thing to themselves.

And that's the key: to get over the cognitive disjuncture, you create a set of intermediary concepts like 'NGO' or 'partnership' or 'empowerment' or 'development' that you can imbue themselves with positive meaning, to save you the mental anguish of actually tracing the logic of what you are doing to its results and power implications. Because the results often ain't pretty. It's circular: I want to help those who are worse off in the world so I go do 'development', and 'development' is good because it's 'development'. The only problem now is that if you recognise all that and still work in an ineffective organisation (which is a lot of them), then that is probably unacceptably cynical.

Anyway I'm sure I've mixed up concepts, and ultimately make a fairly trivial, well-recognised point. But let's finish with a kind-of-relevant section of the Wainaina interview because he's brilliant and illustrates my existential problem perfectly (yes before you ask, development IS about me - that's the premise of the blog). 

"The utter cynicism of it is the assumption that there is no politics in Africa, so you simply bypass it; so you find a really, really good boy who's the best in his class in an English institution, and has symmetrical features, and [an impeccable] past, and so you know that when you know that when you give them a budget line that they'll stay [within it]...as you have defined. And those are what I call applications of power with a capital-P"