Comments

Michael and Bill thank you for your comments. There is a lot there and I take Bill's point on large NGOs crowding out local initiatives. Of course though my point is that it depends on the NGO, their reach, and to some extent the country. Countries with a strong local NGO sector like India Bangladesh or the Philippines see little crowding out; except in the case of Bangladesh where is the big local NGOs like Proshika or BRAC who do the crowding out. On Michaels’ point I sort of answer that in the following section and that is I don't to think it is the closeness to government or otherwise that limits NGOP effectiveness in what they set out to do but rather their world view or what Lissner calls their Weltanschauung or world view. Most NGOs run agendas based on their values, and for the majority these are altruistic or religious values, and for a minority it would be structural social change. My point is that historically it has always been thus. Lissner's Politics of Altruism was written long before the big government donor dollars were there and it reads as relevant today as it was in the 1970s. In Australia the number of NGOs accredited to receive AusAID find has halved in the last 20 years. The accreditation criteria are largely about the foreign aid system and its approaches, but with a strong emphasis on transparent partnership and local control. The Codes of Conduct are trying to move agencies towards more local control and participation in their processes but it is a slow process. I think the issue of about NGOs and what they are on about rather than their closeness or otherwise to government which fluctuates according to the national political cycle (i.e whio is in power). The one big change over the last forty years has been the internationalising of donor NGOs, where most are now part of larger global networks, but this does not mean they are more or less in the thrall of government funding, but perhaps that their world views are reinforced, which may or may not be a good thing.

Thanks Enrique for your post. It's a hard balancing act, especially when there are a small pool of organisations with the skills and resources to both implement and evaluate projects. In the private sector, there are four major audit firms that audit most if not all the large multinational companies. However there are regulations in place to ensure that the same audit firm cannot provide advice on internal controls while also undertaking the external audit. As I recall there are also regulations in place to prevent one audit firm from auditing the same company for more than 5 consecutive years. Perhaps aid donors could consider similar regulations (if they haven't done so already).

Bob, thanks for providing such an excellent synopsis of this important book and also for generating thoughtful and insightful comments by those who have responded with opinions. One of Indonesia's education goals is to make its citizens more competative in a global economy. A useful measure of progress in meeting the goal is PISA results, where Indonesia performs poorly. It might be an interesting study, if not already done, to investigate the methods used among the top performers on a continum ranging from "formalism" to "progressive". S. Korea and Finland were tops in 2009--two countries globally competative but, I assume, with very different cultural traditions. At the same time it might be interesting for an on-going education project to look at PISA results among Indonesian schools. perhaps some correlations between performance and instructional models might emerge. [Puspendik has the data.]

The overview of any huge scheme like this is fairly useless unless it is translated into previously set benchmark achievements. E.g.: 1. What percentage by country of this expenditure is spent in Australia and never reaches the country AusAID is intending to help? 2. Of the amount that is actually spent in the developing nation, how much is spent on consultancies and expatriate salaries? 3. Of each country’s total share, is there a feedback loop to establish the effects and ongoing benefits accruing? If so, who signs off on this loop? Those actually benefiting at the coal face or those in the capital of the local national government whose budget these AusAID funds augment? 4. Against what percentage of each local national budgetary item is the AusAID allocation? 5. What percentage of each country’s AusAID allocation is actually expended in rural (non metropolitan) areas on rural programs benefiting those in rural areas? 6. Finally, did the auditors actually go into the field and check first hand the benefits accrued in each program or did they only check the written reports submitted by AusAID and expatriate consultancies?

Interesting post, Dinuk. I agree on the need to plan evaluations more carefully to find the right approach for each intervention. Also, that more care needs to go into planning to ensure that existing knowledge is incorporated into the interventions. But also important is the development of a system that ensure real independence. Aid is no longer an 'a-political' issue that voters are unaware and interested in. Efforts to assess the quality of Australian aid will be undermined if this is not done properly. Australia should therefore be careful of avoiding the situation we find in the UK where DFID is the main client of the very same consultancies, NGOs and think tanks that are called to evaluate UK aid. KPMG, for example, manages the aid watchdog but also implements hundreds of millions of Pounds-worth of projects. This clientelistic approach means that think tanks like ODI are also now involved in projects as implementing agents for iNGOs and consultancies (such as PWC and KPMG) which makes their oversight roles impossible. And the same is true with smaller consultancies from communications to social development which often find themselves working with organisations that they are also evaluating. 3ie itself, supposedly the enforcers of absolute certainty, is not free from this. True independent voices are few and unpopular. This situation is not helped by the roles played by new foundations like Gates or iNGOs in using researchers to advocate for their own interests (see, for instance, Gates' development progress work) which goes as far as funding influential media outlets like the Guardian for the same purpose. The consequence is a system with few (if any) lines of accountability; one in which all participants are clearly benefiting from the status quo and the conclusion that 'more aid is good'. The public is beginning to react to this and, unless, important changes are made (and many will be big half-baked PR jobs, unfortunately), the baby will be thrown away along with the bathwater. I think the fault here lies mainly with some large bilateral funders such as DFID that have failed to recognise that different organisations play different roles in the aid sector and that their contributions demand certain degree of specialisation and even protection. A system in which consultancies, research centres, think tanks and NGOs are all expected to compete and collaborate with each other can only lead to uncomfortable and dangerous conflicts of interest. Conflicts that are incompatible with the demand for rigour and transparency in project evaluations. Australia would do well to avoid this muddling of roles. It should attempt to strengthen independent research communities with evaluation expertise separate from those tasked with implementing aid policy. Only this will allow Australian to hold its Aid industry to account.

The overview of any huge scheme like this is fairly useless unless it is translated into previously set benchmark achievements. E.g.: 1. What percentage by country of this expenditure is spent in Australia and never reaches the country AusAID is intending to help? 2. Of the amount that is actually spent in the developing nation, how much is spent on consultancies and expatriate salaries? 3. Of each country's total share, is there a feedback loop to establish the effects and ongoing benefits accruing? If so, who signs off on this loop? Those actually benefiting at the coal face or those in the capital of the local national government whose budget these AusAID funds augment? 4. Against what percentage of each local national budgetary item is the AusAID allocation? 5. What percentage of each country's AusAID allocation is actually expended in rural (non metropolitan) areas on rural programs benefiting those in rural areas? 6. Finally, did the auditors actually go into the field and check first hand the benefits accrued in each program or did they only check the written reports submitted by AusAID and expatriate consultancies?

Thanks Darian. Yours is certainly a story of academic perseverance paying off. All the best in your future career in the sector! Jonathan

Tom, I share many of your sympathies but I shied away from writing about them as I was either afraid of spreading too much hypocrisy throughout the paper (as I said, this is built off of my own experiences and not meant to be a prescriptive exercise) or I am simply too naive about the issue (particularly when we talk about image, settling in a developing country, what people are looking for in a development career etc). On the point of volunteering in rural communities, I did make the point that any work done in Indigenous communities is valued far greatly than any other volunteering one does overseas. I stopped myself from going too far into the subject because I also have never volunteered in an indigenous communities. I do, however, know many people that have done it and found it very rewarding, and plan on doing it myself at some point in the years to come. Overall, however, I don't think I share your cynicism about what drives people to work in the development sector. From the people I have met in my (limited) career to date they are all passionate and driven to see positive changes for those less fortunate. Or maybe I am just too well sheltered in my ivory tower think tank environment. If you want to flesh out your ideas further (and you are not prevented by your employer to do so) I would encourage you to write a blog for us to keep the conversation alive. Thanks a lot for your comments! Jonathan

Dear Benjamin, Thanks for your comments. I have heard about the oversight mechanisms you have mentioned, and my understanding is that they have started to become effective in improving the coordination of evaluations across government. However, in a way, this is an example of the overlapping M&E mandates I refer to in my article: as you point out, there are now several coordination mechanisms for M&E, each of which has a similar mandate but has different meeting schedules, reporting lines and composition. The risk is that the coordination of M&E might itself in the future become as complex as the M&E system itself, if it is not properly managed. The Baraza program does indeed look very promising, even though it is still in a pilot phase. As I mention in my article, it will be most useful if it becomes a forum for generating change: if the feedback that community members provide to their representatives is properly responded to, and their concerns are acted upon. Time will tell if the program will achieve this goal, but the early indications are positive. Thanks again for your contributions.

I agree with Bryant. I think the SWP is a sham. I have been working on various projects in the Solomon Islands since 2005. Many of the young people we have trained for the People's Survey and the CSP Snapshot would make excellent short-term workers in Australia. They are keen to gain experience in industry, learn new skills and have a bit of fun at the same time. The agents in Honiara charge the applicants for the processing of their requents and nothing happens. If we are serious about improving the lifestyle of the educated urban youth then Australia needs to follow the example of the NZ seasonal worker scheme. Many young people in Honiara think the scheme here was set up to fail and announced by the PM just to keep friendly with the SI government.

I am in PNG at present. I have a young man here who would be ideal for the SWP; hard worker, speaks English and is very keen to work in Australia to gain experience that he would apply back in PNG. But there is no information whatsoever available in PNG as to how to apply. The PNG contact provided on the one web site does not reply and does answer phone calls. DFAT's web page is full of promise but says in a number of places they do not administer the program and applicants should apply in their home country - but there is no information on who to contact. I think 6 PNGeans have worked in Australia so far. The whole scheme is a sham and has only benefited the crooks who have pretended to be agents, have taken the K1500 or so applicants and run off with the money. The SWP is all talk and no do.

Hey Jonathan, Thanks for sharing your story. I agree that volunteering overseas is very useful to those wanting a career in or related to international development (however that is defined!) but it is not essential. I am 26 and currently doing a PhD having completed my undergrad degree in Agricultural Economics. I have spent several months in South Africa for research purposes and have spent short periods in Cambodia and Laos working (really short- just a week or two) but have never volunteered overseas (Only for Oaktree here in Australia). I am towards the end of my PhD and this year I applied for some short term jobs with the World Bank (where I was successful) and for the grad program with AusAID (where I was also successful). While I have never worked substantially- even in a full-time job in Australia, evidently my PhD research and the skills I have developed as a researcher and an economist are enough to get me interesting paid work. (My grades aren't spectacular either, I just have a credit average at USyd) I'v been at uni for 8 years, so can not claim to have any suggestions for a 'fast tracked' career in development, but I do know that there are many pathways into the sector (if you could even call it a sector). I'm sure you will come across a wide variety of stories.