Gil Zilberfeld, an agile practices expert, has posted on 4 warning signs that Agile is declining. I will not re-iterate his points; you should read the article for yourself. The overall theme though is this. The software establishment - including the managers, the consultants, the trainers, the vendors - has embraced Agile, to the extent that, according to Zilberfeld, more than half of software projects at least nominally use Agile methods. The results are disappointing though, because companies have simply absorbed bits of Agile into their existing top-down management culture. Therefore:
At this point, I feel Agile is declining into what TQM [Total Quality Management] was. A brilliant success in the beginning, and now just a history fact. In a few years, months even, the business side will wake up and say: Agile is snake oil. It doesn't deliver on its promise (and it doesn't matter if it's done wrong). The backlash will be grand.
I am reminded of something I learned from one of the excellent QCon conferences in London, which covers Agile in depth. It was not so much a specific speaker or talk, more a common theme running through many presentations. Software projects generally do not fail for technical reasons. They fail because the team - using the word team in its widest sense, to include all project stakeholders from users to executives - fails to communicate effectively. Many Agile techniques, such as the daily meeting which is part of the Scrum methodology, are designed to facilitate communication. I have also heard recommendations such as moving developers into the same office as managers in order to get them talking to each other. Another example, at a micro level, is Pair Programming, where two developers work side by side on the same code. You cannot do this without communicating your intentions, ideas and solutions to the person alongside you.
Kent Beck, one of the pioneers of Extreme Programming and Test Driven Development, highlights the human factor in software development. Take a look, for example, at his essay on Accountability in Software Development:
... while programming I offer accountability as a way of demonstrating my trustworthiness and encouraging my own best behavior. Pair programming; test-first programming; continuous integration; visible daily, weekly, and quarterly cycles; slack; and estimation are some of the way I make public commitments and render account of my activities. Knowing I will be honest and accountable affects how I do my work, just as knowing that I am hiding and concealing negatively affects how I do my work. Taking responsibility for my choices and actions deflects blame. There is no hidden shame if everything I do is above board.
What is important here is not the techniques in themeselves, but the trustworthiness and accountability they facilitate.
Human problems are harder to solve than technical problems, and seen in this light it is not surprising that companies which adopt bits of Agile methodology without changes in corporate and management culture will miss out on most of the benefits.
Another common experience at software development conferences is to talk to developers who enthuse over the insights they have received, but then lament that they cannot be applied in their workplace. The reasons are old and familiar: inflexible management, longstanding broken processes that nobody seems able to fix, little kingdoms which protect their borders at the expense of the effectiveness of the whole corporation, and so on.
A few thoughts in conclusion. First, if Agile projects are failing, that does not necessarily imply that something is wrong with Agile methodology. It all depends on how it is done and whether the people involved are embracing or resisting the change and communication that goes along with it. Second, irrespective of the methodology, effective communication is key to the success of a project; and if it is not possible to change the methodology or even the tools and technology, working on team communication may still yield amazing results.
