The future is about responsiveness: Jim Highsmith on Agile

Last week I met with Jim Highsmith at an event organised by ThoughtWorks, an IT consultancy for whom he works. Highsmith is one of the authors of the 2001 Agile Manifesto, which is so short I can quote it here:

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

That was over ten years ago, but Highsmith told me that his Agile journey because ten years before that. "I started doing first iterative development projects in the early Nineties," he told me.

Given the Manifesto's statement about tools, I asked Highsmith what he thought about the role of tools in Agile development today. I am aware, of course, that tool vendors frequently highlight their suitability for Agile (rightly or wrongly) as a marketing point.

"Vendors have come at it from two different directions," he said. "There is a set of vendors who basically had products, like IBM and their Rational toolset, who have essentially tried to reposition, redevelop or redesign those to do Agile, and then there are companies that have built up from the bottom, the Rally's, the ThoughtWorks, and built the tools out.

"As you have worked with organisations at scale, some of those tools have changed. So at one Agile team you can use a spreadsheet or just a storyboard, but if you've got a big organisation you have to have some additional complexity than that.

"All that said, I think vendors are overdoing it in some instances in order to appeal to organisations who may not be quite agile enough yet. For example I know one of the vendors was asked one time by one of their big clients 'We want you to do burn down charts by individual.' Their response was: you don't understand Agile and we're not going to do that.

"The Agile Manifesto said people and process over tools, but it didn't say instead of. The tooling is important, particularly around development stuff, automated testing tools, build pipeline tools, things that really help people operate in an agile manner."

How has his thinking about Agile evolved in recent years?

"It has not changed in terms of how to do Agile, but it has changed in terms of scale. Five years ago we were still doing bottom-up Agile implementations. Teams would come in and get dispensation to do an Agile project. In the last five years we've gotten a lot more top-down, we've got VPs of engineering and CIOs and CTOs coming in and saying, we want our organisation to be Agile. The largest one I've dealt with is 25,000 software engineers in China, at one company.

"So you get more of that which has its own problems and issues."

What are the biggest obstacles to Agile?

"One of the things that's happened, first with the movement to the web and now with the movement to mobility, and cloud, and big data, and those kinds of things, is that the technology changes are forcing people to look at business changes. So there is more interest. But I think there's still a lot of inertia in big companies, though you are seeing more and more big banks, big insurance companies, big manufacturing companies, that have realised the future is about responsiveness, that are moving to Agile." he said.

The question of scale came up repeatedly and seems to be a sign of maturity within Agile. It is no longer in any sense an underground movement, and Highsmith understands that change cannot only be bottom-up. His current topic is adaptive leadership, emphasising that top-down is also needed. "It is a mix ... It has got to be driven from the bottom in terms of delivering software, but if you don't also support it from the top you can't scale fast enough."