Why did Agile become so popular?

In a recent post on his blog, Bob Marshall (flowchainsensei.wordpress.com), says that Agile Coaching is Evil.

He then gives the token denouncement that his post is not link bait, but then spends several paragraphs explaining how he thinks that the promises Agile Coaches make and fail to deliver on — even if unknowingly — qualifies as evil.

I’ll let others decide the morality of that, but I can affirm absolutely, by his own qualification, that Mr. Marshall was engaging in link baiting. But I’m also certain that it was deliberate, any protestation to the contrary notwithstanding — and that dishonesty *is* immoral — if not evil.

It appears that the flowchainsensei is marketing himself as a different brand of “coach”, having found the “Agile” appelation no longer to his liking.

It’s a good move, since there is a pretty popular backlash to “Agile” these days and so it’s almost as good marketing to say you can teach people how to work in a “not-Agile” way — the same way 10 years ago Agile folks claimed they could teach people how to work in a “not-Waterfall” way — with the major exception being that Agile was a label people used to describe themselves and “Waterfall” was a label used by Agile (and other) marketers to describe other people’s processes disparagingly.

There has been a strong backlash against “Agile” from the beginning, and contrary to the “Agile” acolytes claims, it wasn’t from management. It was from programmers, and has grown in popularity with manifestos like this.

But why did Agile become so popular?

I think it’s because almost everyone knows what “agile” means and knew their development process wasn’t. With the possible exception of Agile coaches, and definitely not any of the signatories of the Agile Manifesto.

Read it, and tell me if you see anything that fits the definition of the word “agile” in there?

Unless you consider writing documentation an unnecessary hindrance, and thus not writing it makes you more agile.

That single tenet was the most tempting part of the “Agile” process — and responsible for about 90% of the early converts among developers. It catered to the vanity of the “rockstar” developer personality — they were called “cowboys” back then — because they thought they were so good that they were above the process.

It’s common knowledge that most people think they’re better than they actually are, and their lack of experience prevents them from perceiving their lack of experience. They don’t know that they don’t know what they don’t know.

The idea that writing documentation was unnecessary was quickly disavowed by Agilists in discussions with managers (and others to whom accountability or maintenance was important) and they anticipated the need of such double-speak in the initial manifesto with the phrase:

while there is value in the items on the right (such as writing documentation),
we value the items on the left (such as not writing documentation) more.

As anyone who is familiar with decision making knows, when there is more value in one thing than another –and you have to make a choice– you choose the item that has more value. Especially if it costs less too!

If you’ve ever used a product backlog (or a prioritized task list), you’ve seen this in action. Things that get prioritized higher get done, things that have lower priority don’t get done. Because there is always something with a higher priority to do.

Chances are, the Agilists considered the word “agile” to be a generic one meaning essentially “good” and chose the label for it’s general sense of “coolness”. But it is possible that they too realized what so many who instinctively knew they wanted, and could patently see they did not have: that their development process was not agile, it was rigid.

Writing comprehensive documentation is rigid (as well as boring) — there’s no doubt about that — but so is writing unit tests. It’s an exercise is budgeting and resource management (not a development task) to determine if comprehensive unit tests or comprehensive documentation has a better payoff.

I’d guess from a cost/benefit perspective they’re about equal because documentation is easier to edit than code, and writing (and testing) against it is cheaper than writing code. But of course that’s a false dichotomy. There is a (variable) point at which there are diminishing returns for both writing documentation and writing unit tests.

And the choice isn’t between comprehensive documentation or comprehensive unit tests (neither can be completely comprehensive) — and neither one is a substitute for the other. You need both adequate documentation and adequate unit tests. Without the former, you can not hope to achieve the latter — unless your developers (the ones writing the unit tests) are determining the requirements.

I’ll completely skip over a discussion about whether unit tests are adequate, since I’ve addressed that before.

Doing things in short iterations is possibly the most universally implemented (and praised) practice of Agile. But short iterations doesn’t make one agile. It’s often disparaged as “Agilefall” or “Cascade” development. It can give the semblance of agility (like a higher resolution bitmap can give the semblance of curves) and can potentially help you practice limbering up if you’re particularly rigid — not through the iterative development process itself, but from the introspective effort that is supposed to accompany it — which is probably the least practiced part Agile.

But the reason the “retrospective” part is almost universally neglected (or scorned) is because it DOES NOT work. You can be forced to have meetings every two (or three, or one, or four weeks) where you write down in groups what went well and what didn’t, but unless your heart is in it, and there is unanimity on what is right and wrong (and how to fix it), it doesn’t have any benefit.

It’s really a misguided kindergarten-level way of avoiding decision making and responsibility. The technique doesn’t work on 5 year olds, and it doesn’t work any better on grown men and women.

And that’s the crux of it. Agile started out as a way for developers to take power from ineffective management (and hence it’s appeal, and it’s become a way for ineffective management to shift blame.

I agree with Bob Marshall that Agile Coaching is (if not evil), definitely not a good. It’s a case of those who can’t, teaching those who should be able to. And a deliberate lack of management. It’s snake oil, and every snake oil salesman knows they’re selling something of little value, and that’s evil.

They don’t have to know that it doesn’t work — they know that they are expending little effort and claiming large value. That contradicts the laws of thermodynamics. For every lever or pulley, there are a thousand perpetual motion machines for sale. Don’t expect someone at this stage to have invented something that’s better than the wheel. Even if they claim to have invented a better wheel (an incremental, rather than evolutionary change), you’d better check to see how good they are at making conventional wheels before you believe they’ve stumbled onto a better one.

Best case scenario, it’s just someone who doesn’t know how much they don’t know. But they darn well know they don’t know anything about the process they’re claiming to be able to improve.

The real issue is that we have managers who (even if they’re not inattentive) don’t know how to make software. They can’t effectively mentor developers (if they can’t actually teach them how to be developers (from experience) — and they can’t tell how to effective reward or discipline behavior if they don’t know how to manage.

Agile is not agile –it is definitely rigid– all the moreso the more purely it is practiced. The very concept that Agile has a purity measurement (and is said to be ineffective unless practiced 100% purely) smacks of rigidity (not to mention religiousity) and I think that’s a large cause of the backlash.

The solution isn’t an new process, movement, or manifesto. Although to those selling silver bullets, newness is advantageous, because it’s flaws have not been explored and it’s dogmatism is not yet apparent.

The crisis is not one of process but of leadership. And you can’t outsource that to a consultant or a wet behind the ears MBA who’s been pampered his whole life, or someone with a condescending attitude and a bunch of feel good aphorisms, whatever they label them.


4 thoughts on “Why did Agile become so popular?

  1. Great post. Interesting countries you’ve lived in!

    Back to this post I think you’ll find much to ponder (and your readers as well) on similar subjects on my blog.


  2. Just wanted to add that the whole “unit test” thing is a defense mechanism to discover problems related to refactoring (primarily).

    This refactoring is done because no design up front is done; with proper architecture and design (jettisoning “emergent”) design, the whole need for TDD and unit testing disappears.

    As I’m sure you’re well aware, unit testing does not negate the need for human based testing. It just adds another layer of testing to insulate the codebase from the fact that it was never designed in the first place and must be continually refactored.

    Agile is not efficient — it is just short term oriented.

    • I like to think of unit tests as adding domain knowledge to the compiler. They should rightly be considered a part of the build process and not actual tests.

      Long term technical debt buildup is definitely a problem I’ve seen — even on the best run projects.

  3. I’m curious what overall approach you prefer since it seems like you are not in favor of agile…. I have a writeup of my approach on my blog…

    Getting back to “why did agile become so popular” — I think that deserves a blog post on its own — going into the marketing angle of how they sold this concept in such vast amounts given how weak it is

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s