Managing the software development process is broken. Just my opinion. I have yet to put my finger on exactly what needs to be fixed, and guess what? No one else has either. If they did the software dev process wouldn't be broken. Yet corporations and management constant search for the elusive fix, like Francisoco de Orellana searching for the Lost City of Gold. Fred Brooks' classic, The Mythical Man Month, comes the closest to flat-out telling us, "There is no El Dorado."
This is a lead-in post to a DevOps series I'm going to do. DevOps is the latest software development method out there. It is one that I truly believe can deliver on its promises of fixing some of the ills of our industry.
The Cargo Cult
Sometimes people observe a successful outcome and believe that if they replicate the circumstances that they too can reproduce the same outcome. When success isn't forthcoming it is because the circumstances had absolutely nothing to do with the outcome. This is a fallacy called, post hoc ergo propter hoc...after this, therefore because of this. Like any fallacy, understanding when you are susceptible to it is half the battle. And in my opinion, software architects primary responsibility is understanding when their organizations are falling for fallacious concepts.
So why is it a Cargo Cult? World War II brought western wealth to indigenous peoples of once-isolated, tiny Pacific islands. These people had little exposure to the West and led more primitive lives. In many cases these islands didn't see the bloodshed of war, instead they were used as supply depots. Some jungle would be cut down and a landing strip would be made with landing lights to guide the supply planes. The cargo was often food that wasn't always easy to get on these islands. After the war these islands were abandoned by the war powers and suddenly the indigenous people didn't see food and cargo any more. There are documented cases where these people would cut down more trees and build more landing strips and then light bonfires and wave discarded signal flags, yet no planes would land. This baffled the people who assumed that these circumstances were responsible for their food.
And it is this cargo cult behavior that I see today when adopting new software development methods. Management says (or hears) that some process (agile maybe) was involved in some wildly successful software project. Management believes that therefore if they adopt the process that they too will experience success.
Other Cargo Cult Examples
Various Software Development Methods
I've worked in, and think I understand, *at least* the following development methods:
|Method||Dime Tour||Does it work?|
|Waterfall||actually I'm not sure about this one. I've never had any company actually claim to be waterfall. I think that is because "waterfall" has a negative connotation. But some of us know when we've worked in a waterfall environment. And it isn't all bad.||I've worked in waterfall environments that were wildly profitable and fun. And I've seen waterfall projects be death marches too.|
|Scrum/Agile||I lump these together. They are separate and unique but in every environment I've worked in the practitioners claim they practice both.||Works ok but practitioners sometimes adhere to the tenants like it was a religion. It isn't, don't drink the Kool-Aid|
|XP||another variant of agile||Works for me.|
|ICONIX||I started out on this. I guess this is roughly waterfall.||Worked for me.|
|Rational Unified Process||ok, maybe I didn't understand this one when I was forced to use it. Kinda waterfall-ish I guess.||We shipped software with it and were profitable, and that's the metric I use for a working method.|
|Kanban||a just-in-time method. It attempts to focus developers on what is really important and on getting that done. Runs the software dev process like a manager would from a catwalk on a manufacturing floor.||Works well when people understand more than the simple platitudes like "stop starting and start finishing" and really understand how "work works"|
|Microsoft Operations Framework||I actually worked at a place that took MOF and made it into a software engineering process. M$ even came in and did a lecture series on how to make it work.||Seemed ok...we made money...I got home at a decent hour most nights.|
|Inception||this seems to be the "old new thing" right now. Really, it is part of RUP. Inception is the first phase of software development where you try to broadly grasp the problem and approximate how much effort will be required. So, there is an up-front focus on requirements gathering and design. Um, that sounds kinda waterfall-ish to me (but don't tell an Inception Guru that). I worked at a place that spent millions to bring in a bunch of Incepticons (Inception Consultants) to teach us how to do Inception right. They were gone within a year, and so was Inception. We went back to scrum/agile. We would an entire sprint "incepting" only to realize by the end of a release that the product looked nothing like what we incepted. To save face in the wake of wasting millions, management decided that we should use "just-in-time inception" instead, which is a mini-inception at the beginning of each sprint. Um, I call that "basic planning".||Complete failure if you follow the religious dogma.|
|DevOps||a lot like kanban but with an emphasis on developers working with ops people to deliver more supportable products. I'm actually going somewhere with this post...it's going to be the first of many posts on DevOps and how to move towards DevOps.||This is a game-changer.|
So, which one is the best?
Any of these processes can work if practiced properly. I have no affinity to any of them (except DevOps which I'm starting to really like). They each have their good and bad points. I can work and succeed in any of them. Or fail in any of them.
I would NEVER make any of these statements though:
- "Our project succeeded because we finally realized the truths in <insert software development method here> and practiced it. "
- "We failed to meet our objectives because we didn't follow <insert software development method here> the way we should have. "
There are common denominators that each process has. Reliance on structure, process, and measurement for example. But this isn't radical.
HPCs (Highly Paid Consultants)
If you are looking for a new method, maybe because your software team is not as productive as you feel it should be, you should be aware that you'll find lots of HPCs who will want to sell you their expertise in a given method. These shills will ALWAYS exhibit these characteristics:
- "If you adopt this approach then all of your development problems will go away. "
- "We are consultants that would love to come in and train your people on The Way. We don't come cheap but you can't afford not to learn from our experience."
- "Every other process is a fraud and will never work. We have lists of how our process is better than whatever process you currently use. Use this one to ensure success."
- The "Who We Are" slide of their deck is always a mugshot of each of the two HPCs giving the presentation. The first HPC is in the process of a deep belly laugh. This connotes how they will make your failing projects a joy to work on. The second HPC mugshot is a contemplative pose..."we have wisdom."
- One of the slides will tell you how they've been training The Way for the past x years with success. They won't list specific companies or software, but they'll tout their successes with both small and large firms. They don't know how to code in any specific language or domain, but they can manage the process. They'll have lots of photos of their teams huddled around kanban boards collaborating over a bunch of color-coded Post-It notes, trying to determine what they are going to fix next...world hunger or peace? They'll conveniently "forget" to mention how this collaboration is going to work with offshoring.
- They'll always provide you with lots of swag. It may be "planning poker" cards with their logo on it, a board game they devised to use in their training classes, or bags with catchy logos like "Stop Starting...Start Finishing".
All of this is a complete waste of time and money. Just my two cents. A HPC cannot possibly know your business domain, your software, or your people. To assume that what worked previously for the HPC will work again for you is very Cargo Cult-ish.
So, what does work?
I don't like to complain without offering alternatives. This is what I feel are the most important concepts that any method should espouse:
- Never shackle people. If your framework commands, "Thy shalt have daily standups where we discuss 'pulling cards from the left' (Kanban)", but the team likes to do Scrum-style standups, then allow people some latitude. Let the team alter the framework to be assistive, not restrictive.
- No multi-tasking. I call multi-tasking "faulty-tasking", because it DOES NOT WORK. Human beings are not computers, they cannot multi-task. Humans, instead, can context switch. A developer can work on one and only one piece of code at a time. Allow your IT people to complete a task, do not make them context switch.
- Don't rely too much on metrics. Burn down charts and kanban cards and points are nice, but the focus should be on customer satisfaction.
- Don't spend millions on HPCs or special "Kanban TVs" for each team room. It annoys people when large sums of money are spent on this nonsense yet raises were non-existent the last 2 years.
- Ensure everyone is allowed to participate. Too often I've seen cases where a new method was instituted for the onshore workers, but not for the offshore guys...they aren't important after all.
- Practice MBWA. Management By Wandering Around was practiced by HP back in the 1970's. It's simple...managers get out of their offices and visit employees randomly, staying quiet and observing. Good managers will quickly determine lots of little things that are impeding process and can remove those barriers.
- Consider adopting a new framework every other year on a trial basis. Have your people learn the new method and bring it in-house. Then do retrospectives often to see what is working and what should be discarded. It's always a good idea to try new things. I never thought software development management theory could improve but then I started to learn DevOps and realized that it in fact does contain a lot of useful concepts that are obvious, but under-employed by most organizations.
Over the course of my career I've seen the companies I've worked for bring in lots of management frameworks and paradigms to try to solve what is wrong with their IT departments. All of these "tools" have a bit of truth and value, but I hold that you can't solve a people problem with a slick framework. The software development process is broken because fundamentally it is a people problem, not a process problem. There are patterns and anti-patterns that I list above that I've seen work universally. Don't be dogmatic about any process and don't be part of a Cargo Cult.
other management devops