Why New Systems Fail: an interview
My friend Phil Simon published a book in 2008 called Why New Systems Fail. The book explores the many reasons why projects, mainly IT projects, fail and what a wise person can do to prevent and recover from these situations. It’s a hard nosed, low to the ground book, which tend to be my favorites. The book did surprisingly well in its first printing, so the book was reissued recently in an updated and revised edition.
Given failure is one of my favorite topics, I interviewed him about all things failure and his take on why some projects work well, but most don’t.
CONTEST: Phil will send a copy of the book to the person who leaves the best comment or question below.
SB: I’m a fan of failure and its value in teaching us things. But to label a book “Why new systems fail” might seem cynical or depressing to some. Why did you choose such a provocative title?
PS: Some have called me (and others who write routinely about failure) cynical. If you look at the statistics on IT project failures, though, you’ll see that more than three in five fail. So, is that really being pessimistic or realistic? I tried to write the book in a way to maximize success but, to me, it’s irresponsible and misleading to pretend that IT projects tend to go well.
More than just whining about problems, I pepper solutions to implementation and project challenges throughout the book. It’s about avoiding failure, not simply stating that things can go awry.
Consider the analogy of going to the doctor’s office. How can you expect to be cured if you don’t talk about what’s ailing you? How can a doctor prescribe treatment?
There are big differences in culture between projects for profit, and IT or infrastructure projects inside companies. The latter are often more bureaucratic and political. Do you think there is a difference? Are IT projects harder or easier?
I completely agree with you. In short, I believe that generally there is a difference between the two (although there are exceptions).
“For profit” software development projects tend to differ substantially from “implementation” projects. Let’s define the latter as those that involve purchasing a software vendor’s prepackaged product, configuring it, testing it, and implementing it. Those systems never face external customers. They’ll be used exclusively by internal folks in accounting, HR, payroll, etc.
Developing a product to be released to the market can unify a company, particularly if it’s a small outfit. If the product’s people are incentivized (through bonuses, stock options, or recognition), then the development can go (relatively) smoothly. People want to make the product a reality. There can be an “all hands on deck” approach because that product is part and parcel of the company’s identify. Employees want to show off its bells and whistles. Human obstacles to progress are more quickly identified and removed if necessary.
On the other hand, implementations rarely go smoothly for all sorts of reasons covered in the book. At a high level, different people and departments often have vastly different agendas, up to and including killing the implementation itself. I’ve seen it many times. Often, this lack of a singular organizational focus ultimately causes IT projects to fail. Some people don’t want the new system; they like the old one just fine.
As to which is harder, it’s hard for me to unilaterally say. I have worked on relatively simple implementations that have gone fairly smoothly, even though the system was quite complicated. People knew that their jobs would change and they just dealt with that.
It’s one thing for something to fail dramatically (unmitigated disaster), like say the Challenger shuttle disaster, where it’s so bad everyone is forced to confront the causes. But in IT it’s often easy to hide, bury or whitewash projects so they don’t seem as bad as they are. Do you think the ability to hide failure contributes to the frequency of failure? What can a manager do to minimize this?
Absolutely. For obvious reasons, organizations and key people aligned with the success of a project have strong incentives to under-report failures. For example, I have seen people call activating a reporting dashboard “a success” even though it contained only one report and no one ever used it. I have also seen people fudge numbers to make overages less obvious.
As for what managers can do, it’s a very tough question because of all of the constraints. What do you do when the CIO essentially tells you that you have a week to do something that should take a month? What do you do when you recognize that a key person isn’t performing yet have no direct authority over him/her? What if your boss won’t publicly cop to it, but he’s told you privately that he wants the project killed or postponed?
These are all people issues that make up the much of the book.
Why New Systems Fail reads like a handbook and checklist for common mistakes to avoid on IT projects. But in many cases the being aware of problems doesn’t mean you have enough power to prevent them. How do you compare political and organizational problems to project management problems in terms of how often they’re the root cause?
As you know, awareness of an issue and the ability to fix it are two very different things. It’s hard to separate PM issues from cultural/political/organizational ones. For instance, it’s possible to have a great PM and team running up against constant institutional issues that prevent the project from being successful. It’s also possible to have a great culture conducive to success and a poorly planned and executed project within that culture.
I have found in my years as a technology consultant that both of these two extremes are unlikely to be found. Typically, organizations with oodles of politics, an aversion to change, difficult end users, and cultural problems tend to run projects poorly. That includes appropriate staffing levels, comprehensive requirements’ gathering, sufficient data validation and testing, acceptance of new technologies, and all of the things that collectively cause projects to fail.
I have said before that Jack Welch himself could not have rescued some of the unmitigated disasters in the book. Companies with challenging cultures have a tough time managing projects and implementing software on time. Is this a PM issue a cultural one? It’s a chicken-and-egg question.
If you like Phil’s line of thinking, which I do, the book is available from Amazon (also kindle edition) but Phil blogs regularly as well and tweets here – check it out.
CONTEST: Phil will send a copy of the book to the person who leaves the best comment (in his opinion 🙂 or question below.