Second System Syndrome

I don't know about you but I often find myself recreating the same systems over and over and over and over again.  I just finished another SQL script build/deploy project.  The first one I did over 15 years ago.  It was a simple batch file that looped through and executed any .sql scripts it found in its folder using isql.  It worked flawlessly.  It was simple and elegant.  It logged any SELECT and PRINT statements to a .log file using DOS redirection.  Anyone could support it.  

My second foray was for a different job and I tried to improve upon the design.  It now used vbscript which was the latest technology.  It also provided a simple 
API to deploy database objects in a "properties-based" manner.  This means that instead of writing hand-crafted DDL the system let you define what the database object should look like and built the necessary DDL under the covers to get the current state of the object into the desired state.  It was also smart enough to ensure I deployed things in the correct object-dependency order (tables are created before the views, for instance).  

The latest build/deploy project I just created I tried to further improve upon.  It is a total monstrosity.  In my own defense I had a lot of other helpers on the project who gave lots of input into how to make my original designs better.  They (we) took an elegant system and over-complicated it.  This is classic Second System Syndrome...taking an elegant, working system, abandoning that codebase, and rewriting it.  I really think Second System Syndrome is one of the most important software anti-patterns in existence.  

You must recognize Second System Syndrome (SSS) before it gets a foothold in your project.  I once worked on a project where the goal was to rewrite the existing application (expressed in C and SQL Server) using Java with an RDBMS-agnostic data tier.  This was a high performing OLTP system.  First of all, what is wrong with C?  It's high performing right?   The issue was that it wasn't "cool" like Java.  We also wanted to sell our software to shops that refused to run SQL Server.  We had tuned our SQL Server stored procedures and data structures and they were nearly perfect (honestly).  How do we take those stored procs and rewrite them using generic ANSI SQL that was RDBMS-agnostic that would perform equally well on Oracle or SQL Server?  It just can't be done.  Sorry.  The project failed miserably after millions of dollars and years of effort.  

Full rewrites of software applications is rarely a good idea and rarely works.  This is especially true in industries with heavy competition.  Netscape's full rewrite of their browser in the mid-90s failed and MS's Internet Explorer stole most of their market share in the interim.  How many times has MS said that the next version of Windows would fix all of the ills of previous releases?  Think Windows ME and Vista.  Epic fail.  Both were supposed to fix perceived shortcomings of the previous releases.  

There are some ways to mitigate SSS when you first spot it.  

1. Instead of doing a full rewrite, determine what is really wrong with the current system and rewrite that.  Is it performance, which is the usual driver in my experience?  Then fix the performance problems.   Create a "performance release" of your software.  
2. Are there sections of "blackbox code" that people are too scared to fix?  If so, take a deep breath and refactor that code instead.  Gather your requirements first of course.  Usually architects and managers would rather rewrite the blackbox than slowly refactor it.  Avoid this temptation.  
3. There are some times when a full rewrite is appropriate, especially if the architecture is just wrong or inflexible.  When these cases are identified make sure you follow tight development rules on the rewrite.  Prototype.  Practice agile methods, delivering working pieces of the system to other developers and your customers often to get feedback.  Write unit tests...many older systems that need full rewrites have no unit tests, which is part of the problem.  Also, try to use good interfaces that can communicate with older modules whenever possible.  This will even aid in unit test reuse, again, if you have them.