Current development techniques are unrecognisable from the days of punch cards and tick-a-tapes and the constraints of yester-year. Over the decades the legacy of these primitive development environments lingers on, not in technological terms but in mentality terms. The Waterfall generation has now matured and they are now running and directing the projects using what they know best – which is why it won’t die soon.
Many decades ago software development was a very time consuming and expensive undertaking. It was difficult. If you programmed a computer using punch cards then you needed to be very clear up front on your requirements. Debugging punch card systems was notoriously difficult and time consuming. Doing a dry run through your deck of punch cards could aid in reducing bugs but it couldn’t prevent badly defined requirements or misinterpretation of requirements. For this reason alone software development used to be very expensive and extra effort was required upfront to really pin down the full detail of all requirements.
Ok, so punch cards may be a bit extreme so what about COBOL or something slightly more recent such as FORTRAN? Well, even these software development languages often required developers to book processor time on mainframes to test and debug their code. In these environments, extra effort in defining everything upfront mitigated risks as the development environments were very primitive by today’s standards.
In the olden days this approach (Waterfall) was engrained into everyone who worked on the software development lifecycle, from Project Managers and developers, to end users. There were no other ways for decades.
Development environments back then were very different to what’s available today and many of the methodologies were born out of the development hurdles and technical constraints of the day.
Now contrast this with current development capabilities:
- Integrated development environments
- Interactive debugging
- Object Orientated Design
- Real-time compilation and the ability to run software on the developers PC to get instant feedback
- Continuous Integration
- Automated Testing
- Automated Deployment
- Advanced release pipelining supporting rapid deployment
- User / Developer collaboration enabling developers to develop systems WITH the end users
- Object Relational Mapping
- Design Patterns
- Feedback loops measured in minutes / hours rather than months.
So, if the landscape is so unrecognisable now from how it used to be when Waterfall was the de-facto standard then why are lots of organisations still following an approach that is completely unnecessary and in most cases counter productive?
BECAUSE – the developers of yester-year are the Project Managers of today! They haven’t moved on from when they used to develop software. I’ve heard this a lot from PM’s who say they used to be developers – usually COBOL developers?
Singling out Project Managers is unfair as it’s more of a cultural thing across many levels of the organisation following conditioning over many decades. Another big contributor is the end users. The end users of yester-year are the senior business users of today and expect detailed definitions of all requirements up-front ‘because that’s the proper way to do it isn’t it?’
I’ve noticed this to be a particular problem in the UK Public Sector where they employ individuals – usually in positions such as ‘Business Assurance’ or ‘Project Governance Officer’ – to ensure project teams are capturing all requirements up-front and ‘doing things properly’. When challenged as to why it’s important to define all requirements in detail up-front, a common response is “because it’s the process” or “it’s the proper way to do it”. For decades these individuals and those around them have been told they must define all requirements up-front and be very detailed in their spec’s to reduce risk, so for them, this is the only way to develop software.
Another common misconception is that by not doing detailed up-front requirements capture we’re trying to avoid doing documentation. The reality is we’re not avoiding the documentation. I’ve regularly found that we’re not actually reducing the amount of documentation we produce - we’re just doing different documentation. Different in that we’re using other, more efficient, mental models to capture and communicate requirements or solutions such as site maps, wireframes, design mock-ups, user journeys, etc.
So, the biggest hurdle to finally putting Waterfall to rest is letting people know that its ‘ok’ to do things differently and developing software is not as difficult as it used to be – relatively speaking that is.
