It’s interesting to observe the trends in software R&D over the years. There are two different approach in the market place and each promote waste in it’s own way.

This has a very significant impact on the job market and on organisation and this is why I am touching this in this post.

Waterfall

The older one is the “waterfall” approach. In this model, software are development following a very organized step-by step process. Software will go through phases like: Conception, Initiation, Analysis, Design, Construction, Testing, Production/Implementation, and Maintenance. This is the model that is generally favored by large consulting organisation. The problem with it is that software needs are evolving and are hard to nail down and get right the first time. However, since it is done linearly, each time that a change is required, the process will be going back to previous steps and things will be redone.

In this model, projects are typically late, over budget and when it is completed, it generally fall short of the needs. And this is when project are completed. A very large number are never completed. This is however a very good way for large consulting firm to make a lot of money on project. The more changes, the longer it takes, the more money they make.

Scrum

On the flip side, there is the “scrum team”. This is a process where the development is iterative and not really planned much in advanced. It is very flexible as it is easy to change direction at any time.

This style of development is encouraged by the way venture capital and so called “angel” investors operates.

See, typically today, these groups don’t invest in an R&D effort that has not reach a public beta stage and has not demonstrated that there is user “traction”. So, you have to have something working, deployed and used before you get investment.

Since very few people that launch startup have much ressources, they will race with something not very well planned and deploy it. Iterate and deploy. If they get a bit of traction, they will maybe get some investment but more likely than not, what they have developed so far will need to be re-written… and since the investment they get are linked with result, even the new version will be rushed and will eventually need to be re-written.

The impact on the job market

One of the problem that it create in the job market is that it created a lot of instabilities. It is hard to build something useful and well design that will last for a long time. Further, it push for a job market where experience is seen as a liability.

You need to have just the right amount of experience to be able to do the job but not too much as to chalenge the way the things are done. In the waterfall model, a project is canned and the team is disbanded, in the scum model, well, since you start over, you bear the burden of the fail version and the team is disbanded and replaced by another.

I am a big believer in incremental development but I think that it also need to follow a plan and to be implemented with experienced team member that creates an architecture than can be grown, not one that will be thrown away.

Unfortunately, this is not how things are done in R&D these days and I don’t really see that changing anytime soon.

In the meantime, waste is continuing at an eve faster paste. Maybe just a reflection of the time?