Waterfall to Agile Software Development
A Three-Dimensional Framework: Transformation Guidelines
Despite the differences in factors like processes, practices, mentality, scale and maturity between software companies we attempt to provide a minimal framework for guidance. Three categories can be identified, namely, structure, mentality, and processes, and they may provide a basis for generating a transformation recipe.
Centralized or decentralized, flat or tall, a company’s structure defines how resources are distributed. It may be based on services, products, markets, or in geographical areas, among others. Although a detailed analysis is beyond the scope of this article we found that agility requires flattening the structure. Reducing the number of hierarchical levels may help the communication between different levels. Software architecture, analysts, development, testing and deployment departments could be the building blocks of a structure that enjoys and empowers synergies. We’ve found that synergies between all phases of the waterfall SDLC are a prerequisite for the success of future agile scrum teams.
Top-level management’s decision making should be based on a clear and stable direction whilst being dynamic. It should be both proactive and reactive. Middle-level managerial mentality involves exercising ownership whilst leading by example and embracing employee independence and self-organization. There must be a balance between following rules and independence. Which rules and how many rules, needs to be experimented and decided per company. Independence brings diverse viewpoints that are desirable, keeping in mind that everyone must be aligned on a common direction.
Focusing on projects may deviate from focusing on products, which may be detrimental. In large software projects people may focus on their function or profession and not on products. A product-focused mentality relaxes functioning and profession constraints and welcomes learning whatever is necessary in order to build the product. This mentality shift is not easy and it requires training.
Processes determine how things get done and they should be lean and flexible. This includes planning, communicating, predicting, designing, controlling, developing, testing, reporting, documenting, etc. To get things done processes are usually decomposed into rules.
There are interdependencies between the three. Large waterfall companies tend to have tall structures and inflexible processes. Although reducing hierarchy levels may allow for more flexible processes and independence, a mentality that welcomes changes is always a good start.
Waterfall stability and scale need not exist against speed and flexibility, the much desirable agile properties. Waterfall companies with the appropriate structure, processes and mentality may achieve dynamism based on stable steps. This is not an easy task and it may require substantial reformations, restructuring and lay-offs.
We explored basic ingredients of our transformation recipe dictating possible courses of action, difficulties and lessons learned. If a company can make the transformation it will do it via its own recipe, a recipe however, that could be based on guidelines. Agile is not only about following “agile processes”. It is not scrum or sprints that brings agility. They can help only if the appropriate culture and competencies exist.
Turning from waterfall to agile is a change. A waterfall company that does not possess any agile-like characteristics will find the transformation to be a big change. Big changes take time and they must be broken into smaller changes. It is important that even small improvements and contributions are recognized and celebrated.
Agile software development may not be for everyone but it is also important to remember that agility is not binary. There is no dilemma like either you are agile or you are not since the level of adoption is a continuum: some are more agile than others. Some companies may never be fit for this kind of software development whilst for others, possessing a few agile-like characteristics could be what they need. There are cases where it is not possible for a company to be agile in all products. Some products must remain in waterfall production whilst the rest of the products can be agile. In order to decide about a company’s ability for agile software development, training, and time to experiment are necessary.
The worst-case scenario should be avoided: A transformation attempt that was considered to be “catastrophic” since it had a detrimental effect to the company’s overall performance. The outcomes were a waste of resources, a demoralized personnel and a company reverting to the waterfall software development mode. Under such circumstances, it may be useful to remember that a learning organization  may learn more from failing to transform into agile rather than succeeding. Learning has to do with realizing mistakes and improving.
Companies expect tangible and sustainable improvements by transforming to agile. Although intangible improvements could also be important, a waterfall company that submits the effort and the resources to make the waterfall-to-agile transformation expects to improve. If it manages to cope with frequent requirement changes, fast releases and if it is more successful (e.g., attract more customers, increase ROI), then this is a success story. After all, it’s all about learning and improving, irrespective if you’re agile, lean, waterfall, or a hybrid.