Scalability of the agile processes seems to be one of the most frequently discussed issues in the community at the current time. One comment to my last post leads me to the question: what does it mean basically – scalable development process/approach/method?

I worked as the software architect for years before i became the product manager. And what I’m doing in that article is – I’m applying the architecture principals and methods to the development process. Because they are universal. You can compare a development organization with a product you need to design. From that point of view I try to analyse and to focus on the essential requirements for the organization and analyse different possible solutions like Scrum, Kanban, Waterfall, etc for that particular organisation. What I do not like – comparison of different approaches in general, because there is not general answer what is better Java, .NET or unmanaged C++. I hope you agree.

Definition of scalability

I’ve learned before start discussing about something, clarify the core terms to be discussed first. So let us see the definition of scalability the most compact and helpful part of it:

“If the design or system fails when a quantity increases, it does not scale.”

Even true: if you need to change the design for managing the increased quantity, the design does not scale.

Scalability in terms of development organization

If you have an issue, you are looking for a solution and the solution should fits exactly your issue shouldn’t it? It means the architecture of your solution should first of all address the core requirements of your issue. And what if your issue is the management of a large project / productline? In that case the structure of the organization defined by a development process is in fact nothing else as the architecture of your organization.

But what about the quantity in that case? I believe, in terms of the development process it’s about productivity of your organization:  the number of features, releases, etc. per time slot. You have two options for increasing your productivity:

  • you can increase the productivity of your existing resources
  • your can increase your resources

But you can not endless optimize the productivity of your existing resources, because it’s limited by nature. The next step is the increasing number of team-members or of number of teams. At that time the question comes: does your organization design scale? Or with other words is it possible to increase number of resources without changing the organization itself so that the productivity really increases? Your organization has good scalability if its productivity has the linearly dependency to the amount of resources.

Let us try to define it in that way: if a development process or method allows / supports the scalability of your organization – it’s scalable, if not – it’s not scalable.

The core thesis

To be able to handle increased quantity of features/releases by increasing number of resources (employees, teams) it must be possible to parallelize the work. Just adding resources does not help if they are not able to work in parallel to each other. Imagine your have multiple components you can develop independently (parallelization of work items) on the one hand and different development phases: e.g. analysis, implementation, testing, integration (serialization work items) on the other hand – it could be helpful, right? To achieve it you need:

  1. an approach for splitting the whole work into independent work-items
  2. an approach for managing the serial and parallel flow of work items within your organization
  3. a load balancing mechanism to ensure that everybody is supplied with work but nobody is overloaded

Can agile methods help you to improve the scalability of your organization ?

Let us analyze what two most popular agile approaches handle the scalability requirements: SCRUM and Kanban. Even if these two approaches are totally different, it’s possible to compare how they are addressing the same process requirements.

SCRUM addresses the parallelization e.g. by introducing user stories and by enabling every team member for editing every line of code the team is responsible for. It also describes how to serialize the work using the backlog and sprints. BUT it does not work well for teams bigger than 8-10 developers and it has no answers for parallelization of the work between multiple departments, no answers how to control the work over different phases and no answers for load balancing. If I’ll take also my last post into account I’d say SCRUM does not scale if you’ll try to apply it for the management of multiple teams.

Kanban is very an effective tool for a) the organizing of the work-items-flow (serial work items) through your process phases and b) it addresses the load balancing by limitation of the work-items number within one development phase. But Kanban does not address the issue of the parallelization between the multiple teams neither.

Conclusion

Kanban can help to improve scalability of your organization. It cares about the load balancing and the serialization of the work. But Kanban does not care about the way how to create independent work items and additionally you need a development process which enables parallelization of work. Basically Kanban is not a process at all however you need one.

Scrum is not scalalble. It’s a process that can help to improve the team productivity, but does not help you to improve the scalability of your organization => take Scrum as a process for the particular teams but not for the whole organization.

Do you remember: the structure of the organization defined by a development process is the architecture of an organization. And remember: „Architecture is about everything costly to change“ [Grady Booch]. Know if somebody says e.g. Scrum can be adapted to be scalable – do not believe, because Scrum was created for something else.

Separate and Rule

I believe there is only one possible answer – the system analysis and the system architecture. They were und remain the core factor in that field, they care of separation of concerns and enable the parallelization of work. But it is another topic (key words: component based development, service oriented architecture, contract-first development, etc.)

Very interesting point: the waterfall- and V-process defines that analysis and architecture have to be done first before to do the rest Smile