Org Design Models, Part 1: Startup & IT

"You can't always get what you want. But if you try sometimes, well, you might find. You get what you need."

— The Rolling Stones

A map of the management battlefield

What kinds of different organizational design paradigms are known for software product development? How do organizations develop over time, and are there any shortcuts on this journey? What should company management and executives know about all this?

In the role of an organization-agility consultant, I often help my clients — in particular leaders of mature organizations — find a better organizational design: a better fit between the structure of the organization and the needs of the business. This is a non-trivial task with no obvious right answers. What makes it even more difficult is the lack of a clear common basis — terminology, vocabulary — and shared understanding among practitioners. So if this article takes off, it can serve as a map of a management battlefield with its core strategies that practitioners can relate to and build on top of.

This article shares some of the most commonly observed models of how organizations that produce software get typically structured. Such organizations don't necessarily sell the software they produce (though that is one option); they ought to produce some software that is necessary to run the core business — whatever that is. And because nowadays arguably any business comes with some sort of custom software it relies upon, this article can hopefully be useful outside of the classical IT crowd.

I'll share six somewhat distinctive models of organizational design (or management paradigms). Some of them are linked to each other with evolutionary ties — that is, they grow naturally on top of each other as headcount and business desires grow. Others require some sort of transformation of the management mindset to jump out of the status quo.

Since organizational design (how things are structured) is inseparable from process (how things are running), we will consider six schemes that combine the organization and operation of software production:

1. (Simplified) Startup.
2. (Centralized) IT development.
3. (Horizontal) Component development.
4. (Overcomplicated) Project development.
5. (Side) Satellite development.
6. (Vertical) Product development.

In this article I will guide you through a typical evolutionary path from a Startup to a mature Product development organization, with its unavoidable crises on the way. I will describe how one hypothetical organization goes through these models one by one as if they were stages of development — but they are not. It is important to keep in mind that this storyline is just one option from a wider set of possibilities of how companies might decide to develop. I'm following it for the sole purpose of more coherent storytelling. But for you, the reader and leader of an organization, it should be clear that companies are not destined to follow any given path; the path being taken is the sum of all cornerstone decisions made.

So I do believe in the free choices of managers who consciously (or unconsciously) make them and by doing so keep forming their organizations — what we call an "org design". This article is about the causes and consequences of these uneasy management decisions, and also about some of the irrational thinking that kicks in on the way.

This is also not an article about "waterfall vs. agile" — that would have been an overly simplistic view of things. Here you will learn about the broader field of decisions that the designers of organizations are facing.

Startup

You cannot start exploring how mature organizations function without having a quick look at how startups get formed, from the viewpoint of their structure and process. Understanding startups is essential to understanding "mature" organizations, because in the life of every organization there are times when the company wants to add processes, structure, and rules — and times when it wants to "hack around and just get things done". These movements are like a seesaw: today a company wants to be more like an enterprise (dreaming of more standardization and operational efficiency), then more like a startup (dreaming of more results and better alignment), then more like an enterprise again, and so on. These back-and-forth movements are periodic and are provoked by temporary growth crises (more about them below), so understanding what makes the "startup dream" so powerful is important to understanding the dynamics of any mature organization.

In a nutshell, an idealized startup-like style of software product development is an organization of cross-functional work of a small number of people around a clear business strategy. Usually with direct access to clients and end-users. Note how close this is to a Scrum prescription of a cross-functional, colocated, dedicated, full-time, customer-facing team. This is not a coincidence. The goal of Scrum, if you will, is to help organizations return their lost youth…

Nowadays startups are also seen as lean operations — thanks to the Lean Startup movement that has really become the norm and the status quo of how entrepreneurs operate their young ventures. Startups naturally don't have excess resources to waste, so they have to stay laser-focused on a specific mission: for instance, getting traction from a wisely chosen customer segment. (Nothing stops more mature orgs from taking on this lean approach — their resources are limited too — but for a startup this is a clear survival mechanism creating strong alignment and focus, not an artificially fixed "project budget".) From a clear shared focus, other good things are derived: goals, expectations, metrics, milestones — shared by everyone involved, typically by the entire team. Until the moment this unity starts to compete with locally-set goals driven by some organizational changes.

"Sales" is most commonly the first department being formed that immediately starts to drift away from the cross-functional core (R&D in the future). This can be related to the "noisiness" of salespeople's work, their extraversion that happens to be somewhat uncomfortable for the IT crowd, and other common beliefs — but these are just stereotypes. The only reason this starts to happen is because someone decides this is to happen. Remember the "free managerial choice" thing we talked about a few paragraphs back?

Isolation of the sales department can lead to consequences whose negative impact can only be clearly seen in the long run. Examples I've seen include a desire of salespeople to close contracts at any cost — including negative profitability, agreeing to unrealistic customer demands for impossible product functionality, setting wishful deadlines, and so on. From that moment, R&D will be forever blamed for "low quality of estimates and missed deadlines". The consequences such decisions bring are usually hard to fight, because they turn everyone into being extremely busy, overstressed, and unhappy with whatever outcomes. And when everyone is like that, there's no time for clear wide system thinking, and a shortcut-looking mindset kicks in. Then it quickly becomes a habit and a vicious cycle only makes things worse with every loop.

The real task of a competent org designer in such a young company is to try to avoid this split at any short-term cost. This implies practicing long-term thinking at the cost of "quick cheap wins" that are so tempting at times when cash is burning low. Keeping everyone in the same boat of a common business cadence, making everyone believe in shared goals, practicing cross-department collaboration with the clients, holding a focus on the value delivered — these are some vital practices for keeping the startup spirit alive. If management can withstand this first fight and keep the naturally-born product culture and client orientation of a startup, then with upcoming growth such an organization can jump over the next few models described below — Centralized IT Development and Project Work — right into Product Development. And if not — the journey will be long and winding.

It is worth saying that the odds of sustaining the product culture in a growing startup are very low — like 1–5%, in my observation. Why? Being a maverick sounds cool only if you read about one; no one really wants to be one, and what I see all around are mere unthoughtful copy-and-paste solutions. Top management of a company copies the ideas of others, even though the others are not really doing any better. And then joint tuition in "Exec MBA" schools only strengthens their belief in collective righteousness. So it seems all the business forces of the universe work together to keep management isolating software product development from the rest of the organization, from the business. Why? Simply because others are doing so.

And that's the beginning of a downward spiral that is pushing IT to the underground floors of organizations, literally.

(Centralized) IT Development

At this stage of development, the initial business model — or its adapted and improved increment — has already been validated by a positive market response. The company gets its first real investment round. The appetites of shareholders start to grow faster, and with them the stress and pressure on IT. Growth in terms of more headcount is kicking in.

This soon leads to the first management crisis — a crisis of direct management. It can be seen already with 30–50 engineering staff. This is a growth crisis dealing with the fact that the power of personal influencing connections, which used to glue it all together with fewer people despite various issues, is no longer enough to serve as the center of gravity. Smaller cracks in the organization, created earlier, are now opening wider and wider with every new person hired.

The situation worsens as the execs get themselves separated from the realities of work and busy with "strategies" — sometimes out of sheer incompetence in solving real organizational issues (sorry to say). To compensate for the lack of their involvement, a special squad of specialists enters the company doors: process guys "with a good track record and prior enterprise process experience". They start to form a new elite — vice presidents. Each department now has its own VP.

Of course, here we see the emergence of the VP of Engineering. His presence reinforces the IT silo with its opaque processes, lack of agility, and the known problems of "missing deadlines and bad estimates". But now IT has more expensive excuses, stated with the proud voice of an enterprise.

"We've overgrown the startup, we are a mature organization now, we need to focus on processes," says the VPoE. "You want clear estimates from us? Fine, I'll handle it. But to control our estimates we need to limit the continuous flow of change requests coming down on us from the chaotic sales team. We need better engineering processes."

From this moment on, the organization is in a trap. The appetite of the business elites will keep growing (resulting in more pressure). Marketing folks will tap into new prospects and new customer segments (resulting in less focus). But R&D will be fencing itself off from uncomfortable requirement changes (by the way, it's not the requirements that get changed — it's our understanding that grows).

The dynamics described above lead to a mutual lack of trust between the business and the IT side of the org. What would fix the issue is the presence of a business stakeholder holding these two worlds together, acting as a leader and owner — what would introduce a real Product Owner as we see this role in Scrum. But ironically, it is highly unlikely to expect a business stakeholder to risk his skin and step into the IT side. It's too dangerous for the career, as IT constantly fails. And because no business stakeholder will be willing to lead and guide software product development, it will stay in the hands of engineering heads, who will be focused on constant operational issues — fixing software holes and ruling permanent release dramas (due to constant pressure and rush, software quality has been degrading, making it harder to introduce changes). So the IT heads will be busy dealing with all this chaos (for which they were hired in the first place), thus having a lack of strategic and business focus.

Can you spot the vicious cycle here?

An example of companies that might be facing these kinds of challenges: businesses (for instance, e-commerce) that need some custom software to operate — maybe even a lot of it — but unable (yet) to see and embrace software production as their direct activity. Also, businesses that try to limit the amount of software they actually need — a few dozen IT staff plus several vendors.

By the way, using a vendors-only approach might not be such a bad idea here — provided the vendors are real software professionals, they can showcase to the business ways of how to guide software development.

What's next

We've just covered two models:

• (Simplified) Startup.
• (Centralized) IT development.

A real company can stop at this level when there is no need or opportunity for growth. But in the next chapter of this article, we will watch our sample organization passing through the next phases:

• (Horizontal) Component development.
• (Overcomplicated) Project work.
• (Side) Satellite development.
• (Vertical) Product development.

So read on in Part 2.