Scaling Software Engineering Teams

The humble objective of this article is to bring together best practices for scaling software engineering organizations during hyper-growth phases.

Rapidly scaling an organization is a complex and often messy endeavor. It usually takes quite some time before the additional manpower can make a significant impact.

In my experience, the most common problems are:

Narrow focus on hiring, neglecting the impact on the rest of the organization,
Adding people to teams, and expecting immediate benefits,
Taking a (simplistic) people-centric view of the organization
Failing to properly address new challenges caused by rapid growth

This brief, pragmatic guide covers the following four domains: (i) hiring, (ii) onboarding, (iii) organization, and (iv) processes. It does not prescribe how to structure teams, but it does provide some general principles.

The intent is to provide you with a better understanding of the challenges, tips on how to prepare, and how to manage growth - material that could easily fill a book. Depending on the maturity of your organization, some of the suggestions may be irrelevant or just plain obvious. Use and adapt according to your needs.

Besides my own experience, the content has been mainly derived from the books listed in the appendix. More specifically, the Hiring and Processes sections have been heavily influenced by Larson's book.

Assumptions

You already have one or more products and one or more development teams in place
There exists some sort of idea/plan about what you plan to develop over the next few years
You have budget available to fund the growth
Scaling out will result in a substantial increase in headcounts and new teams being formed.

Hiring

When you embark on a large-scale hiring hunt you must also understand the impact on the rest of the organization. You must ensure that the recruitment teams’ current and future capacity have the necessary bandwidth to keep up with your needs. Same goes for IT for purchasing, provisioning, third party products licenses, etc.

Best practices:

Start with the metrics. Make sure the hiring funnel is instrumented and that you have a good understanding of its performance for each phase.
Establish a process to regularly review the metrics with HR and optimize where needed.
Ensure the interviewers are in agreement with the job spec
Design tests for each skill - look for strengths, not weaknesses.
Ensure the interviewers agree on how to rank candidates
Pair new interviewers with experienced ones
Ask candidates for feedback

Tips:

Hiring is quite disruptive. A common recommendation is to work on a team at a time. Fix it, and move on to the next team.
Hiring has a significant impact on the team’s ability to keep up with current priorities. Make time for this additional activity and, if it takes place over several months, allow team members to take a month off from interviewing (rota or similar)
As engineering manager, if you have been approved for a number of roles, then treat this as your top priority.
For candidates you have hired in the past, how does their work performance relate to their interview performance? What elements correlate with success?
The cost to replace an experienced developer is huge. If your churn rate is greater than 5% you already have a big problem. I suggest you start by making sure that the people you worked hard to hire don’t leave the company. Exit interviews may help uncover some of the reasons.

Note: many assume Software Engineers know how to interview but that is often not the case, as most have never been formally trained on interview techniques. This may result in too many rejections or, worse, poor candidates getting hired. Depending on where you stand, it may be an investment worth considering.

Onboarding

The Oxford dictionary defines onboarding as “the action or process of integrating a new employee into an organization”. In software development, that is usually translated to a sufficient understanding of the software architecture, codebase and processes to confidently begin to commit changes and deploy them in production.

Unfortunately, not all managers invest in onboarding and often don’t go much further than presenting the new hire to the team - job done. Onboarding into a new organization is thus often a frustrating experience for new hires; certainly not a cost-effective one for the organization.

So, what are the steps for successful and efficient onboarding? In my opinion, you need a clear idea of where you stand, a plan to improve over time and, perhaps most importantly, a clear statement about what the new hire is expected to learn in order to start contributing.

For each role, break down the various activities new hires must carry out and which skills they must acquire. When possible, establish metrics for each phase
For each phase (eg: environment setup, debugging, learning, etc.) write down the ideal experience and current situation.
Assess what can or should be automated, what may require writing additional documentation, mentoring, pair programming etc.
Design a feedback loop so that new hires can contribute to improve the onboarding experience for future hires.

Best practices:

Each team should have:

Onboarding landing page with links to all available documentation relevant to this phase
A personalized document stating onboarding objectives, a rough plan and means to follow progress
List of all key people that are required for a successful onboarding (HR, IT, DevOps, team members, etc)
Team norms: when is it ok (or not ok) to interrupt someone when stuck, after long long being stuck should reach out, how to reach out (slack, email, in person), etc.

Tips:

First month is key: people have to learn new jargon, understand who does what within the broader organization, settle in if they have relocated and so on. Be proactive and reach out regularly to help them out during this phase.
Broaden the horizon: don’t focus just on onboarding within a given team, but facilitate the development of relationships across the organization.
Ask them to be patient: not everybody is able to resist the temptation of offering advice before having the full context. Teams have a history and operate under a set of constraints and so it helps to fully understand them before offering, potentially non actionable, advice.

Organization

A substantial increase in headcounts means that you may have to re-assess your current organization and/or processes. What used to work before, may no longer work well with so many new hires and additional teams.

The painful reality is that, more often than not, reorgs make matters worse, with teams ending up having to work around the new structure to get stuff done.

This is usually because most take an Org Chart (or people) approach, neglecting the constraints imposed by the underlying system architecture. Research has consistently demonstrated that you just cannot design a new organization without taking into account the underlying system architecture of your product/solution. The system architecture constrains the organization architecture and vice versa, which means that both the systems architect and the people that have been charged to design the new organization must work hand in hand to take into account mutual constraints.

The Org Chart approach also does not take into account technical debt, ability to design, develop, test and ship features in parallel, and so on. As hyper-growth does not usually happen overnight, it is your duty, as engineering leader, to take the necessary measures to future-proof the existing organization.

The “organization” is also often considered a static construct, something that you work on from time to time, and move on to other priorities. Modern organizations are more fluid, focused on removing complexity, reducing dependencies, and increasing autonomy. It thus helps to think in terms of principles and team patterns, and build on that.

My go-to reference for the design of software development organizations is the Team Topologies book. The authors have carried out substantial research and came up with team patterns and interaction modes that provide solid foundations for a successful transformational journey.

Given the complexity of the topic, rather than trying to summarize it here in a few lines, I suggest you get yourself a copy of the book - highly recommended.

Best practices:

Focus on removing complexity and increase autonomy
Move decisions down to the right level of the organization
Create a culture of continuous improvement: be the best at being better
Teams should ideally be formed by a stable group of 5 to 9 people working toward a common goal as a unit.
Team should be stable, but not necessarily static, changing occasionally and when necessary
Every part/component of the software system is owned by exactly one team. No shared ownership of components, libraries or code.
Team may use shared services at runtime, but every running service, application or subsystem is owned by only one team
Weak code ownership. Other teams may submit PRs, but they cannot make the changes themselves
Minimize handoff to other teams

Tips:

Don't assume you know everything that is going on in your organization. Run regular surveys to ensure that all voices are heard - unfiltered
Be clear in your comunication that this is a long journey. Most people unfamiliar with the challenges you face have unrealistic expectations.
You don't have all the answers. Focus on creating a culture of continuos improvement rather than an end result.
Beware of adding too many management layers, as you will lose the voice of the people who built the solution, and are probably the reason you have been hired.
Invest time developing and maintaining the necessary relationships outside the engineering organisation.

Note: As the engineering organization is one of many departments, you must also consider the impact that growth will have on the interactions with all other departments. This is out scope for this article, but equally important.

Processes

As the engineering organization grows in size you will face new issues, and you may find yourself thinking about the need to introduce new policies.

A policy should have one or more goals, and constraints that bring actions into alignment with those goals. Be careful when choosing the constraints, as they will likely introduce local inefficiencies. This implies accepting a “reduced opportunity space”, by finding reasonable tradeoffs.

Best practices:

If a particular issue affects only one team, talk to them and ask them to address the situation, and let them choose how.
Consider whether a norm, a non-binding recommendation, may be sufficient.
The cost of creating and maintaining a policy is high. There is not much point in writing a policy that does not constrain behavior.
Introduce one policy at the time and refine it until it has broad support. Only then you should consider adding another policy
The success of a given policy depends on how you handle exceptions. Collect requests for exceptions and review the policy at regular intervals.

Tips:

Communicate in advance when a policy will be reviewed (eg: once per quarter). It gives a clear signal that yes, requests for exceptions will taken into account, but also it won't require to deal with every request as they arrive.
Expect to spend considerable amount of time explaining policies and fine-tuning them untill they'll become fully accepted and established.

Conclusions

As if scaling an engineering organization wasn't already complex enough, it is often just one of the many priorities a scale-up has to deal with. Conflicts for shared resources, politics, and change of priorities are all going to affect your plans.

Instead of trying to over-organize everything, I suggest you put your efforts in creating a culture of collaboration and continuous self-improvement. Processes matter, but no process will fix a toxic culture.

I hope this article succeeded in providing you with an overview of the challenges you'll face as well as giving some suggestions that hopefully will prove useful during this diffcult, but exciting, journey.

References:

An Elegant Puzzle: Systems of Engineering, by Will Larson
Team Topologies: Organizing Business and Technology Teams for Fast Flow, by Matthew Skelton, Manuel Pais
https://speakerdeck.com/simonrohrer/modern-enterprise-architecture-architecting-for-outcomes