Innovation Death Spiral: Common Organizational Challenges

April 30, 2024 (1y ago)

Research has found that, while innovation is among the top three priorities for many organizations, less than 10 percent seem to be satisfied with their innovation performance.

In this post I will document common reasons organizations perform poorly, and will provide suggestions to correct the trend in a follow up article. More specifically, we'll focus on scale-ups, and their journey from innovators to successful businesses to entities that struggle innovating.

Acquiring other companies to increase the product portfolio, customers’ base and revenues obviously have their merits, it’s just out of scope and won’t be discussed further.

What is innovation and why does it matter?

McKinsey defines innovation as “the systematic practice of developing and marketing breakthrough products and services for adoption by customers.”

It matters because, while your current products or services may be doing well, competitors aren’t sitting idle. Smaller, leaner organizations can bring to market new products and services that can potentially erode, or worse, displace your products and services — and you’ll find yourself from being a market leader to one providing legacy solutions in a blink. To maintain a significant edge from the others, you need to excel in both execution and innovation.

Innovation death spiral

As organizations grow, they add management layers, policies, constraints, and bureaucracy that make working on new, ambitious ideas almost impossible. The products increase in complexity, and technical debt starts showing its ugly face. And it all happens in plain sight, but slowly, pretty much like a boiling frog.

Let’s begin by looking at some of the reasons organizations get into the innovation death spiral:

  • Alignment. The executive team declares Innovation to be the top priority. The same team also communicates the need to hit very ambitious sales targets. Then OKRs are ab-used to add even more objectives, and everybody is already at 120% and burned out.
  • CFOs-run organizations. As organizations grow, they obviously need to put in the required financial discipline. Very often however, the company strategy is not not really aligned with the constraints imposed by the finance department.
  • Managers. Hiring people that have worked in larger organizations will help you transition from a messy startup to a scale up. Managers, by definition, manage - they rarely are disruptive innovators willing to take significant personal risks.
  • Management layers. The more management layers between the builders and the executives, the harder it gets to be heard.
  • Sales-driven Product Management. Everybody loves closing deals but, in the long term, it becomes a real issue when not properly weighted against what else won’t get done.
  • Gated access to customers and prospects. Only a few chosen ones can talk to them, eliminating a critical bi-directional feedback channel between the builders and consumers of products/services.
  • Infinite Backlog. The backlog is so long that there is no perceived need nor time to work on anything else.
  • Short sprints = no time to work on major features. This is dumb, but it happens.
  • Small bets. Similar to the above, a culture of small increments, typical of SaaS organizations.
  • Infinite Estimates. Teams are requested to provide detailed estimates which is a time consuming activity of dubious value and often an indicator of lack of trust.
  • Blame culture. People will stop sticking their neck out if they get blamed when they make mistakes. Innovation is all about making experiments, only some of which will prove valuable.
  • Technical debt. The various types of technical debt (codebase, documentation, flaky tests, manual deployments, etc.) affect team velocity.
  • Cognitive load. Software development is becoming more complex, with many organizations introducing unnecessary complexities prematurely.
  • Team structure. Scaling the engineering organization is a major challenge, one that often introduces unnecessary dependencies, communication issues, mismatched priorities, unclear ownerships, etc.
  • KILO. The cost of Keeping the LIghts On is often not clearly visible and thus not well understood by management.

Moving Forward

It would be tempting to conclude that fewer managers, better software development practices and a healthy culture are the solution, but in practice it's way more complex and subtle. I personally worked in very successful organizations where I inherited a messy code base, and success is generally measured in dollars, not other metrics.

Now comes the hard-part. How do you create the necessary conditions? How do you structure your teams? After all, everybody is already working long hours and they are all fully allocated to existing projects.

I will explore the topic and provide some suggestions in a follow-up article.