Disruptive innovation - or “architectural innovation”?

Energy is where it's happening: 

Or as BP’s chief executive Bob Dudley candidly admitted to the Washington Post recently, “If someone said, ‘Here’s $10bn to invest in renewables,’ we wouldn’t know how to do it.”

Here's a very interesting piece from the FT which challenges the prevailing orthodoxy of 'disruption': 

Why big companies squander good ideas

The FT’s Undercover Economist on the real reasons that corporate innovation dies

Tim Harford

SEPTEMBER 6, 2018

J F C Fuller did not invent the tank.

That distinction should probably fall to E L de Mole, an Australian who approached the British war office in 1912 with a design that was — in the words of historians Kenneth Macksey and John Batchelor — “so convincingly similar to those which finally went into service that one wonders why it was never adopted from the outset”.

But when the British army eventually introduced the tank, it was J F C Fuller, chief staff officer of what would later become the tank corps, who understood what to do with it.

At 39 years old, Fuller was a small man with a neatly trimmed moustache and a hairline that had retreated over his crown and was beginning to march down the back of his head. He could have passed for a butler in a costume drama, but his appearance belied an inner radicalism. (He had been friends — and then enemies — with the occultist Aleister Crowley.)

Late in 1917, after almost 400 British tanks had, with modest success, lumbered across the German lines at the battle of Cambrai, Fuller applied his radical streak to the problem of using the tank effectively.

A new and much faster tank, the Medium D, could travel 200 miles at a speed of 20 miles per hour. Fuller proposed that these tanks would attack the German army’s brain — the string of German headquarters miles behind the front line. A Medium D could roll across the trenches and be on the German command posts in an hour; Fuller’s attack would come from nowhere. Air support would disrupt German road and rail travel.

“Bad news confuses, confusion stimulates panic,” wrote Fuller. His idea was dubbed Plan 1919. By striking suddenly at the German command, Plan 1919 would cause the German army to disintegrate. It would, Fuller declared, be “the winning of the war in a single battle”.

His astonishing idea became “the most famous unused plan in military history”, according to his biographer Brian Holden Reid. But, of course, that is not entirely true. It was used to great effect, in 1940 — by the Germans. J F C Fuller had invented blitzkrieg.

The story might be a historical curiosity, had echoes of it not been repeated so frequently since the British army stuffed Fuller’s plans for blitzkrieg into a desk drawer. Organisations from newspapers to oil majors to computing giants have persistently struggled to embrace new technological opportunities, or recognise new technological threats, even when the threats are mortal or the opportunities are golden. Why do some ideas slip out of the grasp of incumbents, then thrive in the hands of upstarts?

In 1970, the photocopying giant Xerox established the Palo Alto Research Center, or Parc. Xerox Parc then developed the world’s first personal computer, with a graphical user interface, windows, icons and a mouse. Bill Gates of Microsoft and Steve Jobs of Apple observed developments at Xerox Parc with great interest. Xerox still makes photocopiers.

In 1975, a 24-year-old engineer named Steven Sasson built the world’s first digital camera — a patched-together device scavenging a lens from a Super-8 camera, magnetic tape in a portable cassette recorder and a TV screen. Sasson worked for Eastman Kodak, where in 1989 he and his colleagues also constructed the first modern digital SLR camera. Kodak built a sizeable line of business in digital photography, and earned a small fortune from the patents. Yet Kodak could not adjust to a world in which every phone contained a camera. The company filed for bankruptcy in 2012.

In 1999, Sony launched the “Memory Stick Walkman”, one of the world’s first digital music players. Sony was armed with the iconic Walkman brand, some of the world’s best consumer electronics engineers and the talent-soaked Sony-BMG music label. The Memory Stick Walkman went nowhere and, two years later, it was eclipsed by a product that transformed the fortunes of a struggling Apple: the iPod.

And in 1918, Britain had the best tanks in the world, a clear vision of how to use them and, in Fuller, one of the best military strategists to emerge from the British army. The German army was forbidden to use tanks at all; it was scarcely more than a collection of officers, a head without a body. Heinz Guderian, later one of the leading Panzer commanders, had not even seen the inside of a tank until he managed to go on manoeuvres with the Swedish army in 1929. Yet by the late 1930s, the British had conceded technical and tactical superiority to Hitler’s new army.

There is an obvious explanation for all of these failures and missed opportunities: people are idiots. “Now we can get back to some real soldiering,” remarked one senior officer to Fuller at the end of the first world war — as though defending Britain in an existential struggle had been a frivolous distraction from tending to noble horses, bright buckles and shiny boots. The army blocked publication of Fuller’s books for several years; they were seen as insubordinate.

When Steve Jobs visited Xerox Parc in 1979, and saw a windows-and-mouse interface for the first time, he couldn’t contain himself, according to an article by Malcolm Gladwell. “Why aren’t you doing anything with this?” he yelled. “This is the greatest thing. This is revolutionary!” If Jobs had been teleported into the British war office in the 1920s, he might well have said the same thing.

Idiocy is a tempting explanation and not without merit. The top man in the British army, Field Marshal Sir Archibald Montgomery-Massingberd, responded to the threat of Nazi militarisation by increasing the amount spent on forage for horses by a factor of 10. Cavalry officers would be provided with a second horse; tank officers would get a horse too. As I say: people are idiots.

But there is something about the “idiot” theory that feels too glib. Consider Xerox Parc: how is it that a corporation could be smart enough to establish such a superb research centre, but then fail to take advantage? Was Sony really run by idiots in the 1990s? Even Montgomery-Massingberd is too casually caricatured. These organisations stumbled for a reason. Management theorists have a word for it: disruption.

“Disruption describes what happens when firms fail because they keep making the kinds of choices that made them successful,” says Joshua Gans, an economist at the Rotman School of Management in Toronto and author of The Disruption Dilemma. Successful organisations stick to their once-triumphant strategies, even as the world changes around them. More horses! More forage!

Why does this happen? Easily the most famous explanation comes from Clayton Christensen of Harvard Business School. Christensen’s 1997 book, The Innovator’s Dilemma, told a compelling story about how new technologies creep up from below: they are flawed or under-developed at first, so do not appeal to existing customers. Holiday snappers do not want to buy digital cameras the size of a shoebox and the price of a car.

However, Christensen explains, these technologies do find customers: people with unusual needs previously unserved by the incumbent players. The new technology gets better and, one day, the incumbent wakes up to discover that an upstart challenger has several years’ head start — and once-loyal customers have jumped ship.

Christensen’s story is an elegant one and fits some cases brilliantly. But there are many examples that do not fit — such as the failure of Xerox to exploit the cutting-edge research at Parc. The mouse and the graphic user interface aren’t a low-end competitor to the photocopier. They’re from a completely different universe.

The iPod didn’t sneak up on Sony from below: the company had seen the potential of a digital music player and moved quickly. Dominant organisations often see the disruptive technologies coming.

“Kodak and Blockbuster weren’t caught by surprise,” Joshua Gans tells me. “They knew what the future looked like. They didn’t know later than everybody else, they knew ahead of everybody else.” They knew; but they were unable to put together the right response.

There is also a striking counter-example to Christensen’s idea that disruptive technologies begin as flawed or low-quality options. The iPhone was priced as a premium product with never-before-seen capabilities. It devastated Nokia and Research In Motion — now simply named BlackBerry Ltd in an echo of its once-iconic offering.

Christensen has tried to fit the iPhone into his theories. At first he predicted that incumbents would easily respond, and later he recast it as a disruption in a different industry altogether: “It was intended to disrupt the laptop. And that’s the way it happened.”

The laptop? Tell that to Nokia and BlackBerry.

Anyway, is the tank a low-end competitor to the horse? That’s a stretch. When a theory needs to be made this elastic, it may be time to look for another theory.

In 1990, a young economist named Rebecca Henderson published an article with her supervisor Kim Clark that presented a different view of why it is hard to do new things in old organisations. The relevant word is “organisations”.

Dominant organisations are prone to stumble when the new technology requires a new organisational structure. An innovation might be radical but, if it fits the structure that already existed, an incumbent firm has a good chance of carrying its lead from the old world to the new.

Consider, for example, IBM — the giant of mainframe computing. IBM is a survivor. It predates the digital computer by more than three decades. While the performance of computers was being revolutionised by the semiconductor, the integrated circuit, the hard drive and the compiler, IBM maintained a dominant position without breaking stride. This was because the organisational challenge of making and selling a sophisticated mainframe computer to a bank in the 1970s was not greatly different from the organisational challenge of making and selling a mechanical tabulating machine to a bank in the 1930s. Change was constant but manageable.

When computers started to be bought by small businesses, hobbyists and even parents, IBM faced a very different challenge. It did build a successful business in PCs, but was unable to maintain its old dominance, or bring to bear its historical strengths. In fact, the PC division prospered only as long as it was able to snub the rest of the organisation, often partnering with component suppliers and distributors that directly competed with IBM divisions.


Ideas that got stuck: 
Sony's Memory Stick Walkman; a Kodak SLR digital camera (Alamy); a Medium-D tank

Internal politics soon asserted itself. A case study co-authored by Henderson describes the PC division as “smothered by support from the parent company”. Eventually, the IBM PC business was sold off to a Chinese company, Lenovo. What had flummoxed IBM was not the pace of technological change — it had long coped with that — but the fact that its old organisational structures had ceased to be an advantage.

Rather than talk of radical or disruptive innovations, Henderson and Clark used the term “architectural innovation”.

“An architectural innovation is an innovation that changes the relationship between the pieces of the problem,” Henderson tells me. “It can be hard to perceive, because many of the pieces remain the same. But they fit together differently.”

An architectural innovation challenges an old organisation because it demands that the organisation remake itself. And who wants to do that?

The armies of the late 19th century were organised — as armies had long been — around cavalry and infantry. Cavalry units offered mobility. Infantry offered strength in numbers and the ability to dig in defensively.

Three technologies emerged to define the first world war: artillery, barbed wire and the machine gun. They profoundly shaped the battlefield, but also slipped easily into the existing decision-making structures. Barbed wire and machine guns were used to reinforce infantry positions. Artillery could support either cavalry or infantry from a distance.

Tanks, however, were different. In some ways they were like cavalry, since their strength lay partly in their ability to move quickly. In other ways, they fitted with the infantry, fighting alongside foot soldiers. Or perhaps tanks were a new kind of military capability entirely; this was the view taken by J F C Fuller.

These discussions might seem philosophical — but in the light of Henderson’s ideas, they are intensely practical. “You have to find an organisation that will accept the new bit of technology,” says Andrew Mackay. Mackay runs an advisory firm, Complexas, but was also the commander of British and coalition forces in Helmand, Afghanistan, in 2008. “The organisational question is deeply unsexy, but it’s fundamental.”

A more recent example: is the helicopter basically a kind of aeroplane, and therefore an asset of the Royal Air Force? Or something quite different? Who should be in charge of drones today?

So it was with the tank. If it was to prosper, it needed an organisational home. Someone would have to argue for it, someone would have to pay for it, and someone would have to make it all work, technologically and tactically.

Perhaps the two most obvious places to put the tank were as a standalone unit (since it offered quite new capabilities) or in cavalry regiments (since it was highly mobile and the horse was becoming obsolete).

There were traps along either route: the established regiments would resist a standalone structure for tanks, which would compete for resources while the postwar army was shrinking. A new tank regiment would lack both allies and the heft of historical tradition.

After various twists and turns, it was the cavalry that ended up as the organisational home of the tank. And cavalry officers certainly understand a highly mobile strike capability. But they were never really organised around the concept of “mobility”. They were organised around horses. The cavalry officer loved his horse and rode it with skill. His regiment was devoted to feeding and caring for the horses. Would he not resist the usurper tank with every fibre of his being?

Xerox Parc developed or assembled most of the features of a user-friendly personal computer, but Xerox itself did not have the organisational architecture to manufacture and market it. Xerox Parc did develop the laser printer, a product that matched the company’s expertise nicely. As Gladwell pointed out, this easily paid for the entire Parc project.

The laser printer was like artillery or the machine gun for Xerox: it was an exciting new technology, but it was not a challenge to the organisation’s architecture. The personal computer was like the tank.

The same is true for Sony and the Memory Stick Walkman. As Sony expanded, it produced radios and televisions, video recorders and camcorders, computers, game consoles and even acquired a film and music empire. But to keep this sprawl manageable, Sony’s leaders divided it into silos. As Gillian Tett explains in The Silo Effect, the silo that produced the PlayStation had almost nothing to do with the silo that produced portable CD players.

The Memory Stick Walkman was like the tank: it didn’t fit neatly into any category. To be a success, the silos that had been designed to work separately would have to work together. That required an architectural change that Sony tried but failed to achieve.

And for IBM, the shift from a mechanical tabulator to a mainframe digital computer was like the shift from rifles to the machine gun: an awesome step up in firepower, but a modest adjustment to organisational capacity. The tank was like the personal computer: it may have been a logical step forward given the technology available, but it required a different organisational architecture — one that bypassed and threatened the existing power centres of Big Blue. That was the problem.

The politics of organisational change are never easy. In the case of the tank, they were brutal. The British public never wanted to fight another war in mainland Europe, and the tank represented an admission that they might have to. The armed forces were starved of cash in the 1920s and 1930s. In 1932, the British army ordered just nine tanks — delicate four-tonners. The total weight of this entire force was less than a single German Tiger tank. But at a time of declining budgets, who could justify buying more?

It did not help that the tank enthusiasts were often politically naive. Since an architectural innovation requires an organisational overhaul, it is a task requiring skilful diplomacy.

Fuller was no diplomat. His essays and books were dotted with spiky critiques of senior military officers. After a while, even the junior officers who admired his wit began to tire of his “needlessly offensive” lecturing.

Despite alienating the army top brass, Fuller was handed a unique opportunity to advance the cause of tanks in the British army: he was offered the command of a new experimental mechanised force in December 1926.

There was just one problem: he would have to step away from his single-minded focus on the tank, also taking command of an infantry brigade and a garrison. In short, Fuller would have to get into the organisational headaches that surround any architectural innovation. He baulked, and wrote to the head of the army demanding that these other duties be carried out by someone else, eventually threatening to resign. The position was awarded to another officer, and Fuller’s career never recovered. His petulance cost him — and the British army — dearly. Architectural innovations can seem too much like hard work, even for those most committed to seeing them succeed.

Within academia, Rebecca Henderson’s ideas about architectural innovation are widely cited, and she is one of only two academics at Harvard Business School to hold the rank of university professor. The casual observer of business theories, however, is far more likely to have heard of Clayton Christensen, one of the most famous management gurus on the planet.

That may be because Christensen has a single clear theory of how disruption happens — and a solution, too: disrupt yourself before you are disrupted by someone else. That elegance is something we tend to find appealing.

The reality of disruption is less elegant — and harder to solve. Kodak’s position may well have been impossible, no matter what managers had done. If so, the most profitable response would have been to vanish gracefully.

“There are multiple points of failure,” says Henderson. “There’s the problem of reorganisation. There’s the question of whether the new idea will be profitable. There are cognitive filters. There is more than one kind of denial. To navigate successfully through, an incumbent organisation has to overcome every one of these obstacles.”

In an email, she added that the innovators — like Fuller — are often difficult people. “The people who bug large organisations to do new things are socially awkward, slightly fanatical and politically often hopelessly naive.” Another point of failure.

The message of Henderson’s work with Kim Clark and others is that when companies or institutions are faced with an organisationally disruptive innovation, there is no simple solution. There may be no solution at all. “I’m sorry it’s not more management guru-ish,” she tells me, laughing. “But anybody who’s really any good at this will tell you that this is hard.”

Almost a decade after resigning from a senior position in the British army, Andrew Mackay agrees: “I’d love to think that there could be a solution, but I don’t think there is.”

If I had to bet on the most significant disruption occurring today, I would point to the energy industry. Chris Goodall is a longtime observer of the renewable energy scene and author of The Switch, a book about breakthroughs in solar panel technology. Goodall points out that solar photovoltaics have enjoyed a dramatic fall in costs, one that shows no sign of abating.

Solar PV electricity is now cheaper than electricity generated by gas or coal in the sunny climes where most of the planet’s population live. A few more years and that advantage will seem overwhelming, which is great news for the planet and terrible news for incumbents.

Consider General Electric, which this year disappeared from the Dow Jones Industrial Average. In little more than a year, the old industrial titan’s share price had halved. One of the key culprits for its woes was a precipitous collapse in the demand for large gas turbines, that, in turn, was the result of a fall in the cost of solar power cells that had been relentless, predictable and ignored.

This possibility has been clear to the fossil fuel industry for a while. I know: I used to work in long-range scenario planning for Shell International. Back in 2001, my Shell colleagues and I were discussing thin solar films that could be printed cheaply and applied to windows or hung as wallpaper.

We could see the threat of exponentially cheaper solar power — but recall what Joshua Gans said about Kodak and Blockbuster: “They knew what the future looked like. They didn’t know later than everybody else, they knew ahead of everybody else.”

They knew. But they could not act. Because what is an oil company to do in a world of abundant, cheap solar energy? Offshore wind farms play to some oil-company strengths; they know a lot about large metal structures in the North Sea. But solar energy is an architectural innovation. The pieces just don’t fit together like an oil rig or a refinery. As a mass-market, manufactured product it is closer to the skill set of Ikea than Exxon.

The implication of Christensen’s theory is that oil companies should have set up solar subsidiaries decades ago. Many of them did, without much success. The implication of Henderson’s theory is that the oil companies are in big trouble.

Chris Goodall thinks the oil companies should rescue what they can — for example, by developing synthetic hydrocarbons derived from water, atmospheric carbon dioxide and solar energy. Such products would play to oil-company strengths. But for most of their business lines, Goodall says, “The best strategy for the oil companies is almost certainly gradual self-liquidation.”

Or as BP’s chief executive Bob Dudley candidly admitted to the Washington Post recently, “If someone said, ‘Here’s $10bn to invest in renewables,’ we wouldn’t know how to do it.”

Despite all the obstacles, the British army continued to develop both tanks and tank tactics throughout the 1920s and 1930s. Yet the internal politics proved toxic. The Germans, meanwhile, watched and learnt. If the British were hamstrung by their inability to reorganise what was, after all, a victorious army in the first world war, the Germans had the opposite problem: they had barely any army, and no status quo to defend. There was no organisational architecture to get in the way.

When Adolf Hitler came to power in 1933 and began to expand the German army and invest in tanks, he encountered a German military that had been watching, thinking and experimenting for 14 years. On his 50th birthday in 1939, Hitler celebrated with a parade of Germany’s newly reconstructed army through Berlin. “For three hours,” wrote one witness, “a completely mechanised and motorised army roared past the Führer.” This witness was a guest of honour at the celebrations. His name: J F C Fuller.

After quitting the British army in frustration, he had thrown his lot in with the British fascists of Oswald Mosley. He wrote vitriolic attacks on Jews. Some observers wondered whether this was simply an attempt to win favour with the world’s tank superpower, Nazi Germany. One of Fuller’s biographers, Mark Urban, doubts this: “The facility with which Fuller made anti-Jewish jibes in letters and books suggests pleasure rather than duty.”

Nobody doubts, however, that Fuller was obsessed by German tanks. After all, there was one army that had really understood and embraced his ideas: that of Adolf Hitler.

After the parade, Major General Fuller met Hitler himself in a receiving line at the Chancellery. The Führer grasped Fuller’s hand and asked, “I hope you were pleased with your children?”

“Your excellency,” Fuller replied, “They have grown up so quickly that I no longer recognise them.”


Illustrations by Janne Iivonen

Tim Harford is an FT columnist. His latest book, ‘Fifty Things that Made the Modern Economy’, is available in paperback

Head to the Tech Tonic stage at the FT Weekend Festival on Saturday September 8 at 1pm to hear Tim’s talk on “Why brilliant ideas die”

Follow @FTMag on Twitter to find out about our latest stories first. Subscribe to FT Life on YouTube for the latest FT Weekend videos


Why big companies squander good ideas | Financial Times
.
.
.