Tag: Gartner

Photo by Jongsun Lee on Unsplash. SQUIRREL!!! added by the author.

If we could only predict the next boom in cryptocurrency. Or when gene therapies might show up at your local Walgreens. Or when planes will fly themselves. Timing any of these correctly will make individuals rich and shareholders giddy. Timing them wrong will bankrupt investors and sink promising product launches. The most popular model for understanding technology trends such as these, and cashing in when the time is right, is the Gartner Hype Cycle. If ever there were applications where its usefulness should apply, it would be cryptocurrency, gene therapy, and self-flying planes. Unfortunately, the Gartner Hype Cycle can’t help us understand any of these trends. The only enlightenment it can offer is the simple truism that “hype exists.” Sorry Gartner, but PT Barnum proved that theory 100 years ago.

Let’s see how we can do better.

Introduction

Anyone who has sat through a corporate PowerPoint presentation or Investor Pitch Contest has seen the Gartner Hype Cycle. As testament to Gartner’s excellent marketing, most corporate followers can recite it by heart: After a trigger event, hype grows quickly, peaks, crashes just as quickly, and then slowly reaches a balanced equilibrium. It’s so reasonable and intuitive that most of us rarely question its basic premise.

Since 2009, Gartner has published a so-called “Hype Cycle” for all manner of markets, technologies, and trends. In fact, the model has become so popular that “analysts” now publish nearly 60 different hype cycle charts each year – from Supply Chains to Digital Marketing to Government Technology. Here’s the gotcha: When we apply even the most modest level of scrutiny to it, each assumption that underpins the Gartner Hype Cycle falls apart.

Great marketing. Bad science.

But what’s the alternative? In the early stages of any new technology, “hype” seems to be all there is to go on. Entrepreneurs need some mechanism to understand if a hot topic has potential, or if it’s just hot air. Investors need to know when to jump into the water, and when to sit on the beach.

What follows is the answer.

In five easy steps, we’ll dismantle the flawed logic of the Hype Cycle and replace it with a bespoke Attention Dashboard – a set of building blocks you can use to generate meaningful insights on your own. No fancy charts. No breathless anticipation of an analyst’s opinion. No massive checks written to consultants. Less branding and marketing spin. More data and useful conclusions.

To help illustrate the process, we’ll use an emerging technology clearly in its commercial infancy: clustered regularly interspaced short palindromic repeats, or CRISPR, for short.

Step 1: Stop using the word hype. Start using the word attention.

There is something about hype that seems both unseemly and unethical at the same time. Hype is disconnected for substance, truth, and reality. Hype is a deception. And the worst criticism of all: Hype is marketing.

Source: New Oxford American Dictionary, edited for clarity. Hype and Attention mean much the same thing, but hype is an emotional trigger on our lizard brain. Attention is a neutral (even positive) word, allowing us to engage our rational mind.

While not wholly false, when we use the word hype (with all its emotional baggage), we ignore its usefulness. Whether positive or negative, hype draws attention to a product, service, or idea, helping to break through the background noise of daily life and shine a spotlight on it – if ever so brief and fleeting that light may shine. Why are businesses, politicians, and advocacy groups so willing to risk the criticism that comes from hype? Because the biggest risk to achieving your goals isn’t being criticized, it’s being ignored.

Let’s make a helpful edit to this Gartner Hype Cycle chart. If we’re interested in the promise of CRISPR, it’s more useful to think of it as “attention” rather than “hype.” Attention draws interest. Hype turns people off.

Gartner was smart not to brand its “Hype Cycle” as the “Attention Cycle” – the word hype carries more emotional energy, making you feel as though you are privy to forbidden knowledge of how the world works. But while it may be smart branding, and while it may drive business for consultants, using the word hype leads to poor decision-making. Most decent people want to avoid hype, but they’re just fine with attention – especially when they believe in what’s getting the attention.

As an entrepreneur or investor exploring commercial opportunities for CRISPR, the word “hype” clouds our thinking. Using that word implies that the promises of the technology won’t materialize – that CRISPR is doomed to the dustbin of promising, yet ultimately failed, therapies. But that’s an oversimplification. Some specific therapies may not pan out. Others may provide only marginal improvement over current therapies. Still others may be revolutionary, transforming our approach to entire branches of medicine. Like many emerging technologies, CRISPR is pluripotent, meaning that it can morph into many specific applications as it develops. In fact, many versions of the base CRISPR science already are diverging to meet different therapeutic needs. Each of those will have different commercialization pathways. Some may end up meeting the literal definition of hype, but many will not.

What you call something matters. Choose your words carefully.

Step 2: Start focusing on what you want. Stop chasing squirrels.

It may seem tempting to track a wide variety of “emerging technologies” on the same chart, but it doesn’t help you understand them. In fact, it doesn’t help you understand any of them. You’re like a dog let loose in the backyard with 20 squirrels running in every direction. Unless you can pick one, you’re not likely to catch any of them.

It’s the same dynamic with emerging technologies. They’re all squirrels. They all dart around the marketplace at random. If you’re trying to catch “Smart Dust,” “Connected Home,” or “Biochips” by chasing them all, you’re just like our poor dog – exhausted and frustrated. Yes, those technologies may be “emerging,” but each one of them follows different market forces, varying user applications, and independent technology development paths. Plotting them all on the same curve implies they all will follow the same “hype” (read: attention) path, and more importantly, that none will fail or that none morph into something else entirely. That’s a ridiculous assertion on its face. We have examples of many technologies failing to ever reach the “plateau” in anything resembling their initial form. “Cold Fusion” immediately comes to mind, but give yourself 30 seconds, and you could come up with a dozen other examples.

 

Another helpful edit – let’s track related, not unrelated, technologies. But wait … where’s CRISPR on the “Emerging Technologies” list? By almost any measure, CRISPR is “emerging,” yet we don’t see it. The Gartner Hype Cycle is simply a random list of whatever they want you to notice, not what’s important to you.

By definition, emerging technologies change quickly as they mature. In other words, we have non-comparable data points plotted on the same graph. It’s not about comparing apples and oranges. It’s about apples and orangutans.

Instead, let’s zero in on what matters to us. We’re interested in tracking attention specifically on “CRISPR” as a therapeutic technology. To do that, we’ll want to track “CRISPR” as a term (obviously), but also a few other related terms in a topic cluster:

• Gene Therapy: CRISPR falls within this general category. Tracking this topic will ensure you catch other, related developments in the field as they emerge.
• Pfizer: While not specifically focused on CRISPR, Pfizer is a major player in pharmaceuticals, and is likely to play a major role in the future of the technology. Including it ensures we don’t ignore organizations who can (singlehandedly) drive the conversation.
• CRISPR Therapeutics: In addition to a major player, you’ll want to include a smaller organization solely focused on the topic area. You’re watching to see if their fortunes rise and fall with changes in attention … or if they do not.
• Coronary Artery Bypass Surgery: This is an excellent benchmark – a well-understood and mature healthcare topic. No, it’s not used to treat the same conditions as CRISPR might treat, but it provides a reference point for attention. Attention in “Coronary Artery Bypass Surgery” is stable over time, whereas attention in CRISPR ebbs and flows. Comparing two topics in the field helps us understand if attention in CRISPR is high because attention in all healthcare is high, or if the variation is unique to CRISPR.

You can use the same formula with your own topic of interest: (1) Choose the specific term; (2) Add the generic technology/topic; (3) Add a major player in the field; (4) Add a minor player or startup; and finally, (5) Remember to include a benchmark for comparison. Think of it as a study group with “controls” included. It’s good research design. Consulting firms tend to struggle with good science.  Now, you know how to do better.

If you’re not careful, you’ll end up chasing squirrels … and never catch one.

Step 3: Start quantifying attention. Stop fretting over hype.

The Gartner Hype Cycle uses “expectations” as its dependent variable (on the Y-axis) as simply another word for hype. At first glance, it seems intuitive – expectations can rise and fall over time. But take a more careful look at the chart – is the expectation at the “trigger” at the same level as the “trough”? Not exactly. The trigger is more accurately a representation of “no expectations” and the trough is a representation of “negative expectations” – or at least lower than the initial condition (that’s what “disillusionment” means.) At the very least, we should draw the chart like this:

Attempting to correct the scale (and the zero point on the Y-axis) highlights a more pressing issue: The measurement of “hype” as a variable at all. What is hype, exactly? How do you measure it? Is it a measure of the number of articles written within a certain amount of time? Is it a measure of the tone or sentiment of those articles? Is it the authority of the publications? Or the authority of the authors? Or the size of their Twitter following? Without a standard quantification for your variable, you don’t really have one.

 

Whenever you see a chart, be wary of undefined terms or vague ranges masking as quantified variables. In the first case, we’re given no explanation for “expectations” on the Y-axis other than “it goes up and down.” The time series ranges in the legend don’t correspond with any formula – they’re just an analyst’s guess. Ignore them.

In years past, if we wanted to quantify attention, we needed to make educated guesses based on counts of media articles, exposure to advertising, or some other indirect measure. Today, we can use actual behavioral data to track the pattern of changes in attention. Google Trends is a good place to start (and it’s free), and it’s a powerful tool if you know how to use it. In simple terms, Google Trends shows you the change in search volume on a topic (or topics) over a period of time. It will not show actual search volume (in number of searches), but it will show you much more important information in addition to change over time: relative search volume. If you compare multiple terms (up to five) on the same chart, you can see how they compare to each other. This is particularly useful when you benchmark a new term against one you understand quite well.

Hmm. Notice something about the chart once we use actual data from Google Trends? Do you see anything like the curve represented in the Gartner Hype Cycle? No? More on that later.

The blue line represents the topic “CRISPR” over a three-year period. As you can see, interest spikes and wanes over time, but remains high as it relates to the terms and topics we’ve chosen as controls. From this data alone, what conclusions could we draw? Pfizer (the company) and Coronary Artery Bypass Surgery (the topic) draw more consistent attention than CRISPR, although CRISPR occasionally spikes. Attention from CRISPR seems to drive a bit of interest in gene therapy more generally, but it does not create a “halo effect” on any specific startup company. If I were an angel investor, I might tentatively conclude to hedge my bets on several CRISPR-related startups rather than just one. What would have learned from a “static position” on the Gartner Hype Cycle (had CRISPR even been on it)? Not very much. And what you would have learned would have been wrong.

Is Google Trends the only source for attention data? Certainly not. Engagement on social media (likes, comments, and shares) is another one. As is checking the prices for keywords in a pay-per-click auction (Google, Amazon, and LinkedIn all use these real-time systems.) What you want to avoid are “counting” metrics such as “the number of media articles written on a topic.” The media might be paying attention, but what you really want to know is if everyone else is paying attention.

Bottom line: Never settle for pure intuition without bothering to confirm it with data … especially if the data is accessible, accurate, and affordable.

Step 4: Stop imposing arbitrary phases. Start fixing time as a truly independent variable.

Time is the usual suspect for the independent variable, appearing reliably on the X-axis of most charts. But notice something about this one. Time isn’t the variable; phase is the variable – e.g. “Plateau of Productivity.” Be careful, time and phase are not interchangeable. Using one in place of the other isn’t unheard of, certainly. The more-familiar S-curve of Diffusion of Innovations uses product lifecycle stages instead of absolute time measurements as well. However, swapping time for phases is a mental trick that can fool you if you’re not careful.

Never confuse phases for actual passage of time. When time is an independent variable, it can dramatically impact the shape of the “curve” in the plotted data – compressing or extending it. When you’re making a decision, the phase isn’t as important as the velocity through that phase. The only way you know that is by plotting actual time.

We’re used to seeing time on the X-axis. Quantum mechanics aside, time is a fixed, periodic variable. It progresses in an orderly fashion, and at the same rate, from the past into the future. It’s so obvious that we don’t stop to think about it. When you exchange time for phases, you risk being tricked into thinking that phases act in the same way as time: Linear, with fixed intervals, and progressing with consistent velocity. Emerging technologies never evolve in an orderly fashion – they are riddled with fits and starts, sometimes accelerating quickly and sometimes languishing for years. By swapping time (predictable) for phase (unpredictable), we hide the most critical information.

Any time you think about trends, a snapshot in time isn’t as important as the velocity of the movement and what triggers those movements into motion. That’s how investment decisions are made. The arbitrary, non-measurable phases in the Gartner Hype Cycle are not truly independent variables at all.

To see for ourselves, we only need to go back to our example:

Instead of “one” trigger to “start off” the “hype cycle,” we notice at least two spikes in attention in a reasonably stable volume of attention over time. The better question to ask is what triggered those changes?  Although we cannot know what will cause attention to spike in the future (sorry, Gartner), we would be wise to see what drove attention in the past. Luckily, our chart gives us some clues – two spikes in particular: One on or about May 1, 2018 and another in December of that same year.

Sticking with Google (although you could use any news aggregation tool), a time-limited search quickly reveals the answers:

Spike #1: Scientists can use CRISPR to edit genes. Should they?

Spike #2: Marc Thiessen: Gene editing is here. It’s an enormous threat

In the first case, it was a national news program (60 Minutes) that brought the technology to a wide audience. Were there stories about CRISPR in the media before? Of course, but none quite captured the broad-based public attention that 60 Minutes could generate. In an era of “micro-influencers” it seems that big media can still drive big interest.

In the second case, a Chinese scientist claimed to do what the 60 Minutes program hinted at – using CRISPR technology to edit human infant genomes. Whether he actually did it or not is a matter of debate, but news outlets pounced. As with other emerging technologies, the negative attention of a “worst case scenario” can drive interest. It’s notable that positive developments (potential therapies for debilitating disease) failed to result in a similar spike in attention.

Again, if I were that angel investor, I might reconsider my micro-influencer strategy and be prepared to fund my startup to go for the national news. I might also focus on those companies who could court negative attention – either by doing what people fear (or shouldn’t fear in this hypothetical investor’s opinion) or by preventing it.

Beware of the “phase trick” on any chart. Keep time independent of value judgements.

Step 5: Stop believing in generic cycles. Start discovering true attention drivers.

We may admit that we can’t quantify hype, that we can’t predict timing, and that we must stick with comparable data points, but the shape of the Gartner Hype Cycle curve still seems to match our everyday experience. After a trigger, hype rises quickly, peaks with a hysteria, crashes hard, and finally, slowly regains a reasonable middle ground. It all seems so intuitive.

Like so much of our intuition, it turns out that we’re wrong.

When scholars attempted to replicate the hype cycle (here, here, and here as just three of many examples) the resulting curves looked nothing like the intuitive story the curve attempts to tell us. In one particular example, researchers used actual quantitative data (gasp!) to plot the actual “hype cycles” for three emerging technologies: Internet Telephony, Gene Therapy, and High-Temperature Superconductivity. Here’s what they found:

See a pattern? No? Precisely. When scientists put the Hype Cycle – or any hypothesis for that matter – to the test, they’re trying to disprove it. Having an idea is just fine, but that’s just the first step. You must test it, empirically, to decide if your idea actually represents something about the natural world. If it does, you don’t accept it, but rather you continue to try to find its flaws. If it fails, you discard it. That’s how science works. Clearly, that’s not how consulting works. Here’s what one group of researchers said in the introduction to their 2013 paper (emphasis mine):

Given the model’s proclaimed capacity to forecast technological development, an important consideration for organizations in formulating marketing strategies, this paper provides a critical review of the hype cycle model by seeking evidence from Gartner’s own technology databases for the manifestation of hype cycles. The results of our empirical work show incongruences connected with the reports of Gartner, which motivates us to consider possible future directions, whereby the notion of hype or hyped dynamics (though not necessarily the hype cycle model itself) can be captured in existing life cycle models through the identification of peak, disappointment, and recovery patterns.

Let’s translate academic English into standard English: Generic hype cycles are bullshit.

To better understand changes in attention and tone surrounding CRISPR, we must understand who to pay attention to, and who we safely can ignore. We could stick with Google here and set up a “News Alert,”, but it’s more efficient to use a media tracking tool. Brand24 is a functional, low-cost option, and we will use it here. Why use an analysis tool rather than a simple raw news feed from Google? A few reasons:

1. Media attention shifts quickly – you may have very few results for many weeks, and then a flood of them in one afternoon. You want a tool that can help you see the trend line.
2. Influence on attention tends to aggregate – as you already know, certain media outlets, companies, universities, and individuals disproportionately impact what gets attention and when. You need a tool that can show you the “hidden” influencers in the network.
3. Attention tends to multiply iterative and duplicate results – not all attention is created equal. The 60 Minutes piece, for example, spawned thousands of “reactions” from less-influential sources. While important in aggregate (to know what’s getting attention), the flood of information can overwhelm your analysis, leading you to follow the wagging tail, not the happy dog. Your need a tool that can analyze the origin of the topic in real time.

Let’s use Brand24 to collect some basic data over the past 30 days:

Huh, the “influencers” aren’t who we thought they’d be.

Again, our intuition fails us on multiple levels. Is Twitter the biggest source of attention (aka “news”) on CRISPR? No. It’s YouTube. Are the traditional media outlets driving the conversation? No, not recently. It’s the science-explainer YouTube channel Seeker. Although MIT Technology Review clocks in at the number three position, the usual “big media” suspects aren’t present.

What about tone and sub-topics? When you examine the most influential sources in the recent conversation, you notice something unmistakable – they’re all explainers. In other words, although there has been plenty of media coverage, the public remains largely ignorant of the underlying technology. As that angel investor, I would be wary of sentiment and opinions from the general public – they’re interested, but they don’t know enough (yet) to buy a commercially-available product. That investor might be wise to stick to investing in startups offering technical products to an informed, scientific audience – at least until public knowledge grows.

Attention doesn’t follow a set pattern. Take the time to learn what drives changes in attention for your specific topic.

Conclusion

The Gartner Hype Cycle is a brilliant piece of marketing, and it’s a boon to consultants everywhere, but it’s not going to help you make decisions. Specifically, it fails in five key areas:

1. It chooses a loaded, emotional word (hype) instead of a variable you can objectively quantify (attention). Instead of the word hype, use the word attention to eliminate negative bias. You can always track positive and negative attention (sentiment) separately if that’s meaningful.
2. It fails to quantify attention – simply using the word “hype” distracts us from recognizing this error. Instead, use a quantitative measure of behavioral data, such as the number of web searches on a given set of topics.
3. It fails to stabilize time as the independent variable, using branded “phases” to mask the reality of changes over time. Instead, use actual time. Specifically, choose a time scale that makes sense for your area of interest. For fashion trends, that might be a few months. For medical devices, that might be a few years.
4. It fails to measure the correct data, grouping non-comparable trends and technologies onto the same graph until it’s impossible to see the meaningful pattern behind any one of them. Instead, choose your topic cluster (and comparable / benchmark topics) carefully and intentionally.
5. It fails to properly identify the shape of the curve itself, relying on our “intuition” and a “reasonable-sounding story” to trick us into not looking deeper for the real patterns. Instead, use your own data to plot the “attention cycle” unique to your area of interest. When you use your own data, you’ll begin to discover the true influencers and possible triggers. Hint: They never follow a predictable pattern. You’ll need your own data.

In other words, use this:

Real, actionable, accurate, and reliable data on a topic that matters to you.

Not this:

The next time you’re in a meeting and someone whips out a version of this chart, feel free to respond with this one. You’re welcome 😉

As we’ve demonstrated, you don’t need expensive consultants or sophisticated software to build an “Attention Dashboard” for the topic that interests you. In a few easy steps and for less than $50, we were able to draw useful conclusions about the past, present, and future of CRISPR – far more than we could learn by studying the so-called Hype Cycle, or by listening to consultants wax philosophical to the tune of thousands of dollars per hour.

If all else fails, you’re safe to bet against anyone who still believes in hype.