Proliferation, democratization & the cloud: divining the future of the modern data stack

The catchphrase “data is the new oil” illustrates the massive value to be derived from collecting and “refining” raw data in the context of the digital economy. More than a decade after this expression first entered the corporate lexicon, enterprise data is still big business.

During the recent KeyBanc Capital Markets Tech Leadership Forum, Michael Turits, Senior Enterprise Software Analyst, convened a panel of thought leaders in the database industry: Suresh Vittal, Chief Product Officer, Alteryx; Sahir Azam, Chief Product Officer, MongoDB; Arsalan Tavakoli-Shiraji, Co-founder and SVP of Field Engineering, Databricks; and Emil Eifrem, Founder and CEO, Neo4j. These four executives shared insights on the current state of their market, the enterprise shift to the cloud, the democratization of analytics and AI, and the future of the modern data stack.

Multiplying use cases + surging demand = fragmented marketplace

Ten or 15 years ago, a handful of database providers served the bulk of the market’s needs. Today, that handful has exploded into a crowded field of companies and solutions tailored for an ever-growing variety of use cases.

“As born-digital and digital native companies came about, more companies have seen that data has a lot of value,” says Tavakoli-Shiraji. Organizations’ desire to use more of their data rather than discard it, as well as their demand for more advanced analytics capabilities, is driving a proliferation of data sources and use cases: from fraud protection for credit card applications, to predictive maintenance, to enhancing customer experiences in fields like retail.

Azam points out that the evolution of enterprise software has also contributed to the massive proliferation of providers in the database market: “Twenty-five years ago at a bank, the software you ran served maybe 100 to a few thousand people internally. Now, banks are building applications that directly serve millions of customers every day. That’s driven a ton of innovation in terms of what people expect their software to do, and it’s driving monumental shifts in technology requirements.”

The proliferation of service providers in this space is also driven by the growing demand for real-time data processing and analysis. For example, banks evaluating credit card applications need to run complex models in real time, and fast-food retailers serving drive-thru customers require real-time operational intelligence. “The real-time conversation is nuanced,” says Vittal. “What kind of data do you need to power the experience in real time? Does the data need to be refreshed? That impacts your pipelines and how you think about your architecture.”

Enterprises are moving to the cloud, but not all at once

Enterprises in a variety of industries are adopting cloud-based platforms, but the rate at which they are making the transition varies. “We’re all seeing this macro shift to the cloud,” says Eifrem. “But what’s more subtle is that it’s happening at a different pace for different layers in the stack.” He notes that COVID-19 accelerated the demand for Neo4j’s cloud-based offerings.

“We have to live in this world where we’re balancing movement to the cloud with data continuing to grow in a variety of other data sources,” adds Vittal. He notes that Alteryx is currently focused on “bringing the analytics to where the data sits and allowing for the business users to be productive against the cloud environment.”

“The modernization effort of traditional systems, from legacy stacks onto more modern technology, isn’t always on the same timeline as the move to the cloud,” observes Azam. He says MongoDB has “bet aggressively” on a multi-cloud approach that allows enterprises to leverage their data across all surrounding services. Tavakoli-Shiraji of Databricks, which has been cloud-based since its inception, agrees that more and more enterprises are looking for a unified experience across multiple cloud providers.

The democratization of analytics and AI in the enterprise

“We’re in the very early stages of democratizing analytics,” says Vittal. “We think of democratization as upscaling the business analyst and the knowledge worker inside the enterprise, providing them with access to capabilities that were previously unavailable to them due to lack of technical depth or access to the right technologies.”

Azam views democratization in the context of reducing the number of people it takes to integrate artificial intelligence (AI) and machine learning (ML) capabilities in smart applications across an enterprise. “How can you empower the average development team to build better experiences that either automate the business, better serve the customer, or make the business more efficient?”

Tavakoli-Shiraji points to two trends driving the democratization of data: First, enterprises’ desire to more easily automate certain processes and tasks, allowing developers to build models more quickly; and second, the demand for vertical-specific applications that enable less sophisticated end-users in the enterprise to leverage AI and ML for industry-specific use cases. “From a data perspective, a lot of users in the same industries all want the same things,” he explains. “Vertical-specific applications have not historically been as successful, because it was so hard getting access to the data. But now you have platforms where you can see different use cases to build apps for end-users who won’t even know they’re using AI, ML, or a platform underneath.”

Predictions for the future of the modern data stack

Looking ahead, Eifrem predicts fast growth and significant consolidation: “The database market is the biggest market in all of enterprise software, and it’s predicted to grow to $100 billion over the next four to five years.” He anticipates that the 350+ database companies in the marketplace will converge down to a small handful, with a leader and a runner-up in each category.

Vittal believes that with the industry still in the early stages of some major shifts (moving to the cloud, evolving architecture, building out more real-time capabilities, etc.), the current fragmentation in the marketplace will persist for multiple years before giving way to consolidation. Even then, it will be unlikely for a single player to serve the entire tech stack. “Historically, we haven’t seen a single application top-to-bottom because the users at each level of the stack are very, very different. It’s hard to build a model where you serve multiple masters successfully.” Similarly, Tavakoli-Shiraji envisions a market with a few core platforms dominating the space, and a layer of vertical-specific fragmentation on top of those platforms offering a variety of new and different capabilities.

For Azam, the future looks like the best of both worlds from a customer standpoint. “Think about the early phases of cloud adoption with the major hyperscalers. Customers had to make a bit of a trade-off: if you wanted the best consumption-based cloud services with frictionless availability to spin things up and down, that’s the initial value of the infrastructure cloud providers. But if you wanted the best-of-breed depth of capability, you were often going with a software vendor. Now you’re seeing the rise of companies where you don’t have to make that choice. You get a multi-cloud platform that has the best capabilities in a particular domain and delivers the cloud-native experience: consumption, elasticity, scale. This is relatively new in the market, and that’s the next platform.”

The way enterprises use data is evolving quickly, and providers at every layer of the data stack are finding innovative ways to support a complex variety of use cases and capabilities. To learn more about investment opportunities in this dynamic market, contact your KeyBanc Capital Markets investment banker.

To learn more about attending one of our conferences, email the Corporate Access team.

About the 2021 Technology Leadership Forum

Technology companies experienced a tumultuous start to 2021 with uncertainty around industry valuations, tax policies, COVID-19 transitions, supply shortages, inflation concerns and valuation thresholds. To assess the current market dynamics we brought together investors, executives and founders from top private and public companies, and industry thought leaders to provide insights and explore opportunities for technology as we progress toward a "new" normal. Attendees included 1,000+ institutional investors, 180+ private equity/venture capital corporate development investors, 105 public companies and 46 private companies. The agenda included 90+ Fireside Chats/Presentations, and 6 thematic panels.