Market data in the age of AI

Crisil Coalition Greenwich, in partnership with SIX for the third consecutive year, conducted an in-depth study of 50 global buy-side firms to uncover the latest trends and innovations in market data consumption, distribution and technology adoption. The research provides a comprehensive snapshot of the evolving market data landscape, examining the types and frequency of data usage, including the growing demand for real-time and historical tick data. With a particular focus on the transformative impact of emerging technologies, this report delves into the adoption and applications of artificial intelligence and machine learning (AI/ML), as well as the strategic role of cloud infrastructure and application programming interfaces (APIs) in shaping the future of market data delivery and consumption.

Executive Summary

Crisil Coalition Greenwich, in partnership with SIX for the third consecutive year, conducted an in-depth study of 50 global buy-side firms to uncover the latest trends and innovations in market data consumption, distribution and technology adoption.

The research provides a comprehensive snapshot of the evolving market data landscape, examining the types and frequency of data usage, including the growing demand for real-time and historical tick data. With a particular focus on the transformative impact of emerging technologies, this report delves into the adoption and applications of artificial intelligence and machine learning (AI/ML), as well as the strategic role of cloud infrastructure and application programming interfaces (APIs) in shaping the future of market data delivery and consumption.

Key takeaways

• Real-time data usage is on the rise. Use of real-time data is growing and becoming more frequent, with 65% of respondents using it throughout the trading day, up from 54% in 2024. Moreover, the use of real-time data is moving beyond trade execution into both risk and compliance, and portfolio analytics.
• In parallel, overall spending on market data is projected to rise—again. Participants in our study expect market data spending to grow solidly over the next 12 months, echoing our 2023 and 2024 results. Nearly 70% anticipate an increase in budgeted spending of 1% to 5%. In some cases, this increase is expected to be larger, particularly with spending associated with spending on data types such as such as risk and regulatory data, index data, tick history, and crypto data.
• Market data delivery continues to shift toward cloud adoption. Asset managers, wealth managers and private banks have significantly increased cloud adoption since our previous studies. Today, 63% of participants receive data using internet connectivity from the public cloud, versus only 30% in 2023.
• Buy-side transformation through APIs and cloud maintains its positive trajectory. Cloud is increasingly leveraged for streaming data, with 53% of respondents saying that cloud will contribute to enhancing delivery and consumption of streaming data in the next few years. APIs, which are becoming more and more ubiquitous, are giving users a more efficient use of data while maintaining quality and customer experience, thus offering a very good ROI for market participants.
• AI/ML will contribute the most to market data delivery and consumption in the next two to three years. AI/ML continues its stratospheric rise and is seen as more influential than cloud and APIs. While AI/ML is viewed more as a recommendation machine (by 90% of participants) versus a decision-maker, there are signs that this technology is becoming far more mainstream, with concerns about erroneous data on the decline.
• The buy side wants more high-quality tick data. Over three-quarters of buy-side respondents are seeking more and better sources of historical tick data for areas such as market and trade surveillance. Quality of data is their top consideration.

Market data delivery shift to cloud reaches new heights

Cloud adoption for market data delivery continues to rise to a higher altitude. Notably, 63% of respondents report receiving market data using internet connectivity from the public cloud, showing that historical concerns about security have been minimized. In our 2023 study,¹ the use of cloud providers stood at only 30%, showing the rising tide of cloud adoption over the last few years.

And while internet connectivity is more prevalent, dedicated connectivity remains popular. Among users of dedicated connectivity, one notable trend is the reduction in preference for cross-connects in favor of direct cloud connectivity and dedicated point-to-point connectivity.

Moving beyond connectivity to the data providers themselves, market data is received using a variety of popular methods. The top choice is receiving data from market data vendors/redistributors—an increase from last year, when direct-from-the-source was the most popular choice. Redistributors offer value, including aggregating and normalizing different data types and integration into workflows.

We believe that direct-from-the-source will always be applicable, as whole swaths of the market, such as hedge funds, often choose it. Another consideration is that there will always be a need for the data management function. Data professionals at these firms are dealing with multiple asset classes and will still need specialized data providers, direct sourcing of data and many APIs. They will almost always need multiple vendors—often four or five providers if these buy-side firms need specialized data. Aggregators and redistributors can provide multiple datasets through the same pipe, but the challenge is also to maintain quality, manage costs and provide great service and reliability. Buy-side firms will often need backup sources, having more than one vendor even for directly sourced data.

At the same time, primary market data sources such as exchanges remain slightly more popular than market data packaged through third-party applications, which includes trading software like order management/execution management systems (OMS/EMS) and portfolio management systems (PMS). Moreover, on-demand platforms (e.g., AWS, Databricks, Google, or Snowflake) are offering compelling value propositions as well, reflecting the popularity of the data marketplace approach. The ease of adding new datasets, including historical datasets, makes these ondemand platforms (and their pay-for-use model) relatively simple and attractive.

Real-time data expands well outside of trading

A compelling trend the last few years has been the rise of real-time data, as we move into earlier and earlier trading, and even some 24/7 trading environments in certain asset classes. We observe real-time data use becoming more frequent and growing well beyond the front office into the middle office. Sixty-five percent of respondents use real-time data throughout the trading day, with the U.S. slightly more inclined than the U.K./ Europe. This suggests greater alignment in data frequency requirements across the trade life cycle (as opposed to historically only at the point of trade).

Real-time data is more expensive and not needed for certain functions, including back office, where end-of-day or intraday snapshots suffice. However, we believe the uptick in real-time data use suggests that a wider range of firms and functions, particularly in the U.S. and U.K., see the value of this data relative to cost.

The use of real-time data is moving beyond trade execution into both risk and compliance (Europe) and portfolio analytics (U.S./U.K.). Using real-time data for trading is more prevalent in the U.S. and U.K., where there are typically more diverse and latency-sensitive trading strategies. Interestingly, however, wealth managers and private banks told us they use real-time data in risk and compliance more often than in trade execution. This is likely due to respondents being in roles other than trading, including research, risk management, compliance, and operations, which consume more and more intraday and real-time market data. At the same time, it also reflects the growing ubiquity of real-time data. For clarity, we note that real-time data use cases include both real-time streaming and intraday snap valuations. There is a growing interest in these intraday tools, whereas non real-time APIs can be useful for end-of-day use cases.

Tick data sources are rising

Moreover, historical tick data is increasingly used in areas such as market and trade surveillance, where we see 85% of respondents using it. There we observe that good quality tick data typically comes from a smaller set of niche players. Historical tick data is useful in this area because it can improve the actual performance of this function, whereas areas like TCA are driven less by performance and more by a simple obligation to report. A high percentage (76%) are also considering adding additional historical/end-of-day data sources and new markets, beyond what is consumed today.

Users are seeking new sources of tick data as higher frequencies of information grow in importance. Rather than focusing on the replacement of existing data, the addition of more tick data sources is being demanded. While the volumes of tick data are set to increase, it is important to note that quality is still the number one priority. Market participants are clearly willing to pay more for high-quality tick data that passes muster.

Market data in the age of AI

The age of AI/ML is having a major impact on market data delivery and consumption, and strongly impacting perceptions of how market data will change in the coming years. The era of generative AI and agentic AI offers opportunities to reinvent research and investment workflows on the buy side—not just trading signals or algorithmic trading. Wealth managers and private banks see a broad opportunity to reinvent themselves using generative AI, including the automation of internal business processes, enhancing the customer experience, predictive analytics, and better investment decisions.

However, despite the increasing role of AI/ML, our respondents don’t believe that humans will be replaced—rather, machines will make better recommendations and help personnel make better decisions. Whether it be customer service, investment ideas or even trading decisions, humans will continue to be accountable for these decisions and outcomes.

AI/ML application to market data also reinforces the need to get the right data and get it accurately, as AI/ML processes are downstream of quality data. The benefits are clear—professionals can use AI/ML to synthesize large amounts of data, including documents, into manageable pieces of their workflow quickly. But it all depends on getting good data first.

Budgets for market data projected to rise—again.

Market data budgets and spending continue to be reliable and predictable, as market data is the lifeblood of investing and trading decisions. Study participants expect market data spending to grow solidly over the next 12 months, echoing our 2023 and 2024 results. Nearly 70% anticipate an increase in spending in the 1% to 5% range.

In some cases, this increase is expected to be over allocated, particularly with spending on categories such as index data, risk and regulatory data, and crypto data. For example, risk and compliance professionals working at buy-side institutions are using more tick data in dynamic areas such as market abuse. Moreover, crypto data requirements have increased strongly as more traditional institutions trade these assets and build investment products. However, for providers, there will be continual pressure to offer all data types from year to year, as demand will shift based on market conditions, volatility, regulatory environment, and evolving risk and compliance activities. A rising tide appears to be lifting all data types, but not equally from year to year.

Moreover, there is also growing evidence of more devolved decision-making in market data purchasing and licensing, where more decisions are made in the business units and teams, and within a primary region. This could lead to somewhat less scrutiny and budget controls, meaning the market is still very open to spending.

Conclusions and recommendations

Our research reveals a market data landscape that is rapidly evolving to meet the needs of an increasingly complex and technologically advanced ecosystem. The shift toward cloud adoption and APIs, the growing demand for real-time data across functions, and the transformative impact of AI/ML are redefining the way market data is consumed and distributed.

As we look to the future, we believe market data (including its various flavors, such as compliance, risk and regulatory data, pricing, etc.) will continue to play a critical role in informing investment decisions, managing risk and supporting operations. With AI/ML as a key driver of market data delivery and consumption over the next two to three years, it is imperative that firms prioritize investments in these areas to remain competitive.

With market data budgets projected to rise again, firms must be strategic in their allocation of resources. We believe firms will prioritize investments in cloud infrastructure, AI/ML technologies and real-time data capabilities to stay ahead of the curve. Firms will also focus on developing robust data management practices to ensure the quality and accuracy of data as, once again, quality matters more than just cost. By doing so, firms can unlock the full potential of market data to improve decision-making, manage risk and navigate the increasingly complex regulatory environment.

David Easthope advises on market structure and technology globally.

¹http://www.greenwich.com/market-structure-technology/market-data-quality-over-cost

This study is an update to our series of annual studies designed to better understand the trends and challenges of market data consumption by asset managers and wealth managers/private banks in the United States, the United Kingdom and Europe. Between June and July 2025, Crisil Coalition Greenwich employed a detailed questionnaire to gather responses from 50 buy-side firms. The majority of insights stem from professionals in both front-office and middle-office roles, including portfolio management, trading and research, and market data specialists.