Implementing AI in Financial Data Management: Navigating Challenges for Future Rewards
The financial industry is undergoing a transition spurred by the relentless advance of artificial intelligence (AI). Recent research by Gresham polling firms split over the US, the UK and the DACH region revealed that, although nearly all firms are starting to use AI to some degree, there is a pronounced split between piecemeal, local, department-level adoption versus a more comprehensive, enterprise-wide approach. 41% of Gresham's survey said they had extensively deployed AI across business operations.
From Promise to Process
As AI is set to become anchored in financial data management, it brings a promise of revolutionising operations, enhancing the risk management, compliance and finance functions, and personalising customer service. On the one hand AI is just the next level of automation, which will lead to increased operational efficiencies. Combine this with the human interface, and the ability, via prompt engineering, to effectively program the bots without knowing how to code and you have an enormous productivity boost. On the other hand, the enhanced speed in discovering and presenting data should also lead to new insights and redefine processes to impact client data management, risk and front office.
Yet, the path to harnessing AI’s full potential and effectively embed it into capital markets’ business processes is strewn with challenges that firms must navigate to reap the long-term benefits.
Hurdles to AI Adoption
Adopting AI in financial data management isn’t just a matter of installing new software; it represents a major shift in how users and systems interact with data and puts an increased premium on data quality and provisioning capabilities.
AI adoption is not just about acquiring new technologies but also about ensuring that the data feeding into these systems is of high quality. High-quality data is the lifeblood of effective AI systems, and obtaining, processing, and maintaining this data requires significant investment. Consequently, we can expect a shift within the cost base from operations to data and technology. The Gresham survey supports this. In it, nearly two-thirds (63%) of senior decision-makers employed by financial services firms globally said that AI would result in an increase in the cost of data within their organisation.
Moreover, the technological integration of AI within existing systems presents its own set of hurdles. Financial institutions often grapple with legacy systems that are not readily compatible with the latest AI technologies. Ensuring a seamless integration that allows AI to interact effectively with these systems requires thoughtful strategy and careful execution.
Another significant challenge is the availability of skilled personnel. AI systems are complex and require a deep understanding of both the technology and the financial domain to be effectively implemented and managed. In line with this, contrary to expectations perhaps, 40% of the survey sample projected that AI would actually result in an increase in operational headcount in data management. The current market, unfortunately, shows a scarcity of professionals who possess this blend of skills, making the talent gap a critical barrier to AI adoption.
Lastly, there is the aspect of IP and licensing in the context of AI. Gen AI will have a major impact on market data management but there are some dependencies to successful adoption. Content licensing agreements for market and reference data will evolve to refine the notion of ‘derived data’ in the age of gen AI. The legal system is effectively being asked to clarify the boundaries of ‘derivative works’ and the interpretation of ‘fair usage’ doctrine in US IP law. New use case in standard content licensing agreements, possibly along the lines of the derived data notion. Certain AI models will be whitelisted and under permitted use but we can expect more legal work on content licensing agreements. Model suppliers will have to be clear about how and with what content their models have been trained and firms must protect themselves from (inadvertently) supplying them with confidential information. Model developers should maintain the provenance of AI generated content and keep a comprehensive audit trail.
Evolving Notions of Data Quality and Data Governance
The notion of data quality will broaden and adapt for use in AI and machine learning. When it comes to the (meta)data requirements to determine whether data is fit for purpose, the days when data quality was simply about values being right or wrong have long gone. It has become much more subtle but, at high level, comes down to answering the questions “am I allowed to use it?” and “does it make sense to use it?”. On top of the traditional metadata aspects including commercial permissions, legal boundary conditions, timeliness, accuracy and completeness, it is imperative to do thorough red team/adversarial testing on gen AI models.
Fitness for purpose of data when it came to machine learning and statistics so far was mostly about feature engineering and preparing the data, basically giving the models a hand. This could include filling missing values, handling outliers, binning to prevent overfitting etc. With LLMs being language models and AI models as a giant wrapper around (enterprise) content, model testing and model results have become more subtle and fuzzier. The current set of data cataloguing tools and data dictionaries will rapidly look arcane.
Beyond the Barriers
Despite these challenges, the benefits of integrating AI into financial data management are too substantial to ignore. AI has the power to transform the vast amounts of data that financial institutions handle daily into actionable insights, driving operational efficiency and strategic decision-making.
One of the most compelling advantages of AI is its ability to enhance operational efficiency. In fact, 33% of the survey sample saw this as the primary benefit. By automating routine tasks, AI frees up human resources to focus on more complex, value-adding activities. This not only boosts productivity but also allows organizations to allocate their human capital more effectively.
AI’s ability to analyse and interpret large datasets can significantly improve risk management. Through predictive analytics, (which 53% of the sample were either exploring or using), AI can identify potential risks and anomalies that might go unnoticed by human analysts, enabling institutions to take proactive measures to mitigate these risks.
Furthermore, AI can play a crucial role in enhancing customer engagement and satisfaction. By analysing customer data, AI can help financial institutions tailor their products and services to meet the individual needs and preferences of their clients, fostering loyalty and driving business growth.
Looking ahead: A Strategic Approach to AI Adoption
To overcome the challenges and maximize the benefits of AI, financial institutions need to adopt a strategic approach. This involves investing not just in technology but also in people. Enhancing workforce capabilities through targeted training and recruitment is crucial to bridge the talent gap.
Similarly, updating and upgrading technological infrastructure is essential to support the advanced requirements of AI systems. Collaboration with AI and data management experts can also provide invaluable insights and guidance, helping firms to navigate the complex landscape of AI integration.
In conclusion, while the journey toward AI adoption in financial data management is fraught with challenges, the potential rewards are immense. By taking a strategic, informed approach, financial institutions can overcome these hurdles and harness the power of AI to drive efficiency, innovation, and growth. The future of financial data management is undeniably intertwined with AI, and those who navigate this transition wisely will emerge as leaders in the new era of finance.