AI - Taking Stock: The Benefits And Pitfalls Of LLMs And Their Future Applications In Trading

Insights from an early adopter in the industry

Long before the LLMs (large language models) boom, we were one of the early adopters of AI, having developed an AI-powered chatbot and virtual assistant product for trading platforms, based on several AI technologies. Our core business is designing and building trading software for major banks and global capital markets firms, and our AI product was developed to complement our broader offering.

Against this backdrop, the dawn of LLM was undoubtedly a game-changer, however, in truth, it has not always delivered on expectations. Indeed, after seven years, we feel enough time has passed to review the interim results when it comes to AI and LLM, looking at:

• whether the set goals of the LLM adoption have been achieved

• where LLM's impact has been positive and impactful

• where LLM appears, to be useless or even harmful

• possible future applications for LLM in trading.

Firstly, it's important to understand that AI is a much broader term than LLM, and LLM is neither a replacement for other AI methods and models, nor a more advanced version of "traditional AI". But, like the majority of AI industry pioneers, our development team enthusiastically adopted OpenAI’s ChatGPT. This was to make our AI-based chatbot, which interacts with clients of retail brokers, wealth management firms, and other financial institutions, more user-friendly and expand its functionality to include features such as order execution direct from within chats, natural language responsiveness, and multi-channel integration for trading on the go.

As our collective understanding of AI and its potential has developed, we at Devexperts have recognized the need for us to evolve our AI beyond its chatbot capabilities to take on a more proactive, agent-driven role in trading, allowing it, for example, to make decisions derived from user behaviors. We would label this AI a more ‘complete' evolution of the technology, merging the conversational aspects of Gen AI with the intuitive approach of AI agents. This is our predicted new direction of AI applications for trading desks.

Let's start with what worked really well and helped our team to improve the product.

1. Summarization

Yes, language models like ChatGPT are good for summarizing discussions, articles and other long reads. 

When a client's question can't be resolved by a chatbot, and it is escalated to a human, a quick summary explaining the issue is there, saving customer care personnel time when working on a resolution. 

Another use case is generation of a market digest while understanding the client's context (preferred markets, asset classes, time frames), without explicitly asking for details.

• Answers from a knowledge base

LLMs are good at ingesting FAQ knowledge bases and answering questions using the corresponding data. 

It's also possible to instruct AI to adhere to specific communication styles and/or add specific disclaimers. A complex topic can be explained in simple language to a certain extent. 

• Execution of multi-level instructions

Recent advancements allow language models to become "decision-makers" and perform some actions through function calling, for example. 

It can be observed in many use cases. For instance, brokers can automate many business processes such as investor/trader level scoring, KYC, and even upselling products and services – and the accuracy of the AI turns out to be surprisingly good. 

But it's also worth mentioning areas where AI has failed to cope with the task at hand and where further research and model improvement are clearly needed.

1. LLMs are not that good at operating structured data yet (say, trading database or market statistics). There is a possibility to run several models in a pipeline or approach the task as a series of consecutive runs. But the out-of-the-box quality of analysis made by AI is not currently that impressive.
2. It's hard for the language models to adhere to long instructions. The longer and more complex the set of instructions are, the higher the probability that the tool would just ignore some of them. So, our AI assistant requires some fine-tuning, because the raw output of ChatGPT may be misleading.
3. Retrieval agents (that allow us to look for the answer in a knowledge base) struggle to adapt well to the domain-specific use cases. It's hard for them to find relevant context in a myriad of acronyms and complex terminology. Failing to find relevant context results in the response from the model being too generic or completely irrelevant.

As our research and development continues, we have discovered that rather than relying on a single LLM, integrating multiple specialized AI systems yields far better results for traders. By leveraging models trained for specific tasks, such as fundamental analysis, risk management, and execution optimization, we can create an AI-driven experience where trader requests are dynamically routed to the right intelligence layer, all without disrupting users' conversations with chatbots. This approach not only improves accuracy but also ensures AI chats remain seamless and are useful even in highly complex cases. Additionally, this hybrid model opens the door to partnerships between different AI providers for a more well-rounded pool of data for traders.

Development will of course continue, and the issues highlighted will, no doubt, be addressed over time. It is, however, clear that while AI is valuable in enhancing user experience and assisting with customer service provision, it is a far cry from being able to operate independently or, further still, from replacing human capabilities, especially where demands become more complex.

Looking ahead, the true potential of AI in trading applications lies in its ability to offer deeply personalized experiences. By combining historical and real-time data, AI can adapt to individual trading styles and provide insights tailored to each individual user. As technology advances, AI assistants will feel less like tools and more like intuitive market advisors, helping traders make more informed decisions to navigate markets confidently.

Vitaly Kudinov is a Senior Vice President at Devexperts

This article is from an unpaid external contributor. It does not represent Benzinga’s reporting and has not been edited for content or accuracy. No AI-generated content.