Use Case Financial
LLM ChatBots
For one of the most important clients in the Spanish banking sector, QUANT AI Lab, as part of the client’s Cognitive Architecture team, has released a Conversational Assistants service with LLMs. We are responsible for developing, implementing and certifying the release of this capability within the client’s technical architecture, so that the rest of the teams could start releasing internal solutions that use this technology.
The implementation of LLMs in the technical architecture has opened a range of possibilities in optimizing tasks and management within the bank’s internal processes. This service is currently running, and more than 10 technical initiatives are already being launched that will exponentially improve the efficiency of internal processes thanks to the use of this service.
Challenges
The objective of this initiative is to make available in the bank’s architecture a stack of tools and services based on LLMs for the creation of conversational assistants.
Seamless integration
Adapt the environments for the use of LLMs in the official python framework within the bank’s technical architecture
Certification
Certification and integration of Azure cloud services within the client’s architecture.
Solution
Design, develop and implement the flow of microservices that allows the history of conversations to be persisted and manage the flow of information ingestion into the knowledge base.
Creation a stack of tools necessary for the implementation and use of LLM instances for RAG architecture.
Information management of a knowledge base external to the LLM in a RAG architecture.
Resource allocation management based on the application to manage the users consumed by the global service, identifying their cognitive resources and indexes associated with the global configuration of the microservice.
Tech stack
Results
With this development, the first productive framework has been enabled in the client’s technological architecture for the management and use of LLMs in different areas of application.
Process optimization
This is allowing internal developers to create new solutions and optimize current processes by implementing this technology in their solutions.
6.35%
Error for dimensions estimation
