Background & objective of the project
DekaBank is an active member of KI Park e.V. and favours open exchange with other members in order to integrate innovative AI solutions into its work processes. One example is credit analysis, in which analysts extract and structure financial and company data from annual reports. This process is carried out manually to a significant extent - a time-consuming process that requires a high level of expertise and experience.
Between July and August 2024, several workshops and working meetings were held with YukkaLab, Datanising, Agents.inc and ML6. The aim was to develop initial steps to support and automate the analysis steps with the help of AI and thus reduce the manual effort as much as possible, while the control over the result of the analysis should remain with the experts. The companies contributed valuable expertise on AI-supported analysis processes and supported Deka's experts in the targeted further development of their own AI solution.
Development of a specialised RAG system
DekaBank relies on a system based on OpenAI models. The system uses Retrieval Augmented Generation (RAG) and deploys agents that fulfil specific tasks, such as enriching certain texts with references.
One particular challenge was to translate the analysts' knowledge into optimised prompt strategies. The central question was: How can the complex requirements of the analysts be integrated into the AI in such a way that it provides precise answers? Deka trained its analysts accordingly so that the technical experts were able to translate their specialist knowledge into efficient prompt strategies.
Deka organised internal training sessions on the following topics:
- Effective prompting: Optimised formulation of AI queries.
- Understanding RAG: Integration and utilisation of external and internal data.
- Quality control: evaluation and optimisation of AI-generated content.
Experts play a decisive role in assessing the quality of the results. In order to optimise AI in the long term, the development of standardised performance indicators is recommended.
Important lessons learnt from development
- Involve experts at an early stageThe best results are achieved through direct application and continuous feedback.
- Transparency in promptingAnalysts need not only the input and output, but also insight into the underlying context.
- Internalise the learning content: Multiple training courses promote understanding
- Optimise chunking strategy: Tests have shown that longer text sections deliver more reliable results than shorter ones.
- Automated source references improve traceability:Targeted prompting ("Please integrate the sources into the text again") increased the transparency of the AI-generated reports.
- Structured evaluation: Regular feedback helps to optimise model quality in the long term.
Perspectives from the ecosystem
In order to optimise the further development of DekaGPT, external experts from the AI Park network were involved. They contributed valuable suggestions for improvement to make the AI-supported analyses more efficient and practical.
AGENTS.Inc showed how multi-agent systems can be used for AI-supported risk assessment. A robust pipeline allows risks in financial reports to be identified and analysed automatically. This significantly reduces manual review effort and enables more informed, data-driven decisions. Financial institutions benefit from more efficient risk minimisation and strategic planning.
Datanisingemphasises that a standardised evaluation of AI models is essential. Measurable key performance indicators facilitate fine-tuning and enable a targeted switch to alternative models if necessary. Without clear metrics, the optimisation of AI solutions remains a process that is difficult to measure.
YukkaLab emphasises the increasing importance of real-time data for the further development of AI-supported analyses. The integration of real-time data could significantly increase the added value of existing AI applications by combining internal company data with the latest news. This allows market changes to be recorded promptly and integrated into the analysis processes, which further improves the responsiveness and precision of credit research models.
ML6 particularly emphasised the relevance of self-hosted LLMs to ensure data sovereignty and compliance requirements in highly regulated areas. The use of Retrieval Augmented Generation (RAG) enables even more efficient information processing by combining specialised retrieval mechanisms with OpenAI-based LLMs. By using self-hosted models, companies retain full control over their sensitive financial data and can optimally fulfil compliance requirements.
Conclusion: knowledge sharing as the key to successful AI integration
The dialogue with the members of the AI Park network enabled DekaBank to gain valuable insights for its AI strategy. The development of DekaGPT with RAG shows how companies can meaningfully integrate AI into their processes.
Transferring the expertise of experts into AI applications and empowering them to do so is a key success factor. The open exchange of knowledge in the AI Park creates added value for everyone involved and shows how innovative technologies can be advanced more quickly through collaboration.
