Paper ID: 2410.08243 • Published Oct 10, 2024
Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow
Cyrile Delestre, Yoann Sola
TL;DR
Get AI-generated summaries with premium
Get AI-generated summaries with premium
Banking Transaction Flow (BTF) is a sequential data found in a number of
banking activities such as marketing, credit risk or banking fraud. It is a
multimodal data composed of three modalities: a date, a numerical value and a
wording. We propose in this work an application of self-attention mechanism to
the processing of BTFs. We trained two general models on a large amount of BTFs
in a self-supervised way: one RNN-based model and one Transformer-based model.
We proposed a specific tokenization in order to be able to process BTFs. The
performance of these two models was evaluated on two banking downstream tasks:
a transaction categorization task and a credit risk task. The results show that
fine-tuning these two pre-trained models allowed to perform better than the
state-of-the-art approaches for both tasks.