The key to what some call the 'right to be forgotten' is the idea that personal information should be kept for no longer that is needed. So, an individual has the right to request that their personal information be erased. The holder must then either erase it or give a good reason why this cannot happen.
What counts as a good reason? One could be that the information is needed in order to supply a service that a customer still wants or needs. For example, it's not possible to provide a banking service for a person, but also to erase all their personal information from the bank's systems. Another good reason could be that the holder is required by laws or regulations to keep personal information for a set time before it can be erased. This is also true of banks - in fact there are strict rules about the records banks must keep.
For example, banks are required to hold financial records to help in the fight against crimes such as fraud, money laundering, or terrorism. So it's misleading to think of a simple 'right to be forgotten' in relation to banks. You have a right to request personal information to be erased, and for this to be carried out, or to be given a satisfactory reason why it can't be.