How Machine Learning can help fight money laundering

By Peter Munoz, Sr Business Director, and Rexwell Minnis, Director of Software Engineering

When most people think of financial crimes, they think of the ones making headlines--identity theft, stolen credit cards numbers, embezzlement--but these represent only a fraction of what financial institutions should protect against. Money laundering fuels crime syndicates and terrorism worldwide, and while it can be difficult to gauge the impact and immensity of money laundering, the United Nations Office on Drugs and Crime (UNODC) once estimated that criminal proceeds amounted to 3.6 percent of global GDP, with 2.7 percent of global GDP being laundered. This is a monumental figure, representing well over a trillion dollars annually.

That figure is just an estimate, but there is another number we can cite with confidence -- U.S. financial institutions filed 2.5 million Suspicious Activity Reports in 2020, according to the U.S. Treasury Department, and the volume of these reports has been trending up for years.

Best practices to combat the problem have traditionally included monitoring for suspicious behaviors and tactics known to be used in money laundering. But in our global, web-based economy, criminals are constantly adopting sophisticated new techniques. As traditional detection methods become outdated, the financial industry is evolving our Anti-Money Laundering, or AML, tactics to include Machine Learning (ML) to adapt more quickly to changes in customer and criminal behaviors. Putting the ML into AML represents a unique opportunity to take Anti-Money Laundering efforts to the next level and to help prevent financial crime.

Traditionally, banks use rule-based, deterministic tools to monitor transactions for money laundering. But to focus more precisely on just the right combinations of triggers, teams add parameters and conditions over time, which improves performance but also makes the rules more difficult to maintain. And with rapid changes in customer and criminal behavior, we need to be nimble. Rapid adaptability is not the only weakness of a rules-based system. Even with highly complex rules-based triggering logic, the output is often still binary: the activity generates an alert (referral for investigation) or not.  And investigators see those alerts like a table full of wrapped presents -- with little clue about which to open first. 

To address the need for rapid adaptability and the ability to prioritize alerts, in August 2020 Capital One began applying machine learning to suspicious activity monitoring—with AML subject matter experts partnering with data scientists to design a complete system. Our goals were to:

• Use a wide range of data to enable better decisions
• Give more insight to AML investigators
• Prioritize risk-based investigations
• Adapt more quickly to changes in both criminal & customer behavior
• Provide outstanding transparency and governance

Originally starting as an experimental side project, its potential to combat money laundering at scale was quickly apparent. The project was greenlit for primetime and reconstituted with a full time team assigned to develop, document, and productionize the approach.

Powered by a random forest model

The model at the heart of this system leverages a random forest model built in scikit-learn and coded in Python with some PySpark enhancements for scalability. As explainability is a priority, we used open source tools and a model design that enables others to train an identical model and verify our results. That way our stakeholders -- such as federal regulators -- can see and understand the work.

While we initially investigated logistic regression, XGBoost, and recurrent neural networks (RNN) as well, we chose a random forest model because it represented the best balance of model accuracy, performance, and explainability for this use case. As we evolve our model features and design, this may change in future versions.

In practice, we found the random forest model performed favorably across two major metrics:

• Model Training Time - Twice as fast as logistic regression and faster than XGBoost
• Model Performance - Receiver Operating Characteristic (ROC) curves comparable to XGBoost and RNN, and much higher than logistic regression

Features are key to how the model identifies potentially suspicious activity

To build this model, we first create several hundred features --based on customer and transaction attributes -- that could relate to money laundering activity. By applying our features and running the data through a random forest model trained on over a hundred thousand past investigations, we can see if similar transactions were deemed suspicious or not. When presented with a customer's identifiers, the model pulls the relevant customer attributes and transaction data, applies it to the model, and generates a score that represents the likelihood the customer's activity would be deemed suspicious. 

In order to keep things explainable, nimble, and accurate, we regularly audit and prune back the number of features fed into the model. When we have the right combination of features, our model can keep its performance month after month rather than being too heavily customized--overfitted--to the time period used for training.

To help focus on the significant signals in the dataset, we start with a recursive elimination process to reduce the feature dimensionality space, allowing us to determine which are candidate features to prune. Then, through a statistical test, we identify those features that mask the others and eliminate them. We also continuously adjust our model to consider the changing ways customers are doing business. For example, in the era of COVID-19, many customers who previously used cash are now using Peer-to-Peer (P2P) services.

Using alert scores to build risk-based investigative procedures

By producing a score for each alert, our model helps those teams triage the alerts themselves. By assigning alerts into one of three levels, our teams can streamline investigations for low score alerts and prioritize investigations for high score alerts.

For example, one customer selling their home and receiving an unusually large deposit might trigger an alert that the model rates as low-risk.  Another customer depositing cash and then sending funds to a high-risk jurisdiction might trigger an alert that is scored higher and sent for priority review.

With these scores helping to triage the severity of the alerts, the AML subject matter experts  are better able to focus their attention on the riskiest activity. This replaces the traditional first-in, first-out workflow with a risk-based investigative procedure that leads to better, more efficient investigative results.

Our suspicious activity monitoring system doesn’t replace the essential expertise of our AML subject matter experts. Humans still decide what is suspicious, but with the help of machine learning, they can spend less time on false positives and more time on truly suspicious activity.

A collaboration between machine learning and anti-money laundering subject matter experts

But this tool wasn’t just built for our AML teams, it was built with them. This is the product of a true collaboration between Capital One’s Center for Machine Learning (C4ML) and our AML subject matter experts , with team members hailing from five different countries on three different continents. This gave us the domain expertise and international perspective needed to better understand the transactions we investigate. For example, our AML subject matter experts helped our machine learning team understand specific wire transfer patterns common in India and China, the two top sources of foreign students studying in the US and therefore a common source of tuition-related wire transfers. Another member of the team who grew up in Madagascar helped explain why some customers may make foreign ATM withdrawals over the course of weeks, rather than the typical first few days of a vacation, if they are on an extended visit with relatives overseas.

Expertise-driven accuracies like this matter. Our AML professionals are experts and have investigated thousands of cases and filed numerous Suspicious Activity Reports. Naturally, we didn’t want to adopt a redundant or cumbersome tool that would only get in their way. By including them in the process from Day 1 we were able to pinpoint the value that machine learning could provide to their process.

Results of our machine learning-based suspicious activity monitoring system

The vast majority of scores generated by the system result in the expected investigative output. The risk level assigned to individual alerts closely corresponds with a manual ranking process, and the proportion of low, intermediate, and high alerts closely corresponds to historical trends. Overall, the system has proven itself to be less error-prone and more efficient than rules-based systems.

We continuously monitor the results of this system and have taken a multi-pronged approach to detecting potential feedback loops associated with our new procedures. This includes monthly model monitoring, strict quality assurance standards, cross-level testing of alert accuracy, and a monitoring dashboard to review model outputs and operational impacts. This allows us to continually improve the model’s performance, stability, and operational gains.

Financial institutions operate in a heavily regulated environment, and anti-money laundering units have a low tolerance for errors and sometimes a reluctance to be early adopters of new technologies. Many institutions think regulators want to see familiar AML techniques, so they are slow to innovate. By investing heavily in documentation and explainability, Capital One overcame these sticking points to create an industry leading, innovative machine learning solution that enables us to focus more on truly suspicious activity and getting information to law enforcement more quickly.

At Capital One, we’re committed to fighting money laundering and other abuses of the financial system. We think machine learning, when used correctly, represents a powerful new tool in our arsenal against criminal activity.