Capital One’s latest AI research at ICML 2025

Capital One is excited to sponsor the 42nd International Conference on Machine Learning (ICML 2025), taking place July 13-19, 2025 at the Vancouver Convention Centre in Vancouver, Canada. ICML, along with NeurIPS and ICLR, draws thousands of researchers each year and is widely regarded as one of the field’s flagship events. Following a highly competitive ICML 2024, attendees can look forward to an equally robust and engaging program in 2025.

Over the course of the week, ICML 2025 will showcase peer-reviewed papers, interactive tutorials and topic-specific workshops that highlight the latest breakthroughs in machine learning. Our team is dedicated to pushing the boundaries of ML, with a focus on cutting-edge model development, robustness and real-world application. We look forward to sharing our findings and connecting with you at our ICML 2025 workshops.

Research published by Capital One's Enterprise AI teams explores the latest advancements in tabular prediction through novel meta-learning approaches:

This paper introduces an adversarially pre-trained transformer (APT), a new approach for zero-shot meta-learning on tabular prediction tasks. Unlike many models, APT doesn't require real-world datasets for pre-training; instead, it's pre-trained using adversarial synthetic data agents that continuously shift their underlying data distribution, deliberately challenging the model with diverse synthetic datasets. A key innovation is the mixture block model architecture, which enables APT to handle classification tasks with an arbitrary number of classes, overcoming a significant limitation of prior tabular zero-shot learning algorithms like TabPFN. Experiments demonstrate that APT achieves state-of-the-art performance on small tabular classification tasks, without restrictions on dataset characteristics like the number of classes or missing values, all while maintaining an average runtime under one second. For both classification and regression benchmark datasets, adversarial pre-training is shown to enhance TabPFN's performance, with analysis revealing that the adversarial synthetic data agents generate a more diverse collection of data than ordinary random generators, and the mixture block neural design improves generalizability while significantly accelerating pre-training.

• Capital One authors: Yulun Wu, Doron Bergman 

• Details: Thursday, July 17, 2025, 4:30 p.m. PDT

Capital One frequently engages with the broader AI research community through academic partnerships. We're excited to feature two papers accepted at ICML 2025, co-authored through our academic collaborations:

This paper, “Dynamic guardian models: Real-time content moderation with user-defined policies,” proposes specialized classifiers designed for real-time content moderation. Addressing the safety and reliability issues often seen in large language models (LLMs), these models evaluate text based on predefined trustworthiness objectives and assess compliance with user-defined rules across diverse AI-mediated communication contexts. A participatory pipeline produces synthetic datasets for training and evaluation, incorporating diverse perspectives for appropriate AI behavior. 

The methodology uses group relative policy optimization to improve the model's ability to reason through rule violations and articulate justifications. Experiments show these dynamic guardian models match static models in harm detection and identify violations nearly as well as frontier reasoning models in a fraction of the time, ensuring alignment with stakeholder expectations and regulatory standards while providing adaptability.

• Capital One authors: Melissa Kazemi Rad, Bayan Bruss

• Details: This will be presented twice. 
  • Friday, July 18th, 8:30 a.m. PDT: MoFA (Models of Human Feedback for AI Alignment)
  • Friday, July 18th, 8:30 a.m. PDT: The next wave of On-Device Learning for Foundational Models (TTODLer-FM) Workshops

The paper, “Llm-SRBench: A new benchmark for scientific equation discovery with large language models,” introduces a comprehensive benchmark designed to rigorously evaluate the scientific equation discovery capabilities of LLMs. Existing benchmarks often rely on common equations susceptible to LLM memorization, leading to inflated performance metrics that don't truly reflect discovery. 

To counter this, LLM-SRBench features 239 challenging problems across four scientific domains, preventing trivial memorization. It includes two main categories: LSR-Transform, which uses transformed physical models to test reasoning beyond memorized forms, and LSR-Synth, which offers synthetic, data-driven problems requiring genuine discovery. Through extensive evaluation of state-of-the-art open and closed LLMs, the best-performing system achieved only 31.5% symbolic accuracy. These findings highlight the significant challenges in scientific equation discovery, positioning LLM-SRBench as a valuable resource for future research.

• Capital One authors: Kazem Meidani

• Details: Thursday, July 17, 2025, 10:15 a.m. PDT

The position paper, “Out-of-distribution detection methods answer the wrong question,” critically re-examines a common family of OOD detection approaches that rely on predictive uncertainty or supervised model features. The paper argues that these methods are fundamentally asking the wrong questions for OOD detection. It highlights how uncertainty-based methods incorrectly conflate high uncertainty with being OOD, and how feature-based methods mistakenly equate distant feature-space locations with being OOD. The paper details how these issues lead to inherent errors in OOD detection and pinpoints common scenarios where such methods prove ineffective.

• Capital One author: Bayan Bruss

• Details: Tuesday, July 15, 11 a.m. - 1:30 p.m. PDT

Join us at the ICML Expo where Capital One will be leading two insightful workshops designed to explore critical advancements and challenges in AI and machine learning.

This dynamic 90-minute session features a series of engaging lightning talks showcasing the forefront of AI and Machine Learning within the financial services industry. Discover novel in-house innovations, including Grembe, a Capital One-developed system leveraging graph embeddings on transactional data for enhanced financial understanding across fraud detection and customer behavior modeling. Explore MACAW, our multi-agentic conversational AI workflow, an Advanced Quantitative Method that tackles complex financial reasoning. Learn about our research on Fortifying Conversational AI through intelligent input guardrails that enhance the security and reliability of LLM-driven interactions. 

A key highlight will be spotlights on emerging research and innovative concepts from our academic partners as they address critical challenges in AI for finance. This session will provide ICML participants with a concise yet comprehensive overview of impactful applied AI research in finance, fostering conversation and dialogue about notable advances, ongoing research and critical challenges shaping the future of AI in this domain.

• Details: Monday, July 14, 4:30-6 p.m. PDT

• Location: West Ballroom A 

The ability of LLMs to accurately estimate uncertainty is a fundamental bottleneck hindering their safe and effective deployment in high-stakes, industrial-scale applications. Gaps between model confidence and actual correctness can pose an immediate and escalating risk. This workshop convenes leading industry experts and academic researchers to confront these urgent challenges. We will define the state-of-the-art, establish rigorous evaluation standards and consider best practices for ensuring reliable and well-managed AI.

Key topics of focus include: calibration (aligning LLM confidence with true accuracy), confidence-aware generation (enabling LLMs to express uncertainty during content creation), out-of-distribution detection (equipping LLMs to flag inputs outside their training data), uncertainty communication (effective techniques for conveying LLM uncertainty to end-users) and benchmarking (metrics to measure model uncertainty quantification).

• Details: Monday, July 14, 4:30-6 p.m. PDT

• Location: West Ballroom B

We invite you to visit our booth at ICML 2025 to learn more about our research, career opportunities and how we are leveraging AI to transform the financial industry. Our team will be available to discuss our research papers, answer your questions and share insights into our work. 

• Visit our booth: 211

• Explore our research: Dive deep into our latest advancements in AI and machine learning.

• Discover career opportunities: Learn about exciting applied research career paths at Capital One for researchers and engineers passionate about AI and join our world-class team.

• Learn about our student and grad internships: Put your knowledge and skills to work in our 10-week to two year graduate programs innovating new products and creatively solving the problems that impact our customers and our business.

• Engage with our team: Meet our researchers and AI experts, explore how we’re shaping financial services with patented AI and discuss what’s next for AI in finance.