AI: Could the next generation of OpenAI and Google AI Agents be financial?
OpenAI unveiled GPT-4o, and Google provided an update on its Project Astra.
Hi Fintech Futurists —
Today we highlight the following:
AI: The Next Generation of AI Agents with Google's Project Astra and OpenAI's GPT-4o
LONG TAKE: Temenos launches AI and FIS does Banking-as-a-Service, acquiring innovation
PODCAST: Building Artificial Intelligence we can trust using ZK Proofs, with EZKL CEO Jason Morton
CURATED UPDATES: Machine Models, AI Applications in Finance, Infrastructure & Middleware
To support this writing and access our full archive of newsletters, analyses, and guides to building in the Fintech & DeFi industries, subscribe below. Only $12/month.
AI: The Next Generation of AI Agents with Google's Project Astra and OpenAI's GPT-4o
The largest AI companies in the world have both just launched new capabilities in their flagship products — OpenAI unveiled GPT-4o, and Google provided an update on its Project Astra.
In our previous AI analysis, we highlighted Google’s Gemini 1.5 Pro and its ability to manage 1MM tokens. Now, Google doubled Gemini’s context window to 2MM tokens, enhanced its code generation, logical reasoning, audio and image understanding, allowing users to customize chat agent personas. And Google’s Project Astra, set to launch later this year, promises to be their most powerful AI assistant yet.
Meanwhile, OpenAI has introduced GPT-4o, which integrates text, vision, and audio processing within a single framework. GPT-4o can sense and express emotions, making interactions more natural and intuitive, and the model is also significantly faster and more cost-effective, offering 2x the speed, half the cost, and 5x the message limits compared to GPT-4 Turbo.
As a result of stronger inference capabilities, we will see more advanced AI assistants that develop plans and act on behalf of users by understanding their context and preferences, using third-party services, and interacting with other agents or humans. In the context of financial services, one of the intriguing opportunities is AI agents that manage the full financial situation of their clients, including applying new investment ideas and executing transactions. At some point, their interactions become the core of an AI commerce economy, where agents transact with agents.
Plenty of venture capital has gone into AI agent concepts, but not until recently has real intelligence been available for integration into these bots. Below you can see some of the industry market-mapping from Insight Partners and Aura Ventures, showing the vast ecosystems developing around big-tech AI.
The rise of increasingly autonomous and advanced AI assistants is a profound technological shift. While early technologies like Siri employed narrow AI for specific tasks such as text-to-speech, the new generation of AI assistants leverages general-purpose foundation models. Google DeepMind’s recent research on AI assistants defines it as an artificial agent equipped with a natural language interface designed to plan and execute sequences of actions across multiple domains on behalf of the user. To truly qualify as an agent, the AI must be capable of perceiving and acting upon its environment in a goal-directed and autonomous manner.
Developing an autonomous AI agent is an intricate challenge to say the least, integrating the most modern tech and algorithms into a cohesive system. At its heart lies a Transformer-based neural network for natural language understanding and generation, like a GPT model, augmented with sparse and Mixture-of-Experts (MoE) models to enhance scalability. As discussed in our recent AI analysis, Google’s Gemini 1.5 Pro employs an MoE architecture that efficiently processes large datasets by activating only the necessary neural network segments; an architecture also adopted by other major LLMs, with the notable exception of Meta’s Llama 3.
For decision-making, these agents could use reinforcement learning algorithms such as Proximal Policy Optimization (PPO) and Actor-Critic methods. PPO improves the agent's strategy by maximizing expected rewards — think of this as a utility function. It allows the agent to explore various actions to understand its environment and make optimal decisions. Additionally, reinforcement learning from human feedback (RLHF), where human feedback is used to train a reward model that scores example behaviors, helps the agent learn to align its actions with human preferences. This involves updating the model parameters to encourage behavior that humans find acceptable.
Overall, the goal is for these autonomous AI agents to excel in dynamic learning and adaptability, understand context, make accurate predictions, and interact in a more human-like manner. We haven’t even touched upon algorithms that break down complex tasks into manageable sub-tasks, or memory systems that ensure context retention. If you want to create your own AI agent, check out LangChain, Semantic Kernel, Transformers Agents, and AutoGen.
As these capabilities improve, expect to see more financial AI agents. Already, algorithmic trading rules the stock market and roboadvisors manage billions in assets.
But these are just narrow machines for institutional trading. We can see a future world affecting retail investors — for instance, an AI assistant could develop new investment strategies by identifying opportunities and forming hypotheses. A recent paper titled "Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models" has captured significant attention in both industry and academia.
The study investigated whether ChatGPT and other LLMs could predict stock market returns using news headlines, and found that more sophisticated versions of ChatGPT were indeed able to do so. We won’t delve into the econometric approaches in the paper, but imagine discussing the latest stock news with your AI assistant. You could ask it to gauge stock sentiment based on recent headlines, perhaps using techniques like the consensus game from game theory to refine its analysis by having multiple agents agree on the sentiment. The assistant could then compile a long-short portfolio, connect to your brokerage account, assess trading costs, and execute trades, all in one seamless interaction.
While our example might seem far-fetched, it highlights the potential of AI assistants to leverage extensive third-party resources. Of course, challenges remain, such as memory constraints, attention limitations, and the prevalence of hallucinations. However, with rapid advancements — like Google doubling its context window in such a short space of time — what now seems ridiculous might quickly transition from fantasy to plausible reality.
👑 Related Coverage 👑
Blueprint Deep Dive
Long Take: Temenos launches AI and FIS does Banking-as-a-Service, acquiring innovation (link here)
In this article, we discuss the current state of the fintech industry, highlighting the contrast between the decline of new innovations and the rise of operational excellence. We examine the strategic moves by incumbents like Temenos and FIS, who are leveraging acquisitions and to enhance their offerings with Generative AI and Banking-as-a-Service.
Despite the challenges for new entrants, vertical integration and strong user experiences remain viable paths to success. The ongoing competition raises questions about the future of B2B fintech and the potential dominance of established players.
🎙️ Podcast Conversation: Building Artificial Intelligence we can trust using ZK Proofs, with EZKL CEO Jason Morton (link here)
In this conversation, we chat with Jason Morton - CEO at EZKL, Founder at Zkonduit, and an Associate Professor at Penn State. EZKL aims to simplify the implementation of ZK cryptography, allowing developers to focus on building applications instead of delving into complex mathematical theories.
It achieves this by taking a high-level program description and generating the necessary components for ZK verification. This initiative aims to allow developers to concentrate on creating applications without needing to delve deeply into the complex underlying mathematical theories.
Curated Updates
Here are the rest of the updates hitting our radar.
Machine Models
⭐ KAN: Kolmogorov-Arnold Networks - Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, Max Tegmark
Model Explorer: Graph Visualization For Large Model Development - Google Research
Leveraging Language Models For Prudential Supervision - Bank Underground
Do Machine Learning Models Store Protected Content? - Nathan Reitinger
AI Applications in Finance
⭐ Options-driven Volatility Forecasting - University of Oxford
⭐ Quantitative Tools for Time Series Analysis in Natural Language Processing: A Practitioners Guide - W. Benedikt Schmal
Can AI Replace Stock Analysts? Evidence from Deep Learning Financial Statements - G. Nathan Dong
Detecting Multivariate Market Regimes Via Clustering Algorithms - James Mc Greevy, Aitor Muguruza, Zach Issa, Cristopher Salvi, Jonathan Chan, Zan Zuric
Execs Can Game Sentiment Engines, But Can They Fool LLMs? - Risk.net
Application of Deep Learning for Factor Timing in Asset Management - Prabhu Prasad Panda, Maysam Khodayari Gharanchaei, Xilin Chen, Haoshu Lyu
Conformal Prediction via Regression-as-Classification - Etash Guha, Shlok Natarajan, Thomas Möllenhoff, Mohammad Emtiyaz Khan, Eugene Ndiaye
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention - Tsendsuren Munkhdalai, Manaal Faruqui, Siddharth Gopal
Variational Bayesian Last Layers - James Harrison, John Willes, Jasper Snoek
Theoretical Foundations of Deep Selective State-Space Models - Nicola Muca Cirone, Antonio Orvieto, Benjamin Walker, Cristopher Salvi, Terry Lyons
Infrastructure & Middleware
⭐ CoreWeave Secures $7.5B To Expand AI Infrastructure - PYMNTS
⭐ Microsoft Invests $2.2B to Fuel Malaysia’s Cloud, AI infrastructure - CoinGeek
⭐ Nvidia Acquires AI Infrastructure Company Run:AI - AI Business
Amazon and Microsoft Invest in France’s Cloud and AI Infrastructure - Verdict
Tesla spent $1B on AI infrastructure in Q1, Elon Musk posits using Cars as Distributed Compute - Data Center Dynamics
🚀 Level Up
Sign up to the Premium Fintech Blueprint and in addition to receiving our free newsletters, get access to:
All Long Takes with a deep, comprehensive analysis on Fintech, Web3, and AI topics.
Archive Access to an array of in-depth write-ups covering the hottest fintech and DeFi companies.