Long Take: We made ChatGPTs for asset allocation and stock analysis, it hallucinated financial answers & got some right

Pretty bad at math, but very confident about it

Gm Fintech Architects —

Today we are diving into the following topics:

• Summary: OpenAI has recently launched the ability to create custom GPTs, like persistent prompts narrowed on a particular context. We quickly built an asset allocation bot and a fundamental analysis bot, and tested them on financial advice capabilities. The results are mixed, but illustrative of the potential future where data firms plugged into ChatGPT benefit from new market power. Below is our conclusion, and the transcript of the conversations.

• Topics: generative AI, investment management

If you got value from this article, let us know your thoughts! And to recommend new topics and companies for coverage, leave a note below.

---

Long Take

Preface and Conclusion

OpenAI has recently re-packaged their “App Store” concept into the metaphor of custom “GPTs”. We spent some time diving into creating a few custom GPTs of our own, and figuring out what we can learn about the impact on fintech. For our prior take on emerging app stores — why they are needed to turbo-charge platform growth, and who really wins — see here.

Source

In essence, each GPT is a customized access point to the larger language model, with tuning and integrations for particular functionality. You could probably fit all this into a prompt in a general chat window by giving ChatGPT context as to what it needs to do. But the new experience is more intuitive, and it feels like you are interacting with a bunch of robot individuals, rather than some horrific Cthulu machine hivemind wearing theater masks. To each their own.

Source

It feels a bit silly to parse robot text and try and prognosticate. But at the same time, once you play with the machine, some things become painfully obvious.

First off, human knowledge is a public good, not a private one. It is deeply uncomfortable to think that OpenAI will have a monopoly on machine intelligence in the way that Google has a monopoly on access to websites. To that end, open source and Web3 are *desperately needed* as a counterweight, before the new user behaviors set in. Below is what happens when a winning digital user experience begins to find traction, and turns from a proper noun into a verb.

Today, the cost of this for the Internet is advertising in your search results. Tomorrow, the cost of such centralization is inadvertent custodian control of super intelligence.

Another conclusion from playing with the GPT is that, oh boy, will the SEC, FINRA, CFPB, FCA, OCC and the rest of the government alphabet soup try to regulate how artificial intelligence gives investment advice. It is already very clear that LLMs can give real, actionable advice based on solid theory and mathematics. Without even being plugged into very much personal data, the results we got on the investment management side were helpful — even relative to the average Registered Investment Advisor.

While mistakes from such advice are “dangerous”, it is a Promethean misunderstanding to try and put it back in the box. We want to give fire to the people. It must be done. Information must be free. Copyrights must be broken. Napster must rise. BitTorrent must seed. Bitcoin must exchange. This is what technology does, and we must fan its appetite.

See below the current rates of financial literacy globally.

Source

This is a solvable problem with the GPTs, and we have an obligation to solve it in order to improve outcomes for billions of people across the world. ChatGPT makes it trivial to deliver key financial concepts to regular users.

Right now, however, the experience breaks down often, and needs an expert to manage it back on track. Live financial data and historical analytics are missing, even though the GPT has a general set of theories that it can apply. Math goes off the rails, when left to the devices of the LLM, instead of Python based calculation. Below you can see model hallucination rates, which might be comparable to the error rate of an average person, but far above those acceptable for a financial professional.

Source

The robot also lacks the ability to implement its advice — for now. Open banking, integration into brokerages and embedded finance, and of course blockchain-based financial infrastructure will transition these AIs from read-only to execution. As they should.

Lastly, if we were BlackRock or Empower Retirement or Betterment or Coinbase or Bank of America, we would be figuring out how to use ChatGPT as the conversational personalization layer for their underlying trading and investment engines. The homepage featuring performance wiggles and pie charts would be gone. Instead, we would center much of the user experience around inquiry and informed guidance tied to user goals and current situations. While the GPTs didn’t feel particularly customized, if they had our personal investment information, investment philosophy, and up-to-date market context, we think they would make suitable decisions. The same can likely be said for budgeting assistants, or lending and underwriting experiences.

We start with our takeaways to make room for the direct conversations below. Those are comprised of excerpted transcripts and highlighted comments from us, with reflections on what the GPTs said and how useful it would be in the real world.

Enjoy!

Asset Allocator

The first bot we made was a wealth management financial advisor. The full conversation can be accessed here. Below we use code blocks for questions / notes and plain text for answers.

Question: Explain a quantitative method for passive investment … Can you be a bit more concise and use clearer language that can be understood more quickly?

Certainly! A common quantitative method for passive investing is index tracking:

1. Choose an Index: Select a market index like the S&P 500.

2. Replication Approach: Either buy all stocks in the index (full replication) or a representative sample (sampling).

3. Risk-Return Balance: Use mathematical models to balance risk and expected returns, similar to the principles of Modern Portfolio Theory.

4. Regular Rebalancing: Adjust the portfolio periodically to stay aligned with the index.

5. Cost Efficiency: Minimize transaction costs and management fees.

This method is suitable for institutions seeking market exposure with low costs and minimal active management. It involves technology for data analysis and trade execution, aiming to match the performance of the chosen index.

Question: How does this strategy fit into an endowment style asset allocation in 2023?

In 2023, an endowment-style asset allocation typically involves a diverse mix of traditional and alternative investments, aiming for long-term growth and income while managing risk. Index tracking, as a form of passive investment, fits into this strategy in several ways:

1. Core Portfolio Component: Passive index tracking is often used for the core equity and fixed income components of the portfolio, providing broad market exposure.

2. Cost Efficiency: Endowments benefit from the lower costs associated with passive strategies, freeing up resources for other investments.

3. Risk Management: Passive investments in broad indices can provide stability and reduce the portfolio's overall volatility.

4. Long-Term Perspective: Index tracking aligns with the long-term investment horizon of endowments, as it is less about timing the market and more about market participation.

5. Complement to Active Strategies: Passive investments can be a counterbalance to more actively managed and alternative investments in the portfolio, such as private equity, hedge funds, and real assets.

In 2023, endowments are increasingly focusing on sustainable and socially responsible investments. Passive strategies can include ESG (Environmental, Social, Governance) index funds, aligning with these values while maintaining a diversified, long-term approach.

Note: So far, the answer is pretty generic. The model is finding the right things to say, but nothing is really implementable. So the next step is to dive into some detail about an individual situation. Here's our prompt, which we try to derisk with disclosure.