Do you need $ 5 Trillion for AI??
Over the weekend, a news report said (The Wall Street Journal - paywalled),
The OpenAI chief executive officer is in talks with investors including the United Arab Emirates government to raise funds for a wildly ambitious tech initiative that would boost the world’s chip-building capacity, expand its ability to power AI, among other things, and cost several trillion dollars, according to people familiar with the matter. The project could require raising as much as $5 trillion to $7 trillion(!!!)
$ 5 to $ 7 trillion??
The total GDP of the world is about $ 100 trillion.
I don’t know what it means in real terms. There are only two countries in the world with a higher GDP in 2022 above $ 5 trillion. (The US and China). What is the time period for this investment - 5 years, 10 years, 50 years?
Sam Altman obviously is a very savvy and smart CEO.
(About 5 years ago, I attended the live taping of his interview for the Conversations with Tyler podcast with Tyler Cowen in San Francisco, and he was thoughtful and brilliant.)
If this report is true, how much value does OpenAI think they can create and how much of it can they capture?
If OpenAI thinks an investment of this magnitude is needed, how should different businesses, including agrifood businesses think about their investments in AI?
Cost to build models
Companies like Google, and OpenAI backed by Microsoft have spent millions and billions of dollars to build general purpose models like Gemini and ChatGPT. These models are extremely powerful to answer general questions about almost any topic, but struggle if you have very specific questions about a particular specialized topic like agriculture or food.
The next level of specificity can come from domain specific large models called foundation models. They are trained using a massive amount of unlabeled data for a specific domain.
This diverse data could include text, code, images, and more. Unlike traditional AI models trained for specific tasks, foundation models learn general patterns and relationships within the data. This allows them to be adapted and fine-tuned to perform a wide range of tasks, making them incredibly versatile.
Foundation models make building application specific models easier as you are not starting from scratch. For example, if you want to build a computer vision based model which can identify different species of weed, you could start from a foundation model, which has been trained on a large number of images of plants. This foundation model serves as a much more efficient starting point to build the specific weed identification model, compared to starting from scratch.
Here are some key characteristics of foundation models.
Here are some key characteristics of foundation models:
- Broad data: Trained on massive datasets, not just specific tasks.
- Self-supervision: They learn from the data itself without needing labeled examples.
- Adaptability: Can be fine-tuned to perform specific tasks in diverse areas like language translation, text generation, image analysis, and code completion.
- Large scale: Typically have billions or even trillions of parameters, making them complex and powerful.
Training foundation models is also expensive. It makes sense to develop a foundation model for a particular domain, if you think you can create many application specific models based on the foundation model for that domain.
Here are some examples of domain specific foundation models and their areas of expertise.
Healthcare
- MedText: Trained on massive amounts of medical texts and electronic health records to perform tasks like patient diagnosis prediction, symptom analysis, and medical report generation.
- MedBART: Focused on biomedical language understanding, handling tasks like drug-drug interaction prediction, protein function analysis, and scientific summarization.
Finance
- MesaBERT: Specifically trained on financial text data for tasks like financial sentiment analysis, credit risk assessment, and fraud detection.
- BART Finance: Handles financial question answering and document summarization relevant to the financial domain.
It's important to note that the line between general-purpose and domain-specific foundation models can be blurry. While the examples above have a clear domain focus, many general-purpose models like Gemini and ChatGPT can be fine-tuned for specific tasks within a domain, blurring the distinction.
Overall, domain-specific foundation models offer advantages in specialized areas by being pre-trained on relevant data and understanding the nuances of the domain.
I have not seen any major foundation models emerge in the agrifood sector. (unless I am not aware of it. If you know of a foundation model in agriculture or food, I would love to know more). It will be quite expensive to build foundation models for agriculture. One way to recoup your investment in building a foundation model for agriculture is if it enables additional use case specific models to create and capture value.
I don’t anticipate any organization building an agriculture specific foundation model anytime soon, as it is not clear what it would contain.
Could it be a model trained on images of a very diverse and large data set of plant images? This would be a specific foundation model for plants. Can researchers and companies figure out what additional value they can layer on top of the image based foundation model for specific use cases?
I am skeptical we will see the rise of agriculture specific foundation models anytime soon, though we will see a proliferation of task specific AI models over the coming years.
Business models for LLMs
The tech giants investing heavily in developing large language models - Google, Microsoft, Meta, Amazon, etc. - have clear commercial interests. They can integrate the models into existing products like search engines and digital assistants to improve performance. This provides value to users and drives more traffic and ad revenue.
As you can see, given these large tech companies already have access to distribution, any new conversational AI or any other AI capability will be a feature in an existing product. It is much easier to add your new capability as a feature to an existing product with wide distribution.
The tech firms can also monetize the models more directly by offering API access and paid services based on the AI.
Agrifood businesses can take these large language and large multi-modal modes, and use their internal proprietary data sets to build application specific models.
For example, an agribusiness can take a model like OpenAI and combine it with their proprietary data set to create a domain specific model, using proprietary data.
Agrifood companies can build new conversational apps and creative tools that were not possible before. Early enterprise adopters gain competitive advantage by using LLMs for content generation, process automation, and customer service augmentation.
Similar to large tech firms, agrifood companies with an existing customer base, and access to distribution should think about new AI capabilities as features in their existing offering, instead of trying to brand new AI based products (depending on where you are in your AI journey!)
Open Source or Closed?
I often get the question whether an agribusiness wanting to build an LLM type feature should leverage the close source model like ChatGPT or use an open source LLM like Llama 2 from Meta.
First and foremost, an agrifood business should identify practical use cases where LLMs offer clear value, rather than being dazzled or worse, frazzled by the technology.
Thoughtfully integrating LLMs into existing workflows improves outcomes without overestimating their autonomy.
For any agrifood business, as usual there are tradeoffs in using a closed model like ChatGPT or an open source model like Llama 2.
If you go with a closed model like OpenAI’s ChatGPT, you will need fewer technology skill sets as OpenAI’s APIs are and will be much more robust in the future. On the flip side, you will have limited flexibility and maybe some irrational fear of data security.
On the other hand, an open source model like Llama2, will require a larger investment in software resources to help customize the experience, but it will give you the flexibility to create your own unique experience.
Are you a laggard or an over shooter?
As I said last week, being thoughtful and deliberate about AI is critical.
You don’t want to be a laggard. Laggard organizations risk falling behind if they dismiss LLMs as overhyped or avoid AI altogether.
Take a critical and close look at your organization, and see if your organization shows any symptoms of these situations.
Laggard symptoms
- Leadership lacks awareness of large language models and potential applications
- No funding for exploring LLM capabilities and use cases
- Refusal to consider integrating LLMs due to bias concerns alone
- No process to monitor LLM landscape and evolve usage as technology matures
Thoughtfully experimenting with emerging technologies like LLMs, even in limited scopes, keeps organizations competitive. Laggards who wait too long risk sudden obsolescence.
On the other hand, you don’t want to overshoot or you will overpromise and not be able to dollar.
Overshoot symptoms
- Assuming LLMs are near human-level capability
- Launching initiatives without proper testing and controls
- Planning wholesale replacement of human roles versus augmentation
- Rushing LLM adoption simply because competitors are experimenting
LLMs have limitations today and sensible adoption considers ethics, & amplifies human capabilities.
Organizations should look for a strong ROI for this use case, think about how you can tweak or optimize your existing workflows, how you can use your domain expertise to create feedback loops, and get the most benefit out of your investment.
So engage with these new capabilities, and don’t turn your back to them, whether you are a small startup or an established agribusiness.