[UPDATE 05.05.2023 – 12:21 CET] According to information obtained by Bloomberg, the text is indeed from Google. It was published on an internal system at Google in early April and since then shared thousands of times among Googlers.
[04.05.2023 – 23:21 CET] A supposedly leaked internal Google document, which was shared by an anonymous individual on a public Discord server, suggests that the search giant is grappling with the competition posed by open-source artificial intelligence (AI). According to the document shared by Semianalysis, a website that provides deep analysis of the semiconductor industry, both Google and OpenAI are at risk of being outperformed by their open-source counterparts.
The leaked document, which according to Semianalysis has been verified as authentic, reads, “We have no moat, and neither does OpenAI”. It goes on to describe the rapid advancements made by open-source AI projects, which have outpaced the progress of Google and OpenAI in recent months. The document lists several impressive achievements by open-source AI, such as running foundation models on a Pixel 6 at 5 tokens per second, fine-tuning personalized AI on a laptop in an evening, and creating multimodal models in record time.
“Google and OpenAI Not Positioned to Win the AI Arms Race”
The author of the document argues that Google and OpenAI are not positioned to win the AI “arms race” and that the companies should learn from and collaborate with open-source projects. They suggest that people will not pay for restricted models when free, unrestricted alternatives of comparable quality are available. Furthermore, the document emphasizes the need for rapid iteration, stating that smaller models with fewer parameters can be developed more quickly and are ultimately more effective.
The supposed leak also highlights the impact of open-source AI on the wider community, with the release of Meta's LLaMA model sparking a surge of innovation and collaboration among researchers and developers. This phenomenon has been dubbed the “Stable Diffusion moment” for large language models (LLMs), with the rapid progress of open-source AI projects outpacing large players in the industry.
“Google Should Focus On Open-Source Community”
The document's author suggests that directly competing with open-source AI is a losing proposition and that Google should instead establish itself as a leader in the open-source community. They argue that owning the ecosystem like Google has done with Chrome and Android, would provide significant value and control over the direction of AI innovation. To achieve this, Google may need to publish the weights of small universal language model (ULM) variants and cooperate with the broader open-source community.
The leaked document's implications could significantly impact Google and OpenAI's future business strategies. Both companies may need to reevaluate their approach to AI research and development in light of the rapid advancements made by open-source projects. Additionally, Google and OpenAI could face increasing pressure to become more transparent and collaborative with the wider AI community.
As for OpenAI, the document suggests they are also vulnerable to the same challenges posed by open-source AI, with their ability to maintain an edge. The leaked document concludes by stating that open-source alternatives will eventually eclipse both Google and OpenAI unless they adapt their strategies and engage more openly with the community.
The full text of the leaked document (below)
Important: As Semianalysis writes, “the document is only the opinion of a Google employee, not the entire firm”. As them, we do not agree or disagree with what is written below and cannot guarantee its authenticity from our side.
We Have No Moat – And neither does OpenAI
But the uncomfortable truth is, we aren't positioned to win this arms race and neither is OpenAI. While we've been squabbling, a third faction has been quietly eating our lunch.
I'm talking, of course, about open source. Plainly put, they are lapping us. Things we consider “major open problems” are solved and in people's hands today. Just to name a few:
LLMs on a Phone: People are running foundation models on a Pixel 6 at 5 tokens / sec.
Scalable Personal AI: You can finetune a personalized AI on your laptop in an evening.
Responsible Release: This one isn't “solved” so much as “obviated”. There are entire websites full of art models with no restrictions whatsoever, and text is not far behind.
Multimodality: The current multimodal ScienceQA SOTA was trained in an hour.
While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months. This has profound implications for us:
We have no secret sauce. Our best hope is to learn from and collaborate with what others are doing outside Google. We should prioritize enabling 3P integrations.
People will not pay for a restricted model when free, unrestricted alternatives are comparable in quality. We should consider where our value add really is.
Giant models are slowing us down. In the long run, the best models are the ones
A tremendous outpouring of innovation followed, with just days between major developments (see The Timeline for the full breakdown). Here we are, barely a month later, and there are variants with instruction tuning, quantization, quality improvements, human evals, multimodality, RLHF, etc. etc. many of which build on each other.
Most importantly, they have solved the scaling problem to the extent that anyone can tinker. Many of the new ideas are from ordinary people. The barrier to entry for training and experimentation has dropped from the total output of a major research organization to one person, an evening, and a beefy laptop.
Why We Could Have Seen It Coming
In both cases, low-cost public involvement was enabled by a vastly cheaper mechanism for fine tuning called low rank adaptation, or LoRA, combined with a significant breakthrough in scale (latent diffusion for image synthesis, Chinchilla for LLMs). In both cases, access to a sufficiently high-quality model kicked off a flurry of ideas and iteration from individuals and institutions around the world. In both cases, this quickly outpaced the large players.
These contributions were pivotal in the image generation space, setting Stable Diffusion on a different path from Dall-E. Having an open model led to product integrations, marketplaces, user interfaces, and innovations that didn't happen for Dall-E.
The effect was palpable: rapid domination in terms of cultural impact vs the OpenAI solution, which became increasingly irrelevant. Whether the same thing will happen for LLMs remains to be seen, but the broad structural elements are the same.
What We Missed
LoRA is an incredibly powerful technique we should probably be paying more attention to
Retraining models from scratch is the hard path
This means that as new and better datasets and tasks become available, the model can be cheaply kept up to date, without ever having to pay the cost of a full run.
By contrast, training giant models from scratch not only throws away the pretraining, but also any iterative improvements that have been made on top. In the open source world, it doesn't take long before these improvements dominate, making a full retrain extremely costly.
We should be thoughtful about whether each new application or idea really needs a whole new model. If we really do have major architectural improvements that preclude directly reusing model weights, then we should invest in more aggressive forms of distillation that allow us to retain as much of the previous generation's capabilities as possible.
Large models aren't more capable in the long run if we can iterate faster on small models
Data quality scales better than data size
Directly Competing With Open Source Is a Losing Proposition
And we should not expect to be able to catch up. The modern internet runs on open source for a reason. Open source has some significant advantages that we cannot replicate.
We need them more than they need us
But holding on to a competitive advantage in technology becomes even harder now that cutting edge research in LLMs is affordable. Research institutions all over the world are building on each other's work, exploring the solution space in a breadth-first way that far outstrips our own capacity. We can try to hold tightly to our secrets while outside innovation dilutes their value, or we can try to learn from each other.
Individuals are not constrained by licenses to the same degree as corporations
Being your own customer means you understand the use case
Owning the Ecosystem: Letting Open Source Work for Us
The more tightly we control our models, the more attractive we make open alternatives.
Google and OpenAI have both gravitated defensively toward release patterns that allow them to retain tight control over how their models are used. But this control is a fiction. Anyone seeking to use LLMs for unsanctioned purposes can simply take their pick of the freely available models.
Google should establish itself a leader in the open source community, taking the lead by cooperating with, rather than ignoring, the broader conversation. This probably means taking some uncomfortable steps, like publishing the model weights for small ULM variants. This necessarily means relinquishing some control over our models. But this compromise is inevitable. We cannot hope to both drive innovation and control it.
Epilogue: What about OpenAI?
And in the end, OpenAI doesn't matter. They are making the same mistakes we are in their posture relative to open source, and their ability to maintain an edge is necessarily in question. Open source alternatives can and will eventually eclipse them unless they change their stance. In this respect, at least, we can make the first move.
Feb 24, 2023 – LLaMA is Launched
March 3, 2023 – The Inevitable Happens
March 12, 2023 – Language models on a Toaster
March 13, 2023 – Fine Tuning on a Laptop
Suddenly, anyone could fine-tune the model to do anything, kicking off a race to the bottom on low-budget fine-tuning projects. Papers proudly describe their total spend of a few hundred dollars. What's more, the low rank updates can be distributed easily and separately from the original weights, making them independent of the original license from Meta. Anyone can share and apply them.
March 18, 2023 – Now It's Fast
March 19, 2023 – A 13B model achieves “parity” with Bard
March 25, 2023 – Choose Your Own Model
March 28, 2023 – Open Source GPT-3
March 28, 2023 – Multimodal Training in One Hour
April 3, 2023 – Real Humans Can't Tell the Difference Between a 13B Open Model and ChatGPT
They take the crucial step of measuring real human preferences between their model and ChatGPT. While ChatGPT still holds a slight edge, more than 50% of the time users either prefer Koala or have no preference. Training Cost: $100.