This Week In AI - 23.03.11
This is our first update weekly on what’s going on in the world of AI.
AI Research News
Google released PALM-E this week, a multi-modal (vision, language, robot states) large model directed at utility for robotics:
Today we introduce PaLM-E, a new generalist robotics model that overcomes these issues by transferring knowledge from varied visual and language domains to a robotics system …rather than relying on only textual input, with PaLM-E we train the language model to directly ingest raw streams of robot sensor data.
Earlier this week we talked about the recent paper “Enabling Conversational Interaction with Mobile UI using Large Language Models” in the post “Adapting Large Language Models to UI Interactions”.
Running the 64B LLaMA LLM on a notebook computer! This ability to get super-powerful LLMs efficient enough to run on consumer-level platforms for inference is a tipping point. “Large language models are having their Stable Diffusion moment”
AI Tech and Business News
On March 1st, OpenAI opened up the chatGPT and whisper APIs:
We’re releasing ChatGPT in our API. This allows anyone to build AI applications powered by ChatGPT. After many optimizations, we’re pricing it at a 10x lower price than the previous GPT-3.5 model, to allow for many more use cases than before.
ChatGPT-4 release is imminent. Maybe next week. Stay tuned!
First Silvergate, then Silicon Valley Bank shut their doors this week. Crypto bros and VC-backed startups hardest hit. “Regulators close Silicon Valley Bank in largest failure since financial crisis.”
Opinions and Articles
Articles and quotes worth sharing:
Omar Shaya on LLMs + RPA, and some of the startups in the text-to-action space:
LLMs are unleashing a new human-machine interaction model and will redefine the RPA category. A new human-machine interaction model at the intersection of large language models (LLMs) and robotic process automation (RPA).
Itamar Friedman from Codium.AI: “6 important AI near-future breakthroughs, or why the AI hype peak is likely to be ahead of us”
6 technologies are in the works and will very likely mature within the next 3 years, further increasing the (justifiable?) hype around Generative AI and large-language-models (LLMs) specifically. … Soon enough AI-empowered products will be accurate, informative, up-to-date, and efficient.
They point out the six ways that current limits with LLMs are being overcome: Better information grounding and referencing; Efficiently connecting LLMs to tools; LLMs context and input enlargement; maturing the LLM computing ecosystem; LLM fine-tuning and alignment; LLMs reasoning. They expect most of the perceived roadblocks with LLMs to be resolved soon, in 3 years or less.
A bonus is this image capturing the hype-fear-letdown-hype cycle around chatGPT:
Not worth sharing: The NY Times published an Op-Ed by Noam Chomsky on chatGPT that is just factually wrong. I will try not to waste time debunking people with outdated opinions about AI, but report on the actual progress in AI and let that speak for itself.
A Look Back …
Four short years and one pandemic ago, in February 2019, OpenAI shared the news on their GPT-2 model:
We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training.
The rest is LLM and AI History.
Other News …
An asteroid the size of 112 camels barely misses earth. Sweet Meteor of Death averted. I have no idea why camels is the correct measure for asteroids, but that’s how the news reported it.
That’s a wrap! Tell us if you like this, and tell us what you like and don’t like or want more or less of. I hope to make these weekly updates better over time, as with all the content in this newsletter.