This week in AI 23.04.01
More model releases and making things better-than-GPT-4
AI Tech and Product Releases
Sourcegraph open-sources Cody, an AI software coding assistant that is embedded as a plug-in for VSCode. Currently Anthropic’s Claude is the LLM under the hood, but will connect to any LLM in the future. It is wild that we are already at the point of open-sourcing such a wildly powerful application. “Read, write, and understand code 10x faster with AI Cody answers code questions and writes code for you by reading your entire codebase and the code graph.”
Google Partners with AI Startup Replit to Take on Microsoft’s GitHub. Replit’s software development tool will use Google artificial intelligence language models to better write code.
Here come the open source mini-LLMs: Cerebras, AI supercomputer startup, has announced seven open source Large Language Models, dubbed Cerebras-GPT and ranging in size up to 13 billion parameters. It’s open and available on Hugging Face.
I’ve been tracking multiple home-brew efforts for mini-LLMs you can host on your home PC. One effort in that vein is GPT4all, that took the 7B LLaMA model and trained it on 800k GPT-3.5-Turbo instructions to make a chatbot then quantized it so it could be downloaded to your PC. Another is this Alpaca-based bot.
And Databricks releases Dolly, an open source 6B parameter language model.
“Dolly works by taking an existing open source 6 billion parameter model from EleutherAI and modifying it ever so slightly to elicit instruction following capabilities such as brainstorming and text generation not present in the original model, using data from Alpaca,”
LAION Anounces Open Flamingo, an open-source reproduction of DeepMind's multimodal (image plus text) Flamingo model, that enables training and evaluation of large multimodal models (LMMs). Now they have just a 9B model, but “ultimately, we aim to match the power and versatility of GPT-4 in handling visual and text input.”
Genmo announces “Genmo Chat, a creative copilot that uses GPT-4 and a large suite of generative AI tools to create and then edit any video or image you ask for.”
Runway’s recently announced Gen2 - that provides short video straight from text - has a promo video that is the only way to explain this amazing new technology.
AI Research News
“ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge”. ChatDoctor is a next-generation AI doctor model that is based on the LLaMA model.
To address this issue, we collected more than 700 diseases and their corresponding symptoms, recommended medications, and required medical tests, and then generated 5K doctor-patient conversations. By fine-tuning models of doctor-patient conversations, these models emerge with great potential to understand patients' needs, provide informed advice, and offer valuable assistance in a variety of medical-related fields.
Chaining tasks and using iteration is a way to improve results, and so we are seeing a flood of papers extending LLMs and enhancing them with iteration and chaining. One chaining approach is ReAct.
One new paper is “DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents” DERA stands for “dialog enabled resolving agents”, and they use multiple LLM agents in the dialog to get better results: a Researcher, who processes information and identifies crucial problem components, and a Decider, who makes final output judgments. This approach yielded improvements over GPT-4 alone on MedQA questions and human evaluations.
In a similar way, “Reflexion: an autonomous agent with dynamic memory and self-reflection” uses a pattern of reflecting and iterating on answers to yield improved reasoning skills. Reflexion is “an approach that endows an agent with dynamic memory and self-reflection capabilities to enhance its existing reasoning trace and task-specific action choice abilities.”
“Study proves artificial intelligence can respond to complex survey questions like a real human.” In the paper “Out of One, Many” BYU researchers created GPT-3 personas and found that AI personas had voting preferences that matched the human preferences of those personality and demographic types. They found that the GPT-3 model conditioned on human subtypes was “nuanced, multifaceted, and reflects the complex interplay between ideas, attitudes, and sociocultural context that characterize human attitudes”, and it could therefore emulate human samples for survey purposes.
AI Business News
AI as cofounder may become a thing:
A business in China already made an AI a CEO. We will see what this really means as the working model of how human and AIs interact, but consider it just one of many surprising use cases that will crop up.
AI Opinions and Articles
The Call to Pause AI Research, which I already wrote about earlier, is reverberating and gathering support. They claim to have collected over 50,000 signatures.
I’ve seen deepfake pictures and videos related to 2024 already, so, “ChatGPT and LLMs could provide troll farms a way to increase levels of misinformation” can be rated as “plausible” or even likely.
A Look Back …
"We may hope that machines will eventually compete with men in all purely intellectual fields." ~ Alan Turing, 1950