AI Week in Review 23.07.22
Chapyter AI for Jupyter, LLama2 released, Microsoft prices coPilot, FlashAttention gets even better
Top AI Tools
Chapyter adds a GPT-4 extension to the data scientist’s workbench, JupyterLabs notebooks, enabling a natural language interface input to generate and automatically execute Python code. By adding AI code generation to the Jupyter notebook code interpreter engine, this powerful integration provides similar capabilities to the chatGPT code interpreter plug-in, but turns the integration on its head. As an open source project, Chapyter enables integration of any available LLM to power the automated code generation.
AI Tech and Product Releases
Meta released Llama2, an open-source LLM successor to the original Llama. We dug into the details on this new model earlier this week in “Llama 2: A New SOTA in open LLMs.”
Pre-release rumors of an Apple-GPT bubble up: Despite being conspicuously quiet in the generative AI race, Apple is secretly developing their own LLM, called “Ajax.” Apple has banned its employees from using ChatGPT and is using it internally. Speculation is that it may become an LLM-powered Apple consumer product, either a new iOS assistant or perhaps a new version of Siri on the horizon, but there is nothing official from Apple.
Petals is the BitTorrent of LLM inference. This open source technology is able to break up the fine-tuning and inference of a large LLM into smaller chunks and make it a distributed process on multiple consumer devices. It was showcased this week at the ACL 2023 Demonstrations track. This sounds incredible, but you could do inference on any sized LLM on your home PC with enough distributed GPU resources, so long as you have an open source AI model to run (advantage goes to Llama2). BYO GPU card and crank away. Matthew Berman has more on this on YouTube.
Microsoft 365 Copilot will cost how much?! Microsoft announced a $30 per month price tag on their AI_driven CoPilot, which provides extensive generative AI capabilities that is integrated with Microsoft 365 apps like Word, Excel, Teams, and Viva.
Google is currently testing a tool where AI will help write news articles for journalists named ‘Genesis’. Google is pitching it as a writer’s assistant: “In partnership with news publishers, especially smaller publishers, we’re in the earliest stages of exploring ideas to potentially provide AI-enabled tools to help journalists with their work,” a Google spokesperson said. Unsurprisingly, this met resistance from skeptical journalists who believe AI could threaten their positions and lacks the creativity to write high-quality news stories.
OpenAI launches customized instructions for ChatGPT, as a way to personalize the ChatGPT experience. This context is so users don’t have to write the same instruction prompts to the chatbot every time they interact with it, like specifying one’s family size for meal-planning tips, or indicating a preferred tone or programming language in responses.
AI Research News
Google’s DeepMind has published a paper proposing CoDoc (Complementarity-driven Deferral-to-Clinical Workflow), an assistant to human doctors designed to help provide analysis of symptoms and conditions. CoDoc would be able to run on a single computer, be operated with minimal knowledge of machine learning, be trained on small amounts of data, and even compute a level of confidence it has for its own diagnoses. CoDoc could lead to a reduction of false positives in diagnoses, as well as reduce the amount of cases that require consultations from physicians. Once CoDoc has been polished and fully released, it could revolutionize how we think about healthcare.
A study is supporting the contention that GPT-4 and chatGPT performance has degraded, by comparing the reasoning of the model now with its reasoning in March. Differences showing degradation were reported.
Researchers investigating “catastrophic forgetting” in AI learning discover that, like humans, learning a diversity of tasks helps AI learn new tasks better and not forget prior learning.
Stanford Research Introduces FlashAttention-2: A Leap in Speed and Efficiency for Long-Context Language Models. Their paper “FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning” shows impressive speedup on training compared to original FlashAttention and baselines:
The performance of FlashAttention-2 is truly impressive. Benchmarked on an A100 80GB SXM4 GPU, it achieves around 2x speedup compared to its predecessor and up to 9x speedup compared to a standard attention implementation in PyTorch. Moreover, when used for end-to-end training of GPT-style models, FlashAttention-2 unlocks up to 225 TFLOPs/s on A100 GPUs, representing a 1.3x end-to-end speedup over already highly optimized models with FlashAttention.
AI Business and Policy
Microsoft stocks have hit an all-time high in the wake of announcing Github Copilot subscriptions and being a partner in the announcement of Llama2. This AI news confirms Microsoft’s lead in AI, and Microsoft is managing to combine size and speed as it takes advantage of AI as a business.
Harvard School of Law has launched an AI effort titled the Initiative on Artificial Intelligence and the Law (IAIL). With AI becoming increasingly important, concerns around AI are as well, with copyright protection, data privacy and misinformation becoming larger issues. We can hope this will focus on policy and law around AI as we try to properly safeguard against AI’s threats while encouraging ‘responsible’ AI innovation and development.
In the area of security, AI camera systems are being designed for a range of applications. In NYC, they’re being tested to detect fare evaders, using learning models to understand when a person is jumping over a turnstile instead of paying. In hospitals, AI is being developed to detect guns on a person’s body to help prevent violent gun crimes. This technology could lead to more AI powered detectors in schools, malls, and other public areas.
On a related topic, Pennsylvania Senator Bob Casey has rolled out new bills to fence in AI as an effort to protect from surveillance and “Robot Bosses”. The first bill seeks to outlaw determining employment based decisions solely on an algorithm. The second seeks to restrict workplace surveillance.
AI company representatives came to the White House again, and Biden delivered remarks on AI that included the line “I’m the AI.” Was he accidently revealing the truth? You decide.
AI Opinions and Articles
Writers have been taking it on the chin lately due to AI, but now the writers are striking back:
The Authors Guild released an open letter, signed by over 8,000 authors, addressed to the CEOs of OpenAI, Meta, Microsoft, and other companies demanding that authors get the opportunity to give consent and get credit and compensation when their work is used to train AI models. The letter is at least partially retroactive in nature as AI companies have already used writers’ copyrighted content without permission and for profit.
“Millions of copyrighted books, articles, essays, and poetry provide the ‘food’ for AI systems, endless meals for which there has been no bill… It is only fair that you compensate us for using our writings, without which AI would be banal and extremely limited,” - Author Guild open letter
A Look Back …
The most interesting AI research to come out of the top technology company research labs in 2023 has come out of Meta’s AI research team, FAIR.
Born as Facebook AI Research ten years ago, but now called Fundamental AI Research at Meta, this group has been a fundamental driver of some key AI innovations. They’ve led in a number of AI research areas, including self-supervised learning that led to LLMs, and GANs (generative adversarial networks) that led to current image generation AI models.
From their 5-year retrospective in 2018:
FAIR has applied an open model to all aspects of our work, collaborating broadly with the community. Our teams publish cutting-edge research early and often, and open-source our research code, data sets, and tools like PyTorch, fastText, FAISS, and Detectron where possible. The approach has been successful for advancing the state of AI research. This year, FAIR’s researchers have won recognition, including Best Paper awards, at ACL, EMNLP, CVPR, and ECCV, and Test of Time awards at ECCV, ICML, and NeurIPS. We know working in the open allows everyone to make faster progress on AI.