AI Week in Review 23.07.08
LongNet breaks context window limits, AI translates ancient texts, and AI goes to war
Cover art: From an article on robots that stock drinks in convenience stores.
Top Tools
Novel.sh is a Notion-style WYSIWYG editor with AI-powered autocomplete. It went viral last month on Twitter and Hacker News.
Khoj is an open-source personal AI assistant.
AI Tech and Product Releases
OpenAI made several updates this week, broadening access to their tools:
Introducing Superalignment. OpenAI is building a team to solve for alignment that meets the challenge of super-intelligent AI. “Our goal is to build a roughly human-level automated alignment researcher. We can then use vast amounts of compute to scale our efforts, and iteratively align superintelligence.” See also “OpenAI introduces team dedicated to stopping rogue AI.”
OpenAI also announced GPT-4 general availability, giving all paying API customers have access to GPT-4.
Code Interpreter is available to all paid users on ChatGPT ( ChatGPT Plus).
NVidia offers to Train Your AI Model Once and Deploy on Any Cloud with NVIDIA and Run:ai, with the NVidia’s GPU-accelerated Cloud Native Stack Virtual Machine Image (VMI). It “enables you to build, test, and run GPU-accelerated containerized applications orchestrated by Kubernetes.”
Salesforce has announced an update to their CodeGen models, CodeGen2.5: Small, but mighty. CodeGen2.5 is a 7B code-generation model that matches the performance of code-generation models twice its size, in particular matching StarCoder-15B. This continues a trend of smaller-but-better code-generation AI models.
AI Research News
CrunchGPT, A chatGPT assisted framework for scientific machine learning, presents a way to streamline scientific data analysis flows with AI:
We integrate the various stages of Scientific Machine Learning (SciML) under the umbrella of ChatGPT, to formulate CrunchGPT, which plays the role of a conductor orchestrating the entire workflow of SciML based on simple prompts by the user.
In LongNet: Scaling Transformers to 1,000,000,000 Tokens, researchers propose a ‘dilated attention’ approach to expanding context windows that has linear complexity and can replace standard attention in a Transformer based architecture. These features enable greatly expanded context windows.
New AI translates 5,000-year-old cuneiform tablets instantly. A recent study describes a new AI that produces high-quality translations of ancient texts. There are many ancient Akkadian cuneiform tablets, and not enough experts to laboriously translate them. AI can do the tedious
From the APA, AI is changing every aspect of psychology. Here’s what to watch for:
Psychology practice is ripe for AI innovations—including therapeutic chatbots, tools that automate note-taking and other administrative tasks, and more intelligent training and interventions—but clinicians need tools they can understand and trust.
While chatbots lack the context, life experience, and verbal nuances of human therapists, they have the potential to fill gaps in mental health service provision.
“The bottom line is we don’t have enough providers,” Jackson said. “While therapy should be for everyone, not everyone needs it. The chatbots can fill a need.” For some mental health concerns, such as sleep problems or distress linked to chronic pain, training from a chatbot could suffice.
AI Business and Policy News
ChatGPT users drop for the first time, falling 10 percent from May to June. Several explanations give for this have included: Competition from ‘uncensored chatbots’, the novelty of ChatGPT wearing off, and “fewer students need to write academic papers in the summer months.”
Google’s medical AI chatbot is already being tested in hospitals. “Google’s Med-PaLM 2, an AI tool designed to answer questions about medical information, has been in testing at the Mayo Clinic research hospital, among others, since April.” The reports say that inaccuracies were being found in the AI tool, but that on most metrics, the AI was as accurate as doctors.
AI goes to war: Ukraine gained advantage in war against Putin with custom-built AI. The war has been an 'unprecedented testing ground'for low-cost drones, AI-based surveillance and intelligence, and more. “Ukraine has leaned heavily on 'gremlin factories' and its 'IT army' to develop technology from scraps.”
Google UK report highlights AI’s impact on the UK economy.
Top universities pledge to incorporate AI use in teaching: ‘Opportunity rather than a threat.’ This pledge comes as University grapple with responding to the rise of AI, recognizing “the importance of equipping their students and staff with AI literacy skills.”
The AI for Good Global Summit 2023 was just held in Geneva, showcasing AI, robotics and their possible contribution to addressing global issues. In it, AI robots told the UN conference attendees they could run the world better. Cue the cynical comments about UN diplomats and members of Congress.
TechCrunch discusses how generative AI spams up the web, causing headaches for the ad business. AI content generation is being used to build sites that ‘hack’ into automated systems for putting ads on pages. “In its report, Newsguard found close to 400 instances of ads from 141 major brands that appeared on 55 of the junk news sites.”
Meanwhile, SpeedyBrand, which uses generative AI to create SEO-optimized content, just raised $2.5 million to potentially make that problem worse and add more AI content to the web.
AI Opinions and Articles
In the thoughtful essay “Embracing change and resetting expectations”, UCLA Math Professor Terence Tao discusses how he uses GPT-4 in his work. He notes how it can handle “vaguely phrased” prompts, can help draft thoughts and digest papers, and in conversation, act as a “compassionate listener, an enthusiastic sounding board, a creative muse, a translator or teacher, or a devil’s advocate.” In the mathematical workflow, combining GPT-4 with powerful reasoning would make it a useful mathematics assistant:
The 2023-level AI can already generate suggestive hints and promising leads to a working mathematician and participate actively in the decision-making process. When integrated with tools such as formal proof verifiers, internet search, and symbolic math packages, I expect, say, 2026-level AI, when used properly, will be a trustworthy co-author in mathematical research, and in many other fields as well.
AI firms should face prison over creation of fake humans, says Yuval Noah Harari, in a drastic yet thought-provoking proposal:
The creators of AI bots that masquerade as people should face harsh criminal sentences comparable to those who trade in counterfeit currency, the Israeli historian and author Yuval Noah Harari has said.
He also called for sanctions, including prison sentences, to apply to tech company executives who fail to guard against fake profiles on their social media platforms.
Generative AI’s Future Will Be as the Ultimate Human Sidekick, Says Foundation Capital’s Iyengar.
A Look Back …
In the recent Atlantic essay “Gödel, Escher, Bach, and AI,” author and Professor Douglas Hofstadter reflects on this moment in AI. “A dazzlingly fast chatbot cannot replace the authentic and reflective voice of a thinking, living human being.”
Hofstader wrote the tour-de-force work Gödel, Escher, Bach: an Eternal Golden Braid in 1979. It was a ground-breaking journey into the world of computing, genetics, algorithms, philosophy, cognitive science, mathematics, and more. It was also about the nature of minds, and so it had something to say about AI, although it never contemplated AI as we know it today. I read this work when I was just starting in my career in computers decades ago, and it made a profound impression on me. It is worth a read or even a re-read.
