AI Week In Review 24.06.15
Apple Intelligence, Luma AI Dream Machine, Stable Diffusion 3 Medium weights released, Nvidia Nemotron 340B, Google GenType, Mixture of Agents, Samba 3.8B, Mistral and Sakana AI get funding.
AI Tech and Product Releases
Apple’s WWDC was the biggest AI news this week with their Apple Intelligence anouncement, which we covered in depth in Apple gets Intelligence. They wove AI into a number of features:
AI-enhanced writing and summarization in emails, notifications, notebooks, etc.
Notes and Phone app users can now record, transcribe, and summarize audio.
Enhanced Siri, with on-screen awareness, AI-enabled actions to control apps via Siri, and access to AI models, from local AI models to GPT-4o on-demand.
AI image generation with and custom emoji creation with Genmoji.
Apple led with practical use cases and stressed personalization and privacy in this rollout, and they are building these AI features on an infrastructure of local AI models, private cloud-hosted AI models, and a sync-up with OpenAI.
Luma dropped DreamMachine, an AI video generation that made waves for its Sora-level quality that users can try out now; many amazing video results coming out from this release. I shared details in Luma AI Video Gen - The Dream Gets Real. Users can generate 5 seconds of 1280x720 video per generation.
Stability AI announced the release of Stable Diffusion 3 Medium, and made weights available on HuggingFace. It’s released under Stability Non-Commercial Research Community License, so not quite open source.
As this YouTube review by AI Search details, SD3 Medium is great at landscapes, but (possibly due to being ‘censored’) it falls apart at human anatomy. You can play with it online at FAL.
NVIDIA has released the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. These models are aimed at generating synthetic data for building other large language models (LLMs) - AI models to create data to make other AI models.
NVidia has shared a Technical Report on Nemotron 340B, detailing architecture, training, and performance. The flagship 340B model surpasses Llama-3-70b across challenging benchmarks like Arena-Hard-Auto.
OpenAI adds function-calling support in fine-tuning of models.
As of today, we have added support for the `tools` parameter in the Fine-tuning API, which means you can now also fine-tune for parallel function calling.
Top Tools & Hacks
Google has just released GenType, a specialized AI image generation tool for fonts, where you create a font from a text prompt. If you need an artistic font type or creative twist in some lettering, you can try it out in Google Labs.
AI Research News
Our AI Research Review for this week covered benchmarks, including the new released benchmarks: ML-Agent-Bench, LiveBench, MMLU-Redux, McEval for coding, CS-Bench for CS, WildBench, and Test of Time for temporal reasoning.
Some other items that caught our attention:
Together AI’s Mixture of Agents beats GPT-4o. Mixture of Agents (MoA) harnesses multiple LLMs in a 3-layered system to answer a query better. Together’s MoA achieved 65.1% on AlpacaEval 2.0, surpassing GPT-4o (57.5%).
Samba 3.8B is a simple Mamba plus Sliding Window Attention hybrid architecture that outperforms Phi3-mini on major benchmarks. Samba has a 3.7x higher throughput than Transformers with grouped-query attention.
New low-bit transformer architecture could enable powerful LLMs without GPUs, by avoiding need to do matrix multiplications.
AI Business and Policy
The Verge announces The AI upgrade cycle is here: “The new Apple Intelligence features coming to iOS 18 could be impressive, but they might also just be driving another upgrade cycle.” Investors like that story and bid Apple up to exceed Microsoft.
Apple, Microsoft and Nvidia are all hitting new highs at over $3 trillion in market cap each, on the strength of the emerging AI economic opportunity. Rounding out the top five are Google and Amazon, also driving forward with AI.
The AI boom is accelerating. Revenue at OpenAI doubled in the last 6 month and is now at $3.4B annualized.
Sorry, Europe, not Meta AI for you. Meta pauses AI plans for Europe after Irish regulator request:
Meta will not launch its Meta AI models in Europe for now after the Irish Government’s Data Protection Commission told it to delay its plan to harness data from Facebook and Instagram users, the US social media company said on Friday.
Paris-based Mistral AI has announced a €600 million Series B funding round ($640 million) of both equity and debt, co-led by General Catalyst. The funding raises the company's valuation to almost €6 billion ($6 billion). The company, known for its open-source AI initiatives, has garnered support from several prominent investors, including Lightspeed and a16z. Mistral AI has released seven top-ranking generative AI models, solidifying its position at the forefront of AI innovation. The funding will further enable Mistral AI to advance its mission of making frontier AI accessible to all.
Sakana AI, a Tokyo-based LLM developer founded last year by former Google researchers, is raising approximately $100 million at a $1 billion valuation. This company is getting support from both US-based venture funds and within Japan's technology and startup ecosystem, supported by the government. Sakana AI is behind the Evolutionary Model Merge concept.
Italian AI company iGenius is seeking to raise €650 million in a funding round, aiming for a post-money valuation of approximately €1.7 billion. The company has already raised money from investors including Angel Capital Management, the family office of Angelo Moratti, and Eurizon Asset Management, a unit of Italian bank Intesa Sanpaolo SpA, at a pre-money valuation of €1.05 billion.
These three examples - Mistral, Sakana, and iGenius - show that national pride and FOMO (fear of missing out) by Governments and local investors will lead to ‘home-team’ support for the rightly situated players in different nations.
Hugging Face acquired Argilla, a small AI model development company. Hugging Face CEO says he hears from 10 AI founders a week who want to sell their startups. So expect consolidation to continue.
Amazon makes $230 million investment in AWS credits for AI startups. Early-stage generative AI startups get free access to computing power and AI models. The AWS Generative AI Accelerator program will support 80 early-stage companies globally.
Elon drops lawsuit against OpenAI.
AI Opinions and Articles
Scale is a meritocracy, and we must always remain one.
- Alexandr Wang, CEO of Scale AI
Scale AI's CEO defends Meritocracy at Scale on X and in a company blog post, saying the company will prioritize hiring for MEI - 'Merit, excellence and intelligence' not DEI:
“That means we hire only the best person for the job, we seek out and demand excellence, and we unapologetically prefer people who are very smart.”
His honesty is refreshing, because any startup that doesn’t hire for excellence will be an also-ran.
Pope urges world leaders to keep ‘human control’ over AI, calls for autonomous weapons ban. Pope Francis expressed some AI skepticism and concern over AI in warfare.
“We need to ensure and safeguard a space for proper human control over the choices made by artificial intelligence programs: Human dignity itself depends on it. Precisely in this regard, allow me to insist: In light of the tragedy that is armed conflict, it is urgent to reconsider the development and use of devices like the so-called ‘lethal autonomous weapons’ and ultimately ban their use.” - Pope Francis
It’s a nice but vain hope that we won’t use a new technology for war; it’s inevitable. The ‘good guys’ are already applying AI in war, so that the ‘bad guys’ don’t have a technology advantage. AI is going to war.