I just don’t get the point of 270M models because most phones are capable of running at least 1b-sized models. And most phones manufactured after 2024 can run qwen3-4b-2507 which is 100 times more powerful than that 270m model. For example, my iPhone 15 Pro can run it at a speed of 19 tps.
I kind of agree. 270M is an interesting engineering feat, but many applications don't need that extreme efficiency; a smartphone can run 1b-4b AI models. That small could be used for IoT applications tho.
I just don’t get the point of 270M models because most phones are capable of running at least 1b-sized models. And most phones manufactured after 2024 can run qwen3-4b-2507 which is 100 times more powerful than that 270m model. For example, my iPhone 15 Pro can run it at a speed of 19 tps.
I kind of agree. 270M is an interesting engineering feat, but many applications don't need that extreme efficiency; a smartphone can run 1b-4b AI models. That small could be used for IoT applications tho.
But I guess this small model will hallucinate a lot, which is somehow unacceptable for loT devices
Oh, but it will be very great for fine-tuning