Apple's OpenELM and Google's Gemma 3n push small models onto phones

The pitch for small language models is simple: keep inference on the device, skip the round trip to a data centre, and keep the user's data where it is.

Two releases show how far that has moved from research demo to developer tooling. On 24 April 2024, Apple introduced OpenELM, a set of eight models released on Hugging Face under an Apple Sample Code License, as Ars Technica reported. They range from 270 million to 3 billion parameters, split into four pretrained versions and four instruction-tuned versions. Ars Technica noted that the licence carries restrictions, so the release may not meet the usual definition of open source, though the source code is available. The models use a 2048-token context window. They were trained on RefinedWeb, a de-duplicated version of PILE, a subset of RedPajama and a subset of Dolma v1.6, which Apple says totals around 1.8 trillion tokens. Apple also published the code for CoreNet, the library used to train them, plus reproducible training recipes.

What the numbers say

Parameter counts are the crude scoreboard here. Apple's smallest OpenELM is roughly 14 times smaller than Microsoft's Phi-3-mini at 3.8 billion parameters, which Ars Technica covered the day before.

Meta's largest released Llama 3 model has 70 billion parameters, with a 400 billion version in the works. OpenAI's GPT-3 from 2020 shipped with 175 billion. Apple's white paper claims a 2.36 percent accuracy improvement over Allen AI's OLMo 1B while using half as many pre-training tokens. That is a vendor-reported benchmark on a single comparison, not independent proof. Ars Technica reported it as a claim from the paper. Apple also cautions that because the training data was publicly sourced, the models may produce inaccurate, harmful, biased or objectionable output.

"The reproducibility and transparency of large language models are crucial for advancing open research, ensuring the trustworthiness of results, and enabling investigations into data and model biases, as well as potential risks."

That line comes from Apple's OpenELM paper abstract, quoted by Ars Technica. The company says the release aims to empower and enrich the open research community. It has not put this generation of models into its consumer devices. Ars Technica reported that iOS 18, expected to be shown in June at WWDC, was rumoured to bring on-device AI features, with the possibility that Apple would hire Google or OpenAI for heavier processing.

Google widens the toolbox

Google's AI Edge team has taken a different route. In a post dated 20 May 2025, it said it expanded on-device support from four initial models to more than a dozen, including Gemma 3 and Gemma 3n, hosted on a LiteRT Hugging Face community.

Gemma 3 1B is 529MB and can run up to 2,585 tokens per second pre-fill on a mobile GPU. Google says this is enough to process a page of content in under a second. Gemma 3n arrives as an early preview, with 2B and 4B parameter variants supporting text, image, video and audio inputs. Text and image are on Hugging Face, with audio to follow. The company also released a retrieval augmented generation library and a function calling library, both available on Android first. This allows an on-device model to pull relevant context from a thousand pages or photos and trigger app functions from natural language.

One caveat sits over all of this. Google's own quantization guidance says int4 can cut model size by 2.5 to 4 times versus bf16 while reducing latency and peak memory. That trade is not free: lower precision costs some accuracy, which is the dial every on-device deployment has to set.