Abstract
We introduce two multilingual, multimodal foundation language models thatpower Apple Intelligence features across Apple devices and services: i a3B-parameter on-device model optimized for Apple silicon through architecturalinnovations such as KV-cache sharing and 2-bit quantization-aware training; andii a scalable server model built on a novel Parallel-Track Mixture-of-ExpertsPT-MoE transformer that combines track parallelism, mixture-of-experts sparsecomputation, and interleaved global-local attention to deliver high qualitywith competitive cost on Apple's Private Cloud Compute platform. Both modelsare trained on large-scale multilingual and multimodal datasets sourced viaresponsible web crawling, licensed corpora, and high-quality synthetic data,then further refined with supervised fine-tuning and reinforcement learning ona new asynchronous platform. The resulting models support several additionallanguages while understanding images and executing tool calls. In publicbenchmarks and human evaluations, both the server model and the on-device modelmatch or surpass comparably sized open baselines. A new Swift-centric Foundation Models framework exposes guided generation,constrained tool calling, and LoRA adapter fine-tuning, allowing developers tointegrate these capabilities with a few lines of code. The latest advancementsin Apple Intelligence models are grounded in our Responsible AI approach withsafeguards like content filtering and locale-specific evaluation, as well asour commitment to protecting our users' privacy with innovations like PrivateCloud Compute.