What Is Google Gemma? Open Models, Benchmarks & How to Use It

What Is Google Gemma?

Gemma is Google DeepMind's family of open-weight language models. The name comes from the Latin word for "gemstone," and the connection to Gemini is more than cosmetic. Gemma models are distilled from the same research infrastructure and training data pipelines that produce Google's flagship Gemini models. The difference is in the delivery: where Gemini is a closed API product, Gemma ships as downloadable weights under the Apache 2.0 license.

That distinction matters for practitioners. Apache 2.0 means no usage caps, no monthly active user limits, no mandatory attribution, and no restrictions on commercial deployment. You can fine-tune Gemma, merge it with other models, deploy it on your own infrastructure, and sell products built on top of it. The license is identical to what you would find on TensorFlow or Kubernetes.

Since the first Gemma models arrived in February 2024, the family has grown to cover everything from 2-billion-parameter edge models that run on mobile phones to 31-billion-parameter dense transformers that compete with models five times their size on reasoning benchmarks. The community has responded: Gemma has surpassed 150 million downloads on Hugging Face, with over 70,000 community-created variants ranging from medical assistants to code generators to safety classifiers.

How Mixture-of-Experts Works in Gemma 4

The Gemma 4 26B MoE is the most architecturally interesting model in the family. Traditional dense transformers pass every token through every parameter. A 26-billion-parameter dense model would require 26 billion operations per token. The MoE approach changes this equation fundamentally.

Gemma 4 26B contains 128 expert sub-networks. For each token, a learned router selects only the most relevant experts. The result: just 3.8 billion parameters activate per token, while the full 26 billion remain available as specialized knowledge pools. This is not an approximation or a quality trade-off. The model achieves roughly the same intelligence as a 27-billion-parameter dense model while consuming compute equivalent to a 4-billion-parameter model.

The practical impact is significant. Inference costs drop by roughly 7x compared to an equivalently capable dense model. Memory requirements during inference scale with the active parameter count, not the total. A quantized version of the 26B MoE fits in 16GB of VRAM, making it accessible to anyone with a mid-range GPU.

Multimodal Capabilities

Every Gemma 4 model processes multiple input types natively. This is not a bolted-on vision adapter; the multimodal capability is trained into the model from the start.

Audio support on the edge models is a deliberate design choice. E2B and E4B are intended for on-device applications where voice interaction is a primary interface: smart speakers, wearables, mobile assistants. The larger models focus on document understanding and video analysis workflows where audio processing is typically handled by a dedicated pipeline stage.

Gemma vs Llama

Gemma and Llama are the two dominant open model families, and they make fundamentally different architectural bets. Gemma uses a deeper, thinner architecture with more transformer layers and smaller hidden dimensions. Llama goes wider and shallower, with fewer layers but larger hidden dimensions. Neither approach is universally better, but they produce different performance profiles.

The licensing difference is equally significant. Gemma's Apache 2.0 license has no usage restrictions. Llama's community license includes a 700 million monthly active user cap, above which you need a separate commercial agreement with Meta. For most organizations this cap is irrelevant, but for platform companies or widely-distributed applications, it is a real constraint that Apache 2.0 avoids entirely.

When to Pick Gemma

• You need the smallest possible model for edge deployment (Gemma's 2B and 4B variants)
• Your application needs multimodal input including audio (edge models)
• You need Apache 2.0 licensing without any user caps
• Cost per token is a primary concern (Gemma 3 4B pricing is aggressive)
• You want MoE efficiency for serving large-model intelligence on consumer hardware

When to Pick Llama

• You need models larger than 31B parameters (Llama 3.1 405B)
• Your existing infrastructure is optimized for Llama's architecture
• You need the widest community ecosystem for a specific task (Llama has more third-party tooling in some verticals)
• The 700M MAU cap does not apply to your use case

Running Gemma Locally

One of Gemma's strongest selling points is how accessible it is for local deployment. The model weights are available directly from Hugging Face and through Google's Kaggle model hub.

Edge models (E2B, E4B): These run on consumer laptops, Raspberry Pi-class devices, and mobile phones. The E2B model requires roughly 4GB of RAM in quantized form. No dedicated GPU needed for inference at modest throughput.

26B MoE: The MoE architecture makes this model surprisingly accessible. With QLoRA quantization via Unsloth, the 26B MoE fits in 16GB of VRAM. That puts it within reach of an NVIDIA RTX 4060 Ti or equivalent. For fine-tuning, Unsloth's memory-efficient approach means you can train on consumer hardware that would not support the full-precision model.

31B Dense: Requires 24GB+ VRAM for quantized inference (RTX 4090, A5000, or equivalent). For full-precision, plan for 48GB+ (A6000 or dual GPU setup).

All models work with the standard open-source inference stack: ollama, llama.cpp, vLLM, and Hugging Face transformers. The Apache 2.0 license means no registration, no API keys, and no usage reporting required for local deployment.

Limitations & Honest Caveats

Video Resources