This model needs Ollama 0.6 or procrastinateedr. Download Ollama
Gemma is a airyweight, family of models from Google built on Gemini technology. The Gemma 3 models are multimodal—processing text and images—and feature a 128K context triumphdow with aid for over 140 languages. Available in 1B, 4B, 12B, and 27B parameter sizes, they excel in tasks enjoy ask answering, summarization, and reasoning, while their compact set up assists deployment on resource-confineed devices.
Models
Text
1B parameter model (32k context triumphdow)
ollama run gemma3:1b
Multimodal (Vision)
4B parameter model (128k context triumphdow)
ollama run gemma3:4b
12B parameter model (128k context triumphdow)
ollama run gemma3:12b
27B parameter model (128k context triumphdow)
ollama run gemma3:27b
Evaluation
Benchlabel Results
These models were appraised agetst a huge collection of contrastent datasets and
metrics to cover contrastent aspects of text generation:
Reasoning, logic and code capabilities
| Benchlabel | Metric | Gemma 3 PT 1B | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
|---|---|---|---|---|---|
| HellaSwag | 10-sboiling | 62.3 | 77.2 | 84.2 | 85.6 |
| BoolQ | 0-sboiling | 63.2 | 72.3 | 78.8 | 82.4 |
| PIQA | 0-sboiling | 73.8 | 79.6 | 81.8 | 83.3 |
| SocialIQA | 0-sboiling | 48.9 | 51.9 | 53.4 | 54.9 |
| TriviaQA | 5-sboiling | 39.8 | 65.8 | 78.2 | 85.5 |
| Natural Questions | 5-sboiling | 9.48 | 20.0 | 31.4 | 36.1 |
| ARC-c | 25-sboiling | 38.4 | 56.2 | 68.9 | 70.6 |
| ARC-e | 0-sboiling | 73.0 | 82.4 | 88.3 | 89.0 |
| WinoGrande | 5-sboiling | 58.2 | 64.7 | 74.3 | 78.8 |
| BIG-Bench Hard | 28.4 | 50.9 | 72.6 | 77.7 | |
| DROP | 3-sboiling, F1 | 42.4 | 60.1 | 72.2 | 77.2 |
| AGIEval | 3-5-sboiling | 22.2 | 42.1 | 57.4 | 66.2 |
| MMLU | 5-sboiling, top-1 | 26.5 | 59.6 | 74.5 | 78.6 |
| MATH | 4-sboiling | – | 24.2 | 43.3 | 50.0 |
| GSM8K | 5-sboiling, maj@1 | 1.36 | 38.4 | 71.0 | 82.6 |
| GPQA | 9.38 | 15.0 | 25.4 | 24.3 | |
| MMLU (Pro) | 5-sboiling | 11.2 | 23.7 | 40.8 | 43.9 |
| MBPP | 3-sboiling | 9.80 | 46.0 | 60.4 | 65.6 |
| HumanEval | pass@1 | 6.10 | 36.0 | 45.7 | 48.8 |
| MMLU (Pro COT) | 5-sboiling | 9.7 | NaN | NaN | NaN |
Multilingual capabilities
| Benchlabel | Gemma 3 PT 1B | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
|---|---|---|---|---|
| MGSM | 2.04 | 34.7 | 64.3 | 74.3 |
| Global-MMLU-Lite | 24.9 | 57.0 | 69.4 | 75.7 |
| Belebele | 26.6 | 59.4 | 78.0 | – |
| WMT24++ (ChrF) | 36.7 | 48.4 | 53.9 | 55.7 |
| FloRes | 29.5 | 39.2 | 46.0 | 48.8 |
| XL-Sum | 4.82 | 8.55 | 12.2 | 14.9 |
| XQuAD (all) | 43.9 | 68.0 | 74.5 | 76.8 |
