OLMo 1.7 7B

OLMo 1.7 7B needs an estimated 5.3 GB at Q4, 8.9 GB at Q8 and 16.3 GB at FP16, based on its 6.9B parameters. All figures are estimates, covering weights plus about 15% and 0.5 GB for a short context.

Yes. A 24 GB GPU fits at FP16 (estimated 16.3 GB), a 16 GB or 12 GB GPU fits at Q8 (estimated 8.9 GB) and an 8 GB GPU fits at Q4 (estimated 5.3 GB).

Yes, it runs at an estimated 5.3 GB at Q4. Assuming macOS lets the GPU use about 75% of unified memory, the smallest common Mac memory size that fits Q4 is 8 GB.

Yes, OLMo 1.7 7B has open weights, so you can self-host it for full control of your data.

Run OLMo 1.7 7B locally

No, OLMo 1.7 7B only accepts text. It cannot read images, audio, video or PDFs.

Closest alternatives, matched on model kind and release date (no comparable price): OpenELM 1.1B by Apple, OpenELM 270M by Apple and Hermes 2 Pro Llama 3 8B by Nous.

OpenELM 1.1BOLMo 1.7 7B vs OpenELM 1.1BOpenELM 270MOLMo 1.7 7B vs OpenELM 270MHermes 2 Pro Llama 3 8BOLMo 1.7 7B vs Hermes 2 Pro Llama 3 8B

OLMo 1.7 7B is open weights, licensed under Apache 2.0.

Yes, OLMo 1.7 7B is licensed under Apache 2.0, a permissive license that allows commercial use.

The weights are available in the Hugging Face repository allenai/OLMo-7B-0424-hf.

Run OLMo 1.7 7B locally

OLMo 1.7 7B was made by Ai2 and released on April 17, 2024.

More from Ai2

Yes, OLMo 1.7 7B has open weights, so it is free to use.