OLMo 7B

OLMo 7B needs an estimated 5.3 GB of VRAM at Q4, 8.9 GB at Q8 and 16.3 GB at FP16. These are estimates based on the 6.9B parameter count, covering weights plus about 15% and 0.5 GB for a short context.

Yes, OLMo 7B fits on all four GPU sizes. A 24 GB GPU fits at FP16 (estimated 16.3 GB), 16 GB and 12 GB GPUs fit at Q8 (estimated 8.9 GB) and an 8 GB GPU fits at Q4 (estimated 5.3 GB).

Yes. At Q4 OLMo 7B needs an estimated 5.3 GB, and assuming macOS lets the GPU use about 75% of unified memory, the smallest common Mac memory size that fits Q4 is an estimated 8 GB.

Yes, you can self-host OLMo 7B for full control of your data because it has open weights.

Run OLMo 7B locally

No, OLMo 7B cannot read images, audio, video or PDFs. Its only input modality is text.

BGE Reranker V2 M3 by BAAI, SOLAR 10.7B Instruct v1.0 by Upstage and Mixtral 8x7B Instruct v0.1 by Mistral AI. They are matched on model kind and release date, with no comparable price.

BGE Reranker V2 M3OLMo 7B vs BGE Reranker V2 M3SOLAR 10.7B Instruct v1.0OLMo 7B vs SOLAR 10.7B Instruct v1.0Mixtral 8x7B Instruct v0.1OLMo 7B vs Mixtral 8x7B Instruct v0.1

OLMo 7B has open weights. The license is Apache 2.0.

Yes, you can use OLMo 7B commercially. It is licensed under Apache 2.0, a permissive license that allows commercial use.

The OLMo 7B weights can be downloaded from the Hugging Face repository allenai/OLMo-7B.

Run OLMo 7B locally

OLMo 7B was made by Ai2 and released on February 1, 2024.

More from Ai2

Yes, OLMo 7B is free to use because it has open weights.