OpenELM 1.1B

By our estimate, OpenELM 1.1B needs about 1.3 GB at Q4, 1.8 GB at Q8 and 3 GB at FP16 of VRAM. That covers the weights plus headroom for a short context, and longer contexts need more.

Yes, by our estimates OpenELM 1.1B fits on an 8 GB GPU at FP16 (about 3 GB) and at higher precision on larger cards.

Yes, a Mac with at least 8 GB of unified memory should run OpenELM 1.1B at Q4, which needs an estimated 1.3 GB. This assumes macOS gives the GPU about 75% of unified memory.

Yes, OpenELM 1.1B has open weights, so you can run it on your own hardware and keep prompts on your own machines.

Run OpenELM 1.1B locally

No, OpenELM 1.1B takes text input only.

Similar models from other labs include OLMo 1.7 7B (Ai2), Hermes 2 Pro Llama 3 8B (Nous) and Mistral 7B Instruct v0.3 (Mistral AI). They are the closest matches by kind and release date.

OLMo 1.7 7BOpenELM 1.1B vs OLMo 1.7 7BHermes 2 Pro Llama 3 8BOpenELM 1.1B vs Hermes 2 Pro Llama 3 8BMistral 7B Instruct v0.3OpenELM 1.1B vs Mistral 7B Instruct v0.3

OpenELM 1.1B has open weights under the apple-amlr license, so you can download and run it yourself.

The OpenELM 1.1B weights are on Hugging Face at apple/OpenELM-1_1B.

Run OpenELM 1.1B locally

OpenELM 1.1B was made by Apple. It was released on April 23, 2024.

More from Apple

Yes, OpenELM 1.1B has open weights, so you can run it on your own hardware without API fees.