MiniMax-M2.7-Abliterated-Heretic-MLX-3bit
on a Mac: what it actually needs
For Youssof Altoukhi. Worked out from your repo, no sign-up,
nothing to click.
3-bit
62 layers
8 KV heads
head dim 128
4 likes
153 downloads
105.3 GB
measureddownload size, summed from the
21 safetensors files in your repo
>128 GB Mac
computedthe smallest machine that holds it
at 8k context, assuming MLX can wire 75% of unified memory
What it needs as context grows
4k context1.04 GB KV + 105.3 GB weights + 1.2 GB runtime = 107.5 GB, over 128 GB
8k context2.08 GB KV + 105.3 GB weights + 1.2 GB runtime = 108.6 GB, over 128 GB
32k context8.32 GB KV + 105.3 GB weights + 1.2 GB runtime = 114.8 GB, over 128 GB
128k context33.29 GB KV + 105.3 GB weights + 1.2 GB runtime = 139.8 GB, over 128 GB
KV cache = 2 tensors x 62 layers x 8 KV heads
x 128 head dim x 2 bytes (fp16) x tokens. Read straight off your
config.json. Runtime overhead is the 1.2 GB I see for Python, MLX, the
tokenizer and activations on my own runs.
Mac by Mac
| Context | Needs | 8 | 16 | 18 | 24 | 32 | 36 | 48 | 64 | 96 | 128 |
| 4k | 107.5 GB | no | no | no | no | no | no | no | no | no | no |
| 8k | 108.6 GB | no | no | no | no | no | no | no | no | no | no |
| 32k | 114.8 GB | no | no | no | no | no | no | no | no | no | no |
| 128k | 139.8 GB | no | no | no | no | no | no | no | no | no | no |
Columns are unified memory in GB. "Tight" means it fits but leaves under
2 GB for anything else, so expect swap once a browser is open.
Run it
pip install mlx-lm
mlx_lm.generate --model Youssofal/MiniMax-M2.7-Abliterated-Heretic-MLX-3bit \
--prompt "hello" --max-tokens 100
What is not on this page
not measuredTokens per second. That needs a
real run on real silicon and I will not print a number I have not seen. If you want it,
reply and I will run MiniMax-M2.7-Abliterated-Heretic-MLX-3bit on an M5 and add the result here with the
command I used.