Default to Ornith-1.5-35B-A3B with 256k/512k/1m context switch #1

Merged

thesimonharms merged 1 commit from cursor/ornith-15-35b-a3b-d534 into master

2026-08-20 14:10:51 -04:00

thesimonharms commented

2026-08-20 13:52:34 -04:00

(Migrated from github.com)

Makes malaikat ready to run Ornith-1.5-35B-A3B on the Strix Halo APU, with a one-flag context switch at launch.

Run it

python3 scripts/download_ornith15.py   # ~22 GB Q4_K_M into the platform models dir
./malaikat serve -config ./coding.yaml -c 256k
./malaikat serve -config ./coding.yaml -c 512k
./malaikat serve -config ./coding.yaml -c 1m

`-c`	Tokens	YaRN
`256k`	262144	off (native)
`512k`	524288	2×
`1m`	1048576	4×

512k/1m pass --rope-scaling yarn plus a qwen35moe.context_length override so llama.cpp does not cap the slot at the trained 256k window.

coding.yaml is now Ornith. Previous Unsloth Qwen3.6 MoE config remains in coding-qwen36.yaml.

Notes

Official Ornith-1.5-35B-Q4_K_M.gguf from ornith-ai/Ornith-1.5-35B-A3B-GGUF.
MTP stays on (draft-mtp, n-max 3). Use -no-mtp if this GGUF’s next-n tensors are missing.
1M context needs a lot of unified memory for KV even with hybrid attention; start at 256k.

Makes malaikat ready to run **Ornith-1.5-35B-A3B** on the Strix Halo APU, with a one-flag context switch at launch. ## Run it ```bash python3 scripts/download_ornith15.py # ~22 GB Q4_K_M into the platform models dir ./malaikat serve -config ./coding.yaml -c 256k ./malaikat serve -config ./coding.yaml -c 512k ./malaikat serve -config ./coding.yaml -c 1m ``` | `-c` | Tokens | YaRN | |------|-------:|------| | `256k` | 262144 | off (native) | | `512k` | 524288 | 2× | | `1m` | 1048576 | 4× | 512k/1m pass `--rope-scaling yarn` plus a `qwen35moe.context_length` override so llama.cpp does not cap the slot at the trained 256k window. `coding.yaml` is now Ornith. Previous Unsloth Qwen3.6 MoE config remains in `coding-qwen36.yaml`. ## Notes - Official `Ornith-1.5-35B-Q4_K_M.gguf` from `ornith-ai/Ornith-1.5-35B-A3B-GGUF`. - MTP stays on (`draft-mtp`, n-max 3). Use `-no-mtp` if this GGUF’s next-n tensors are missing. - 1M context needs a lot of unified memory for KV even with hybrid attention; start at 256k.  <div><a href="https://cursor.com/agents/bc-cb778e71-8f89-4e32-9bab-53ae8eb2d534?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/background-agent?bcId=bc-cb778e71-8f89-4e32-9bab-53ae8eb2d534&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div>