Mentor's 500M base, fully fine-tuned on 10,000 retrieval-augmented examples — a golden passage among distractors — to answer from the right source and refuse when the answer is absent.
The 500M base given the RAFT habit through a full fine-tune (all 517.8M parameters): answer from the passage among distractors, refuse when it is not there. Uses the same 10,000-example RAFT dataset as the Gemma RAFT model.
2 epochs (1,875 steps). On the same 500 held-out questions and the arena's blind 0–10 rubric judge it scores 4.76/10 (groundedness 70.8%, fabrication 0.6%, token-F1 0.155). Token-F1 is low because RAFT answers quote-first and terse, which lexical overlap punishes; the judge score is the fairer read.
| Class | LlamaForCausalLM |
| Layers | 24 |
| Hidden size | 1,280 |
| Attention | 20 heads · dim 64 · full MHA |
| Feed-forward | SwiGLU · inner 3,456 |
| Positional | RoPE · θ 10,000 |
| Norm | RMSNorm · ε 1e-5 |
| Context | 1,024 tokens |
| Vocabulary | 32,768 |
| Embeddings | tied input/output |
| Pretrained by | Dr. Sreedath Panat |
| Init from | slm-500m base |
| Method | RAFT fine-tune (full) |
| Trainable params | 517.8M — all |
| Training data | 10,000 RAFT examples · 2 epochs |
| Optimizer steps | 1,875 |
our cost begins at fine-tuning — the base is imported.
| Stage | Detail | Cost |
|---|---|---|
| 500M fine-tuning | SFT + RAFT trained as one Modal app ($2.67 for both) — not separable per model | $2.67 |
| Total | $2.67 |
Figures are Modal GPU usage (time × rate) across this model's lineage; shared datasets are charged at this model's share. Whether base pretraining is included is stated above — it is for the models pretrained here, and excluded for imported bases. Evaluation-derived metrics come from an independent blind-judge harness on a frozen, decontaminated held-out set. Serving is billed separately and scales to zero.