SGT: Semantic Generative Tuning for Unified Multimodal Models

[](https://song2yu.github.io/SGT/) [](https://arxiv.org/pdf/2605.18714) [](https://huggingface.co/datasets/Two-hot/SAM-SGT) **SGT (Semantic Generative Tuning)** is the first systematic investigation into generative post-training for Unified Multimodal Models (UMMs). By leveraging **image segmentation as a generative proxy**, SGT bridges the gap between visual understanding and generation, enabling true synergy between the two capabilities within a single architecture. If you find our project or paper useful, we would greatly appreciate it if you could star this repository or cite our work. Existing UMMs optimize understanding and generation independently — this leads to misaligned representations and missed synergies. Previous pixel-level alignment methods over-emphasize texture and fail to provide structural semantic guidance. SGT takes a different approach: use **high-level segmentation** as the generative training objective. This simple yet effective proxy: