Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generation

  • 2024-09-06 18:14:53
  • Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, Ying Shan
  • 0

Abstract

We present Open-MAGVIT2, a family of auto-regressive image generation modelsranging from 300M to 1.5B. The Open-MAGVIT2 project produces an open-sourcereplication of Google's MAGVIT-v2 tokenizer, a tokenizer with a super-largecodebook (i.e., $2^{18}$ codes), and achieves the state-of-the-artreconstruction performance (1.17 rFID) on ImageNet $256 \times 256$.Furthermore, we explore its application in plain auto-regressive models andvalidate scalability properties. To assist auto-regressive models in predictingwith a super-large vocabulary, we factorize it into two sub-vocabulary ofdifferent sizes by asymmetric token factorization, and further introduce "nextsub-token prediction" to enhance sub-token interaction for better generationquality. We release all models and codes to foster innovation and creativity inthe field of auto-regressive visual generation.

 

Quick Read (beta)

loading the full paper ...