ByteDance has finally open-sourced its diffusion language model; with 2.3 billion parameters, it directly takes on the AR approach. This wave of text generation in the potential space is pretty interesting.

ByteDance Seed open-source Cola DLM is a model that performs text diffusion at the potential semantic level. Text VAE maps text to a continuous latent space, while block-causal DiT learns the latent prior through Flow Matching, ultimately with a conditional decoder restoring the latent variables back into text. Total parameters are approximately 2.3 billion (DiT 1.8 billion, VAE 500 million). In 8 evaluation metrics, it competes with and outperforms baseline AR/LLaDA models of similar scale, but it is still a research-oriented checkpoint, not fine-tuned with instructions or RLHF. The current repository only includes the text pipeline, with future plans to extend to text-image.