Overview
Cola DLM is a research-oriented, hierarchical continuous-latent diffusion language model for text generation. The released checkpoint corresponds to the 2000 EFLOPs training-compute point, not a parameter count, in the paper’s scaling experiment. It has roughly 2 billion parameters. (arxiv.org)
Architecture and capabilities
- A Text VAE maps text to continuous latent sequences and decodes them back into tokens. A block-causal Diffusion Transformer models the latent prior using Flow Matching. (github.com)
- The model supports prompt-conditioned text generation and benchmark-style question answering. Its open-source inference implementation reports zero-shot accuracy of 50.80% on LAMBADA and 19.30% on MMLU; results may differ slightly from the paper’s internal implementation. (huggingface.co)
- The research paper discusses potential extensions to continuous modalities; that does not establish image-generation capability for this released text checkpoint. (arxiv.org)
Intended use and availability
The checkpoint is intended primarily for research into latent-variable language models, diffusion for text, Flow Matching and benchmark evaluation. Weights and inference code are available under Apache License 2.0. The repository also provides an adapter for self-hosted, OpenAI-compatible, non-streaming chat completions; this is not evidence of a ByteDance-hosted API service. (huggingface.co)
Limitations
- The checkpoint is not instruction-tuned and has not undergone RLHF or systematic safety alignment. It is not a production-ready chatbot or suitable for safety-critical decisions.
- Training was primarily on English text; other languages have not been well evaluated. Outputs may contain factual errors, hallucinations, bias or offensive content, and quality can depend on prompt format and length.
- Generation uses mutable KV caches; service implementations should serialize generation within a process unless cache handling is isolated. (huggingface.co)
Pricing
As of 2026-11-16, official input and output API prices for this checkpoint could not be verified in either CNY on the domestic Volcengine site or USD on the international BytePlus site. No token unit price, billing tier or hosted API availability is established for this model. Self-hosting may incur infrastructure costs. Pricing references checked: Volcengine model pricing and BytePlus ModelArk pricing. (console.volcengine.com)
Publication and official sources
The paper’s first version was submitted on 2026-05-07; this is the paper publication date, not a verified date for the weight release. (arxiv.org)