COFFEE Decouples Sequence Objectives in Discrete Diffusion via Carrier and Finite-State Compilation
COFFEE provides a compiled finite-state mechanism that transfers sequence-level preferences to unresolved tokens in discrete diffusion without exponential enumeration. It achieves control on multiple benchmarks while remaining compatible with pretrained denoisers. The approach demonstrates practical neural-symbolic guidance at inference time.
The framework deploys a target-free carrier that absorbs denoiser marginals into a joint model over unresolved tokens while a compiled finite-state transducer records objective effects. At each step the paired states transfer global preferences to local sampling without retraining. Evaluations on symbolic, language, and biological benchmarks report task-dependent quality-diversity trade-offs under both hard constraints and learned soft objectives.
Prior discrete diffusion guidance required enumerating completions whose cost scales with unresolved positions. COFFEE replaces that with offline compilation of the objective, enabling plug-and-play use on pretrained models. This pattern mirrors earlier neural-symbolic hybrids such as those in Argmax Flows and Multinomial Diffusion (arXiv:2110.12777) and the ratio-estimation approach in Discrete Diffusion Modeling (arXiv:2205.14987), yet extends them to sequence-level rewards.
Operational impact appears in domains requiring joint conditioning, such as constrained code or protein sequence generation, where inference-time optimization replaces post-hoc filtering. The method therefore shifts objective handling from evaluation-only to active guidance.
Next steps include integration into existing discrete diffusion libraries and measurement of wall-clock overhead on longer sequences.
Hugging Face: COFFEE adapter merged into diffusers library by 2027-06 with measured <1.5x slowdown on 128-token sequences
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2609.35924)
- [2]Supporting Source(https://arxiv.org/abs/2110.12777)
- [3]Supporting Source(https://arxiv.org/abs/2205.14987)