The paper presents Transformer Neural Autoregressive Flows (T-NAFs), a novel solution to challenges in Normalizing Flows (NFs). NFs are a sequence of invertible transformations used in machine learning for density estimation. The most efficient members of the NF family are Neural Autoregressive Flows (NAFs) and Block Neural Autoregressive Flows (B-NAFs), but they face scalability issues and training instability. T-NAFs use transformers to treat each dimension of a random variable as a separate input token. The experimental results demonstrate that T-NAFs consistently match or outperform NAFs and B-NAFs across multiple datasets. They achieve these results using significantly fewer parameters than previous approaches.
Publication date: 3 Jan 2024
Project Page: https://arxiv.org/abs/2401.01855
Paper: https://arxiv.org/pdf/2401.01855
