The paper ‘Jack of All Trades, Master of Some’ introduces a transformer-based model, JAT, optimized for handling multi-modal data types and sequential decision-making tasks. The model demonstrates its versatility by performing well on different Reinforcement Learning benchmarks, as well as on Computer Vision and Natural Language Processing tasks. Importantly, JAT uses a single set of weights for all tasks, marking a significant step towards a more general, cross-domain AI model design. The JAT model is also the first of its kind to be fully open-sourced.
Publication date: 16 Feb 2024
Project Page: https://huggingface.co/jat-project/jat
Paper: https://arxiv.org/pdf/2402.09844
