Jet constituents provide a more complete description of the radiation pattern inside a jet than observables describing the global jet properties like the number of tracks or the jet width. An approach to tag quark- and gluon-initiated jets using their constituents (Particle Flow Objects) reconstructed in ATLAS is presented. Monte Carlo simulations of proton–proton collisions at center of mass energy of 13 TeV are used. Transformer architectures are used to learn long-range dependencies between jet constituent kinematic variables. Several variations of transformers are studied, and their performance is compared to taggers based on global jet properties and other jet constituent taggers. A new Deep Learning architecture (DeParT) is presented, slightly outperforming other networks. All models are evaluated on simulated data from multiple Monte Carlo generators to study their model-dependence.