Please refer to Rosetta ViT, NVIDIA's project that enables seamless training of LLMs, CV models and multimodal models in JAX, for information about running Vision Transformer models and experiments on GPUs.