- Follow INSTALL.md for all dependencies
- Download the datasets and unpack them (see ./tools/unpack_*.sh)
- Re-code the videos if necessary(see ./tools/recode_videos.sh)
- Convert the datasets into our format (see ./tools/convert_dataset.py).
- Download pre-trained models (see this link)
- Try to train on EPIC-Kitchens. You will need to set up the dataset folders and pre-trained model file in the json config
python ./train_joint.py ./exp_configs/epic-kitchens/epic-kitchens-action_trainval_i3d_csn152_vae.json --prec_bn- [Optional]: Distributed training on multiple GPUs using synchronized batch norm. You can use the same config file. Learning rate will be automatically re-scaled.
python -m torch.distributed.launch --nproc_per_node=NUM_GPUS train.py ./exp_configs/epic-kitchens/epic-kitchens-action_trainval_i3d_csn152_vae.json -p 5 --prec_bn --sync_bn- Evaluate a trained model. Models are saved in ./ckpt/your_exp/.
python ./eval_joint.py ./exp_configs/epic-kitchens/epic-kitchens-action_trainval_i3d_csn152_vae.json --resume your_model- Visualize the training (requires tensorboard)
tensorboard --logdir ./ckpt/your_exp/logs