Image-Text Retrieval via Contrastive Learning with Auxiliary Generative Features and Support-set Regularization

Model framework

Requirement

h5py==3.2.1
numpy==1.20.3
scipy==1.6.2
torch==1.4.0
torchvision==0.5.0
tqdm==4.61.2

File Description

• main.py, python file used to train student model;
• main_binary.py python file used to train binary model;
• scripts/, training scripts;
• src/ , code used to train model;
• utils/ , code used for pre-process;
• evaluate/ , code used for evaluating;
• final_results , store the evaluate results for 3 datasets;
• assests/ , assets files for README;

Usage

1. run scprits/{dataset_name}_run.sh to train student model;
2. run scripts/{dataset_name}_run_binary.sh to train binary model;
3. run evaluate/evaluate_{dataset_name}.py to obtain the final results.

Data

• Processed data will soon be available at Google Drive;
• Original data are from:
  • MS COCO;
  • MIRFLICKR-25K;
  • NUS-WIDE;

Checkpoints

• Model checkpoints could be downloaded from Google Drive;