Additional resources

This release collects some additional resources relevant to the paper.

• Pre-trained model on training split

This .pth file contains the weights of a model trained on the training set only. The performance is slightly better than the mean results reported in the paper.

python eval-acc.py logs/pretrained-on-train.pth returns

number (single)	: 49.59% +- nan
number (pair)	: 23.37% +- nan
count (single)	: 57.29% +- nan
count (pair)	: 26.96% +- nan
all (single)	: 65.42% +- nan
all (pair)	: 37.25% +- nan

Use the --resume <path> command-line option for train.py to load it.

• Pre-trained model on trainval split

This .pth file contains the weights of a model trained on both training and validation set, ready for evaluation on the VQA evaluation server. The performance of this is basically the same on test-dev (I didn't check test-standard) as the results in the paper.
Using python train.py --test --resume logs/pretrained-on-trainval.pth to generate a results.json file and uploading this to the evaluation server test-dev split, this gives us:

test-dev
yes/no	83.22
number	51.51
other	58.87
overall	68.07

• Poster
  Presented at ICLR 2018

• results.json
  This is the submission file that was used in the paper containing answers to the VQA test questions. When uploaded to the test server, this gives the following results:

test-dev
yes/no	83.14
number	51.62
other	58.97
overall	68.09

test-standard
yes/no	83.56
number	51.39
other	59.11
overall	68.41