ValueError: The provided metric AccuracyNovel for keeping the best model is not computed by the evaluation routine.

[vieozhu]

Can you give me some advice about this Error?

2019-07-31 08:51:00,403 - algorithms.Algorithm - INFO   - Training: miniImageNet_Conv32CosineClassifier
 20%|█▉        | 197/1000 [00:05<00:21, 37.46it/s]2019-07-31 08:51:06,434 - algorithms.Algorithm 
100%|██████████| 1000/1000 [00:27<00:00, 35.99it/s]
2019-07-31 08:51:28,193 - algorithms.Algorithm - INFO   - ==> Training stats: {'loss': 2.469}
2019-07-31 08:51:28,200 - algorithms.Algorithm - INFO   - Evaluating: miniImageNet_Conv32CosineClassifier
2019-07-31 08:51:28,200 - algorithms.Algorithm - INFO   - ==> Dataset: MiniImageNet_val [2000 batches]
  0%|          | 0/2000 [00:00<?, ?it/s]/data1/zjj/meta-code/withoutForgetting/algorithms/FewShot.py:185: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  images_test_var = Variable(images_test, volatile=is_volatile)
/data1/withoutForgetting/algorithms/FewShot.py:190: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  images_train_var = Variable(images_train, volatile=is_volatile)
100%|██████████| 2000/2000 [01:14<00:00, 26.36it/s]
2019-07-31 08:52:43,083 - algorithms.Algorithm - INFO   - ==> Results: {'loss': 3.1131, 'AccuracyNovel_cnf': 0.4092}
2019-07-31 08:52:43,084 - algorithms.Algorithm - INFO   - ==> Evaluation stats: {'loss': 3.1131, 'AccuracyNovel_cnf': 0.4092}
Traceback (most recent call last):
  File "train.py", line 110, in <module>
    algorithm.solve(dloader_train, dloader_test)
  File "/data1/withoutForgetting/algorithms/Algorithm.py", line 288, in solve
    self.keep_record_of_best_model(eval_stats, self.curr_epoch)
  File "/data1/withoutForgetting/algorithms/Algorithm.py", line 359, in keep_record_of_best_model
    .format(metric_name))
ValueError: The provided metric AccuracyNovel for keeping the best model is not computed by the evaluation routine.

The error occurred in：

    def keep_record_of_best_model(self, eval_stats, current_epoch):
        if self.keep_best_model_metric_name is not None:
            metric_name = self.keep_best_model_metric_name
            if (metric_name not in eval_stats):
                raise ValueError('The provided metric {0} for keeping the best '
                                 'model is not computed by the evaluation routine.'
                                 .format(metric_name))
            metric_val = eval_stats[metric_name]

Thanks a lot！

[Lan1991Xu]

change back to pytorch version==0.4

[Franklin-Yao]

This issue is caused by different versions of pytorch. Higher versions of Pytorch will return a tensor and lower version will return the type of float for this line:
https://github.com/gidariss/FewShotWithoutForgetting/blob/master/algorithms/FewShot.py#L20
Then in the update method(https://github.com/gidariss/FewShotWithoutForgetting/blob/master/utils.py#L154), AccuracyNovel will be ignored.

[xuxf1992]

This issue is caused by different versions of pytorch. Higher versions of Pytorch will return a tensor and lower version will return the type of float for this line:
https://github.com/gidariss/FewShotWithoutForgetting/blob/master/algorithms/FewShot.py#L20
Then in the update method(https://github.com/gidariss/FewShotWithoutForgetting/blob/master/utils.py#L154), AccuracyNovel will be ignored.

nice!!!

[ContestantsD]

This issue is caused by different versions of pytorch. Higher versions of Pytorch will return a tensor and lower version will return the type of float for this line: https://github.com/gidariss/FewShotWithoutForgetting/blob/master/algorithms/FewShot.py#L20 Then in the update method(https://github.com/gidariss/FewShotWithoutForgetting/blob/master/utils.py#L154), AccuracyNovel will be ignored.

Hi, bro, i use the same pytorch version like u, but i still get the error ,could you help me ?