I have a bug about class Confusion.add()

[oYoungCo]

Thank you for your work, i'm interested in your work. Then, I run this project on SearchSnippets, but when I get pred_labels and true_labels then to execute confusion.add(pred_labels, true_labels), I get a problem:self.conf_flat.index_add(0, indices, ones) {IndexError}index out of range in self. The two input parameters:output, target about add() are output={Tensor:(12195,)}tensor([6, 6, 7, ..., 0, 0, 0], device='cuda:0'), target={Tensor:(12195,)}tensor([1, 1, 1, ..., 8, 8, 8]), respectively. I am confused about this problem and i have no idea about the role of add(). Could you please give me some advice?

[l770943527]

I have the same error, I guess this is for calculating the acc of cluster and ground-truth label, here is a link may be helpful https://smorbieu.gitlab.io/accuracy-from-classification-to-clustering-evaluation/

[oYoungCo]

@l770943527 Thanks for your help. I will try it. I have a question: can the method proposed in this paper be used for incremental clustering?

[MrRace]

@Dejiao2018 I use searchsnippets.csv to run the project, but when run to this line in utils/metric.py:
self.conf_flat.index_add(0, indices, ones)
It comes error:

  File "main.py", line 95, in <module>
    run(get_args(sys.argv[1:]))
  File "main.py", line 45, in run
    cluster_centers = get_kmeans_centers(sbert, train_loader, args.num_classes)
  File "/home/nlp_clustering/sccl/utils/kmeans.py", line 43, in get_kmeans_centers
    confusion.add(pred_labels, true_labels)
  File "/home/nlp_clustering/sccl/utils/metric.py", line 101, in add
    self._conf_flat.index_add_(0, indices, ones)
IndexError: index out of range in self

[MrRace]

@l770943527 Thanks for your help. I will try it. I have a question: can the method proposed in this paper be used for incremental clustering?

Same error, have you fix it out?

[remember00000]

Same error, have you fix it out?

[ScarletYarn]

I also have the same error :(
Is this caused by the data I'm using? I'm using searchsnippets downloaded from Kaggle.
Do u guys get some data that can be used?

[336655asd]

There is nothing to datasets, it's just a bug. I read all the codes and find that metrics.py has many problems. You just need to write your own evaluation code.

[urospet]

Same error, have you fix it out?

[urospet]

I used my own accuracy method, here is a working copy : https://www.kaggle.com/urospetricevic/sccl-reproduction

[oYoungCo]

There is nothing to datasets, it's just a bug. I read all the codes and find that metrics.py has many problems. You just need to write your own evaluation code.

yes, you are right.