Is there a confusion matrix in flair for evaluating relation extraction models?

[geheim01]

We are currently evaluating our relation extraction model.
We achieve an accuracy of 28% with 47 relation classes that we use, many of which are not really selective.

Now, in order to determine which relations might be better merged, we would like to output a confusion matrix. Unfortunately, I could not find a way in Flair Training to do this.

Is there a method that can do this that I just haven't found yet?

[grinay]

@geheim01 did you found answer to your question?

[geheim01]

@grinay I managed to build my own confusion matrix based on the output of the training test process in the test.tsv file.

[alanakbik]

@geheim01 we have a new relation extractor that should give better accuracy on this branch if you like to test it: #2748