Using non glove embeddings giving error: RuntimeError: The expanded size of the tensor (2048) must match the existing size (2148) at non-singleton dimension 1

[xijianlim]

Im using a SequenceTagger model but everytime I use word embeddings other than 'glove' im getting the following error:

RuntimeError: The expanded size of the tensor (2048) must match the existing size (2148) at non-singleton dimension 1. Target sizes: [9, 2048]. Tensor sizes: [9, 2148]

Ive tried this using stacked and non-stacked embeddings:

from flair.models import SequenceTagger

tagger: SequenceTagger = SequenceTagger(hidden_size=500,
                                        embeddings=XLNetEmbeddings(),
                                        tag_dictionary=tag,
                                        tag_type='ner',
                                        use_crf=True)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus_train)

# 7. start training
trainer.train(model_path,
              learning_rate=0.1,
              mini_batch_size=128,
              max_epochs=1)

Ive also tried the suggested stacked embeddings as per the documents and have the same kind of error:

embedding_types: List[TokenEmbeddings] = [

    WordEmbeddings('glove'),

    # comment in this line to use character embeddings
    CharacterEmbeddings(),

    # comment in these lines to use flair embeddings
    FlairEmbeddings('news-forward'),
    FlairEmbeddings('news-backward'),
]

embeddings: StackedEmbeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger: SequenceTagger = SequenceTagger(hidden_size=256,
                                        embeddings=embeddings,
                                        tag_dictionary=tag,
                                        tag_type='ner',
                                        use_crf=True)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus_train)

# 7. start training
trainer.train(model_path,
              learning_rate=0.1,
              mini_batch_size=32,
              max_epochs=1)

Is there

[apohllo]

I had the same error due to name clash in the stacked embeddings. I had two embeddings of the same type computed for different fields in the input. This caused:

The expanded size of the tensor (xxx) must match the existing size (xxx) at non-singleton dimension 1

As a workaround I added a name option to the embedding classes, which allows me to set the name explicitly. Yet, there might be a better solution, where the StackedEmbeddings class takes care of the naming of the embeddings. E.g. a number prefix could be used to distinguish between the different "copies" of given embedding.

[djstrong]

It is fixed on master with commit: e1393f6#diff-9b198b0f5b0c7ad771c9239623c9ad8c

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.