Does llama only use decoders? Why don't you use a more efficient method?

[gyunggyung]

Thanks for sharing this really good material. I have a lot of questions.

First, I'd like to say that I hope you ignore much of the mockery. Everyone, including me, is a bunch of people who do crappy work and scream at their keyboards compared to you.

1. The model seems to only use decoders, why?

# https://github.com/facebookresearch/llama/blob/main/llama/model.py#L223
    def forward(self, tokens: torch.Tensor, start_pos: int):
        _bsz, seqlen = tokens.shape
        h = self.tok_embeddings(tokens)
        self.freqs_cis = self.freqs_cis.to(h.device)
        freqs_cis = self.freqs_cis[start_pos : start_pos + seqlen]

        mask = None
        if seqlen > 1:
            mask = torch.full((1, 1, seqlen, seqlen), float("-inf"), device=tokens.device)
            mask = torch.triu(mask, diagonal=start_pos + 1).type_as(h)

        for layer in self.layers:
            h = layer(h, start_pos, freqs_cis, mask)
        h = self.norm(h)
        output = self.output(h[:, -1, :])  # only compute last logits
        return output.float()

2. Is RMS the best way to go? I like the simplicity of it, but I'm curious.
3. For some tasks, compared to your model, Minerva outperforms. why? Is it just the one in the paper?
4. Why isn't the structure of your model described in the paper?
5. By any chance, what structure do you have in mind for your next model?
6. Amazon, Deepmind, and other great companies are showing that the encoder decoder structure is much better. Why do you guys only use decoders?
7. What model would you apply to Facebook, Instagram, Snapchat, etc.?
8. What do you think is your advantage over Bart or Prometheus? Especially over Bart, I don't know what it is, except full disclosure.
9. I sent an application to write the model. When will I be able to use it? I don't see a clear advantage yet.
10. What do you think of the derivative models that people have created? They are emerging very quickly.

Thank you so much. Your competition amuses me. I hope more companies continue to open up their models.

But I don't know why Yann LeCun was left out of the paper.

[gyunggyung]

@likethesky @Celebio @colesbury @pdollar
Can you answer?

[jspisak]

assigning to @AurRod for followup - these are some good questions but a bit deep for GitHub..

[jspisak]

Closing for now but feel free to open if needed.