roberta-base-coqa

Los siguientes hiperparámetros fueron utilizados durante el entrenamiento:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Adam con betas=(0.9,0.999) y epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.06
num_epochs: 10.0

Ejemplo de datos:

{
  "text": "Where do I live?",
  "context": "My name is Wolfgang and I live in Berlin"
},
{
  "text": "Where do I live?",
  "context": "My name is Sarah and I live in London"
},
{
  "text": "What's my name?",
  "context": "My name is Clara and I live in Berkeley."
},
{
  "text": "Which name is also used to describe the Amazon rainforest in English?",
  "context": "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America..."
}

Versiones de Framework utilizadas:

Transformers 4.21.3
Pytorch 1.7.1
Datasets 1.18.3
Tokenizers 0.11.6