commit files to HF hub

README.md

@@ -0,0 +1,103 @@
+
+---
+language: es
+tags:
+- summarization
+- spanish
+- encoder-decoder
+- beto
+license: apache-2.0
+datasets:
+- mlsum - es
+model-index:
+- name: beto2beto-mlsum
+  results:
+  - task: 
+      name: Abstractive Text Summarization
+      type: abstractive-text-summarization
+    dataset:
+      name: "MLSUM: MultiLingual SUMmarization dataset (Spanish)" 
+      type: mlsum
+    metrics:
+       - name: Validation ROGUE-1
+         type: rogue-1
+         value: 26.1256
+       - name: Validation ROGUE-2
+         type: rogue-2
+         value: 9.2552
+       - name: Validation ROGUE-L
+         type: rogue-l
+         value: 21.4899
+       - name: Validation ROGUE-Lsum
+         type: rogue-lsum
+         value: 21.8194
+       - name: Test ROGUE-1
+         type: rogue-1
+         value: 25.8639
+       - name: Test ROGUE-2
+         type: rogue-2
+         value: 8.911
+       - name: Test ROGUE-L
+         type: rogue-l
+         value: 21.2426
+       - name: Test ROGUE-Lsum
+         type: rogue-lsum
+         value: 21.5859
+widget:
+- text: | 
+    La chocotorta, el tradicional y práctico antojo dulce de los argentinos, fue elegida como el mejor postre del mundo por críticos de restaurants internacionales, a casi 40 años de su creación. El ránking Taste Atlas ubicó primero en su lista al postre insignia local de galletitas, queso crema y dulce de leche, por delante del helado de pistacho italiano y la tarta alemana de manzana. “Este postre argentino sin hornear fue influenciado por la cocina italiana y se inspiró en el famoso tiramisú italiano. Está elaborado con tres ingredientes básicos argentinos: galletas de chocolate, dulce de leche y queso crema”, explica la página web que exhorta a los turistas de todo el mundo a que prueben la chocotorta. En la votación, superó también a los waffles belgas y el zserbó húngaro. A nivel local le sigue el alfajor, con 4,2 puntos contra los 4,7 de la torta. En el texto que acompaña al listón dorado de “postre número uno", los expertos enseñan además cómo se hacen las chocotortas, paso por paso. “Las galletas se ablandan en leche y se cubren con una combinación de queso crema y dulce de leche. Las formas de la chocotorta pueden variar, mientras que las galletas se pueden remojar con leche con chocolate, café o incluso licor de café”, detallan. Por último, adjudican su creación a una “campaña de márketing” diseñada para promover las galletitas icónicas que le dan su nombre. La chocotorta, infaltable en los cumpleaños argentinos, fue creada en 1982 por una creativa de las agencias más importantes del país, Marité Mabragaña.
+
+---
+## beto2beto-mlsum
+This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
+For more information look at:
+- [🤗 Transformers Documentation: Amazon SageMaker](https://huggingface.co/transformers/sagemaker.html)
+- [Example Notebooks](https://github.com/huggingface/notebooks/tree/master/sagemaker)
+- [Amazon SageMaker documentation for Hugging Face](https://docs.aws.amazon.com/sagemaker/latest/dg/hugging-face.html)
+- [Python SDK SageMaker documentation for Hugging Face](https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/index.html)
+- [Deep Learning Container](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-training-containers)
+## Hyperparameters
+    {
+    "dataset_config": "es",
+    "dataset_name": "mlsum",
+    "do_eval": true,
+    "do_predict": true,
+    "do_train": true,
+    "fp16": true,
+    "max_target_length": 64,
+    "num_train_epochs": 10,
+    "per_device_eval_batch_size": 4,
+    "per_device_train_batch_size": 4,
+    "predict_with_generate": true,
+    "sagemaker_container_log_level": 20,
+    "sagemaker_program": "run_summarization.py",
+    "seed": 7,
+    "summary_column": "summary",
+    "text_column": "text"
+}
+## Usage
+## Results
+| key | value |
+| --- | ----- |
+| eval_loss | 2.5021677017211914 |
+| eval_rouge1 | 26.1256 |
+| eval_rouge2 | 9.2552 |
+| eval_rougeL | 21.4899 |
+| eval_rougeLsum | 21.8194 |
+| eval_gen_len | 19.2394 |
+| eval_runtime | 1276.236 |
+| eval_samples_per_second | 8.116 |
+| epoch | 10.0 |
+| eval_mem_cpu_alloc_delta | 270336 |
+| eval_mem_gpu_alloc_delta | 0 |
+| eval_mem_cpu_peaked_delta | 4534272 |
+| eval_mem_gpu_peaked_delta | 1424628736 |
+| eval_loss | 2.57672381401062 |
+| eval_rouge1 | 25.8639 |
+| eval_rouge2 | 8.911 |
+| eval_rougeL | 21.2426 |
+| eval_rougeLsum | 21.5859 |
+| eval_gen_len | 19.2463 |
+| eval_runtime | 1713.2216 |
+| eval_samples_per_second | 8.125 |
+| epoch | 10.0 |

config.json

@@ -0,0 +1,160 @@
+{
+  "_name_or_path": "LeoCordoba/int-model",
+  "architectures": [
+    "EncoderDecoderModel"
+  ],
+  "decoder": {
+    "_name_or_path": "dccuchile/bert-base-spanish-wwm-uncased",
+    "add_cross_attention": true,
+    "architectures": [
+      "BertForMaskedLM"
+    ],
+    "attention_probs_dropout_prob": 0.1,
+    "bad_words_ids": null,
+    "bos_token_id": null,
+    "chunk_size_feed_forward": 0,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "gradient_checkpointing": false,
+    "hidden_act": "gelu",
+    "hidden_dropout_prob": 0.1,
+    "hidden_size": 768,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "is_decoder": true,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-12,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "max_position_embeddings": 512,
+    "min_length": 0,
+    "model_type": "bert",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": 12,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 12,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_past": true,
+    "output_scores": false,
+    "pad_token_id": 1,
+    "position_embedding_type": "absolute",
+    "prefix": null,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "sep_token_id": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torchscript": false,
+    "transformers_version": "4.5.1",
+    "type_vocab_size": 2,
+    "use_bfloat16": false,
+    "use_cache": true,
+    "vocab_size": 31002
+  },
+  "decoder_start_token_id": 4,
+  "encoder": {
+    "_name_or_path": "dccuchile/bert-base-spanish-wwm-uncased",
+    "add_cross_attention": false,
+    "architectures": [
+      "BertForMaskedLM"
+    ],
+    "attention_probs_dropout_prob": 0.1,
+    "bad_words_ids": null,
+    "bos_token_id": null,
+    "chunk_size_feed_forward": 0,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "gradient_checkpointing": false,
+    "hidden_act": "gelu",
+    "hidden_dropout_prob": 0.1,
+    "hidden_size": 768,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-12,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "max_position_embeddings": 512,
+    "min_length": 0,
+    "model_type": "bert",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": 12,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 12,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_past": true,
+    "output_scores": false,
+    "pad_token_id": 1,
+    "position_embedding_type": "absolute",
+    "prefix": null,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "sep_token_id": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torchscript": false,
+    "transformers_version": "4.5.1",
+    "type_vocab_size": 2,
+    "use_bfloat16": false,
+    "use_cache": true,
+    "vocab_size": 31002
+  },
+  "eos_token_id": 5,
+  "is_encoder_decoder": true,
+  "model_type": "encoder-decoder",
+  "pad_token_id": 1,
+  "vocab_size": 31002
+}

eval_data.json

@@ -0,0 +1 @@
+{"eval_loss": 2.5021677017211914, "eval_rouge1": 26.1256, "eval_rouge2": 9.2552, "eval_rougeL": 21.4899, "eval_rougeLsum": 21.8194, "eval_gen_len": 19.2394, "eval_runtime": 1276.236, "eval_samples_per_second": 8.116, "epoch": 10.0, "eval_mem_cpu_alloc_delta": 270336, "eval_mem_gpu_alloc_delta": 0, "eval_mem_cpu_peaked_delta": 4534272, "eval_mem_gpu_peaked_delta": 1424628736}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d66d07aae8617248a17d8a931d272c159dd42362d7f5cbd7909bada68ecad87c
+size 992553531

test_data.json

@@ -0,0 +1 @@
+{"eval_loss": 2.57672381401062, "eval_rouge1": 25.8639, "eval_rouge2": 8.911, "eval_rougeL": 21.2426, "eval_rougeLsum": 21.5859, "eval_gen_len": 19.2463, "eval_runtime": 1713.2216, "eval_samples_per_second": 8.125, "epoch": 10.0}

trainer_state.json

@@ -0,0 +1,215 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.999519461797213,
+  "global_step": 7280,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.48,
+      "learning_rate": 4.656593406593407e-05,
+      "loss": 1.5943,
+      "step": 500
+    },
+    {
+      "epoch": 0.96,
+      "learning_rate": 4.3131868131868134e-05,
+      "loss": 1.6732,
+      "step": 1000
+    },
+    {
+      "epoch": 1.0,
+      "eval_gen_len": 19.2321,
+      "eval_loss": 2.4691147804260254,
+      "eval_rouge1": 25.8418,
+      "eval_rouge2": 8.7011,
+      "eval_rougeL": 21.1569,
+      "eval_rougeLsum": 21.4706,
+      "eval_runtime": 1188.0405,
+      "eval_samples_per_second": 8.719,
+      "step": 1040
+    },
+    {
+      "epoch": 1.44,
+      "learning_rate": 3.96978021978022e-05,
+      "loss": 1.9331,
+      "step": 1500
+    },
+    {
+      "epoch": 1.92,
+      "learning_rate": 3.6263736263736266e-05,
+      "loss": 1.9543,
+      "step": 2000
+    },
+    {
+      "epoch": 2.0,
+      "eval_gen_len": 19.2853,
+      "eval_loss": 2.4091384410858154,
+      "eval_rouge1": 26.1697,
+      "eval_rouge2": 9.222,
+      "eval_rougeL": 21.5166,
+      "eval_rougeLsum": 21.8758,
+      "eval_runtime": 1168.6884,
+      "eval_samples_per_second": 8.863,
+      "step": 2080
+    },
+    {
+      "epoch": 2.4,
+      "learning_rate": 3.282967032967033e-05,
+      "loss": 1.8371,
+      "step": 2500
+    },
+    {
+      "epoch": 2.88,
+      "learning_rate": 2.9395604395604398e-05,
+      "loss": 1.8282,
+      "step": 3000
+    },
+    {
+      "epoch": 3.0,
+      "eval_gen_len": 19.2576,
+      "eval_loss": 2.42261004447937,
+      "eval_rouge1": 26.1165,
+      "eval_rouge2": 9.2438,
+      "eval_rougeL": 21.5311,
+      "eval_rougeLsum": 21.8577,
+      "eval_runtime": 1236.3561,
+      "eval_samples_per_second": 8.378,
+      "step": 3120
+    },
+    {
+      "epoch": 3.37,
+      "learning_rate": 2.5961538461538464e-05,
+      "loss": 1.7419,
+      "step": 3500
+    },
+    {
+      "epoch": 3.85,
+      "learning_rate": 2.252747252747253e-05,
+      "loss": 1.7287,
+      "step": 4000
+    },
+    {
+      "epoch": 4.0,
+      "eval_gen_len": 19.2905,
+      "eval_loss": 2.4440300464630127,
+      "eval_rouge1": 26.2529,
+      "eval_rouge2": 9.3031,
+      "eval_rougeL": 21.5955,
+      "eval_rougeLsum": 21.9465,
+      "eval_runtime": 1217.0172,
+      "eval_samples_per_second": 8.511,
+      "step": 4160
+    },
+    {
+      "epoch": 4.33,
+      "learning_rate": 1.9093406593406592e-05,
+      "loss": 1.6642,
+      "step": 4500
+    },
+    {
+      "epoch": 4.81,
+      "learning_rate": 1.565934065934066e-05,
+      "loss": 1.6481,
+      "step": 5000
+    },
+    {
+      "epoch": 5.0,
+      "eval_gen_len": 19.1729,
+      "eval_loss": 2.471247673034668,
+      "eval_rouge1": 26.1789,
+      "eval_rouge2": 9.2847,
+      "eval_rougeL": 21.5357,
+      "eval_rougeLsum": 21.8938,
+      "eval_runtime": 1247.3952,
+      "eval_samples_per_second": 8.304,
+      "step": 5200
+    },
+    {
+      "epoch": 5.29,
+      "learning_rate": 1.2225274725274726e-05,
+      "loss": 1.6071,
+      "step": 5500
+    },
+    {
+      "epoch": 5.77,
+      "learning_rate": 8.791208791208792e-06,
+      "loss": 1.5796,
+      "step": 6000
+    },
+    {
+      "epoch": 6.0,
+      "eval_gen_len": 19.2078,
+      "eval_loss": 2.485071897506714,
+      "eval_rouge1": 26.07,
+      "eval_rouge2": 9.1429,
+      "eval_rougeL": 21.414,
+      "eval_rougeLsum": 21.7483,
+      "eval_runtime": 1301.5565,
+      "eval_samples_per_second": 7.958,
+      "step": 6240
+    },
+    {
+      "epoch": 6.25,
+      "learning_rate": 5.357142857142857e-06,
+      "loss": 1.5558,
+      "step": 6500
+    },
+    {
+      "epoch": 6.73,
+      "learning_rate": 1.9230769230769234e-06,
+      "loss": 1.5319,
+      "step": 7000
+    },
+    {
+      "epoch": 7.0,
+      "eval_gen_len": 19.2394,
+      "eval_loss": 2.5021677017211914,
+      "eval_rouge1": 26.1256,
+      "eval_rouge2": 9.2552,
+      "eval_rougeL": 21.4899,
+      "eval_rougeLsum": 21.8194,
+      "eval_runtime": 1250.4536,
+      "eval_samples_per_second": 8.283,
+      "step": 7280
+    },
+    {
+      "epoch": 7.0,
+      "step": 7280,
+      "total_flos": 1.5986426003154386e+18,
+      "train_runtime": 137981.946,
+      "train_samples_per_second": 0.053
+    },
+    {
+      "epoch": 7.0,
+      "eval_gen_len": 19.2394,
+      "eval_loss": 2.5021677017211914,
+      "eval_rouge1": 26.1256,
+      "eval_rouge2": 9.2552,
+      "eval_rougeL": 21.4899,
+      "eval_rougeLsum": 21.8194,
+      "eval_runtime": 1276.236,
+      "eval_samples_per_second": 8.116,
+      "step": 7280
+    },
+    {
+      "epoch": 7.0,
+      "eval_gen_len": 19.2463,
+      "eval_loss": 2.57672381401062,
+      "eval_rouge1": 25.8639,
+      "eval_rouge2": 8.911,
+      "eval_rougeL": 21.2426,
+      "eval_rougeLsum": 21.5859,
+      "eval_runtime": 1713.2216,
+      "eval_samples_per_second": 8.125,
+      "step": 7280
+    }
+  ],
+  "max_steps": 7280,
+  "num_train_epochs": 7,
+  "total_flos": 1.5986426003154386e+18,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90dff6dcad2ecaa16e932c89b750519a537b76cf7b09f64406a691cc86994327
+size 2139