Upload folder using huggingface_hub

README.md

@@ -0,0 +1,151 @@
+---
+language: code
+thumbnail: https://cdn-media.huggingface.co/CodeBERTa/CodeBERTa.png
+datasets:
+- code_search_net
+library_name: transformers.js
+base_model:
+- huggingface/CodeBERTa-small-v1
+pipeline_tag: fill-mask
+---
+
+
+
+# CodeBERTa-small-v1 (ONNX)
+
+
+This is an ONNX version of [huggingface/CodeBERTa-small-v1](https://huggingface.co/huggingface/CodeBERTa-small-v1). It was automatically converted and uploaded using [this Hugging Face Space](https://huggingface.co/spaces/onnx-community/convert-to-onnx).
+
+
+## Usage with Transformers.js
+
+
+See the pipeline documentation for `fill-mask`: https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.FillMaskPipeline
+
+
+---
+
+
+# CodeBERTa
+
+CodeBERTa is a RoBERTa-like model trained on the [CodeSearchNet](https://github.blog/2019-09-26-introducing-the-codesearchnet-challenge/) dataset from GitHub.
+
+Supported languages:
+
+```shell
+"go"
+"java"
+"javascript"
+"php"
+"python"
+"ruby"
+```
+
+The **tokenizer** is a Byte-level BPE tokenizer trained on the corpus using Hugging Face `tokenizers`.
+
+Because it is trained on a corpus of code (vs. natural language), it encodes the corpus efficiently (the sequences are between 33% to 50% shorter, compared to the same corpus tokenized by gpt2/roberta).
+
+The (small) **model** is a 6-layer, 84M parameters, RoBERTa-like Transformer model – that’s the same number of layers & heads as DistilBERT – initialized from the default initialization settings and trained from scratch on the full corpus (~2M functions) for 5 epochs.
+
+### Tensorboard for this training ⤵️
+
+[![tb](https://cdn-media.huggingface.co/CodeBERTa/tensorboard.png)](https://tensorboard.dev/experiment/irRI7jXGQlqmlxXS0I07ew/#scalars)
+
+## Quick start: masked language modeling prediction
+
+```python
+PHP_CODE = """
+public static <mask> set(string $key, $value) {
+	if (!in_array($key, self::$allowedKeys)) {
+		throw new \InvalidArgumentException('Invalid key given');
+	}
+	self::$storedValues[$key] = $value;
+}
+""".lstrip()
+```
+
+### Does the model know how to complete simple PHP code?
+
+```python
+from transformers import pipeline
+
+fill_mask = pipeline(
+    "fill-mask",
+    model="huggingface/CodeBERTa-small-v1",
+    tokenizer="huggingface/CodeBERTa-small-v1"
+)
+
+fill_mask(PHP_CODE)
+
+## Top 5 predictions:
+# 
+' function' # prob 0.9999827146530151
+'function'  # 
+' void'     # 
+' def'      # 
+' final'    # 
+```
+
+### Yes! That was easy 🎉 What about some Python (warning: this is going to be meta)
+
+```python
+PYTHON_CODE = """
+def pipeline(
+    task: str,
+    model: Optional = None,
+    framework: Optional[<mask>] = None,
+    **kwargs
+) -> Pipeline:
+	pass
+""".lstrip()
+```
+
+Results:
+```python
+'framework', 'Framework', ' framework', 'None', 'str'
+```
+
+> This program can auto-complete itself! 😱
+
+### Just for fun, let's try to mask natural language (not code):
+
+```python
+fill_mask("My name is <mask>.")
+
+# {'sequence': '<s> My name is undefined.</s>', 'score': 0.2548016905784607, 'token': 3353}
+# {'sequence': '<s> My name is required.</s>', 'score': 0.07290805131196976, 'token': 2371}
+# {'sequence': '<s> My name is null.</s>', 'score': 0.06323737651109695, 'token': 469}
+# {'sequence': '<s> My name is name.</s>', 'score': 0.021919190883636475, 'token': 652}
+# {'sequence': '<s> My name is disabled.</s>', 'score': 0.019681859761476517, 'token': 7434}
+```
+
+This (kind of) works because code contains comments (which contain natural language).
+
+Of course, the most frequent name for a Computer scientist must be undefined 🤓.
+
+
+## Downstream task: [programming language identification](https://huggingface.co/huggingface/CodeBERTa-language-id)
+
+See the model card for **[`huggingface/CodeBERTa-language-id`](https://huggingface.co/huggingface/CodeBERTa-language-id)** 🤯.
+
+<br>
+
+## CodeSearchNet citation
+
+<details>
+
+```bibtex
+@article{husain_codesearchnet_2019,
+	title = {{CodeSearchNet} {Challenge}: {Evaluating} the {State} of {Semantic} {Code} {Search}},
+	shorttitle = {{CodeSearchNet} {Challenge}},
+	url = {http://arxiv.org/abs/1909.09436},
+	urldate = {2020-03-12},
+	journal = {arXiv:1909.09436 [cs, stat]},
+	author = {Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc},
+	month = sep,
+	year = {2019},
+	note = {arXiv: 1909.09436},
+}
+```
+
+</details>

config.json

@@ -0,0 +1,28 @@
+{
+  "_attn_implementation_autoset": true,
+  "_name_or_path": "huggingface/CodeBERTa-small-v1",
+  "architectures": [
+    "RobertaForMaskedLM"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 52000
+}

onnx/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab0701ac96e4778d1574c2a25ab2d7ad6b3359c01369bdf335a53a218c3e9250
+size 334190701

onnx/model_bnb4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:512cd7f47b72b4e25c5efb284792660f149f1462dcbae3d2532e459a43ad5656
+size 186187506

onnx/model_fp16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c52d1653d684dfbde3cc9224d0fd631eb49037e8deaa0698e01458350f39f525
+size 167182219

onnx/model_int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:985639ea19b9cfca4690fac46ea49e2cc0fd8195ba6c8afad02e2820c3f56d28
+size 84286447

onnx/model_q4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8049218fb07a102fcdc9fe6ba6381ac6ca1bba2c051d3ae9b6bde428ef4dcede
+size 188878307

onnx/model_q4f16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6fb9a2af6fd4b868d2dd5d4d439041e781996c4dfa239021fa7a4547f59bc98
+size 105293108

onnx/model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:985639ea19b9cfca4690fac46ea49e2cc0fd8195ba6c8afad02e2820c3f56d28
+size 84286447

onnx/model_uint8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e20be90b446bce8ac59e68d2612e5840869c6604cf3d20497d7e2e4d44b9edfd
+size 84286447

quantize_config.json

@@ -0,0 +1,18 @@
+{
+    "modes": [
+        "fp16",
+        "q8",
+        "int8",
+        "uint8",
+        "q4",
+        "q4f16",
+        "bnb4"
+    ],
+    "per_channel": true,
+    "reduce_range": true,
+    "block_size": null,
+    "is_symmetric": true,
+    "accuracy_level": null,
+    "quant_type": 1,
+    "op_block_list": null
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,59 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "max_len": 512,
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<unk>"
+}