StarCoder2? LLM ?? ??? ????

Reading Time: 4 minutes

??? ??? ??? ?????? ???? ??? ?? ??? ? ????. ??? ??? ?? ???? ?? ?? ??(LLM)? ??? ?? ?? ??? ????? ????? ??? ?? ????. ??? ??? ????? ??? GitHub ?????? ??? ?? ??? ???? ??? ??? ?? ?? ??? ??, ?? ? ???? ? ????.

? ?????? ?? ?? ????? ??? ???? ?? ??? ???? ???? ??? ???? ??? StarCoder2? ?? ?? LLM? ?? ?? ??? ?????. ?? ?? NVIDIA AI ????? ?? ? ?????? ???? ??? ??? ?? ??? ??? ?? ??? ???????? ????? ??? ???.

StarCoder2

BigCode? NVIDIA? ???? ??? StarCoder2? ???? ?? ?? ??? ?? LLM???. ?? ??, ?? ???, ?? ?? ??, ???? ??? ?? ?? ??? ?? ? ? ??? ??? ???? ??????? ??? ??? ? ????.

StarCoder2 ????? 3B, 7B ? 15B ???? ??? ???? ?? ?? ??? ???? ??? ???? ???? ??? ???? ??? ? ????. ? ?????? 15B ??? ??? ?? ????.

??

15B ??? ?? ????? ?????? ?? ?? ?? LLM?? ??? ???? ?? ??? ??? ?????. ??? ???? ????? ???? 30%???. ?? ????2? ??? ?? ??? ?????? ??? ?? ??? ????? ?????? ??????? ?????.

*?? 1. HumanEval ?????? ??? ???? ???? StarCoder2 15B*

16,000? ??? ???? ??? ?? Starcoder ??? ? ? ?? ???? ??? ?? ??? ???? ?? ??? ? ? ???? ??? ?? ???? ??? ? ????.

??? ?? ??? ??? ?? ??

??? ??? ?? ??? ???? ????, GitHub?? ??? ???? 1? ?? ??? ?????. ???? 600? ??? ????? ??, Git ??, GitHub ??, Jupyter Notebook? ?????. ? ??? ??, ??, ??? ??? ???? ??? ?? ??? ?????. ?? ??? ??? ??? ???? ???? ??? ??? ? ????.

StarCoder2 ??? BigCode Open RAIL-M ????? ?? ????? ?? ????? ??? ?? ??? ? ??? ??? ?? ?? ? ??? ??? ???? ????? ???? ? ????.

StarCoder2 ????

StarCoder2? ??? ???? ? NVIDIA? ??? ??? AI ??? ?? ???? ???? ?????? ???????? ??, ?????? ? ??? ? ??? NVIDIA AI Foundation Models and Endpoints? ??? ?????.

StarCoder2? NVIDIA AI ????????? ??? ? ???, Nemotron-3, Mixtral 8X7B, Llama 70B ? Stable Diffusion? ?? ?? ?? ??? ??? ? ????.

? ???? NVIDIA NeMo? ?? ??????? ? ??? .nemo ???? ????, NVIDIA TensorRT-LLM?? ??? ????? ????.

TensorRT-LLM?? ?? ?????

NVIDIA? ??? ?? ?? ?? ??? ??, ??? ? ???? ?? ?? ?? ?????? TensorRT-LLM?? ??? ???????. ?? ?? ?? ? ? ?? ???? ? ?? ?? ??? ???? ??? ????? ??? ??? ??? ? ????.

??? ?? ??? ??? ??? ???? ?? ????, ?? ?? ?? ? ????? ?? ??? ?? ?? ?? ?? ??, ????? ??, ??? ?? ?? ???????. ?? ??? ??? ??? ??? ??? ????? TensorRT-LLM GitHub? ?????.

??? ??? ?????? ?? ?? ??

?? NGC ????? ??? ??????? ??? ?????? ???? ?????? ?? StarCoder2? ??? ? ????. ??? ???? ???? ???? ???? ??? ??? ??? ? ????.

*?? 2. ??? ?????? ??? ???? StarCoder2 ??? ??*

API ??

API? ???? ??? ????? ??? ??? ?? ???. NGC ????? ????? NVIDIA ???? ???? ???? ? ????. ? ???? ???? ??????? API ?????? ???? ???? ??? ??? ? ????.

StarCoder2 AI Playground?? ?????? REST ??? ??? ? ?? ?? ?? ?? ?????? ??? ? ????. ?? ????? ?? ?????? ?? Python? ?????. ?? ???? ?? ??? ???? ?? Python ??? ??? ? ?? ??? ??? ?????.

NGC ???? API ? ??

API ??? ? ??? ?????. ???? ?? ?? ?? ?? ?????? ???? ?????.

???? API ?? ?????:

# Will be used to issue requests to the endpoint 
API_KEY = “nvapi-xxxx“

?? ?? ???

StarCoder2? ?? ????? ???? ????? ??? ???? ?? ? ?? ???? ???? ???? ???? ? ????.

?? ????

import requests

invoke_url = "https://api.nvcf.nvidia.com/v2/nvcf/pexec/functions/835ffbbf-4023-4cef-8f48-09cb5baabe5f"
fetch_url_format = "https://api.nvcf.nvidia.com/v2/nvcf/pexec/status/"

headers = {
    "Authorization": "Bearer {}".format(API_KEY),
    "Accept": "application/json",
}

payload = {
  "prompt": "X_train, y_train, X_test, y_test = train_test_split(X, y, test_size=0.1) #Train a logistic regression model, predict the labels on the test set and compute the accuracy score",
  "temperature": 0.1,
  "top_p": 0.7,
  "max_tokens": 512,
  "seed": 42,
  "stream": False
}

# re-use connections
session = requests.Session()

response = session.post(invoke_url, headers=headers, json=payload)

while response.status_code == 202:
    request_id = response.headers.get("NVCF-REQID")
    fetch_url = fetch_url_format + request_id
    response = session.get(fetch_url, headers=headers)

response.raise_for_status()
response_body = response.json()

# The response body contains additional metadata along with completion text. Visualizing just the completion.
print(response_body['choices'][0]['text'])

? ???? StarCoder2? ???? ?? ???? ?? ??? ???? ??? ??? ?? ???? ???? Python ??? ?????.

?? ??? ?? ? ??

?????. ???? ??? ??? ??? ???? ?? ????. ??? ??? ?? ??? ??? ? ??? ??? ? ??? ?? ??? ??? ????? ???. NVIDIA? NeMo? ?? ??? ?? ??????? ? ??? ?????.

??? ?? .nemo ???? ???? ???? LLM? ?? ???? ??? ????, RLHF? ?? ?? ?? ?????? ??, ???? ??? ? ?? ?????? ????? ? ??? NeMo ??? ??? ? ????.

? ????? ???? ??? ?? ??(PEFT) ??? ???? ? ??? ???????? ??? ??? ? ????.

???? ?? ?? ??

NVIDIA Triton Inference Server? ???? ???? AI ?? ????? ??? ????? ????? ?? ?? AI ?? ?? ??????. ?? ?? ??? ?? ?? ???? ???? ???, ??? AI ??? ????? ???? ? ??? ??? ????, AI ?? ? ?? ??? ?? ? ????.

NVIDIA Triton ?? ??? ??????? ??, ??, ??? ? ?? ???? ?? NVIDIA AI ??????? ?????. Triton ?? ??? ???? StarCoder2 ??? ????? ?? ?? CSP? ??? ? ????.

? ?????? TensorRT-LLM? ???? ????? ??? Triton Inference Server? ???? ??? ???? ??? ??? ?????.

??????? AI ?????? ?? ??

???? ??? ?? AI ??? ??? ??? ??? ? ??, ??? ? ?????? ??? ?? ?????.

?? ??????? ??? AI? ???? ????? ????? ???? NVIDIA AI Enterprise? ??? AI ?? ??? ?? ?? ??? ???? ???? ?????. ???? ??????? ??, ?? ? ???? ?? AI ?????, ????? ? ??? ???? ?? ??????? ?????? ??? ?? ???? ??? ? ????.

????

??? ????? ?? API? ?? StarCoder2 ??? ??? ?? ??????? ???? ??? ?, TensorRT-LLM?? ??? ????? NVIDIA NeMo? ?????????.

?????? ??????? ???? ??, NVIDIA AI Enterprise? ?? ??? ?????? ???? ??????? ?????? ???? ?? ?????? ?????? ??? ??? ? ????.

?? ???