NVIDIA NIM?? ??? AI? ???? ?? ?? ???

??????? ???? ?????? ???? NVIDIA NIM ?? ???????? ?????? ????? ????, ??? ? NVIDIA?? ???? ???? AI ??? ?? ??? ? ??? ?????. NVIDIA AI Enterprise? ??? NIM? ???? ???? ??? ???? ??? ???? ?? ?? ??? ??? AI ???? ?? ??? ??? ? ??? ?????.

???? ?? ????? ???? ???? ?? ?????? ???? NVIDIA GPU ????? ?????? ? PC? 5? ??? NIM? ?? ??? ? ????. ?? ????? ???? ???? ?? NVIDIA API ?????? NIM API? ???? ?????? ??????? ??? ? ????.

• ???? NVIDIA ?? ???? ?? ? ??? ??? ? ?? ?? ??? ????? ?????.
• ?? ??? ?????? ???? ???? ??? ??? ?????.
• LoRA? ?? ??? ???? ?? ??? ??? ???? ??? ???? ??? ? ????.
• ??? ?? ?? API? ???? ???? AI ?? ?????? ?????.
• LangChain, LlamaIndex, Haystack ? ?? ?? ?? ??? AI ?????? ?????? ?? ?????.

? ?????? NVIDIA NIM?? ?? ??? Docker ?? ??? ??? ?????. Haystack,?LangChain, ?LlamaIndex? ?? ?? ?? ?? ??? AI ?????? ??????? NIM ??????? API? ??? ? ????. NIM ??? ?? ?? ???? NIM ???? ?????.

5? ?? NIM? ???? ??

???? ?? ?? ?? ??? ?????? ???? NIM ???? ?? ??? ?????. NIM? ?????? ????? NVIDIA AI ?????? ????? ?????.

?? ??? ???? ?? ????? ?????:

# Choose a container name for bookkeeping
export CONTAINER_NAME=meta-llama3-8b-instruct
 
# Choose a LLM NIM Image from NGC
export IMG_NAME="nvcr.io/nim/meta/llama3-8b-instruct:24.05"
 
# Choose a path on your system to cache the downloaded models
export LOCAL_NIM_CACHE="~/.cache/nim"
mkdir -p "$LOCAL_NIM_CACHE"
 
# Start the LLM NIM
docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus all \
  -e NGC_API_KEY \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -u $(id -u) \
  -p 8000:8000 \
  $IMG_NAME

???? ?? ??? ??????:

curl -X 'POST' \
    'http://0.0.0.0:8000/v1/completions' \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
      "model": "meta-llama3-8b-instruct",
      "prompt": "Once upon a time",
      "max_tokens": 64
    }'

?? ???? ???? ???? ??? ?? ??? AI ??????? ???? ??? ? ????.

NVIDIA?? ????? NIM? ?? ??? NVIDIA API ??????? ??? ? ????.

NIM? ??????? ???? ??

?? ??? ?? ???? ???, ?? ???? ?? NIM? ????? ?? ?? NVIDIA API ????? NVIDIA ??? API ?????? ???? ???? ? ????. ?? ??? ????.

NIM ????? ????

OpenAI ??? ??? ?? ? ???? ??? ? ????. ??? ??????? stream? True? ???? ???.

OpenAI ?????? ?? Python ???? NIM? ????? ??? ?? ???:

• NIM? ???? ?? API ?? ??? ??? ????.

base_url? NIM? ?? ?? ??? ?????? ???.

from openai import OpenAI
 
client = OpenAI(
  base_url = "http://nim-address:8000/v1,
)
 
completion = client.chat.completions.create(
  model="meta/llama3-70b-instruct",
  messages=[{"role":"user","content":""}],
  temperature=0.5,
  top_p=1,
  max_tokens=1024,
  stream=True
)
 
for chunk in completion:
  if chunk.choices[0].delta.content is not None:
    print(chunk.choices[0].delta.content, end="")

NIM? ?? Haystack,?LangChain, ?LlamaIndex? ?? ?????? ??????? ????, ?? ??? ?? ?? ??? ??? ??? AI ??????? ???? ?? ?????? ???? ??? ? ??? ???? ?? ??? ?????.

? ?????? ???? ???? NIM ???? ?????:

• ?? ?? AI ??? NVIDIA NIM? ??? Haystack RAG ?????
• NVIDIA NIM? ??? LangChain RAG ????
• NVIDIA NIM? ??? LlamaIndex RAG ?????

NIM ????

??? ????? ??? ?? ??? ?? NVIDIA NIM? ???? ??? ???? ??? AI ????? ? ??????? ???? ? ??? ? ????. NIM? ? ?? ????? LoRA ???? ??????? LLM? ?? ???????? ???? ??? ?????.

NIM? ????? ???? ?????. API ????? ??? ???? ??, ??, 3D, ??? ??? ?? ?? ?? NVIDIA NIM ???????? ?????.

?? ???

GTC ??: ??? AI ?? ?? ? ??? ??: NVIDIA? ?????? ???? ?? ????
GTC ??: GenAI ?????? ?? NVIDIA AI ???? ??? ????
NGC ????: GenAI SD NIM
NGC ???? DiffDock? NVIDIA NIM
NGC ???? ESMFold? NVIDIA NIM
??? NVIDIA? ?? ??? AI? ??? ????