?????? ??? ?? ???? ‘??’? ??? ???? ??? ?? ?????. ?? ?? ??? ?? ???, ???? ?? ??? ?? ???? ???? ?? ?????? ?? ?????.
?? ????? ?? ????. ?? ?? ? ?? ?? ???? ?? OS?? ??? ? ??? ? ? ?? ??? ??? ??? ? ????. ??? ?? ?? ??, ?? ?? ?? ??, ???? ?? ??? ?? ?? ?? ?? ?? ???? ??? ??? ? ????.
??? ???? ??? ??? ????? ???? ?? ??? ?????. ??? ??? ?? NVIDIA Nsight ?? ???? ??? ? ??? ?? ??? ???? ??? ????.
VM? ????? ????
?? ? ?? ??? ???????.
?? 1?? ? ? ??? ?? ??(VM)? ????? ?? ?? ????? ??????? ???? ??? ? ?? ?? ??? ??? ??? ? ?????. ??? ???? ????? ??? ? ?? ???? Linux ??????? ???? ??? ??? ??? ?? ???? ?? ?? ??? ????? ????. VM? ???? VM? ?? ?? ??? ???? ?? ????? VM ??? ??? ???? ? ?????.
??? ??? VM? ?????? ????? ?? ??? ?? ???? ????? ??? ? ????? ???? ?? ?????. ????? ‘?? ??’ ????? ?? ?????? ??, ????? VM???.
????? HPC ? AI ?????? ???? ???? ???? ?? ?? ???? ?????. ? ?? ????? ?? HPC ???????? ??? ??? ?? ? ?? ?????? ??? ?? ????? ??? ???? ???? ?? ??? ???????.
???? ??? ????? ?? ??? ??? ????? ??? ??? ???? ?? ??? ?? ???? ??? ? ????. ??? ?????? ????? ??? ?? ??????, ?????? DGX ??, ???? ?? HPC ???? ? ????? ?? ??? ? ????.
Docker, Singularity ? Podman? ???? ?? ???? ???? ??? ? ????. ?? ? ?? ???? ??? ?? ?? ????? ?? ?? ??, ?? ?? ? ?? ???? ?? ??? ?? ?????. ?? ????? Docker? Singularity? ?? ????? ???? ? ?? ???? ??? ?????.
????? ?? ?? ???? ??? ????(CSP)? ??? ? ????. ??? ?? ??? CSP? ?? ?? ??? ?? ? ???, ?? ? GPU ???? ?????.
???? ???? ???? ???? ???? ?????? ???? ???? ??? ????. ?? NVIDIA Nsight Systems ?? NVIDIA Nsight Compute? ?? ??? ?????? ? ???? ? ? ????. ? ?????? Nsight Systems? ??? ?? ?? ??? ??? ????. Nsight Compute? ?? ?? ???? ?????.
???? ?? Nsight Systems ?? ? ??(Docker/Singularity)
???? ???? Nsight Systems? ???? ???? ?? ??? ????. ?? ????? ????? ?? ???? ? ???? ???? ??? ? ????. ?? ??? ??? ?? ???? ?? ?? ???? ????? ???? ??? ??? ? ????.
? ???? Docker ?? Singularity ????? ?? ??? ??? ?????. ???? ??? ?? ?? Docker ?? Singularity ??? ?????.
?????? ??? ??
Nsight Systems? CPU ??? ????? Linux ??? ?? ?? ???? ???? ? ????? ?????. ??? ???? ??? ? ?? ???(IP) ??? ????? ?? ???? perf paranoid ??? ≤2?? ???. ?? ??? ??? ??? ?????.
cat /proc/sys/kernel/perf_event_paranoid??? >2? ?? ?? ??? ???? ????? paranoid ??? ?????(? ??? ?).
sudo sh -c 'echo 2 >/proc/sys/kernel/perf_event_paranoid'??? ?? ????? ?? ??? ?????.
sudo sh -c 'echo kernel.perf_event_paranoid=2 > /etc/sysctl.d/local.conf'Docker ?????? ???? ?? Nsight Systems ???? ??? ?? Linux ?? ??? ?? perf_event_open ??? ??? ?????? ???? ??? ?????. Singularity? ???? ?? ??? ??? ???? ????.
perf_event_open syscall? ????? ???? ? ??? ????. –cap-add=SYS_ADMIN ???? ???? ???? ? ????. ???? ?? ??? ???? ?? seccomp ?? ???? ???? ???? ???? ?? ????.
?? ??? ??(seccomp)? ??????? ???? ???? ? ??? ? ?? Linux ??? ?????. ? ??? seccomp ??? ?? ??? ???? ???? ??? ? ????. seccomp ??? ????? ?? ??? ?????.
$ grep CONFIG_SECCOMP= /boot/config-$(uname -r)???? ?? ?? ????? ???.
CONFIG_SECCOMP=ySeccomp ????? seccomp 2.2.1? ???? ?? ?? ?????? ???? ????(?: Ubuntu 14.04, Debian Wheezy ?? Debian Jessie). ?? ????? seccomp? ????? ???? ?? ?? ?? Linux ????? ?????? ???.
Docker ??? ??? ?? seccomp ??? ??(default.json)? ???????. perf_event_open? ?? CAP_SYS_ADMIN ?? ???? ??? ??? ?? perf_event_open ??? ?????. ‘syscalls’ ??? ?? ?? ???? ?? ??? default_with_perf.json?? ?????.
{
"name": "perf_event_open",
"action": "SCMP_ACT_ALLOW",
"args": []
},? seccomp ???? ????? Docker ????? ??? ? ?? ???? ?????.
--security-opt seccomp=default_with_perf.jsonNsight ???? ?? ????? ???? ?????? ????? ???? ???? status ??? ???? ??? ?????.
$ nsys status -e??? ?? ?? ???? ??? ????? ??? ? ? ????.
NGC ? CUDA ??? Nsight ??? ??
?????? Nsight Systems? ?????? ?? ??? ??? NVIDIA GPU Cloud(NGC) ?????? ???? ? ??? ?????? ????. NGC 19.11 TensorFlow ????? ?? ??? ???? ? ???? ?? Nsight Systems? ???? ??? ?? ?? ?????.
NGC ????? Nsight Systems? ?? ???? ?? ?? ?? ????? ??? CUDA ?? ??? ?? ?? ?? ? ??? ???? ?????. ????? ???? CUDA ?? ??? ???? ???? ??? nvidia-smi? ???? CUDA ?? ??? ? ? ????.
CUDA 10.1:
CUDA 10.2:
??? Nsight Systems? ? ????? CUDA ???? ??? ? ?? ???? ?? ??? ??? ?? ????. ?? ??? ????? ?? ???? ???? ????? ?????.
?? Docker ????? Nsight ??? ??
?? Nsight ???? ?? Docker ???? ???? ???? ?? ??? ???????. ?? Dockerfile? ?? Dockerfile ?? ??? ?????. ???? ???? RHEL ???? Debian ????? ?? ??? ?? ??? ?????. ???? ???? ???? ?? Linux ?? ??? ???? ???? ??? ???? ???? /etc/os-release? ?????. ???? NGC ???? Ubuntu? ???? ???.
NVIDIA Nsight Systems 2020.2.1
RUN rpm --import https://developer.download.nvidia.com/devtools/repos/rhel8/x86_64/nvidia.pub && \
yum install -y yum-utils && \
yum-config-manager --add-repo https://developer.download.nvidia.com/devtools/repos/rhel8/x86_64/ && \
yum install -y \
nsight-systems-2020.2.1 && \
rm -rf /var/cache/yum/*RHEL ?? ???? ???? Nsight Systems? ???? Dockerfile ?? ??
NVIDIA Nsight Systems 2020.2.1
RUN apt-get update -y && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
apt-transport-https \
ca-certificates \
gnupg \
wget && \
rm -rf /var/lib/apt/lists/*
RUN wget -qO - https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/nvidia.pub | apt-key add - && \
echo "deb https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/ /" >> /etc/apt/sources.list.d/nsight.list && \
apt-get update -y && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
nsight-systems-2020.2.1 && \
rm -rf /var/lib/apt/lists/*Debian ?? ???? ???? Nsight ???? ???? Dockerfile ?? ??
? ??? ?? ??? ?? Nsight Systems ??? 2020.2.1???.
NGC ?? Docker Hub?? ???? ????? ?? ???? ???? ???? ? ???? ?? Dockerfile? ?? ? ????. Dockerfile? ??? ???? ???? ?? ???? ?????. ?? ?? NGC?? 0.9 RAPIDS ???? ????? ??? ?????.
ARG IMAGE=nvcr.io/nvidia/rapidsai/rapidsai:0.9-cuda9.2-base-ubuntu18.04
FROM $IMAGE
# NVIDIA Nsight Systems 2020.2.1
RUN apt-get update -y && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
apt-transport-https \
ca-certificates \
gnupg \
wget && \
rm -rf /var/lib/apt/lists/*
RUN wget -qO - https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/nvidia.pub | apt-key add - && \
echo "deb https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/ /" >> /etc/apt/sources.list.d/nsight.list && \
apt-get update -y && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
nsight-systems-2020.2.1 && \
rm -rf /var/lib/apt/lists/* NGC RAPIDS ???? ???? Nsight ???? ???? ?? Dockerfile ??? ?????.
Nsight Systems? ???? ?? ??? ?? ?? ?????.
HPC Container Maker?? Nsight ???? ?? ? ???? ?? Dockerfile ??? ???? nsight_systems ?? ??? ????.
Docker ????? ??? ? Nsight Systems CLI(nsys)? ???? Docker ???? ??? ???? ?????. ??? ??? ???? ???? Nsight ???? ???? ?? ?? ?? ??? ??????. ?? ??? docker cp? ???? ?????? ???? ??? ?? ?? CLI ??? ????? ???? NVIDIA Nsight Systems GUI? ??? ??? ? ????.
?? Singularity ????? Nsight ??? ??
?? Singularity ???? ???? Nsight ???? ???? ??? ??? ???? Docker? ??? ????? ?????.
?? Singularity ?? ??? ?? Singularity ?? ??? ?????. ???? ???? RHEL ???? Debian ????? ?? ??? ??? ?????. ???? ???? ???? ?? Linux ?? ??? ???? ???? ??? ???? ???? /etc/os-release? ?????. ???? NGC ???? Ubuntu? ???? ???.
%post
# NVIDIA Nsight Systems 2020.2.1
rpm --import https://developer.download.nvidia.com/devtools/repos/rhel8/x86_64/nvidia.pub
yum install -y yum-utils
yum-config-manager --add-repo https://developer.download.nvidia.com/devtools/repos/rhel8/x86_64/
yum install -y \
nsight-systems-2020.2.1
rm -rf /var/cache/yum/* RHEL ?? ???? ???? Nsight Systems? ???? Singularity Definition ?? ??
%post # NVIDIA Nsight Systems 2020.2.1 apt-get update -y DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ apt-transport-https \ ca-certificates \ gnupg \ wget rm -rf /var/lib/apt/lists/* wget -qO - https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/nvidia.pub | apt-key add - echo "deb https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/ /" >> /etc/apt/sources.list.d/nsight.list apt-get update -y DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ nsight-systems-2020.2.1 rm -rf /var/lib/apt/lists/*
Debian ?? ???? ???? Nsight ???? ???? Singularity Definition ?? ??
? ??? ?? ??? ?? Nsight Systems ??? 2020.2.1???. ?? ?? ???? ????? ?? ??? ???? NSight Systems ??? ??? ???? ??? ?????.
NGC ?? Singularity ???? ??????? ???? ????? ?? ???? ???? ???? ? ???? ?? Singularity ?? ??? ????. ? ?? Singularity ?? ??? ???? ???? ???? ?? ???? ?????. ?? ?? namd-2.13.sif?? NGC?? ????? ?? NAMD ???? ???? ?? ?? ?? ??? ?????.
Bootstrap: localimage
From: namd-2.13.sif
%post
# NVIDIA Nsight Systems 2020.2.1
apt-get update -y
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
apt-transport-https \
ca-certificates \
gnupg \
wget
rm -rf /var/lib/apt/lists/*
wget -qO - https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/nvidia.pub | apt-key add -
echo "deb https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/ /" >> /etc/apt/sources.list.d/nsight.list
apt-get update -y
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
nsight-systems-2020.2.1
rm -rf /var/lib/apt/lists/* Nsight Systems? ???? ?? ??? ?? ?? ?????.
HPC Container Maker?? Nsight ???? ?? ? ???? ?? Singularity ?? ?? ??? ???? nsight_systems ?? ??? ????.
Singularity ????? ??? ? Nsight Systems CLI(nsys)? ???? Singularity ???? ??? ???? ?????. ?? ??? ?? CLI ??? ????? ???? Nsight Systems GUI? ??? ??? ? ????.
Nsight ??? ??? ??? ????? ??
?? ? ????? Docker ?? Singularity ???? ???? Nsight ???? ???? ??? ??????. ? ? ?? ??? ??????? ?? ??? ???? ???? ???? ?? ??? ??? ? ????.
????? Nsight Systems ??? ????? ?????? ???? ? ????. ??? ? ?? ??? ???? ???? Nsight Systems CLI ???? ???? ?? ? ???? ? ???? ???? ???? Nsight ???? ???? ? ??? ?? ?? ?????? ??? ??? ? ????.
?? ?? ??? Nsight Systems ??? ???? /opt/nvidia/nsight-systems/2020.2.1? ?? ?? ?? ??? ??? ???? ????? ???? ???? Nsight ???? ???? ? ? ????.
?? ???? ???? Nsight Systems CLI(nsys)? ??? ? ??? nsys ?? ??? ?? ??? ???? ???.
$ sudo docker run --runtime=nvidia -v /opt/nvidia/nsight-systems/2020.2.1:/opt/nvidia/nsight-systems/2020.2.1:ro ...$ singularity run --nv -B /opt/nvidia/nsight-systems/2020.2.1:/opt/nvidia/nsight-systems/2020.2.1 ...?????? Nsight Systems GUI ????
?? ??? ??? ?????? Nsight Systems CLI? ???? ?????? ???? ????? ???? ???? ??? ?? ?? ????? GUI?? ?? ????? ????. ??? Nsight Systems GUI? ?????? ??? ?? ????. ??? X11 ??? ???? ???? ???? ???? ? ?? ??? Singularity? ??? ? ?? ??? ?? ?????.
???? ???? Nsight Systems GUI ?? ??? ???? ?? ?? X11 ? ?? ???? ???? ???. ??? CUDA 10.1, Ubuntu 16.04 ? Nsight Systems 2020.2.1? ?? ?? Singularity ?? ?????. Nsight Systems ?? ??? Nsight Systems? Singularity ???? ???? ???? ??? ??? ????? ?????.
BootStrap: docker
From: nvidia/cudagl:10.1-base-ubuntu16.04
%post apt-get update -y
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
apt-transport-https \
ca-certificates \
dbus \
fontconfig \
gnupg \
libasound2 \
libfreetype6 \
libglib2.0-0 \
libnss3 \
libsqlite3-0 \
libx11-xcb1 \
libxcb-glx0 \
libxcb-xkb1 \
libxcomposite1 \
libxcursor1 \
libxdamage1 \
libxi6 \
libxml2 \
libxrandr2 \
libxrender1 \
libxtst6 \
openssh-client \
wget \
xcb \
xkb-data
wget -qO - https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub | apt-key add -
echo "deb https://developer.download.nvidia.com/devtools/repos/ubuntu2004/amd64/ /" >> /etc/apt/sources.list.d/nsight.list
apt-get update -y
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
nsight-systems-2020.2.1
rm -rf /var/lib/apt/lists/*
%environment
export LC_ALL=C
export __GLX_VENDOR_LIBRARY_NAME=nvidia
%runscript
exec nsight-sys "$@"Nsight Systems GUI ???? ???? ???? ?? ?? ???? ?? Singularity.def?? ??? ??? ?? ?? ???? ?????.
$ sudo singularity build nsys-gui.sif Singularity.def????? ???? Nsight Systems GUI? ???? ?????.
$ singularity run --nv nsys-gui.sif
Singularity Nsight Systems GUI ????? ?? ?? ??? ???? ???? ????? ‘????’ ??????? ? ??? ? ??? ????.
??? ??? ?? ?? ??? ??? ‘localhost’? ??? ???? ???? ? ?? ??? ???? ???? ?? ??? ?????. ?????? ?? ?? ??? ???? ????? ?????. ?? ?? Nsight Systems? ?? ?? ?? ??? ?????? ???? ?????.
Singularity? ???? ??? ??? ? ????? ???? ?????? Nsight Systems ???? ???? ????? ???? ???? ??? ? ????.
?? GUI ????? ???? ??? ??? ???? ???? ?? ????. Nsight Systems? ?? ?? ?? ?? ????? ??, ????, ????? ?????.
?????? Nsight Systems ??
GPU ?? ???? ????? ???? ???? ?? ????(? ??)? CSP?? ??? ? ???? ??? ????. ????? ??? ? Nsight Systems? Pascal GPU? ????? ?? ?????.
NVIDIA? ?? ?? ???? ???? ?? ?? ???? ??? ????? ??? ??? ?? ????. ?? ????? ?? ??? ???? ?? ??? ? ?? ?? ??? ??? NVIDIA GPU Cloud ???? ?????.
CSP? ?? ??? ???? ???? GPU ?? ????? ???? ??? ?????.
????? ??? ? Nsight ???? ???? ?? ?? ??? ???? ?? ????? ?????. ??? ??? ???? ???? ?????. Nsight ???? ?? ????? ???? ?????? ????? status ??? ???? ??? ?????.
nsys status -e??? ?? ?? ???? ??? ????? ??? ? ? ????.
?? ?????!
? ???? ??? Nsight ?? ???? ????, ??? ????, ??? ??? ??? ???? ??? ???? ??? ??? ? ? ?????. ?? ??? ? ??? ??? ?? ???? NVIDIA ??? ???? ?????.
?? ???:
- NVVP ? Nvprof?? NVIDIA Nsight ??? ????????
- NVIDIA Visual Profiler / nvprof?? Nsight ????? ??
- Nsight Compute? ???? ?? ??
NVIDIA CUDA ?? ?? ????? ?? Nsight Systems ? Nsight Compute? ???????. ?? Nsight Systems ???, Nsight Compute ??? ?? Nsight Graphics ????? NVIDIA CUDA ???? ?? ?? ?? ??? ??? ?? ??? ??? ?? Nsight ??? ?? ? ????.
??? ??? ???? ??? ??? ?? GPU ?? ????(GTC)? ??? ?? ?? ??? ?????.
NVIDIA Nsight Systems
- Blue Waters ??? ??? – 2018? 11? – NVIDIA Nsight Systems ??
- NVIDIA Nsight Systems? ???? HPC ????? ? ??? ?? ???
- Nsight Tools? ???? NAMD ?? ?? ????? ???? ???
- NVIDIA GPU? ?? Facebook AI ???? ???
- NVIDIA ??? ???? DNN ???? ?? ??
NVIDIA Nsight Compute
- Nsight Compute? ?? OptiX ?????
- NVIDIA Nsight Compute? ??? CUDA ?? ?????
- Nsight Compute ?? Nvprof? ???? ? ?? ???? ?? ??? ?? ??
NVIDIA Nsight ???
?? ??? GPU ??? ??? ???? Nsight Systems ? ??? ??? ???? ??? ? NVIDIA Nsight ??? ???? ?? ????? ?????.
??? ?????? NVIDIA Nsight Systems ?? NVIDIA Nsight Compute? ???? NVIDIA ??? ?????. ???? nsight-systems-feedback@nvidia.com ?? nsight-compute-feedback@nvidia.com? ?????. ??? ???????? ???? ??? ???? ?? ?? ?? ???? ??? ?????.
? ???? ??? SDK? ???? ?? ???, ?? ???, ?? ??, ??, ?? ??, ???? NVIDIA ??? ???? ??? ??? ??? ??? ? ????. ?? ??? ???? NVIDIA? ?? ????? ???? ? ??? ??? ??? ?????? ???? ??? ??? ???.
