• <xmp id="om0om">
    •
    • <table id="om0om"><noscript id="om0om"></noscript></table>
  • Related Resources

    ?? AI ??????? GPU ?? SDK? NVIDIA Riva? ?????

    2022? 2? 3?
    By , , and
    +1
    Discuss (0)
    Reading Time: 4 minutes

    ?? AI? ??? ?? ??, ?? ?????? ?? ?????, ?? ?? ? ??? ?? ? ??? ??????? ?????. ?? AI?? ?? ?? ??(ASR)? ?? ?? ??(TTS) ????? ??????. ASR ?????? ?? ???? ???? ????, TTS ?????? ???? ???? ????.

    ? ?? ??? ?? AI ???? ??? ??? ???? ??????. ?? AI ??????? ????? ??? ?? ??? ??? ???, ???? ??? ?? ??? ?? ??? ??? ??????? ??? ?, ??? ?? ??? ?????. ?? ????? ??? ????? ????? ???? ???? ??? ??? ?? ?? 300 ???(ms)? ?? ???? ??. NVIDIA Riva? ?? AI ??? ??? ?? ? ?? ????? ?????, ??? ??? ????? ???? ?? ??? ??? ?????.

    Riva SDK

    NVIDIA Riva? ?? AI ?????? ??? ?? GPU ?? SDK???. ??? AI? ?? ??? ?? ??? ?????? ????. ?? ?????? API? ??? ??? ???? ?????, ??? ??? ? ????.

    Diagram shows workflow starting from pretrained models in NGC, TAO toolkit for retraining the models and Riva for optimized speech AI skills to generate high-performance inference.
    ?? 1. Riva? ?? ?????? ?? ?????

    Riva SDK?? ?? ??? ??/?? ??? ? ??? ??? ??????? ?? ??? NVIDIA TAO Toolkit, ??? ?? ??? ?? ??, ?? ??? ???? ?? ? ?? ???? ??? ????.

    Riva? ???? ??? ???? ???? ????? ??? ?? ??? ??? ??? ??? ??? ?? ? ????. ??? ???? ?? CPU ?? ????? 25?? ??? ??? ???? ?? ??? 150 ???? ????? ???.

    ??? AI ???? gRPC ?????(endpoint)? ?? ??? ???? ?? ?? ??? ?? ?? ??? ??? ??(NLP), ?? ?? ?? ????? ?????. ??? AI ????? ?? ?? ??? ??/?? ?????? ???? ?? ???? ????. ?? ?? ??? ??? ??? ? ???? ?????? ?? ??? ??? ??? ??? ?? ??? ?? ????.

    Riva? NVIDIA Triton Inference Server ? ?? ??? ??? ??? ??? ???? ???? ??, ???? ??? ??, ???? ??? ???? ??? ??? ?????.

    Riva ?? ??

    Riva? ??? ??(transcription)? ?? ????? ?? ?? ??? ??? ?? ??? ?? ??? ??? ???? ???? ?????. Riva? ?? ?? ??? ???? ?? ??? ??????? ??? ??? ?????. ??, ????(podcast), ???? ??? ???? ?? ??? ????? ?? ???? ??? ??? ???? ?????.

    Riva? ?? ?? ???? ?? ?? ??? ??, ??????(non-autoregressive) ??? ??? ??? ??? ??? ?????. NVIDIA V100 GPU ?? Tacotron 2/WaveGlow ??? ??? ?, Riva? ?????? ???? NVIDIA A100 GPU??? 12? ??? ??? ????. ? ??? 30? ??? ?? ??? ???? ???? ? ???? ?? ???? ?? ?????? ??? ????? ??? ??? ??? ? ????.

    Diagram shows capabilities in Riva Speech Recognition and Speech Synthesis services such as support for both batch and streaming mode, multiple sampling rates (8,16,22,32, 44, 48 kHz), and models such as Jasper, QuartzNet, CitriNet, Tacotron2, WaveGlow, Fastpitch, and HiFiGAN.
    ?? 2. Riva ??? ??

    GPU? ??? ??? ??? ????? Riva? NVIDIA Triton Inference Server? ??? ?? ???? ??? ?????(ensemble pipeline)? NVIDIA TensorRT? ?? ????? ????? ?????.

    Riva? ????? gRPC ?????? ???? ??? API? ?? ????? ???? ??? ???? ????. ?? 3? Riva ???? ?? ??? ?????. gRPC API? ??(Docker) ?????? ???? API ??? ?????. ??? ????? ??/NLP ??? ??? ??? ????.

    Diagram shows the Riva client applications such as desktop, mobile, and laptop interactions with Riva speech recognition and speech synthesis pipelines.
    ?? 3. Riva ??? ?????

    ?? API ??? NVIDIA Triton? ?? ??? ??? ??? ?????.

    NVIDIA Triton? ?? GPU?? ?? ???? ??? ?????? ???? ??? ?? ??? ??? ???? ???(backend) ?????.

    ??? AI ??????? ??, ????? ??? ??? ???? ???? ?? ?? ?????. ?? ?? ??? ?????? ???? ??? ????. GPU? ??? ???? ??? ?????? ?? ???(batch size)? ??? ??, ? ?? ??? ???? ? ? ??? ??? ??? ?? ??? ???? ???.

    ?? NVIDIA Triton? ?? ?? ? ??? ????? ?? ??? ?????.

    Riva? NGC?? ??? ??? ????? ?????? ??? ????? ?? ????(bare-metal)? ?? ?????, ?? ???? ?? ??(Helm chart)? ?????? ??? ? ????.

    ???? Riva? ?????? ??? ??? ???????. ???(Python) ?????? ??? ??? API ??? ?? ????? ???? Riva ???? ??? ? ???? ????. ?? ?? ?? TTS Riva ???? ??? ???? 3?? ??? ??? ????.

    ?? Riva API? ???? ???:

    import src.riva_proto.riva_tts_pb2 as rtts 
import src.riva_proto.riva_tts_pb2_grpc as rtts_srv 
import src.riva_proto.riva_audio_pb2 as ri

    ???? Riva ?????? gRPC ??? ?????:

    channel = grpc.insecure_channel('localhost:50051') 
riva_tts = rtts_srv.RivaSpeechSynthesisStub(channel)

    ?? TTS ??? ?????:

    req = rtts.SynthesizeSpeechRequest() 
req.text = "We know what we are, but not what we may be?" 
req.language_code = "en-US" 
req.encoding = ri.AudioEncoding.LINEAR_PCM 
req.sample_rate_hz = 22050 
req.voice_name = "ljspeech" 
resp = riva_tts.Synthesize(req) 
audio_samples = np.frombuffer(resp.audio, dtype=np.float32)

    ???? ???? ?? ????????

    NVIDIA TAO Toolkit? ??? ??? Riva?? ??? ? ?? ????(?? 4). ?? ??? ??? ??? ??????? ??? ?? ??? ? ??.

    Diagram shows the workflow of TAO toolkit starting from NGC pretrained model to adding your custom data and deploying it as a Riva skill.
    ??4. NVIDIA TAO Toolkit ?????

    ?? ASR? ???? ???? ???? ???? ???? ?? ??, ?? ASR ???? ???? ???? ??? ??? ??? ?? ??? ???? ?? ????? ???? ??? ????? ??? ? ????.

    ?? ??? BERT ??? ???? ?? 1??? ?????? ?????. ??? ?????? ?? ??? ?? ??? ?? ??? ?????.

    • ?? ?? ?? ???? ? ???.
    • ?? ??? ???? ??.

    ?????? ???? ??? ??? ????? ??? ??? ?????. ??? ???? ?? ???? ???? ??? ???? ?? ??? ????? ?? NVIDIA Triton? ?? ???? ?????.

    NVIDIA Riva Speech Skills ???????? ?? ???? ??? ?? ?? ??? ??? ??? ????. ? ???? TAO Toolkit? ??? ??????? ??? ??? ? ? ??? ??? ?? ????.

    Riva?? ?? ????

    Riva? ??? ??? AI? ?? ?????. ? ??? ??? ?? ???? ????? ??? ? ??? ?? ??? ??? ??? ?? ??? ?? ?????(?? 5).

    Diagram shows the workflow for deploying TAO model as a Riva skill and highlights the TensorRT optimizations and Triton Inference Server under the hood.
    ?? 5. Riva??? ??? ?? ??? ???? ???? ??? ?????

    NGC ????? ?? ?? ??? ??? ?? ??? ?? ??? ?????. ??? ??? ?? ??? ??, ???? ?? ??? ??? ???? ??? ? ????.

    ??

    Riva? NVIDIA Developer Program? ?????? ?? ??? ???? ??? ??? ?? ?????, ??? ??? ??? ?????. ??? ??? ??? ???? AI ????? ??? ?? ???? Riva Enterprise? ??? ? ????.

    ? ??? ??? Riva Getting Started?? ?????.

    Discuss (0)
    +1

    Tags

    Riva | TensorRT | Conversational AI / NLP | featured | Speech Synthesis | technical walkthrough | Triton

    ??? ??

    Davide Onofrio ???
    Davide Onofrio is a Senior Deep Learning Software Technical Marketing Engineer at NVIDIA. He is focused on development and presentation of deep learning technical developer focused-content at NVIDIA. Davide has several years of experience working as a computer vision and machine learning engineer in Biometrics, VR and Automotive industry. His educational background includes a PhD in signal processing at the Politecnico di Milano.
    Vinh Nguyen ???
    Vinh Nguyen is a deep learning engineer and data scientist, having published more than 50 scientific articles attracting more than 2500 citations. At NVIDIA, his work spans a wide range of deep learning and AI applications, including speech, language and vision processing, and recommender systems.
    Siddharth Sharma ???
    Siddharth Sharma is a Senior Technical Marketing Manager for Accelerated Computing at NVIDIA. Before joining NVIDIA, Siddharth was a product marketing manager for Simulink and Stateflow at Mathworks, working closely with automotive and aerospace companies to adopt model-based designs for creating control software.
    Alex Qi ???
    Alex Qi is a product manager for the NVIDIA AI Software Group. Her focus is on AI software and applications for a conversation AI framework (Riva) and AI/ML for multimedia streaming (Maxine). Prior to joining NVIDIA, she had extensive experience in leading challenging technical projects in technology and engineering organizations across various roles, such as data Scientist, Computational Modeling and Design Engineering. Alex holds a dual-degree masters' from the Massachusetts Institute of Technology: an MBA from the MIT Sloan School of Management, and a master of science from the School of Engineering - Mechanical Engineering, where she focused her studies on robotics and artificial intelligence.

    ??

    ?? ???

    人人超碰97caoporen国产