Paresh Kharya – NVIDIA Technical Blog News and tutorials for developers, data scientists, and IT admins 2023-02-10T22:26:05Z http://www.open-lab.net/blog/feed/ Paresh Kharya <![CDATA[Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, the World��s Largest and Most Powerful Generative Language Model]]> http://www.open-lab.net/blog/?p=38456 2023-02-10T22:26:05Z 2021-10-11T13:00:00Z We are excited to introduce the DeepSpeed- and Megatron-powered Megatron-Turing Natural Language Generation model (MT-NLG), the largest and the most powerful...]]>

We are excited to introduce the DeepSpeed- and Megatron-powered Megatron-Turing Natural Language Generation model (MT-NLG), the largest and the most powerful monolithic transformer language model trained to date, with 530 billion parameters. It is the result of a joint effort between Microsoft and NVIDIA to advance the state of the art in AI for natural language generation.

Source

]]> 1 Paresh Kharya <![CDATA[Introducing the NVIDIA OpenACC Toolkit]]> http://www.open-lab.net/blog/parallelforall/?p=5569 2022-11-28T18:20:54Z 2015-07-13T07:01:55Z Programmability is crucial to accelerated computing, and NVIDIA's CUDA Toolkit has been critical to the success of GPU computing. Over three million CUDA...]]>

Programmability is crucial to accelerated computing, and NVIDIA’s CUDA Toolkit has been critical to the success of GPU computing. Over three million CUDA Toolkits have been downloaded since its first launch. However, there are many scientists and researchers yet to benefit from GPU computing. These scientists have limited time to learn and apply a parallel programming language, and they often have…

Source

]]> 2 ���˳���97caoporen����