Microsoft funding in ChatGPT doesn’t simply contain cash sunk into its maker, OpenAI, however an enormous {hardware} funding in knowledge facilities as properly which exhibits that for now, AI options are only for the very high tier firms.
The partnership between Microsoft and OpenAI dates again to 2019, when Microsoft invested $1 billion within the AI developer. It upped the ante in January with the funding of a further $10 billion.
However ChatGPT has to run on one thing, and that’s Azure {hardware} in Microsoft knowledge facilities. How a lot has not been disclosed, however in keeping with a report by Bloomberg, Microsoft had already spent “a number of hundred million {dollars}” in {hardware} used to coach ChatGPT.
In a pair of weblog posts, Microsoft detailed what went into constructing the AI infrastructure to run ChatGPT as a part of the Bing service. It already supplied digital machines for AI processing constructed on Nvidia’s A100 GPU, known as ND A100 v4. Now it’s introducing the ND H100 v5 VM primarily based on newer {hardware} and providing VM sizes starting from eight to 1000’s of NVIDIA H100 GPUs.
In his weblog publish, Matt Vegas, principal product supervisor of Azure HPC+AI, wrote prospects will see considerably quicker efficiency for AI fashions over the ND A100 v4 VMs. The brand new VMs are powered by Nvidia H100 Tensor Core GPUs (“Hopper” technology) interconnected through subsequent gen NVSwitch and NVLink 4.0, Nvidia’s 400 Gb/s Quantum-2 CX7 InfiniBand networking, 4th Gen Intel Xeon Scalable processors (“Sapphire Rapids”) with PCIe Gen5 interconnects and DDR5 reminiscence.
Simply how a lot {hardware} he didn’t say, however he did say that Microsoft is delivering a number of exaFLOPs of supercomputing energy to Azure prospects. There is just one exaFLOP supercomputer that we all know of, as reported by the most recent TOP500 semiannual checklist of the world’s quickest: Frontier on the Oak Ridge Nationwide Labs. However that’s the factor in regards to the TOP500; not everybody stories their supercomputers, so there could also be different techniques on the market simply as highly effective as Frontier, however we simply don’t find out about them.
In a separate weblog publish, Microsoft talked about how the corporate began working with OpenAI to assist create the supercomputers which might be wanted for ChatGPT’s giant language mannequin(and for Microsoft’s personal Bing Chat. That meant linking up 1000’s of GPUs collectively in a brand new manner that even Nvidia hadn’t considered, in keeping with Nidhi Chappell, Microsoft head of product for Azure high-performance computing and AI..
“This isn’t one thing that you just simply purchase a complete bunch of GPUs, hook them collectively, they usually’ll begin working collectively. There’s a variety of system-level optimization to get the perfect efficiency, and that comes with a variety of expertise over many generations,” Chappell stated.
To coach a big language mannequin, the workload is partitioned throughout 1000’s of GPUs in a cluster and at sure steps within the course of, the GPUs alternate data on the work they’ve achieved. An InfiniBand community pushes the info round at excessive velocity, for the reason that validation step should be accomplished earlier than the GPUs can begin the subsequent step of processing.
The Azure infrastructure is optimized for large-language mannequin coaching, nevertheless it took years of incremental enhancements to its AI platform to get there. The mixture of GPUs, networking {hardware} and virtualization software program required to ship Bing AI is immense and is unfold out throughout 60 Azure areas world wide.
ND H100 v5 cases can be found for preview and can turn into a normal providing within the Azure portfolio, however Microsoft has not stated when. events can request entry to the brand new VMs.
Copyright © 2023 IDG Communications, Inc.
