Recently, I noticed that my wife and neighbors have been buzzing with questions about the new AI modeling efficiencies found with DeepSeek—a dinner table topic that really piqued my interest as an AI network engineer.
Not long ago, headlines and watercooler discussions were dominated by the $500 billion Stargate project. And just a few days back, Grok 3 made its entrance to the AI arena, using massive training hardware infrastructure that has 10 times the processing power that previous models had access to.
It’s been a whirlwind of exciting AI developments, each one pushing the boundaries of what we once thought was possible with AI model scaling. In seeing many real and perceived inflection points over the decades, perhaps we need to take stock of what is happening and take in a deep breath of fresh air.
There are economic, technical and political inflections underway in this multi-trillion-dollar AI opportunity. Where technological inflections once seemed to come every number of years (or even decades), it seems this now happens on a weekly basis.
The reality is that the current trajectory for AI infrastructure scaling is not sustainable. It can take years to bring a new electrical power plant online. There is a shortage of GPUs industry-wide, and the demand is increasing. Infrastructure costs are skyrocketing. ROI can be challenging. The bottom line is that this industry must improve its efficiency – both for innovation and for the planet.
The need for technological inflections is necessary at every layer of an AI stack – from AI data center infrastructure (compute, storage, networking) to the model layers, and the application layers. These will come at a semi-regular pace (but much faster than what we are traditionally used to). I look forward to this accelerating and bewildering rate of innovation in the industry.
Having said that, we must be more efficient. With machine learning, AI workloads fail at an alarming rate (in some cases 45% of the time) for reasons spanning hardware, firmware, and software. With a failure, a rollback and restart is needed, and depending on the situation, may require human intervention. The nature of machine learning algorithms is that all GPUs must complete their work before they can continue to the next workload. Any weak links in the network, whether marginal optics, sub-optimal switch configurations, or architecture choices can cause $100M in AI infrastructure to sit idle waiting on one GPU to receive the data it needs to complete its work.
How do we know if we are making the best possible use of the infrastructure in place? We know it’s working, but how well is it optimized? As we await the next major inflection to solve the problems we see ahead of us, there are significant opportunities to better utilize what is already in place without building additional infrastructure. Keysight solutions can emulate realistic AI workloads, proactively identifying infrastructure bottlenecks and optimizing AI Data Center infrastructure performance. These offers enable operators to evaluate new algorithms, components, and protocols to improve AI training performance without investing in expensive, large-scale deployments.
As one of my colleagues said, “Keep Calm and Train on”.