Broadcom has launched VeloRAIN, an AI-optimized networking system specifically designed to manage encrypted traffic and maintain high-speed data flows at the edge of networks, where data is processed closer to its source rather than in a centralized data center.
VeloRAIN Takes on Encrypted Traffic Challenges
VeloRAIN, now part of the VeloCloud SD-WAN portfolio, brings AI and machine learning capabilities to network traffic management. For years, managing encrypted AI traffic has been a hurdle for optimizing network quality, but VeloRAIN aims to decode and prioritize these complex data flows efficiently. “By harnessing the advanced capabilities of VeloRAIN, AI workloads from distributed inferencing and agentic peer-to-peer applications,” remarked Sanjay Uppal, Broadcom’s VeloCloud head, who explained that VeloRAIN is engineered to maximize quality-of-service (QoS) in data-heavy environments.
This advancement includes tools for stabilizing wireless network performance, especially on 5G and satellite connections. Channel estimation techniques make wireless data transfers more stable, aiming to provide fiber-like speeds even in unpredictable conditions.
DABS and Policy-Driven Optimization
One of the most advanced features in VeloRAIN is Dynamic Application-Based Slicing (DABS). By assigning network resources based on the needs of each AI application, DABS ensures that critical data flows remain uninterrupted. It also introduces user-centric policy frameworks that prioritize network access depending on user roles, ensuring high-priority traffic is managed effectively. Admins can automate these policies to maintain streamlined operations.
Broadcom also released new hardware, including the VeloCloud Edge 4100 and 5100, high-performance edge appliances designed for large enterprises, regional hubs, and data centers. The Edge 4100 handles up to 30 Gbps, a significant performance boost over older models, while the Edge 5100, with its 100 Gbps capability, targets large-scale operations. These devices simplify network setups by reducing the number of appliances required, while offering enhanced tunnel capacity for complex networking needs.
Broadcom’s Custom Chips and Strategic Partnerships
In addition to providing networking advancements, Broadcom is deeply involved in OpenAI’s custom chip project. The collaboration focuses on inference chips, used for AI tasks like real-time user queries rather than model training. Broadcom’s prior work with Google in developing Tensor Processing Units (TPUs) positions it well to manage the data traffic that occurs when multiple AI chips operate simultaneously.
Initially, OpenAI explored building its fabrication plant with TSMC and even designed early chips with Marvell. However, working with Broadcom allows OpenAI to capitalize on existing expertise in chip-to-chip communication, making their infrastructure more efficient and less costly. This strategic shift is vital, given OpenAI’s projected $5 billion loss for the year and the rising costs of running expansive AI models.
OpenAI’s Shift from a $7 Trillion Foundry Vision
In a pivot that’s reshaping its hardware strategy, OpenAI recently ditched its massive $7 trillion foundry initiative, a project meant to secure its AI infrastructure through a global network of chip manufacturing plants. The plan, initially proposed to combat AI hardware shortages, faced insurmountable logistical and cost barriers. Instead, OpenAI has turned to partnerships with TSMC and Broadcom to develop custom AI chips optimized for its models, such as ChatGPT (source: Reuters report).
TSMC’s A16 process node, set to be production-ready by 2026, is at the heart of this new approach. The A16 technology, built on a 1.6-nanometer process, offers higher efficiency and performance, which is crucial for AI applications. Apple has also reserved capacity for this tech, aiming to use it in AI-driven features integrated into its devices.
Nvidia, AMD, and the Competition for AI Hardware
Despite diversifying its chip suppliers, OpenAI has maintained a complex but essential relationship with Nvidia. By refraining from poaching Nvidia’s engineers, OpenAI keeps access to Nvidia’s high-performing Blackwell GPUs, crucial for ongoing model training. Meanwhile, Microsoft has bolstered its AI offerings by integrating AMD’s MI300X chips into Azure, and AMD is on track to generate $4.5 billion in AI hardware revenue in 2024.
Microsoft’s own Cobalt and Maia chips, based on Arm architecture, have been gaining traction among enterprises, signaling how the AI hardware landscape is increasingly competitive.
ByteDance Navigates Sanctions with Broadcom
In June, ByteDance took steps to sidestep U.S. trade restrictions by designing a 5nm AI chip in partnership with Broadcom, with TSMC slated for production. The collaboration aims to ensure ByteDance remains compliant with U.S. regulations while advancing its AI technology. Already, ByteDance leases Nvidia hardware from U.S.-based cloud providers to comply with sanctions, showing the company’s careful navigation of geopolitical tensions.
ByteDance’s move could serve as a model for other Chinese tech firms facing similar challenges, and Broadcom’s role in this project highlights its expanding global influence.
Last Updated on November 7, 2024 2:12 pm CET
