# Beyond the Rack: How Co-Packaged Optics (CPO) and Next-Gen Storage Are Solving AI's Data Movement Problem
At CIOE 2026 in Shenzhen, a fascinating shift in conversation emerged among data center architects. Five years ago, discussions centered on GPU TFLOPS and CPU core counts. Today? The obsession is PB/s (petabytes per second) and nanosecond latency budgets.
Why? Because in the age of trillion-parameter AI models and real-time inference, the bottleneck has moved from computation to data movement. A modern H200 GPU can process 4.8 PFLOPS—but it's starving for data. Training a GPT-4 class model requires shuffling 45 petabytes of training data across thousands of GPUs, billions of times over.
This article explores two converging technologies showcased at CIOE 2026 that are redefining AI infrastructure: 1. Co-Packaged Optics (CPO) and Near-Package Optics (NPO) – eliminating electrical bottlenecks 2. Next-generation enterprise storage – SSDs and memory systems built for AI's insatiable appetite
Traditional optical modules connect to switches via electrical traces on a PCB. As data rates climb past 1.6Tbps, these traces become insurmountable barriers:
- Signal loss: 30-40 dB/m at 112Gbps PAM4 (56GHz) - Power consumption: Retimers and DSPs burn 15-20W per module - Latency: Electrical SerDes adds 100-200ns round-trip - Density: QSFP-DD faceplate limits to ~51Tbps per switch ASIC
Enter Co-Packaged Optics: integrating optical engines inside the same package as the switching silicon.
Walking through the exhibition, CPO had transitioned from conference papers to shipping silicon:
#### 1. 6.4T NPO Optical Engines (华工正源/Huagong Zhongyuan) - Configuration: 8× 800G lanes using quantum dot DWDM lasers - Power efficiency: 3.5 pJ/bit (vs. 8 pJ/bit for pluggable modules) - Form factor: Integrated into switch package substrate - Status: Sampling to Tier-1 hyperscalers in Q4 2026
#### 2. Silicon Photonics Platforms (GlobalFoundries) Their 45nm CMOS-compatible SiPh process enables: - Monolithic integration of lasers, modulators, and photodetectors - Wafer-scale testing reducing assembly costs by 60% - Multi-wavelength sources for DWDM applications
#### 3. Light联芯科's Hardware Breakthrough The Chinese startup unveiled the ZW1000 maskless programmable lithography system for silicon photonics: - Rapid prototyping: Weeks instead of months for new designs - Low-volume production: Economical for <10K units/year - Flexibility: Reconfigurable waveguides for custom applications
This addresses CPO's biggest challenge: design iteration cycles. Traditional photomask costs ($200K+) made experimentation prohibitively expensive.
Not all applications demand CPO's complexity. Here's when each technology makes sense:
| Architecture | Best For | Bandwidth | Power | Cost | Availability | |--------------|----------|-----------|-------|------|--------------| | Pluggable (QSFP-DD) | Spine-leaf networks | 800G-1.6T | 15-20W | $ | Now | | LPO (Linear Pluggable) | Rack-to-rack <100m | 800G | 8-12W | $$ | Now | | NPO (Near-Package) | AI training clusters | 3.2-6.4T | 5-8W | $$$ | 2027 | | CPO (Co-Packaged) | Switch fabrics | 12.8T+ | 3-5W | $$$$ | 2028 |
Key Insight from CIOE: For 2026-2027 deployments, hybrid architectures dominate: - CPO for spine switches (51.2Tbps+ per ASIC) - Pluggable 800G for leaf switches - LPO for compute fabric GPU interconnects
For engineers evaluating CPO solutions, understanding the architecture is critical:
#### Traditional Pluggable Module Path: ``` GPU → PCIe → NIC ASIC → PCB traces (30cm) → Retimer → Module DSP → Laser → Fiber → Photodiode → DSP → Retimer → PCB traces → Switch ASIC ``` Total latency: ~800ns Power: ~35W for 800G bidirectional
#### CPO Path: ``` Switch ASIC → On-package waveguide (3mm) → Modulator → Fiber → Photodiode → Waveguide → Switch ASIC ``` Total latency: ~200ns Power: ~8W for 800G bidirectional
The latency reduction is transformative for synchronous parallel training (e.g., ZeRO-3, FSDP), where collective communication primitives (all-reduce, all-gather) dominate execution time.
Despite compelling technical advantages, CPO faces hurdles:
1. Supply chain immaturity: Only 3-4 qualified vendors globally 2. Switch ASIC redesign: Requires co-development with Broadcom/Nvidia/Marvell 3. Thermal management: 400W+ switch ASICs + optical engines = complex cooling 4. Repair economics: Dead optical engine = scrap entire switch module ($50K+)
Industry consensus at CIOE: CPO will capture 15-20% of 2028 switch market, concentrated in hyperscale AI clusters. Pluggable modules remain dominant for enterprise and telco.
While everyone fixates on GPUs, CIOE's enterprise storage exhibits told an equally important story: AI's data pipeline is only as fast as its slowest component.
Training a 1 trillion-parameter model on 10TB of data requires: - Dataset loading: 10TB read from storage → GPU memory - Checkpointing: 2TB model weights written every epoch (10-100× per training run) - Shuffle operations: Random access patterns across entire dataset - Gradient synchronization: Distributed writes during backpropagation
Traditional SAS HDDs? Laughably inadequate. Even SATA SSDs struggle. This is where NVMe PCIe 5.0 SSDs and DDR5 persistent memory enter the picture.
#### 1. PCIe 5.0 Enterprise SSDs: The New Baseline Topstar's exhibit included Micron 7500 Series and Kingston DC1500M drives demonstrating: - Sequential read: 14,000 MB/s (vs. 3,500 MB/s for PCIe 3.0) - Random read IOPS: 2.5M (crucial for distributed training data loading) - Endurance: 3 DWPD (Drive Writes Per Day) for 5 years = 82PB total writes on a 15TB drive
Real-world impact: A 16-drive NVMe JBOF can saturate dual 100GbE links for dataset streaming—eliminating storage as a training bottleneck.
#### 2. DDR5-5600 Server Memory: Beyond Capacity AI workloads demand both capacity (512GB-2TB per server) and bandwidth (>400GB/s per socket). DDR5-5600 delivers: - Bandwidth: 44.8 GB/s per DIMM (vs. 32 GB/s for DDR4-3200) - Capacity: 128GB RDIMM modules becoming standard - Power efficiency: 1.1V vs. 1.2V for DDR4 (10% reduction)
At CIOE, we saw Samsung's 512GB DDR5-5600 RDIMMs – enabling 2TB per socket (16× DIMMs) for LLM inference servers. This allows entire 70B-parameter models to remain resident in DRAM, eliminating model swapping overhead.
#### 3. CXL Memory Expansion: The Game Changer Compute Express Link (CXL) 2.0 enables memory pooling across servers: - Shared memory pool: Multiple CPUs access common DRAM/persistent memory - Latency: <200ns for remote memory access (vs. milliseconds for NVMe) - Use case: LLM inference clusters share KV-cache across nodes
While still emerging (limited production in 2026), CXL memory expansion could reduce inference cluster costs by 30% by eliminating per-node memory overprovisioning.
Based on CIOE discussions with data center architects, the optimal AI storage stack is:
#### Tier 1: GPU HBM (High Bandwidth Memory) - Capacity: 80-192GB per GPU (H200/B200) - Bandwidth: 4-5 TB/s - Purpose: Active training batches, model weights during forward/backward pass
#### Tier 2: Host DRAM (DDR5) - Capacity: 1-2TB per server - Bandwidth: 400-800 GB/s per socket - Purpose: Dataset prefetching, gradient accumulation, inference KV-cache
#### Tier 3: NVMe SSD Storage - Capacity: 60-240TB per server (4-16× 15TB drives) - Bandwidth: 50-200 GB/s (RAID 0 across drives) - Purpose: Full dataset storage, checkpointing, long-term model versioning
Critical ratio: Aim for Host DRAM : NVMe = 1:50 (e.g., 2TB DRAM, 100TB NVMe). This ensures dataset prefetching never stalls GPU training.
Consider a 1,000-GPU AI training cluster: - GPU cost: $30M (1,000× $30K per H100) - Annual power: $5M ($150/GPU × 1,000 × 8,760 hours × $0.12/kWh) - Opportunity cost: $5,000/hour for idle cluster ($43M/year at 100% utilization)
Now imagine storage bottlenecks reduce effective GPU utilization to 85% (15% idle waiting for data): - Wasted GPU time: $6.45M/year - Cost to eliminate bottleneck: $500K (premium NVMe + DDR5 + 100GbE networking)
ROI: 1,200% in year one. This is why hyperscalers don't hesitate to deploy $2,000 PCIe 5.0 SSDs.
As both a CIOE 2026 participant and industry supplier, Topstar Technology uniquely positions customers for AI infrastructure success:
#### Optical Interconnect - 800G/1.6T QSFP-DD/OSFP modules - 400G/200G solutions for hybrid architectures - AOC/DAC cables for rack-scale connectivity - MPO/MTP fiber harnesses
#### Enterprise Storage - SSDs: Micron, Kingston, Samsung enterprise NVMe (PCIe 4.0/5.0) - Server Memory: DDR5-4800/5600 RDIMMs and LRDIMMs - GPU Memory: HBM2e/HBM3 for AI accelerators (OEM sourcing)
#### AI Server Components - NVIDIA H100/H200/B100 GPUs (authorized distributor) - Mellanox ConnectX-7 NICs (400GbE for RDMA) - Broadcom switches and SmartNICs
#### 1. Single-Vendor Simplicity Stop managing 10 different suppliers. We deliver complete AI rack solutions: - Bill of Materials (BOM) optimization - Compatibility pre-validation - Consolidated shipping and support
#### 2. Flexible Sourcing Options - OEM-grade components for hyperscale deployments - Original brand modules (Cisco/Arista/Mellanox compatible) - White-box solutions for cost-sensitive projects
#### 3. Technical Co-Development Our engineers collaborate on: - Network topology design (Clos, Fat-tree, Dragonfly) - Storage tier sizing based on workload profiling - Power and cooling budgets for rack-level deployments
#### 4. Rapid Deployment Timelines In an industry with 16-24 week lead times, we maintain: - Strategic component inventory ($5M+ stock) - 4-8 week delivery for standard configurations - Expedited options for urgent deployments
A recent customer project illustrates our end-to-end capability:
Requirements: - 500× NVIDIA H100 GPUs - 100TB dataset storage per node - <10μs network latency for all-reduce operations
Topstar Solution: - Networking: - 500× Mellanox ConnectX-7 400GbE NICs - 64× 800G QSFP-DD optical modules for spine switches - 1,024× 400G QSFP-DD modules for leaf switches - Storage: - 4,000× Kingston DC1500M 15TB NVMe SSDs (8 per server) - RAID controllers with 8GB cache per server - Memory: - 32,000× Samsung DDR5-5600 64GB RDIMMs (64 per server, 4TB/node)
Deployment Timeline: - Week 1-2: BOM finalization and procurement - Week 3-6: Component delivery and integration - Week 7-8: Rack installation and network commissioning - Week 9: Workload validation and handoff
Result: Cluster achieved 94.2% GPU utilization during week-long training runs—among the highest in the industry.
Based on CIOE 2026 insights and industry roadmaps, here's what's coming:
- 3.2T optical modules enter volume production - DDR6 sampling begins (7,200 MT/s, 60GB/s per DIMM) - PCIe 6.0 SSDs deliver 28GB/s sequential reads - CXL 3.0 enables memory pooling across racks
- 12.8T switch ASICs with integrated CPO - 400G per-lane signaling (vs. today's 200G max) - Optical circuit switching for dynamic bandwidth allocation - First commercial photonic AI accelerators (analog optical computing)
- Rack-level optical switching replacing electrical distribution - Silicon photonics becomes cheaper than copper at >100Gbps - Persistent memory (CXL-attached) replaces NVMe for warm storage - Quantum-entangled secure links for inter-DC AI model synchronization
The AI models capturing headlines—GPT-5, Gemini Ultra, Claude Opus—are only as capable as the infrastructure beneath them. At CIOE 2026, the message was unambiguous: the era of AI is being built one optical module, one SSD, and one memory DIMM at a time.
For organizations deploying AI infrastructure, three imperatives emerged:
1. Don't wait for perfect technology – 800G and DDR5 are production-ready now 2. Design for data movement, not just compute – Storage and networking are co-equal with GPUs 3. Partner with suppliers who understand the stack – Optical, storage, and compute must be architected together
Topstar Technology stands at this intersection—bringing together cutting-edge components, deep technical expertise, and the supply chain agility to deliver on aggressive deployment timelines. Whether you're building your first AI cluster or scaling to 100,000 GPUs, we're ready to be your infrastructure partner.
The future of intelligence is being built today. Let's build it together.
---
Topstar is a leading supplier of AI infrastructure components, specializing in: - Optical Modules: 800G/1.6T QSFP-DD, 400G OSFP, CPO/NPO solutions - Enterprise Storage: NVMe SSDs (PCIe 4.0/5.0), Enterprise SATA, U.2/U.3 form factors - Server Memory: DDR5 RDIMMs/LRDIMMs, HBM for AI accelerators - GPU & Networking: NVIDIA AI GPUs, Mellanox SmartNICs, Broadcom switches
With 15+ years serving hyperscale data centers, telecommunications carriers, and enterprise customers, we combine OEM-grade quality with white-box economics.
📧 Engineering Inquiries: dsale@topsfp.com 🌐 Product Catalog: www.opticalmodulemanufacturers.com | www.qsfp28module.com 📱 Connect: [Facebook](https://www.facebook.com/topstar.sfp/) | [LinkedIn](https://www.linkedin.com/company/13177171/)
---
Keywords: Co-packaged optics, CPO, NPO, AI data center, enterprise SSD, DDR5 server memory, NVMe PCIe 5.0, silicon photonics, optical interconnect, GPU storage, CIOE 2026, high-speed networking, AI infrastructure
SEO Tags: #CoPackagedOptics #CPO #NPO #AIInfrastructure #EnterpriseSSD #DDR5Memory #DataCenter #CIOE2026 #SiliconPhotonics #GPUStorage #OpticalModules #NVMe
اتصل شخص: Ms. Vicky Tian
الهاتف :: +86 19860146913
الفاكس: 86-0755-82552969
Arabic
