Cloud Migration Lessons from Huawei's Ascend Communication Optimizations

Why Cloud Migration Is More Than Just Moving Data

When most people think about cloud migration, they picture lifting virtual machines, shifting databases, or replatforming a legacy monolith. But for AI workloads, the migration story is different. It's not just about where your data lives — it's about how your models talk to the hardware underneath. And that communication layer can make or break your entire deployment.

Huawei's 2012 Lab recently shared some hard-won lessons from optimizing their Pangu model's training and inference on their own Ascend 950 platform. The talk, delivered at the AICon conference in Shenzhen, focused on two stubborn bottlenecks: AllToAll communication in Mixture-of-Experts (MoE) models and the host-to-device transfer of KV caches in ultra-long-context inference. Both are classic cloud migration pain points, even if they don't look like the typical migration checklist items.

The core takeaway? When you move a large AI model to a new cloud or hardware environment, you're not just porting code. You're renegotiating the contract between your software and the physical infrastructure. And that means understanding the quirks of your new home.

The AllToAll Bottleneck: A Cloud Migration Story

Let's start with AllToAll. In MoE models, tokens need to be routed to different experts, and that requires a lot of communication between GPUs or NPUs. The numbers are stark: AllToAll can eat up more than 30% of the end-to-end time in MoE training. That's not a minor overhead — it's a dominant cost center.

When Huawei moved Pangu to Ascend 950, they couldn't just use the same communication libraries that worked on other platforms. The network topology was different, and the hardware had a dedicated communication accelerator called CCU. The generic solution, DeepEP, simply didn't work well on their older Ascend 910A3. So they had to build custom communication operators that were tailored to the Ascend's specific topology and capabilities.

The result? A 10% performance improvement in the EP communication domain's AllToAll. That's not a trivial gain. In a training run that takes weeks, shaving off 10% of the communication time can translate into days of saved compute.

When Host-to-Device Becomes the New Wall

But training isn't the only place where communication bites. In inference, especially with 1M-token contexts, there's a new bottleneck: moving the KV cache from host memory to the device. This host-to-device (H2D) transfer can dominate the time to first token (TTFT). If you've ever waited for a chat AI to start responding to a massive prompt, you've felt this delay.

Huawei's solution involved two prongs. On the hardware side, the Ascend 950 gives each NPU its own dedicated H2D path. On the software side, they built something called Omni Cache, which handles the KV cache offloading and reloading more efficiently. Together, these optimizations cut TTFT by more than 10%.

This is a perfect example of why cloud migration isn't just about picking a provider. It's about understanding that the hardware you land on has its own communication patterns. If you ignore those, you'll leave performance on the table.

Hardware Affinity: The Secret Sauce

The Huawei engineers talk about "hardware affinity" — designing your software to match the specific strengths of the underlying hardware. For instance, the Ascend 950 has a dedicated communication engine, CCU, that can accelerate collective operations. But you have to write your operators to actually use it. The same code that runs on NVIDIA H20 or Ascend 910A2 might not just be slower — it could be actively worse, as the team found when they tried to apply their Ascend 950 optimizations elsewhere.

This is a crucial lesson for anyone migrating AI workloads to the cloud. You can't assume that a "write once, run anywhere" approach will work. The hardware beneath your model is not a neutral substrate. It has its own personality, its own bottlenecks, and its own accelerators.

Practical Takeaways for Your Cloud Migration

So what can you do if you're planning to move an AI workload to a new cloud environment? Here are a few concrete steps, inspired by Huawei's playbook:

• Profile your communication before you migrate. Run a small-scale test to see where your AllToAll or H2D transfers are spending time. You might be surprised.
• Don't assume your existing libraries will work. DeepEP was useless on the Ascend 910A3. Your favorite communication library might not be tuned for your new hardware either.
• Look for dedicated accelerators. Many modern chips have special units for communication, like the CCU on Ascend 950. Learn how to use them, or you're wasting money.
• Design for the topology. The network shape — how NPUs are connected — matters more than you think. A tree topology and a mesh topology require different communication strategies.
• Consider custom operators. If you're hitting a stubborn bottleneck, writing a custom communication operator might be worth it. It's not as scary as it sounds, and the payoff can be significant.

Beyond the Basics: Fusion and Overlapping

The Huawei team also hints at their future direction: fusing operators and using multi-stream parallelism to hide communication behind computation. The goal is to make communication time completely invisible — to overlap it with actual compute so that the end-to-end time is dominated by useful work, not by waiting.

This is a universal principle for cloud migration. Whether you're moving a batch pipeline or a real-time inference service, you want to minimize the time your resources spend idle. Communication is often the hidden idle time. By overlapping communication with computation, you can squeeze more throughput out of the same hardware.

The Takeaway: Cloud Migration Is a Hardware Conversation

Huawei's experience with Pangu on Ascend 950 is a reminder that cloud migration is not just a software exercise. It's a conversation between your code and the physical infrastructure. When you move to a new cloud, you're not just changing where your data lives — you're changing the rules of the game.

The 10% improvements they achieved might seem modest, but in the world of large-scale AI, a 10% reduction in communication time can be the difference between a viable product and a financial drain. And the same principle applies to any cloud migration: the more you understand your new environment, the better you can tune your system to thrive there.

So before you lift and shift your next workload, ask yourself: have you profiled the communication paths? Do you know the hardware's quirks? Are you ready to write custom code if needed? If not, you might be leaving performance — and money — on the table.