Avoid long-blocking H2D copies in ViT

[maxyanghu]

Purpose

Two host-to-device copies on the per-iteration critical path can take a very
long time to return. Despite non_blocking=True, the cudaMemcpyAsync call
blocks the calling thread, and the time it takes tracks the amount of GPU work
that has already been queued. That undoes the host run-ahead that asynchronous
scheduling and CUDA graphs exist to build up.

Neither call site looks suspicious at a glance, which is why this is easy to
miss:

M-RoPE positions (vllm/v1/worker/gpu_model_runner.py). The staging buffer
is pinned. However mrope_positions is allocated as [3, max_num_tokens + 1],
where the dummy trailing column is deliberately there to keep the tensor
non-contiguous for torch.compile. That makes cpu[:, :N] a strided view of a
pinned buffer. Using CudaMemcpyAsync with non-contiguous buffer leads to a silent synchronization.
Copying the three rows individually — each row being contiguous
within that same pinned allocation — makes the synchronization go away.

Before M-RoPE fix:

After M-RoPE fix:

As shown above, the new three HtoDs became truly pinned and the synchronization goes away. There is still a very long cudamemcpyasync because we haven't fixed the following vision position ids copies yet.

Vision position ids (vllm/model_executor/models/qwen3_vl.py).
rot_pos_ids() builds its per-image tensors from numpy, so the concatenated
pos_ids is not pinned. Calling .pin_memory() before
.to(device, non_blocking=True) makes the long copies go away. pin_memory()
is a host-side copy only and does not change what is transferred.

Before vision position ids fix:

After vision position ids fix:

After these two fixes, cudaMemcpyAsync synchronizations are totally eliminated.

Scope

The M-RoPE change affects models using M-RoPE (Qwen2-VL, Qwen2.5-VL, Qwen3-VL)
on the V1 model runner. Model Runner V2 stages positions through UVA buffers and
does not perform this copy, so it is unaffected.

The vision position id change is specific to Qwen3-VL and is independent of the
model runner.

Test Plan

Existing Qwen-VL correctness tests. The changes affect only how the transfer is
issued, not what is transferred.

Test Result

Output is unchanged. Profiling shows the long-blocking copies are no longer
present at either call site.

What is measured, and what is not

A microbenchmark that issues a single copy behind a fixed, deep queue of GPU
work, sweeping only the copy size, shows a payload threshold. Below it the
cudaMemcpyAsync call returns in microseconds. Above it the call takes roughly
as long as the GPU work already queued, and stays at that cost as the payload
grows further.

Being explicit about the limits of that result:

• The root cause is not established. We can show the threshold exists and is
  reproducible; we cannot yet say why.
• The observation is that above the threshold the call becomes very long. We
  have not demonstrated that it is specifically a stream synchronization.
• The threshold was found in a microbenchmark. It has not yet been confirmed
  end to end on a production model configuration.
• We have not established what PyTorch does internally for a strided host
  source, so no claim is made about that path here.

What is directly observable is the part this PR relies on: at both sites, issuing
the transfer from pinned, contiguous memory removes the long-blocking copies.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[Isotr0py]

LGTM, thanks!

[Isotr0py]

/ci run

[github-actions]

✅ Triggered Buildkite CI #83395 for commit bb74c9bffca1.

[mergify]

Hi @maxyanghu, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[mgoin]

/ci run

[github-actions]

✅ Triggered Buildkite CI #83439 for commit 7e8568beb862.

[Isotr0py]

/ci retry

[github-actions]

✅ Queued 3 failed job(s) for retry in Buildkite CI #83439.