BlockLock Pins Fixed Geometry for Target Requests, Delivers Bitwise Match on Five Models
BlockLock enforces fixed batch geometry for selected requests without full kernel replacement or prefill isolation. It achieves bitwise reproducibility on five dense models across three GPU generations while cutting latency 53.5–55 % versus vLLM Batch-Invariant. The approach limits constraints to protected paths and needs no new operator implementations.
BlockLock implements two modes. BlockLock-Rows locks row width for the protected sequence and continues batched decode for all requests. BlockLock-Pack anchors terminal packing with an isolated dummy segment then extracts the protected KV state for isolated decode. Both operate under a stated per-deployment contract that isolates only the protected path.
On NVIDIA A6000, A100 and H100 the method was measured against tinyBenchmarks and BFCL. BlockLock-Rows reduced latency 53.5–55.0 % and raised throughput 115.2–122.2 % relative to vLLM Batch-Invariant at equal delivered work. Target sequences and Top20-logit bits matched exactly in all reported runs. Initial two-GPU tensor-parallel tests on A6000 pairs reproduced the same bitwise result.
Existing batch-invariant kernels force every operator to identical shapes; isolating prefill with fixed-shape decode discards shared compute. BlockLock constrains geometry only for protected requests and requires no custom kernels, directly addressing the continuous-batching nondeterminism documented in vLLM PagedAttention and the numerical sensitivity shown in FlashAttention-2 evaluations.
Production inference stacks can adopt BlockLock-Rows first for single-GPU latency-sensitive services, then extend Pack mode for multi-GPU tensor-parallel workloads where KV extraction overhead remains below 4 %.
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2611.06977)
- [2]Supporting Source(https://arxiv.org/abs/2309.08872)
- [3]Supporting Source(https://arxiv.org/abs/2307.08691)