https://blog.tensorplay.cn/ Recent content on TensorPlay Blog Hugo -- 0.157.0 en-us Sun, 11 Oct 2026 16:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-02-dispatch-codegen/ Sun, 11 Oct 2026 16:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-02-dispatch-codegen/ How a + b reaches real instructions: two YAML contracts, one code generator, and a lock-free dispatcher that routes each operator to its kernel. https://blog.tensorplay.cn/en/posts/tensorplay-01-tensor-system/ Sun, 11 Oct 2026 15:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-01-tensor-system/ How TensorPlay builds a tensor across DataPtr, Storage, TensorImpl, and Tensor: the move-only deleter, caching allocator, version guard, and zero-copy views. https://blog.tensorplay.cn/en/posts/tensorplay-stax-06/ Sun, 11 Oct 2026 14:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-stax-06/ References for the whole chapter, plus appendices: a source-code index, measurement scripts, and worked solutions for selected exercises. https://blog.tensorplay.cn/en/posts/tensorplay-stax-05/ Sun, 11 Oct 2026 13:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-stax-05/ Content-addressed caching, parallel compilation, fallback semantics, design comparisons against PyTorch 2 and XLA, measurement models, exercises, and labs. https://blog.tensorplay.cn/en/posts/tensorplay-stax-04/ Sun, 11 Oct 2026 12:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-stax-04/ GPU execution and Triton programming models, reduction kernels, multi-reduction fusion, launch configurations, epilogue fusion, and CUDA Graphs. https://blog.tensorplay.cn/en/posts/tensorplay-stax-03/ Sun, 11 Oct 2026 11:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-stax-03/ Buffer dependencies, the greedy fusion algorithm, buffer reuse, affine dependences and tiling, plus the full CPU codegen path from vectorization to wrappers. https://blog.tensorplay.cn/en/posts/tensorplay-stax-02/ Sun, 11 Oct 2026 10:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-stax-02/ Reverse-mode autodiff over the operator-level graph, the minimum-cut model for save-versus-recompute, and lowering to a loop-level intermediate representation. https://blog.tensorplay.cn/en/posts/tensorplay-stax-01/ Sun, 11 Oct 2026 09:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-stax-01/ Just-in-time compilation as program specialization: guards, specialization keys, tracing-time proxy capture, data-dependent control flow, and graph rewrites. https://blog.tensorplay.cn/en/posts/tensorplay-stax-guide/ Sun, 11 Oct 2026 08:00:00 +0800 https://blog.tensorplay.cn/en/posts/tensorplay-stax-guide/ A systematic tour of Stax, TensorPlay&#39;s compiler backend: specialization, graph capture, transformations, autodiff, loop-level IR, scheduling, and codegen. https://blog.tensorplay.cn/en/posts/talk-linear-attention/ Tue, 10 Mar 2026 12:00:00 +0800 https://blog.tensorplay.cn/en/posts/talk-linear-attention/ <h2 id="introduction">Introduction</h2> <p>Since Vaswani et al. proposed the Transformer architecture in 2017, the attention mechanism based on Softmax Attention has become the core component of sequence modeling, supporting the rapid development of Large Language Models (LLMs). However, Softmax Attention inherently suffers from <strong>quadratic computational complexity</strong>: as the sequence length $L$ increases, the computational and memory overhead of $O(L^2d)$ grows quadratically, which has become a core bottleneck for long-sequence modeling and efficient inference and training of LLMs.</p>