AMD / VIDEO GEN

TeaCache for LTX2 on AMD: Why Unified Memory Changes the Design

LTX2 (LTXAV) audio-video generation on AMD APUs has a different bottleneck than it does on a discrete card, and the existing TeaCache implementations are built for the discrete case — when they run at all. Halo-TeaCache is a lean, single-file ComfyUI node that caches LTX2's 48-layer dual-stream transformer without ever touching the CPU.

What TeaCache actually does

Diffusion sampling is a loop. Each denoising step feeds a slightly-less-noisy latent through the full transformer stack and gets a residual back. The observation behind TeaCache is that consecutive steps often produce very similar intermediate representations — the model is doing near-identical work twice in a row, and paying full price for it both times.

So instead of running the stack, you estimate whether this step's input is close enough to the last one that the output would be close too. If it is, you skip the computation entirely and re-add the residual you already have. If it isn't, you run the full pass and refresh the cache.

In Halo-TeaCache the loop looks like this, per the repo's own description:

1. Before each denoising step, computes a modulated input from the video timestep embedding
2. Compares L1 distance to previous step's modulated input using polynomial coefficients
3. If distance is below threshold → skip all 48 transformer layers, add cached residual
4. If above → run full computation, cache the new residual
5. Both video and audio residuals are cached together (they're coupled via cross-attention)

That last point is the LTX2-specific part. LTXAV is a dual-stream model: video and audio move through the transformer together, coupled through cross-attention. You cannot cache one and recompute the other — the two residuals are stored and restored as a pair, keyed by cond/uncond slice so classifier-free guidance branches get their own cache state.

The AMD unified memory argument

Most TeaCache implementations expose a cache_device toggle: keep the cached residuals in VRAM for speed, or push them to system RAM when VRAM is tight. On a discrete GPU that's a real tradeoff, because moving a cache off-device means paying a PCIe transfer every time you touch it.

On AMD APUs, the tradeoff evaporates. As the README puts it:

On AMD APUs (Strix Halo, etc.), CPU and GPU share the same physical memory. There's no PCIe transfer penalty for keeping the cache on "GPU" — it's all the same address space. This eliminates the cache_device toggle that other TeaCache implementations need.

This is not a micro-optimization; it's a design simplification that removes an entire axis of configuration, an entire class of device-mismatch bugs, and a lot of code. Halo-TeaCache is roughly 250 lines against the original's roughly 1000. There is no offload path because on this hardware an offload path is a fiction — "system RAM" and "VRAM" are the same pool of bytes with different labels on them.

Avoiding CPU offload matters for a second reason too. Every device toggle is a place where a tensor can end up on the wrong device mid-sample, and ROCm users already spend enough time debugging that category of failure. Keeping the cache resident and unconditional means the fast path and the correct path are the same path.

Why the original crashes on LTX2

The other reason this exists: the original TeaCache patches the model's full forward method. Against LTXAV that crashes. Halo-TeaCache instead patches _process_transformer_blocks — the repo calls this "surgical" versus "full method" — which is exactly the boundary where the residual is well-defined and where the video and audio tensors are both in hand. The swap is done with unittest.mock.patch scoped to the sampling call, so nothing is permanently monkeypatched onto the model object.

Summarized from the repo's comparison table:

Measured performance

The repo reports these figures on AMD Strix Halo, LTX2 19B fp8, 121 frames @ 24fps:

Note the shape of that: a cache hit isn't free, it's ~10.7s against ~16.5s, and hits are a fraction of the steps. This is an honest, moderate win on a real workload rather than a headline multiplier — which is what you'd expect from a technique that only skips work when the math says the skip is safe.

Installing it

It's a ComfyUI custom node with no additional dependencies:

cd ComfyUI/custom_nodes/
git clone https://github.com/bkpaine1/Halo-TeaCache.git
# Restart ComfyUI

Then in your workflow: add the Halo-TeaCache node, and connect your LTX2 model through it before the CFGGuider/Sampler. It takes a MODEL in and returns a MODEL out, so it drops into the model path like any other patcher node.

The three knobs

Setting rel_l1_thresh to 0 disables the node entirely and passes the model straight through — useful for A/B comparisons without rewiring the graph.

The start_percent default of 0.15 encodes something worth understanding. Early denoising steps decide the coarse structure of the video; that's where a bad skip does the most visible damage. Late steps are refining detail, where a reused residual is far more forgiving. Gating the cache to the back end of the schedule is how you buy speed out of the cheap part of the sampler.

The repo's tuning guidance:

• Blurry output? Lower rel_l1_thresh (try 0.10-0.12)
• Want more speed? Raise rel_l1_thresh (try 0.25-0.30)
• Quality on early steps matters most — keep start_percent at 0.10-0.20

Compatibility

Models: LTX2 (LTXAV) and LTXv (LTXVModel). The node checks the diffusion model's class name and warns if it sees anything other than LTXAVModel or LTXVModel, then proceeds anyway — so an unexpected architecture gets a console warning rather than a hard stop.

Hardware: AMD APUs with unified memory are what it's designed for; the README notes discrete GPUs work fine too. You simply don't get the specific benefit of the eliminated offload path, because on a discrete card there was something real to offload. ComfyUI: tested with latest (Jan 2026).

The polynomial coefficients used for the L1 distance rescaling are the LTXV baseline, reused for LTXAV on the grounds that it's the same transformer architecture. That's the one place the implementation is leaning on an inherited constant rather than something derived for LTX2 specifically — worth knowing if you're tuning and the threshold behavior feels off from what you'd expect.

Why lean matters

The AMD ROCm video-generation stack is not a place where you want a thousand lines of device-juggling logic between you and the sampler. Every abstraction that exists to paper over a discrete-GPU constraint is, on unified memory, pure surface area for bugs. Halo-TeaCache is one file, one node, three parameters, MIT licensed. If it breaks you can read the whole thing in a sitting and find out why.

Source and full code: github.com/bkpaine1/Halo-TeaCache