xocialize commited on
Commit
5590eab
·
verified ·
1 Parent(s): 320ed33

Add lossless_decode subsection alongside memory_mode (PR #7)

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md CHANGED
@@ -53,6 +53,12 @@ The `lance-mlx` source repo's [`memory_mode`](https://github.com/xocialize/lance
53
 
54
  `relay` produces **byte-identical output** to `parallel` (MD5-verified on real Lance-3B-bf16) — it sheds the UND tower after prefill and frees the GEN tower before VAE decode, so peak memory ≈ heaviest single phase rather than the sum of all three. Default `auto` resolves by `mx.device_info()`'s recommended working-set size with the split at ~18 GiB. The same envelope and modes apply to [`mlx-community/Lance-3B-Video-bf16`](https://huggingface.co/mlx-community/Lance-3B-Video-bf16) for video tasks.
55
 
 
 
 
 
 
 
56
  For VQA-only use cases on small Macs, the compressed [`mlx-community/Lance-3B-AWQ-INT4`](https://huggingface.co/mlx-community/Lance-3B-AWQ-INT4) variant is also available (3.3 GB LLM, 6-9× faster long-form decode; VQA only, not for image generation).
57
 
58
  ## Quickstart
 
53
 
54
  `relay` produces **byte-identical output** to `parallel` (MD5-verified on real Lance-3B-bf16) — it sheds the UND tower after prefill and frees the GEN tower before VAE decode, so peak memory ≈ heaviest single phase rather than the sum of all three. Default `auto` resolves by `mx.device_info()`'s recommended working-set size with the split at ~18 GiB. The same envelope and modes apply to [`mlx-community/Lance-3B-Video-bf16`](https://huggingface.co/mlx-community/Lance-3B-Video-bf16) for video tasks.
55
 
56
+ ### Lossless streaming VAE decode (`lossless_decode`, 2026-06-05)
57
+
58
+ `generate(..., lossless_decode=True)` (default since 2026-06-05) uses a **bit-identical** streaming VAE decode that's lighter than the naive whole `dec(z)` in every measured config — via temporal causal-cache streaming (flat in frame count) and spatial halo-tile + crop (no blend ⇒ exact). 50-case bit-identity test in [`lance-mlx`](https://github.com/xocialize/lance-mlx) with a negative control verifies the guarantee. `lossless_decode=False` keeps the lossy trapezoidal-blend tiling from the prior path (~1.5–4.8 / 255 off the reference) for the one config where lossless exceeds 16 GB (768² video; lossless on 1024² image fits 16 GB cleanly at ~12.2 GB).
59
+
60
+ Full envelope and `ri_phys`-measured numbers (true OS-committed footprint, supersedes the older `mx.get_peak_memory()` quotes which under-report ~2×) in the source repo's [`LIMITS.md`](https://github.com/xocialize/lance-mlx/blob/main/LIMITS.md).
61
+
62
  For VQA-only use cases on small Macs, the compressed [`mlx-community/Lance-3B-AWQ-INT4`](https://huggingface.co/mlx-community/Lance-3B-AWQ-INT4) variant is also available (3.3 GB LLM, 6-9× faster long-form decode; VQA only, not for image generation).
63
 
64
  ## Quickstart