QVAC-19998 feat(ltx): full-GPU LTX-2.3 video support (Metal)#13
Conversation
|
The default build path looks broken on this branch. Verified on a clean checkout of df47d5e: cmake .. with default flags (the flow docs/build.md documents) — exit 1: CMake Error at CMakeLists.txt:312 (add_subdirectory): Questions: Is consuming this repo directly (without the vcpkg overlay) unsupported on this branch? If yes, can we either flip SD_USE_SYSTEM_GGML default to ON, or replace the add_subdirectory(ggml) fallback with a message(FATAL_ERROR ...) pointing users at -DSD_USE_SYSTEM_GGML=ON + the qvac-ext-ggml package? |
Addresses review feedback on PR #13. After removing the ggml submodule, a plain `cmake ..` defaulted SD_USE_SYSTEM_GGML=OFF and hit add_subdirectory(ggml) on a now-missing directory, failing with a confusing CMake error. - Default SD_USE_SYSTEM_GGML to ON (system/vcpkg ggml is the only supported path on this branch). - Replace the add_subdirectory(ggml) fallback with explicit FATAL_ERROR messages pointing at the qvac-ext-ggml vcpkg port and the vcpkg toolchain file (both for missing system ggml and for an explicit -DSD_USE_SYSTEM_GGML=OFF with no submodule present). - Update docs/build.md: drop the --recursive / git submodule update instructions and document the system/vcpkg ggml workflow. Note that the port exports GGML_MAX_NAME=128 as a PUBLIC compile definition so consumers inherit it automatically. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks @dev-nid — all four points addressed in 6a13b9c:
|
|
I see this PR is adding stuff not related to LTX. I think the problem is that 2026-06-04 is not the correct rebase of 2026-03-01 |
Addresses review feedback on PR #13. After removing the ggml submodule, a plain `cmake ..` defaulted SD_USE_SYSTEM_GGML=OFF and hit add_subdirectory(ggml) on a now-missing directory, failing with a confusing CMake error. - Default SD_USE_SYSTEM_GGML to ON (system/vcpkg ggml is the only supported path on this branch). - Replace the add_subdirectory(ggml) fallback with explicit FATAL_ERROR messages pointing at the qvac-ext-ggml vcpkg port and the vcpkg toolchain file (both for missing system ggml and for an explicit -DSD_USE_SYSTEM_GGML=OFF with no submodule present). - Update docs/build.md: drop the --recursive / git submodule update instructions and document the system/vcpkg ggml workflow. Note that the port exports GGML_MAX_NAME=128 as a PUBLIC compile definition so consumers inherit it automatically. Co-authored-by: Cursor <cursoragent@cursor.com>
ggml is now provided entirely by the vcpkg ggml port (tetherto/qvac-ext-ggml). The submodule is not needed and was only present for non-vcpkg local builds. Co-authored-by: Cursor <cursoragent@cursor.com>
The ../ggml/src/ggml-impl.h path was a remnant of the git submodule. All used symbols (GGML_MAX_DIMS, GGML_MAX_SRC, GGML_ASSERT, ggml_graph_n_nodes, ggml_graph_node) are in the public ggml.h. Co-authored-by: Cursor <cursoragent@cursor.com>
ggml_cgraph is now opaque (ggml-impl.h no longer vendored). Replace direct member access with the public leaf accessors exported by qvac-ext-ggml 2026-06-06, fixing 'member access into incomplete type ggml_cgraph'. Co-authored-by: Cursor <cursoragent@cursor.com>
SDContextParams::to_sd_ctx_params_t left preferred_gpu_backend out of the sd_ctx_params_t aggregate initializer, so it zero-initialized to SD_BACKEND_PREF_CPU and the whole LTX pipeline ran on CPU even on Metal machines. Add SD_BACKEND_PREF_GPU explicitly; it is only honored when --backend is unset, so existing --backend overrides are unaffected. Co-authored-by: Cursor <cursoragent@cursor.com>
With the LTX-2 additions, the stable-diffusion.cpp translation unit exceeds MSVC's COFF 2^16 section limit and fails to compile on Windows with fatal error C1128. /bigobj raises the limit; clang/gcc are unaffected. Co-authored-by: Cursor <cursoragent@cursor.com>
Addresses review feedback on PR #13. After removing the ggml submodule, a plain `cmake ..` defaulted SD_USE_SYSTEM_GGML=OFF and hit add_subdirectory(ggml) on a now-missing directory, failing with a confusing CMake error. - Default SD_USE_SYSTEM_GGML to ON (system/vcpkg ggml is the only supported path on this branch). - Replace the add_subdirectory(ggml) fallback with explicit FATAL_ERROR messages pointing at the qvac-ext-ggml vcpkg port and the vcpkg toolchain file (both for missing system ggml and for an explicit -DSD_USE_SYSTEM_GGML=OFF with no submodule present). - Update docs/build.md: drop the --recursive / git submodule update instructions and document the system/vcpkg ggml workflow. Note that the port exports GGML_MAX_NAME=128 as a PUBLIC compile definition so consumers inherit it automatically. Co-authored-by: Cursor <cursoragent@cursor.com>
|
The base branch 2026-06-04 seems incorrect. Its not a clean branch with upstream-only work (contains an additional commit that should be on this PR). |
It seems correct now, right ? |
Why would Flux rope be part of my PR in your diagram? Flux Rope is replayed ontop of the rebased branch because it's what we added separate from leejit. This PR is only LTX additions, and the base branch 2026-06-04 is the target which should contain rope additions, not only things from leejit. |
|
ggml submodule repointed to the canonical The
Context on the base-branch discussion (re @jesusmb1995 / @gianni-cor): this line intentionally depends on qvac-ext-ggml — the fused Flux RoPE path in |
Resolve the ggml submodule conflict introduced by the base repoint (55352fa). The base re-added/repointed the vendored ggml submodule, while this branch removes it in favor of the vcpkg system ggml port (47dbd6b). Keep the submodule removed: ggml is provided by the qvac-ext-ggml vcpkg port (pinned to 2026-06-06@805e8e1b), so the merged tree matches the 2026-06-04-ltx head. Co-authored-by: Cursor <cursoragent@cursor.com>
Reverses the submodule removal direction: instead of consuming ggml only from the qvac-ext-ggml vcpkg port, keep the vendored ggml submodule so the repo can also be built standalone without vcpkg. - Re-add the ggml submodule pinned to qvac-ext-ggml 2026-06-06@805e8e1b (the merged compute branch: Metal fused Flux RoPE/conv2d, Wan IM2COL_3D/PAD, coopmat1 f32 flash-attn fixes, public graph_leaf API). - CMake: default SD_USE_SYSTEM_GGML back to OFF so a plain `cmake ..` after a --recursive clone builds the vendored submodule via add_subdirectory. System/vcpkg ggml is still fully supported via -DSD_USE_SYSTEM_GGML=ON (the diffusion-cpp vcpkg port passes this explicitly, so it is unaffected by the default). Update the stale "submodule removed" error. - docs/build.md: restore the --recursive / submodule-update instructions and document both ggml paths (vendored default vs system/vcpkg). Co-authored-by: Cursor <cursoragent@cursor.com>
|
Update: keeping the vendored ggml submodule (standalone builds) — pushed Reversing the earlier "remove the vendored ggml submodule" direction. Rather than making the vcpkg system-ggml port the only supported path, this branch now keeps the
This also resolves the earlier review points on this branch: with |
Review StatusCurrent Status: ❌ PENDING Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member. |
…ABLE Addresses dev-nid review on PR #13. ggml_rope_flux(ctx, v_in, nullptr) was unclear: with a null position tensor it applies no rotation (V is never RoPE-rotated) and is just a fused-kernel equivalent of the permute(0,2,1,3)+reshape_3d in the fallback branch. - Add a comment documenting the null-pe (permute-only, no rotation) semantics. - Gate the fused V path on GGML_ROPE_FLUX_DISABLE, matching the q/k fused path in rope.hpp, so the whole fused-RoPE kernel family can be disabled together; when disabled it falls back to the explicit permute+reshape. Co-authored-by: Cursor <cursoragent@cursor.com>
Summary
Splits the LTX / custom-ggml delta out of the
2026-06-04fork base.2026-06-04(base) = upstream1f9ee88+ the 5 general, upstream-ggml-compatible patches: vcpkg port infra, Flux qkv assert, ESRGAN device API, upscaler defaults, Wan I2V VAE tiling bypass. Itsggmlsubmodule stays at stock leejet/ggml0ce7ad3.This PR adds the 6 commits that depend on / switch to the unified qvac ggml:
feat: use fused Flux RoPE when available— requiresGGML_OP_ROPE_FLUX(not in stock leejet ggml); bumps submodule to qvac-ext-ggmlchore: remove ggml git submodule — use vcpkg SD_USE_SYSTEM_GGMLfix: remove ggml-impl.h include — symbols are in public ggml.hggml_graph_cut: use public ggml_graph_leaf/n_leafs/add_leaf APIcli: default preferred_gpu_backend to GPU— fixes the LTX pipeline running on CPUcmake: add /bigobj for MSVC— build fix for the larger LTX/system-ggml objectsLTX-2.3 model support itself comes from upstream; these are the qvac-side build/packaging reconciliation + the custom-ggml dependency.
Notes
2026-06-04-ltxtip (df47d5e) is unchanged and byte-for-byte identical to the original fork tip.Test plan
2026-06-04-ltx