ComfyUI-DyPE

ComfyUI custom node pack for ultra-high-resolution generation (4K and beyond) with Diffusion Transformers — FLUX, Qwen Image, Z-Image, Anima/Cosmos, Krea-2.

Training-free methods that push pre-trained DiT models far beyond their native resolution — no retraining, no workflow changes. Patch the model once after your loader and generate at 2K, 4K and above.

_{A simple, single-node integration to patch your model for high-resolution generation.}

• Multi-Architecture — FLUX, Nunchaku, Qwen Image, Krea-2, Z-Image, Anima/Cosmos
• High-Resolution Generation — 4096×4096 and beyond
• Single-Node Integration — place after your model loader, done
• Full Compatibility — works with existing workflows, samplers and optimization nodes
• Zero Overhead — adjustments happen on-the-fly with negligible performance impact

⟔ ▲ ⟓

Two families: model patches alter how your own KSampler run attends (no image input) — best for native high-res generation; cascades consume an existing latent/image and refine it.

Dynamic Position Extrapolation (paper, code). Adjusts positional encodings at each denoising step to match the current stage of generation — low-frequency structure early, fine detail later. Training-free, no additional sampling cost.

Usage: Load model → add DyPE (under WMNodes/image) → connect MODEL → set width/height to match your latent → connect to KSampler.

⟔ ▲ ⟓

Spectral-Energy Guided Attention (code). Content-aware RoPE sharpening derived from the latent's frequency spectrum. Use as an alternative to DyPE on FLUX/Qwen.

Usage: Add the SEGA node after your model loader → set width/height to match your latent → tune mscale_alpha and spread_min/spread_max.

⟔ ▲ ⟓

Spatial Position Alignment, from the HRDiT paper (arXiv 2608.07003). A static, training-free patch that fixes high-resolution spatial disorder — repeated structures and positional collisions when pushing past native resolution. Resolution-aware (automatic no-op ≤ 1024px) with bounded overhead at 2K/4K. Mechanism: bundles token positions into groups of N, slides the bundle boundary per axis (2s − 1 variants), and averages the attention outputs across variants — never the RoPE matrices themselves.

Usage: Add the SPA (HRDiT) node after your model loader → set width/height → leave model_type: auto → connect to KSampler. Recommended bundle_size: 3 at 2K, 5 at 4K (0 = auto).

⟔ ▲ ⟓

Head-Adaptive attention Pruning, from the same HRDiT paper — the speed half complementing SPA (the quality half). Each attention head only sees the keys it actually needs, via a pre-calibrated scope plan executed through block-sparse attention. Composable with SPA in any order.

A ready-to-use FLUX scope plan ships at configs/scope_plan_flux.json.

Usage: Add the HAP (HRDiT) node after your model loader → point scope_plan_path at a plan JSON → connect to KSampler (optionally through an SPA node first).

# Self-contained dry run (no GPU needed):
python calibration/calibrate_hap.py --dry_run --out tmp/scope_plan_toy.json

# Real-model calibration:
python calibration/calibrate_hap.py --model_path /path/to/flux.safetensors \
    --model_type flux --width 4096 --height 4096 --num_prompts 30 \
    --out configs/scope_plan_flux_4k.json

⟔ ▲ ⟓

Cascade-based refinement node. Generates at native resolution first, then progressively adds detail through coarse-to-fine cascade refinements — producing crisp 4K output without regenerating the whole image from noise. Works with any ComfyUI model (SDXL, SD1.5, FLUX, Qwen, …).

Usage: Generate a base latent at native resolution → connect model, vae, positive, negative and the base latent_image → set num_cascade_stages (1 = 2× upscale, 2 = 4×, 3 = 8×) → decode the output latent.

⟔ ▲ ⟓

Tuning-free higher-resolution generation via scale-fused attention and self-cascade upscaling (paper, code). Supports FLUX-family DiTs (auto-detected); base-resolution inputs pass through untouched.

⟔ ▲ ⟓

Training-free high-resolution upscaling for rectified-flow models (FLUX, Qwen-Image, Krea2, Z-Image, …) via flow-aligned guidance (paper, NeurIPS 2025). The base-resolution sampling runs once, recording every per-step clean prediction; each upscale stage then reuses that time-matched trajectory as a virtual reference — initialization alignment seeds the stage from it, direction alignment keeps low frequencies true to it, acceleration alignment matches its detail-generation rhythm. Structure survives; high-res detail is synthesized fresh.

Usage: connect model (flow models only), vae, positive, negative and a base latent at native resolution (e.g. EmptySD3LatentImage) → set noise_seed + scale_factor → decode. The cascade noises the latent to the first sigma itself — an empty latent + seed reproduces the reference pipeline's from-noise start. Chain DyPE (ntk) before the loader for RoPE extrapolation at the scaled resolution.

⟔ ▲ ⟓

All nodes registered by this pack (V3 schema ids):

⟔ ▲ ⟓

Via ComfyUI Manager: Search ComfyUI-DyPE → Install.

Manual install:

cd ComfyUI/custom_nodes/
git clone https://github.com/wildminder/ComfyUI-DyPE.git

Restart ComfyUI. No further dependency installation is required.

⟔ ▲ ⟓

⟔ ▲ ⟓

• Restructured the pack layout + unified the node category. All node definitions now live in a dedicated nodes/ folder (nodes/dype.py, sega.py, spa.py, hap.py, hap_calibrate.py, freescale.py, pixelrush.py, hiflow.py); src/ holds engines/implementation only and the pack __init__.py just registers the extension. All 8 nodes moved to the single WMNodes/image menu category (previously split across two menu paths). No node ids, inputs, defaults, or behavior changed — workflows keep loading. Also merges PR #41 (FreeScale fp16 antialiased-bicubic crash fix).

• Fixed HiFlow Krea2/Qwen-Image noising crash (user-reported torch.cat size mismatch, "Expected size 1 but got size 16"): the v2.12.1 model-space noising called the model's process_latent_in on the 4D core tensor, but Wan21's per-channel mean/std stats are shaped [1,C,1,1,1] — a 4D tensor against 5D stats broadcasts silently to [B,C,C,H,W] garbage (the model reads T=16=channels). The node now wraps the noising conversions ndim-transparently: unsqueeze → convert in true 5D model space → squeeze back, so the cascade's σ-mix runs on 4D tensors with correctly-normalized values. The node-test mock now uses Wan21-faithful stats (replicating the broadcast hazard — the earlier affine mock masked the bug class).

• HiFlow: 3D-latent image model support — Krea2 and Qwen-Image work now (plan 2026-09-07, user-reported Krea2 "does not support 3D-latent (video) models" rejection). These models are image models with a 5D Wan21-style latent layout [B,C,1,H,W] (Qwen VAE) — the old gate conflated 5D tensors with video. The gate now accepts latent_dimensions=3 image models and rejects only actual multi-frame (T>1) input; the node bridges 5D↔4D around the 4D core (the PixelRush convention): latents squeeze on entry and re-expand on output, the model-call adapter unsqueezes before process_latent_in (Wan21's per-channel mean/std stats broadcast on 5D only), and the VAE adapters speak the Qwen-VAE latent_dim=3 boundary (decode frame-slices the [B,T,H,W,3] image; encode lets the VAE do its own not_video unsqueeze). Qwen-Image gains real (previously gate-blocked) support from the same fix; Anima inherits it, untested on real runs.

• HiFlow: target_resolution replaced by scale_factor (user request — the absolute pixel target was unintuitive). scale_factor is relative to the input latent: 2 doubles each side, 1 returns the base unchanged, 0.5 halves it via a single refinement stage. Scales now apply per side (the absolute form over-upscaled the short side of non-square images), upscales keep the paper's 2×-stage quantization (a 1.5 scale runs one 2× stage), and downscale scales (0.25–1) run one guided stage at the smaller size. Example workflow updated.

• Fixed HiFlow img2img noising space (user-reported "drastic changes at any usable denoise; only 0.05 looks right"): the σ-mix σ·ε + (1−σ)·content now runs in MODEL space (convert the content with process_latent_in first, convert the mix back), matching ComfyUI's KSampler pipeline (samplers.py converts the content before the σ-mix). Mixing in VAE space scaled the noise by the latent format's scale_factor (Flux/Z-Image: 0.3611 — 2.77× under-noised) and added spurious shift offsets, so the model aggressively "corrected" every img2img input. The guided-stage initialization σ-mix got the same fix. The sampler itself (rectified-flow Euler) and scheduler spacing (model-table "simple") were already faithful — the defect was the space mix, not the routine.

• HiFlow img2img: denoise parameter (user-reported "connecting the real latent does nothing"): with the full flow schedule the base start σ=1 zeroes the content weight, so a sampler latent connected to the node was silently ignored. The KSampler convention now applies — denoise < 1 truncates the base schedule so the walk enters below σ=1 and keeps (1−σ_start) of the input latent (an empty latent always runs the full schedule; the node warns when a content latent meets denoise=1.0).

• HiFlow realigned with the authors' implementation (plan 2026-09-03-realignment, user-reported Z-Image "burned and blurred" output identical in both upsampling modes): the base stage now starts from noised latent instead of the raw input (an EmptySD3LatentImage was being sampled verbatim as all-zeros "noise" — the root cause); stage initialization anchors on the previous chain's final image (always pixel round-tripped) instead of the time-matched reference; the reference velocity derives from the walk's own state; trajectories store the raw (uncorrected) x0 so guidance doesn't compound across stages; α/β follow the code's linear-in-index schedule, not the paper's σ/σ_entry (which over-locks low frequencies late on shifted schedules). New noise_seed input drives the base and per-stage init noise reproducibly.

• New HiFlow node (plan 2026-09-03): training-free high-resolution upscaling for rectified-flow models (FLUX, Qwen-Image, …) via flow-aligned guidance (arXiv:2504.06232). The base-resolution trajectory is recorded per-step and guides each upscale stage through initialization, direction and acceleration alignment. Non-flow and video models are rejected with a pointer to PixelRush.

• Fixed the PixelRush noise-injection λ convention (user-reported "structure visible but completely noisy, soft blurred patches"). The injection now uses slerp(eps_random, eps_refined, λ) — λ weights the model's prediction (0.95 = 95% prediction + 5% noise). The previous order (slerp(eps_pred, eps_random, λ)) made λ=0.95 mean 99.6% pure random noise: at real scales per-pixel noise std ≈ 1.17 vs signal ≈ 1.0, which rendered through the Gaussian feather as the reported soft-patch noise. The additive legacy mode uses the same convention (eps_refined + (1−λ)·eps_random). This was exactly the argument-order caveat pixelrush-correct.txt flagged for verification against the authors' implementation.

• PixelRush realigned with the corrected theory (plan 2026-09-02): standard raw-vector SLERP (with collinear lerp fallback) for the noise injection — the paper's slerp(eps_pred, eps_random, λ) is now the default, with the 2026-08-13 additive injection kept as an opt-in (noise_injection).
• Fixed the VAE/model space mixing in the forward/reverse steps: adapters now convert via process_latent_in/out, so the model sees noise at the scale its timestep claims. For SDXL the previous code under-noised 7.7× — the root cause behind the "compressed look" that the additive hack had papered over.
• Generic DDIM transitions (ddim_deterministic_step between arbitrary timesteps, predict_x0_from_epsilon); analytic Gaussian feather mask (σ default 24, gaussian_kernel_size input removed).
• Optional refiner_model input — use a separate distilled refiner (e.g. SDXL-Turbo) as in the paper; the base model drives the partial inversion.
• Bug fixes: empty-negative conditioning no longer amplifies eps by cfg_scale (CFG is skipped); alpha_k NameError with partially-provided adapters; empty positive now raises a clear error.

• Qwen2D VAE support disabled by default. User reports showed that with the Qwen2D VAE interception installed, loading certain non-Qwen2D (video-style) VAE checkpoints crashed with a size-mismatch error whose traceback passed through this pack's delegation frame — breaking workflows that never used the Qwen2D VAE. The patch now installs only when the environment variable DYPE_ENABLE_QWEN2D_VAE=1 is set. If you relied on the Qwen2D VAE (Anzhc/Qwen2D-VAE checkpoint with FreeScale/PixelRush on Krea-2/Qwen/Anima), set that variable in your ComfyUI environment to restore the previous behavior.

• Fixed graph-build and execution crashes when resolution inputs are None (validate_inputs now passes through uninitialized state; execute falls back to 1024)
• Fixed PixelRush crash on float16: antialiased bicubic upsample casts to float32 and restores the original dtype

• Fixed valid resolutions being rejected at graph build
• Validation errors are now reported once, for the right input

• New HAP Calibrate node: calibrate HAP directly in-graph
• HAP accepts calibrated plans either by file or by direct connection
• CLI calibration tooling completed

• Fixed crashes on Anima/Cosmos models
• Safer automatic fallbacks instead of hard errors
• SPA and HAP nodes now work in any order

• New HAP node: sparse-attention acceleration (up to ~5× faster attention at 4K)
• One-click scope-plan calibration pipeline (in-graph + CLI)
• New optional attention scaling and per-layer filtering controls
• SPA and HAP can be composed together

• Reworked SPA bundle-size control to match the paper
• Much faster SPA runs (up to ~10× less overhead at strong settings)
• Automatic no-op at/below native resolution

• New SPA node (HRDiT)

• Fixed "totally noisy" output on SDXL models

• New SEGA node
• Video-model latent support

• Anima/Cosmos support
• Krea-2 support
• Stability fixes and new example workflows

• Z-Image quality improvements

• Experimental Z-Image support

• Qwen Image and Nunchaku support
• Modular codebase refactor for easier future model support

• New vision_yarn method for better aspect-ratio handling
• Sharper results with fewer artifacts
• New start-sigma control

• Initial release: core DyPE for FLUX with yarn and ntk methods

⟔ ▲ ⟓

• Noam Issachar, Guy Yariv and co-authors — DyPE (paper)
• The SEGA authors — SEGA
• The HRDiT team — HRDiT (code) — basis for SPA & HAP
• The PixelRush authors — PixelRush
• The HiFlow authors — HiFlow (code)
• Yanhong Zeng et al. — FreeScale (paper)
• The ComfyUI team — for the platform

⟔ ▲ ⟓

══════════════════════════════════