DiTailed: Ensuring Visual Object Consistency in Text-Image-to-Image Flow Matching Models

The object consistency problem

Even with straightforward editing prompts specifying background removal or basic rotations given an image, state-of-the-art models frequently fail to maintain visual object consistency: geometry is distorted, textures are altered, and fine-grained details disappear.

Visual object inconsistencies in current models. (Left) pillow patterns are not preserved; (center) cup holder and rivets are missing or poorly reconstructed; (right) sofa geometry is entirely altered with inconsistent texture.

ABO-Edit: a benchmark for visual object consistency

ABO-Edit is built on the Amazon Berkeley Objects (ABO) dataset and contains over 12,000 triplets of source lifestyle images, editing prompts with degree-level rotation angles, and ground-truth targets rendered from artist-designed 3D models using Blender at 1024×1024 resolution. Each product is rendered from three viewpoints (left, front, right), enabling evaluation of visual consistency under orientation changes. All samples are validated through human quality control via crowdsourcing.

ABO-Edit samples. Source lifestyle images (left) paired with ground-truth targets rendered at precisely specified rotation angles. The dataset covers diverse product types including furniture, wall art, and decorative items, with fine-grained angular annotations.

What happens inside the model?

In rectified flow editing models, the predicted velocity is computed solely from the noisy input, while the conditioning output h_L^cond is discarded at inference. Yet this representation remains coupled to the generation process: it participates in attention computations across all transformer layers and receives gradients during training. This raises a question: what information does it encode, and how does it evolve during denoising?

We decode h_L^cond using the VAE decoder at various timesteps and discover two distinct behavioral patterns. Models in the FLUX family maintain representations of the conditioning image throughout denoising. In contrast, Qwen-based models encode a prediction of the final generated image in h_L^cond, even at high noise levels (t ≈ 1) — a property that is never explicitly supervised during training.

Internal representations across models. FLUX models maintain representations of the conditioning image throughout denoising, whereas Qwen models encode the final prediction in the conditioning embedding space even at high noise levels (t ≈ 1). Our auxiliary loss (bottom row) enforces this behavior at early timestep, preserving the original geometry.

Remarkably, this latent prediction exhibits higher similarity to the final output than the velocity estimate at early denoising steps, providing insight into the effective number of inference steps required for convergence, as shown in the following Figure:

Temporal evolution of predictions during denoising. With our auxiliary loss, the model's prediction in the conditioning space (orange) exhibits higher similarity to the final output than the velocity estimate (green) at early timesteps. At high noise (t=1), the gap is substantial: 0.823 vs 0.723.

FlowMirror: when conditioning mirrors the generation

The observations above reveal that certain model families naturally encode the final output in h_L^cond, and that these predictions stabilize early in denoising. Leveraging these insights, we hypothesize that explicitly supervising h_L^cond to match the target amplifies this beneficial behavior: it encourages the model to form accurate output representations early, while providing auxiliary gradient signal without additional parameters.

We thus propose a parameter-free multi-scale auxiliary loss that supervises the conditioning output using the VAE latent of the target image. Operating entirely in latent space (without invoking the decoder), the loss decomposes into high-frequency (detail preservation) and low-frequency (structural) components via a Laplacian-pyramid decomposition. Crucially, this incurs negligible computational overhead and requires no architectural changes.

Overview of FlowMirror. Given noisy input, conditioning image embedding, timestep, and text embeddings, the DiT predicts velocity while producing the conditioning output h_L^cond. We supervise it against the target latent using a parameter-free, multi-scale decomposition that captures both fine details and global structure.

Degree-level viewpoint control

Fine-tuning on ABO-Edit enables precise angular control (mean error < 2.5°) while maintaining visual consistency across generated views.

Precise spatial control. Our fine-tuned model generates novel views at varying rotation angles with fine-grained angular accuracy and strong visual consistency across all orientations.

Quantitative results

FlowMirror consistently improves metrics across two architectures (FLUX and Qwen) with lower variance, indicating more stable optimization.

All fine-tuned models trained 3× with different seeds. Mean reported. Best per section in highlighted rows. Human evaluation: our method preferred 4.4 percentage points more often than baseline.