How AI garment rendering works, explained without the hype

“Virtual try-on” covers two families of technology that share almost nothing beyond the name. One moves pixels around; the other draws new ones. They fail differently and are worth very different things to a fashion catalogue.

This is the explainer we wish existed when we started: what each approach does, why composing a whole outfit is harder than swapping one garment, and the questions that separate a demo from a product.

Two different things get sold as virtual try-on

The dividing line is whether the system reuses the garment’s original pixels or generates new ones. Overlay systems cut the garment out of the product photograph and warp it onto a picture of a person. Generative systems read that photograph as a description of the garment and paint it onto the body from scratch.

Everything else follows. A warped cut-out preserves the garment perfectly and the body badly; a generated garment sits on the body correctly and has to work harder to stay faithful to the original.

What a 2D overlay actually does

Three stages. It segments the garment out of the product shot, estimates keypoints on the target person — shoulders, hips, elbows — then deforms the garment image so its anchor points land on the body’s. The deformation is a smooth warp with no understanding of what it is warping.

The result reads as a sticker for a specific reason: the fabric keeps the shape it had in the studio. Shadows in the cut-out belong to the studio lighting, not the body underneath. The garment cannot fall behind an arm, gather at a waistband or bunch where a bag strap crosses it, because nothing in the pipeline knows those things exist.

Overlays are not worthless. For small rigid items — glasses, watches, some hats — the geometry barely changes and an overlay looks fine. For anything that drapes, they hit a ceiling quickly.

What a generative model does differently

It is trained on paired data: a photograph of a garment on its own, and a photograph of someone wearing that same garment. Given enough pairs, the model learns the mapping between them — not as a rule set, but as a statistical relationship between how a garment looks laid out and how it looks worn.

At inference it is conditioned on three things: the body and its pose, the garment, and a mask of the region to redraw. Because the pixels are generated rather than transported, the output can include detail that was never in the source photograph — a shadow under a collar, a fold where an elbow bends, the way a hem falls against a hip.

The trade is fidelity. The model has to reproduce the garment’s identity — exact colour, print scale, hardware — while inventing everything about how it sits, and that tension is where most visible failures live.

How a model learns drape and texture

Not by simulating physics. Cloth simulation exists and is excellent, but it needs a 3D garment, material parameters and a body mesh — assets a fashion catalogue does not have and will not build for ten thousand SKUs. Generative try-on learns the appearance directly instead.

The model learns families instead. Enough satin under studio light teaches it that satin produces long highlights that shift with the body’s curve; enough ribbed knitwear teaches it that ribbing compresses across the bust and stretches at the sleeve. Coverage of the training distribution matters more than any architectural choice.

It also explains the failure modes, which are consistent across the field:

• Sheer fabric, where the model has to decide how much of the body shows through.
• Heavy unstructured drape — a bias-cut slip has many plausible shapes and the model has to pick one.
• Text and logos, distorted or invented because the model treats them as texture rather than as symbols.
• Fine hardware: buckles, chains and buttons at small pixel sizes.
• Poses far from the training distribution, where limbs cross the torso or the body is foreshortened.

A vendor who names these before you find them is a better sign than one whose demo only shows structured cotton on a straight-on pose.

Why multiple garments are harder than one

Composing a top, a bottom, shoes and a bag into one image is not four times the work of one garment. The difficulty is in the interactions, and those grow faster than the pieces.

• Layer order. A shirt can be tucked or untucked, and the model has to commit to one and render the waistband accordingly.
• Occlusion. A bag strap crosses a shoulder, a hem covers a boot shaft, a jacket hides most of what is under it. Every garment has to know what is in front of it.
• Proportion. Rendered separately, a shoe at one scale and trousers at another read as a collage rather than an outfit.
• Light and colour. All four pieces need one light direction and one white balance, or the composite falls apart even when each element is convincing alone.
• Contact. Where fabric meets fabric, or a strap presses into a shoulder, the shadow has to belong to both objects.

The practical consequence: a system that renders the full look in one pass holds together better than one that composites separate renders, because the single pass has every piece in context while it draws. That is what makes outfit-level try-on possible at all — swapping one piece and re-rendering the whole look only makes sense if the whole look was composed together in the first place.

What “photorealistic” should mean

It should mean testable properties, not a feeling about a screenshot. Five parts, each checkable in about ten seconds on a real render.

1. 01Garment identity is preserved: the colour matches the product page, the print is at the right scale, the hardware is right.
2. 02Occlusion is correct: what should be in front is in front, and edges are clean where two garments meet.
3. 03Light is consistent — one direction across the whole image, including the shadow the body casts on the garment.
4. 04Fabric behaves plausibly. Folds appear where the body bends, and a stiff fabric does not ripple like silk.
5. 05The body is intact. No melted shoulder line, no waist narrowed to fit the garment rather than the other way round.

A render that passes all five is useful even if it is not indistinguishable from photography. One that fails the last is unusable however good the fabric looks.

Questions to ask any try-on vendor

These separate a controlled demo from something that survives your catalogue. Ask them in this order.

• Is this an overlay or a generative model? If the answer is vague, it is an overlay.
• Can I upload three of my own products right now and see the output? Curated demo catalogues hide category weaknesses.
• What does it do with sheer fabric, printed text, and a bias-cut dress?
• Can it render more than one garment on the same body, in one image? If yes, is that one pass or a composite?
• What input does it need per SKU, and what happens to a product with only a flat-lay photograph?
• How long does a render take at PDP traffic levels, and what does the shopper see while waiting?
• What happens to a shopper’s uploaded photograph — retention, training use, deletion on request?
• What is the failure behaviour on a poor render? Is there a confidence threshold, and can a bad render reach a shopper?
• What is the unit of billing — a render, a session, a SKU — and what does a busy month actually cost?

The last two get asked least and matter most after month one. Our own pipeline, step by step answers most of this list, and the glossary defines the terms.