Inconsistent body generation and saree transfer

#1
by aniket2025 - opened

I have used the fashn vton code to change saree among different models. The code is given as:

Importing Required Packages

from fashn_vton import TryOnPipeline
from PIL import Image
import numpy as np

The Function to get the VITON result

def get_viton_result(model_image: str, cloth_image: str):

garment = Image.open(cloth_image).convert("RGB")
target_w, target_h = garment.size
person = Image.open(model_image).convert("RGB").resize((target_w, target_h), Image.Resampling.LANCZOS)

# Initialize pipeline (auto-detects GPU)
pipeline = TryOnPipeline(weights_dir = r"C:\Users\Webbies\Jupyter_Notebooks\TITAN\Fashion_VITON\fashn-vton-1.5\weights")

# Run inference
result = pipeline(
    person_image = person,
    garment_image = garment,
    category = "one-pieces",
    garment_photo_type = "model",
    num_timesteps = 50,
    guidance_scale = 1.5,
    seed = 42
)

# Save output
image = result.images[0].resize((target_w, target_h), Image.Resampling.LANCZOS)
image_np = np.array(image)

return image_np

slim_bg

Image_1

image

The slim_bg is the slim person (model in black saree). the image_1 (the blue saree with black border) is the saree image and the remaining image is the result. I want to know why the result image is generated as this?

There are gaps in the result image from which the background is shown. The gaps does not exsists in any of the person or cloth image. Any way to solve this issue?

Hi,

Due to how this model is trained, it is difficult for this model to handle sarees, specifically the dupatta part
It can handle Indian ethnic wear like kurta better, because the silhouette shape is more predictable

Ok. And what about body shape preservation with accuracy? The result mimics the hand position of the input person image but did not show good accuracy

Sign up or log in to comment