Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/img/inpainting_demo.gif filter=lfs diff=lfs merge=lfs -text
+hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/img/test_01.png filter=lfs diff=lfs merge=lfs -text
+hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/img/test_02.png filter=lfs diff=lfs merge=lfs -text
+hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/img/test_03.png filter=lfs diff=lfs merge=lfs -text

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/.mv

@@ -0,0 +1 @@
+Revision:master,CreatedAt:1755071739

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/README.md

@@ -0,0 +1,173 @@
+---
+tasks:
+- image-inpainting
+widgets:
+    - task: image-inpainting
+      inputs:
+      - type: image
+        name: image
+        validator:
+          max_size: 5M
+          max_resolution: 1920*1080
+      - name: mask
+      parameters:
+      - name: prompt
+        title: Prompt
+        type: string
+      examples:
+        - name: 1
+          title: 示例1
+          inputs:
+            - data: git://img/test_01.png
+          parameters:
+            - name: prompt
+              type: string
+              value: background
+        - name: 2
+          title: 示例2
+          inputs:
+            - data: git://img/test_02.png
+          parameters:
+            - name: prompt
+              type: string
+              value: background
+        - name: 3
+          title: 示例3
+          inputs:
+            - data: git://img/test_03.png
+          parameters:
+            - name: prompt
+              type: string
+              value: background
+      inferencespec:
+        cpu: 4
+        memory: 16000
+        gpu: 1
+        gpu_memory: 16000
+
+domain:
+- cv
+frameworks:
+- pytorch
+customized-quickstart: True
+finetune-support: False
+license: Apache License 2.0
+tags:
+- Image Inpainting
+- Stable Diffusion
+- stablediffusion model
+- 图像填充
+- 图像修复
+- 图像修补
+
+---
+# Stable Diffusion v2 for Image Inpainting 图像填充模型
+该模型为图像填充模型，输入一个抹除部分内容的图像，实现端到端的图像填充，返回填充后的完整图像。
+
+模型效果如下:
+
+<img src="./img/inpainting_demo.gif">
+
+## 模型描述
+该模型基于Stable Diffusion v2与diffusers进行构建。
+## 模型期望使用方式和适用范围
+
+1. 该模型适用于多种场景的图像输入，给定图像（Image）和需要修补填充区域的掩码（Mask），生成修补填充后的新图像；
+2. 该模型推理时对机器GPU显存有一定要求；在FP16模式下并开启enable_attention_slicing选项时，对于16G显存的显卡，建议的最大输入分辨率为1920x1080；FP32模式建议使用含较大显存（如32G及以上）GPU的机器进行推理。如果没有GPU显卡或显存不足够，可以尝试使用CPU模式进行推理。
+### 如何使用Demo Service
+通过在页面右侧绘制Mask，即可快速体验模型效果：
+- 建议点击右上角的最大化按钮后再绘制Mask，充分抹除物体可以带来更好的Inpainting效果；
+- 希望抹除物体并还原背景时，Prompt默认为"background"；希望生成其他物体时，可以更改Prompt来描述希望生成的物体；Prompt需要英文输入；
+- 建议上传体验图像的分辨率不超过1280x720，更大尺寸的图像推理时需要更大的GPU显存和更长的推理时间，可以在Notebook或本地调用Pipeline体验。
+### 如何使用Pipeline
+在 ModelScope 框架上，提供输入图像和掩码，即可以通过简单的 Pipeline 调用来使用Stable Diffusion v2图像填充模型。
+#### 推理代码范例
+```python
+import cv2
+import torch
+from modelscope.outputs import OutputKeys
+from modelscope.pipelines import pipeline
+from modelscope.utils.constant import Tasks
+
+input_location = 'https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/image_inpainting/image_inpainting_1.png'
+input_mask_location = 'https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/image_inpainting/image_inpainting_mask_1.png'
+prompt = 'background'
+output_image_path = './result.png'
+
+input = {
+    'image': input_location,
+    'mask': input_mask_location,
+    'prompt': prompt
+}
+image_inpainting = pipeline(
+    Tasks.image_inpainting,
+    model='damo/cv_stable-diffusion-v2_image-inpainting_base',
+    device='gpu',
+    torch_dtype=torch.float16,
+    enable_attention_slicing=True, model_revision='master')
+output = image_inpainting(input)[OutputKeys.OUTPUT_IMG]
+cv2.imwrite(output_image_path, output)
+print('pipeline: the output image path is {}'.format(output_image_path))
+```
+
+如果遇到报错类似于：
+```text
+No module named 'transformers.models.qwen3'
+```
+请考虑卸载awq:
+```shell
+pip uninstall autoawq
+```
+
+#### 推理代码说明
+
+- Pipeline初始化参数
+  - 可缺省参数device，默认值为'gpu'，可设置为'cpu'。
+  - 可缺省参数torch_dtype，默认值为torch.float16，可设置为torch.float32。
+  - 可缺省参数enable_attention_slicing，默认值为True，开启将减少GPU显存占用，可关闭。
+
+- Pipeline调用参数
+  - 输入要求：输入字典中必须指定的字段有'image'，'mask'；其他可选输入字段及其默认值包括：
+```python
+'prompt': 'background',
+'num_inference_steps': 50,
+'guidance_scale': 7.5,
+'negative_prompt': None,
+'num_images_per_prompt': 1,
+'eta': 0.0
+```
+  - 额外参数：
+    - prompt参数也支持在Pipeline调用时作为单独参数传入；但如果input中存在prompt字段，将会优先使用input中的prompt。
+
+- 由于GPU显存限制，本项目默认支持开启FP16推理，可以在构建pipeline时传入参数torch_dtype=torch.float32来使用FP32；同时torch_dtype参数可缺省，默认值为torch.float16。
+- 本项目支持使用CPU进行推理，可以在构建pipeline时传入参数device='cpu'；CPU模式下torch_dtype仅支持torch.float32。
+
+### 模型局限性以及可能的偏差
+
+- 实际测试中，FP16模式下生成的图像较FP32模式质量有所下降。
+- 在一些背景较为简单平滑的场景下，Stable Diffusion可能生成一些无意义的前景物体，可以通过调整Prompt和模型参数进行消除。
+- 在一些场景下，指定某些不同的Prompt时，Stable Diffusion可能生成错误的前景物体；可以生成多次，取效果较好的结果。
+- 目前模型推理前会Resize输入图像以匹配输入尺寸要求。
+
+## 训练介绍
+本模型根据diffusers开源库构建，由Stability-AI从 stable-diffusion-2-base (512-base-ema.ckpt) 微调 200k steps。 并使用了LAMA中提出的掩码生成策略。请参考[模型来源](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting)。
+
+## 说明与引用
+本算法模型源自一些开源项目：
+
+- [https://github.com/Stability-AI/stablediffusion](https://github.com/Stability-AI/stablediffusion)
+- [https://github.com/huggingface/diffusers](https://github.com/huggingface/diffusers)
+- [https://huggingface.co/stabilityai/stable-diffusion-2-inpainting](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting)
+
+如果你觉得这个模型对你有所帮助，请考虑引用下面的相关论文：
+```
+@misc{rombach2021highresolution,
+      title={High-Resolution Image Synthesis with Latent Diffusion Models}, 
+      author={Robin Rombach and Andreas Blattmann and Dominik Lorenz and Patrick Esser and Björn Ommer},
+      year={2021},
+      eprint={2112.10752},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```
+

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/configuration.json

@@ -0,0 +1,17 @@
+{
+    "framework": "pytorch",
+    "task": "image-inpainting",
+    "pipeline": {
+        "type": "image-inpainting-sdv2"
+    },
+    "model": {
+        "type": "image-inpainting-sdv2"
+    },
+    "modelsetting": {
+        "num_inference_steps": 50,
+        "guidance_scale": 7.5,
+        "num_images_per_prompt": 1,
+        "eta": 0.0
+    },
+    "allow_remote": true
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/feature_extractor/preprocessor_config.json

@@ -0,0 +1,20 @@
+{
+  "crop_size": 224,
+  "do_center_crop": true,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_resize": true,
+  "feature_extractor_type": "CLIPFeatureExtractor",
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "resample": 3,
+  "size": 224
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/model_index.json

@@ -0,0 +1,33 @@
+{
+  "_class_name": "StableDiffusionInpaintPipeline",
+  "_diffusers_version": "0.8.0",
+  "feature_extractor": [
+    "transformers",
+    "CLIPFeatureExtractor"
+  ],
+  "safety_checker": [
+    null,
+    null
+  ],
+  "scheduler": [
+    "diffusers",
+    "PNDMScheduler"
+  ],
+  "text_encoder": [
+    "transformers",
+    "CLIPTextModel"
+  ],
+  "tokenizer": [
+    "transformers",
+    "CLIPTokenizer"
+  ],
+  "unet": [
+    "diffusers",
+    "UNet2DConditionModel"
+  ],
+  "vae": [
+    "diffusers",
+    "AutoencoderKL"
+ ],
+  "requires_safety_checker": false
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/requirements.txt

@@ -0,0 +1,3 @@
+huggingface-hub==0.25.*
+transformers==4.48.3
+diffusers==0.28.0

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/scheduler/scheduler_config.json

@@ -0,0 +1,13 @@
+{
+  "_class_name": "PNDMScheduler",
+  "_diffusers_version": "0.8.0",
+  "beta_end": 0.012,
+  "beta_schedule": "scaled_linear",
+  "beta_start": 0.00085,
+  "clip_sample": false,
+  "num_train_timesteps": 1000,
+  "set_alpha_to_one": false,
+  "skip_prk_steps": true,
+  "steps_offset": 1,
+  "trained_betas": null
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/text_encoder/config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": "./hf-models/stable-diffusion-v2-inpainting/text_encoder",
+  "architectures": [
+    "CLIPTextModel"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dropout": 0.0,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_size": 1024,
+  "initializer_factor": 1.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 77,
+  "model_type": "clip_text_model",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 23,
+  "pad_token_id": 1,
+  "projection_dim": 512,
+  "torch_dtype": "float32",
+  "transformers_version": "4.25.0.dev0",
+  "vocab_size": 49408
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/text_encoder/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9c787e9388134c1a25dc69934a51a32a2683b38b8a9b017e1f3a692b8ed6b98
+size 1361679905

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/tokenizer/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "!",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/tokenizer/tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+  "add_prefix_space": false,
+  "bos_token": {
+    "__type": "AddedToken",
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "do_lower_case": true,
+  "eos_token": {
+    "__type": "AddedToken",
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "errors": "replace",
+  "model_max_length": 77,
+  "name_or_path": "./hf-models/stable-diffusion-v2-inpainting/tokenizer",
+  "pad_token": "<|endoftext|>",
+  "special_tokens_map_file": "./special_tokens_map.json",
+  "tokenizer_class": "CLIPTokenizer",
+  "unk_token": {
+    "__type": "AddedToken",
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/unet/config.json

@@ -0,0 +1,44 @@
+{
+  "_class_name": "UNet2DConditionModel",
+  "_diffusers_version": "0.8.0",
+  "_name_or_path": "./hf-models/stable-diffusion-v2-inpainting/unet",
+  "act_fn": "silu",
+  "attention_head_dim": [
+    5,
+    10,
+    20,
+    20
+  ],
+  "block_out_channels": [
+    320,
+    640,
+    1280,
+    1280
+  ],
+  "center_input_sample": false,
+  "cross_attention_dim": 1024,
+  "down_block_types": [
+    "CrossAttnDownBlock2D",
+    "CrossAttnDownBlock2D",
+    "CrossAttnDownBlock2D",
+    "DownBlock2D"
+  ],
+  "downsample_padding": 1,
+  "dual_cross_attention": false,
+  "flip_sin_to_cos": true,
+  "freq_shift": 0,
+  "in_channels": 9,
+  "layers_per_block": 2,
+  "mid_block_scale_factor": 1,
+  "norm_eps": 1e-05,
+  "norm_num_groups": 32,
+  "out_channels": 4,
+  "sample_size": 64,
+  "up_block_types": [
+    "UpBlock2D",
+    "CrossAttnUpBlock2D",
+    "CrossAttnUpBlock2D",
+    "CrossAttnUpBlock2D"
+  ],
+  "use_linear_projection": true
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/unet/diffusion_pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f2f6cff77df1279d280950c6566ecbac4c3e822d17c25e5aef97ef6dde1bdb7
+size 3463992293

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/vae/config.json

@@ -0,0 +1,30 @@
+{
+  "_class_name": "AutoencoderKL",
+  "_diffusers_version": "0.8.0",
+  "_name_or_path": "./hf-models/stable-diffusion-v2-inpainting/vae",
+  "act_fn": "silu",
+  "block_out_channels": [
+    128,
+    256,
+    512,
+    512
+  ],
+  "down_block_types": [
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D"
+  ],
+  "in_channels": 3,
+  "latent_channels": 4,
+  "layers_per_block": 2,
+  "norm_num_groups": 32,
+  "out_channels": 3,
+  "sample_size": 512,
+  "up_block_types": [
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D"
+  ]
+}

hub/models/iic/cv_stable-diffusion-v2_image-inpainting_base/vae/diffusion_pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4302e1efa25f3a47ceb7536bc335715ad9d1f203e90c2d25507600d74006e89
+size 334715313