latent_diffusion
nv-mzephyr commited on
Commit
bf75067
·
verified ·
1 Parent(s): 4056b94

Updating README.md with metadata and consistency improvements

Browse files
Files changed (1) hide show
  1. README.md +36 -50
README.md CHANGED
@@ -1,24 +1,35 @@
1
  ---
2
  license: other
3
  license_name: nvidia-open-model-license-agreement
4
- license_link: >-
5
- https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
 
 
 
 
 
 
 
6
  ---
7
 
8
- # Model Overview
 
 
9
 
10
  ## Description:
11
- NVIDIA MAISI (Medical AI for Synthetic Imaging) is a state-of-the-art three-dimensional (3D) Latent Diffusion Model designed for generating high-quality synthetic CT images with or without anatomical annotations. This AI model excels in data augmentation and creating realistic medical imaging data to supplement limited datasets due to privacy concerns or rare conditions. It can also significantly enhance the performance of other medical imaging AI models by generating diverse and realistic training data.
 
 
12
 
13
- MAISI offers several key features:
 
 
 
 
14
 
15
- Generates high-resolution 3D CT images up to 512 × 512 × 768 voxels
16
- Supports variable voxel sizes ranging from 0.5mm to 5.0mm
17
- Capable of annotating up to 127 anatomical classes, including organs and tumors
18
- Allows controllable anatomy size for 10 specific classes
19
- Produces paired segmentation masks
20
 
21
- By providing these capabilities, MAISI is a valuable tool for researchers advancing AI applications in healthcare. However, it is important to note that this model is intended for research purposes only and not for clinical usage.
22
 
23
  ## Terms of Use
24
 
@@ -91,50 +102,14 @@ Inference Engine: Triton <br>
91
  **[Preferred/Supported] Operating System(s):** <br>
92
  * Linux <br>
93
 
94
- ## Model Version(s):
95
- 0.3.1 <br>
96
-
97
- # Training & Evaluation:
98
- ## Training Dataset:
99
- Internal ONLY
100
- ~35 Datasets
101
- Name, JIRA/SWIPAT, Commercial, and # of Data Tracked
102
- "MAISI" Sheet: https://docs.google.com/spreadsheets/d/14frhzELquSF_-tF7yGFDBHmSdnp-9-5pmbONQx8iQWk/edit?usp=sharing
103
- https://docs.google.com/spreadsheets/d/1hmv-O-f6tdgndsRnoqCgcunR2uQ9IySDhZWmjsXwgbM/edit?usp=sharing
104
-
105
- ## Evaluation Dataset:
106
- Internal ONLY
107
- ~35 Datasets
108
- Name, JIRA/SWIPAT, Commercial, and # of Data Tracked
109
- "MAISI" Sheet: https://docs.google.com/spreadsheets/d/14frhzELquSF_-tF7yGFDBHmSdnp-9-5pmbONQx8iQWk/edit?usp=sharing
110
- https://docs.google.com/spreadsheets/d/1hmv-O-f6tdgndsRnoqCgcunR2uQ9IySDhZWmjsXwgbM/edit?usp=sharing
111
-
112
- ** Data Collection Method by dataset <br>
113
- * Hybrid: Human, Automatic/Sensors <br>
114
-
115
- ** Labeling Method by dataset <br>
116
- * Hybrid: Human, Automatic/Sensors <br>
117
-
118
- **Properties:** Custom internal and public datasets of 60,000 3D volumes from multiple scanner types. <br>
119
-
120
- ## Evaluation Dataset:
121
-
122
- ** Data Collection Method by dataset <br>
123
- * Hybrid: Human, Automatic/Sensors <br>
124
-
125
- ** Labeling Method by dataset <br>
126
- * Hybrid: Human, Automatic/Sensors <br>
127
-
128
- **Properties:** Custom internal and public datasets of organs from multiple scanner types. <br>
129
-
130
  ## Inference:
131
  **Engine:** PyTorch<br>
132
  **Test Hardware:**
133
- A100 with at least 80GB memory for 512x512x512 images<br>
134
- H100 with at least 80GB memory for 512x512x512 images<br>
135
 
136
  ## Additional Information:
137
- The current list of classes available within MAISI:
138
  "liver": 1,
139
  "spleen": 3,
140
  "pancreas": 4,
@@ -260,6 +235,17 @@ The current list of classes available within MAISI:
260
  "bone lesion": 128,
261
  "airway": 132
262
 
 
 
 
 
 
 
 
 
 
 
 
263
  ## Ethical Considerations:
264
  NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
265
 
 
1
  ---
2
  license: other
3
  license_name: nvidia-open-model-license-agreement
4
+ license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
5
+ pipeline_tag: image-to-image
6
+ library_name: monai
7
+ tags:
8
+ - nvidia
9
+ - medical-imaging
10
+ - ct
11
+ - synthetic-data
12
+ - generation
13
  ---
14
 
15
+ # NV-Generate-CT
16
+
17
+ ![Generation Demo](https://raw.githubusercontent.com/NVIDIA-Medtech/.github/main/profile/generate.gif)
18
 
19
  ## Description:
20
+ NVIDIA NV-Generate-CT is a state-of-the-art three-dimensional (3D) Latent Diffusion Model designed for generating high-quality synthetic CT images with or without anatomical annotations. This AI model excels in data augmentation and creating realistic medical imaging data to supplement limited datasets due to privacy concerns or rare conditions. It can also significantly enhance the performance of other medical imaging AI models by generating diverse and realistic training data.
21
+
22
+ NV-Generate-CT offers several key features:
23
 
24
+ - Generates high-resolution 3D CT images up to 512 × 512 × 768 voxels
25
+ - Supports variable voxel sizes ranging from 0.5mm to 5.0mm
26
+ - Capable of annotating up to 127 anatomical classes, including organs and tumors
27
+ - Allows controllable anatomy size for 10 specific classes
28
+ - Produces paired segmentation masks
29
 
30
+ By providing these capabilities, NV-Generate-CT is a valuable tool for researchers advancing AI applications in healthcare. However, it is important to note that this model is intended for research purposes only and not for clinical usage.
 
 
 
 
31
 
32
+ **Training & Fine-tuning**: Visit [GitHub](https://github.com/NVIDIA-Medtech/NV-Generate-CTMR) for training scripts, ControlNet fine-tuning, VAE training, and advanced configuration guides with comprehensive documentation.
33
 
34
  ## Terms of Use
35
 
 
102
  **[Preferred/Supported] Operating System(s):** <br>
103
  * Linux <br>
104
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
105
  ## Inference:
106
  **Engine:** PyTorch<br>
107
  **Test Hardware:**
108
+ A100<br>
109
+ H100<br>
110
 
111
  ## Additional Information:
112
+ The current list of classes available:
113
  "liver": 1,
114
  "spleen": 3,
115
  "pancreas": 4,
 
235
  "bone lesion": 128,
236
  "airway": 132
237
 
238
+ ## Resources
239
+
240
+ - **Training & Development**: [GitHub Repository](https://github.com/NVIDIA-Medtech/NV-Generate-CTMR) - Complete training pipeline (VAE, diffusion model, ControlNet), fine-tuning guides, and comprehensive development documentation
241
+ - **Interactive Demo**: [MAISI on build.nvidia.com](https://build.nvidia.com/nvidia/maisi) - Try toy examples online with instant generation
242
+ - **Sister Model**: [NV-Generate-MR](https://huggingface.co/nvidia/NV-Generate-MR) - MR image generation variant
243
+ - **Research Papers**:
244
+ - [MAISI: Medical AI for Synthetic Imaging (WACV 2025)](https://arxiv.org/pdf/2409.11169)
245
+ - [MAISI-v2: Accelerated 3D High-Resolution Medical Image Synthesis with Rectified Flow](https://arxiv.org/pdf/2508.05772)
246
+ - **Built with**: [MONAI](https://monai.io/) - Medical Open Network for AI
247
+ - **Clara Medical Collection**: [View all NVIDIA medical AI models](https://huggingface.co/collections/nvidia/clara-medical)
248
+
249
  ## Ethical Considerations:
250
  NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
251