InstructDesign-Kontext

Fine-tuned FLUX.1 Kontext [dev] model for transforming web interfaces with natural language instructions.

Model Description

InstructDesign-Kontext is a LoRA fine-tuned version of FLUX.1 Kontext [dev] specialized in:

• Web interface transformations
• UI/UX redesigns
• Style transfers (dark mode, glassmorphism, etc.)
• Device mockup generation
• Design system applications

Training Details

• Base Model: FLUX.1 Kontext [dev]
• Training Method: LoRA fine-tuning
• Dataset: 937 web interface transformation pairs
• Training Steps: 10,000
• Training Time: ~2 weeks (iterative)

File Structure

├── unet/
│   └── flux1-kontext-dev.safetensors (23GB)
├── loras/
│   └── flux_kontext_lora_v4_consolidated_000010000.safetensors (2.6GB)
├── clip/
│   ├── clip_l.safetensors (235MB)
│   └── t5xxl_fp8_e4m3fn_scaled.safetensors (4.9GB)
├── vae/
│   └── ae.safetensors (320MB)
└── workflow/
    └── workflow_api.json

Usage with ComfyUI

1. Download Models

Download all model files to your ComfyUI models directory:

# Main model
ComfyUI/models/diffusion_models/flux1-kontext-dev.safetensors

# LoRA
ComfyUI/models/loras/flux_kontext_lora_v4_consolidated_000010000.safetensors

# CLIP encoders
ComfyUI/models/clip/clip_l.safetensors
ComfyUI/models/clip/t5xxl_fp8_e4m3fn_scaled.safetensors

# VAE
ComfyUI/models/vae/ae.safetensors

2. Load Workflow

Import the workflow/workflow_api.json into ComfyUI.

3. Configure Parameters

Default optimal settings:

• Steps: 20
• CFG: 1.0
• Guidance: 5.0
• Sampler: er_sde
• Scheduler: kl_optimal
• LoRA Strength: 1.0

API Usage

See the GitHub repository for full API implementation.

# Example API request
import requests

response = requests.post("http://localhost:8000/api/generate", json={
    "positive_prompt": "Transform this interface to dark mode with purple accents",
    "input_image": "website.png",
    "seed": 42,
    "steps": 20,
    "cfg": 1.0,
    "guidance": 5.0
})

Transformation Capabilities

Style Transformations

• Dark/light mode conversion
• Glassmorphism effects
• Neumorphic design
• Material Design application
• iOS/Android style transfer

UI/UX Improvements

• Layout modernization
• Accessibility enhancements
• Color scheme updates
• Typography improvements
• Interactive element additions

Mockup Generation

• Device frame placement
• Environment contextualization
• Billboard/poster mockups
• Multi-device presentations

Example Prompts

"Transform this website to dark mode with purple gradient accents"
"Apply glassmorphism effect with blurred backgrounds"
"Convert to material design 3 with rounded corners"
"Place this interface on an iPhone 15 Pro in a coffee shop"
"Modernize this dated interface with current design trends"

Performance

• Inference Time: ~45-70 seconds (NVIDIA GPU 16GB+)
• VRAM Requirements: 16GB minimum, 24GB recommended
• Quality: 85%+ instruction adherence rate

Docker Deployment

Complete Docker setup available in the GitHub repository with:

• ComfyUI container
• FastAPI wrapper
• Queue management
• Auto-scaling workers

Limitations

• Requires significant VRAM (16GB+)
• Best results with clear, specific instructions
• Limited to single image transformations (no batch in single inference)

Citation

If you use this model, please cite:

@misc{instructdesign-kontext,
  title={InstructDesign-Kontext: Web Interface Transformation with FLUX},
  author={tercumantanumut},
  year={2024},
  publisher={Hugging Face}
}

License

This model is subject to the FLUX.1 [dev] Non-Commercial License.

Acknowledgments

• FLUX.1 team for the base model
• ComfyUI for the inference framework
• 10 weeks of iterative development