Stable Diffusion

Latent diffusion models (LDMs) are a type of image generation technique that work by iteratively "de-noising" data in a latent representation space, and then decoding the representation into a full image. This is in contrast to other popular image synthesis methods such as generative adversarial networks (GANs) and the auto-regressive technique used by DALL-E. The Stable Diffusion model is created by a collaboration between engineers and researchers from CompVis, Stability AI, and LAION and released under a Creative ML OpenRAIL-M license, wich means that it can be used for commercial and non-commercial purposes.

The release of this file is the culmination of many hours of collective effort to compress the visual information of humanity into a few gigabytes. Furthermore, the model also supports image-to-image style transfer, as well as upscaling and generating an images from a simple sketch. Included is also an AI-based Safty Classifier, which understands concepts and other factors in generations to remove outputs that may not be desired by the model user.

General
Relese date	August 22, 2022
Research Paper	https://ommer-lab.com/research/latent-diffusion-models/
Type	Deep learning text to image model

Stable diffusion Tutorials

👉 Discover more Stable Diffusion Tutorials on lablab.build

Knowledge Base

Find out how it is working!

Research Paper The Stable Diffusion paper describes the model and its training process in detail.
Stable Diffusion Demo You can play around with Stable Diffusion on Hugging Face
GitHub Repository Visit the Stable Diffusion v2 repository on GitHub
dreamstudio Online stable diffusion interface with a lot of optional configurations

Models

There are plenty of Stable Diffusion models, which are taiolred to deliver various art styles, animation styles and more. We encourage you to experiment with many of them and choose the one which you like the most. Here are some of the finest ones:

Boilerplates

Get started with technology!

Stable Diffusion Colab A Colab notebook that shows how to use Stable Diffusion with GPU!
Stable Diffusion & Whisper Colab A Colab notebook that shows how to use Stable Diffusion with Whisper!
Stable Diffusion Interpolation V2.2 A Colab notebook about Stable Diffusion Interpolation!

Fresno LUX: A Living City

Fresno LUX is a prototype for a living intelligence layer across an entire city. Fresno already contains enormous amounts of intelligence: residents know where problems exist, nonprofits know where help is needed, businesses know where opportunities exist, educators understand gaps in learning, researchers generate knowledge, and government maintains critical infrastructure and data. The problem is that much of this intelligence is fragmented across disconnected systems. Fresno LUX connects those pieces. The platform creates a shared city graph where people, organizations, resources, needs, projects, observations, experiments, and outcomes become connected nodes. Instead of simply collecting data, LUX continuously asks: What is happening? What do we know? What is missing? What can we test? What changed? A community problem can become an observable signal. That signal can become an experiment. The experiment produces evidence. Evidence changes the system's understanding. That updated understanding informs the next action. Observe → Connect → Understand → Act → Measure → Learn → Converge. The goal is to move Fresno from disconnected information toward collective intelligence—a system where knowledge can travel across boundaries and successful solutions can be discovered, tested, measured, and replicated. LUX is designed to make the invisible connections visible: the relationship between a neighborhood and its resources, between a student and educational opportunity, between a business and workforce capacity, between environmental conditions and community health, and between an idea and measurable real-world impact. This hackathon prototype demonstrates the beginning of a Living Fresno Graph: a city that can observe itself, learn from evidence, and coordinate toward better outcomes.

Dragon Forge 3D: Agentic 3D creation studio

3D content creation is widely used across games, architecture, simulation, entertainment, visualization, and other digital applications. However, producing 3D content remains complex and fragmented, requiring specialized knowledge and multiple software tools. The main limitation is often not imagination, but the gap between what users can imagine and what they can practically create. Agentic Dragon Forge 3D is an agentic 3D creation studio designed to address this gap by integrating the 3D creation workflow into an intelligent, conversational environment. Users interact with an AI Agent through natural language to describe what they want to create, while the system coordinates research, visual reference discovery, concept development, preview generation, 3D generation, inspection, and refinement. The system is organized around two core components: FORGE and STAGE. FORGE focuses on creating usable 3D assets. The Agent can research the subject, identify relevant references, generate visual previews, create 3D models, and support retexturing, rigging, animation, and real-world scaling. The workflow is not fixed; the Agent can move between stages, repeat steps, or adapt the process according to user requirements and feedback. STAGE extends the workflow from individual assets to complete environments and worlds. It supports natural environments and architectural spaces, allowing users to assemble and manipulate assets within a scene. Users can either instruct the Agent through natural language or directly control the interactive interface. This creates a continuous workflow from imagination to 3D assets and from assets to worlds, while keeping the resulting 3D content usable beyond the platform for games, architecture, visualization, AR, and 3D printing. The system is built on Natively Builder and integrates OpenAI for agent intelligence and image generation, Serper API for web and image research, and Meshy API for 3D generation, retexturing, rigging, and animation.

ReelBudget

The Vision Pre-production is one of the most expensive and time-consuming phases of video creation. Whether you are an independent filmmaker pitching a short film or an enterprise marketing team designing a high-budget commercial, the process requires weeks of sketching, planning, and budgeting before a single frame is ever shot. Reelbudget was built to completely automate this process, acting as your autonomous AI creative director and line-producer. How it Works Users simply input a raw text creative brief. Our multi-agent backend instantly parses the creative intent and breaks the script down into structured scenes, characters, and semantic camera directions. The AMD MI300X Advantage To bring these storyboards to life, we needed massive, uncompromised compute power. We proudly deployed our True AI Video Engine on the AMD MI300X Cloud Instance. Rather than relying on simple static images, our pipeline leverages the MI300X's incredible speed to run two state-of-the-art models on the fly: Stable Diffusion XL generates hyper-realistic, anatomically correct base frames. Stable Video Diffusion instantly animates those frames into true moving video clips, physically matching the AI's camera directions. The system then automatically stitches these clips together, burns in dialogue subtitles, and overlays a background soundtrack, outputting a highly polished, ready-to-pitch movie animatic. Feasibility & Cost Estimation Finally, our AI Feasibility Agent analyzes the generated shot list to provide accurate, real-world cost estimations and suggests cheaper production alternatives, saving businesses thousands of dollars in pre-production overhead. Reelbudget proves that with AMD hardware, AI can handle the heavy lifting of video production, allowing creators to focus entirely on the story.

Gemma Captioner: Four Voices, One Truth

Most video captioning agents write all four styles in a single pass, so one visual mistake poisons every caption. And the funnier a caption tries to be, the more it tends to invent: a tech joke reaches for packets and uptime that were never there, a sarcastic line flips what actually happened. Track 2 scores both accuracy and tone, so a caption that reads well can still be wrong. We built it the other way around. Instead of adding style at the end, the agent uses Google Gemma-4-31B to look at the video once. It turns sampled frames into a grounded record of what is actually there: it reads clearly legible on-screen text like signs and logos, names concrete colors and counts, and flags whatever it cannot be sure of. Any detail a caption asserts that the frozen facts do not support is rejected and rewritten, so unsupported claims never survive. Only then do four parallel Gemma-4 text passes turn the same verified facts into formal, sarcastic, humorous-tech, and humorous-non-tech voices. A joke can change the tone but never the events, because the humor voice has to map its metaphor onto a real visible detail. The other hard part is speed. Long UHD clips are the trap, because a clip that finishes too late is graded as a placeholder. So the agent decodes only about ten frames per clip, aligned to the exact timestamps the judge samples for accuracy, rather than the whole video, and it captions multiple clips concurrently. A complete results.json sits on disk from the first moment and each clip upgrades its own entry as it finishes, so a slow or failed call degrades to a grounded fallback instead of a zero. The full set of clips finishes well inside the 10-minute budget. Gemma does the real work. Every scene fact and every caption comes from Gemma-4-31B: take it out and there is no product. It ships as a public linux/amd64 Docker image that runs headless, reading /input/tasks.json and writing /output/results.json.

ForgeAi

ForgeAI is a hardware-aware AI model optimization platform that automatically finds the fastest, most efficient version of a model for a specific GPU — starting with AMD MI300X. Instead of manually tuning models for each accelerator, ForgeAI runs a 7-phase optimization pipeline: architecture search finds the best candidate structures, knowledge distillation transfers accuracy from a teacher model, pruning removes redundant weights, quantization compresses from FP32 to INT8, benchmarking measures real performance on target hardware, Pareto analysis identifies optimal latency-accuracy tradeoffs, and Optuna hyperparameter tuning auto-optimizes across 6 parameters with 50 trials and early stopping. The platform consists of a FastAPI backend with 9 optimization modules, a Next.js 14 frontend, and WebSocket-based live progress streaming. Users upload a PyTorch checkpoint, select target hardware, set constraints (max latency, max memory, min accuracy), and watch the pipeline execute in real time. Results include a Pareto frontier chart, before/after performance comparison, and export to ONNX or TorchScript. ForgeAI targets the $100B+ AI inference market where hardware-specific optimization is still done manually. Unlike Neural Magic and OpenVINO (CPU-focused, tool-by-tool), ForgeAI is AMD-native, full-pipeline, and open source under Apache 2.0.

Framure Forge: Text-to-PBR Materials on AMD GPU

Framure Forge turns a text prompt into a game-ready PBR material. You describe a surface — "weathered copper, oxidized patina" — and SDXL, running on an AMD Radeon PRO W7900 (gfx1100) through ROCm, renders the albedo. The normal and roughness maps are then derived on the CPU, and the full set is applied live to hero objects in a Three.js showroom, where you can inspect each map and download a zip ready for any engine. It is built for indie game developers, 3D and environment artists, archviz, and students learning PBR — anyone who needs surfaces fast without the asset-store rabbit hole or per-texture licensing. The output is standard PBR maps: nothing proprietary, nothing locked in. It works with Unreal, Unity, Blender, and Godot. The AMD GPU does the real work: a warm 1024px generation takes about ten seconds, verified with rocm-smi and a live /health endpoint reporting gfx1100 and model_loaded:true. The seven materials were proven end-to-end by importing them into Unreal Engine 5. Both frontend and backend are containerized. The public demo runs in a transparent mock mode so judges can try it instantly, with a popover that says so.