How to: Local Stable Diffusion

Stable Diffusion is a powerful tool for generating images, but many users rely on cloud services to run their models. What if you could take full control and run Stable Diffusion on your local machine? Enter Automatic1111, a popular web-based UI that makes managing local models easy and highly customizable. This guide will take you through the setup and explain key concepts like LoRA and ADetailer, helping you get the most out of your local Stable Diffusion setup.

Setting Up Automatic1111

First things first, you'll need to install Automatic1111, which provides a sleek interface for running Stable Diffusion on your local machine.

Prerequisites

  • A GPU: While you can technically run Stable Diffusion on a CPU, it’s incredibly slow. A decent GPU is essential for reasonable performance.
  • Python: Make sure Python is installed on your system. You'll need at least version 3.10.
  • Stable Diffusion model files: You’ll need a Stable Diffusion model checkpoint to run. These are typically .ckpt or .safetensors files, which you can download from various sources online, like Hugging Face.

Installation Steps

  1. Clone the repository: First, navigate to the folder where you want to install the Automatic1111 UI. Open a terminal and clone the GitHub repo:

    git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
cd stable-diffusion-webui

  2. Install the dependencies: Next, install the required dependencies using:

    pip install -r requirements.txt

  3. Download the Stable Diffusion model: You need to place your downloaded .ckpt or .safetensors model file into the models/Stable-diffusion/ folder within the stable-diffusion-webui directory.

  4. Run the web UI: Finally, start the web UI by running:

    python webui.py

After running the script, you should be able to access the web UI by opening a browser and navigating to http://127.0.0.1:7860/.

Key Concepts: LoRA and ADetailer

Now that you've got Automatic1111 running, let’s dive into some advanced features that can take your image generation to the next level: LoRA and ADetailer.

LoRA (Low-Rank Adaptation)

LoRA (Low-Rank Adaptation) is a technique designed to fine-tune large models like Stable Diffusion without consuming too many resources. Instead of retraining the entire model, LoRA injects smaller, task-specific weights into the model, effectively "adapting" it for new tasks. This is useful when you want to train the model on specific styles, subjects, or artistic directions but don’t have the computational power to retrain the full model.

With LoRA, you can:

  • Train on smaller datasets
  • Reduce the computational cost of fine-tuning
  • Easily swap different styles or customizations by loading different LoRA adapters

To use LoRA in Automatic1111:

  1. Upload your LoRA weights (typically .ckpt files) into the models/Lora folder.
  2. In the Automatic1111 UI, you can select the LoRA model from the dropdown and apply it to your base Stable Diffusion model.

This allows for incredible flexibility in applying specific styles or targeted customizations to your images without fully re-training.

ADetailer: Enhancing Image Details

ADetailer is another powerful tool that can significantly improve the quality of generated images. When using Stable Diffusion, you might notice that the finer details, such as hands, eyes, or textures, can sometimes be off. ADetailer helps by focusing on these specific regions and enhancing the clarity and accuracy of the details.

How does it work? ADetailer runs a secondary pass over the generated image, specifically focusing on areas that need refinement. It works particularly well on images where complex details, like facial expressions or hand positions, are crucial to the overall composition.

To use ADetailer in Automatic1111:

  1. Once the initial image is generated, use the ADetailer settings to specify which areas you want to refine (like hands, faces, etc.).
  2. Run the refinement pass, and ADetailer will enhance those specific details without compromising the overall image.

Customization and Fine-Tuning

One of the greatest benefits of running Stable Diffusion locally through Automatic1111 is the ability to fully customize the workflow. You can:

  • Experiment with different samplers: Automatic1111 offers a variety of samplers, such as Euler, DPM, and DDIM. Each one will affect the quality and speed of the generated images in different ways.
  • Adjust hyperparameters: Fine-tune settings like CFG Scale, Steps, and Denoising to control how closely the image adheres to the prompt.
  • Use ControlNet for advanced control: ControlNet allows for input conditioning, enabling precise control over the generated image by providing control images like pose references.

Cost and Performance Considerations

Running Stable Diffusion locally is resource-intensive, especially when generating high-quality images at higher resolutions. However, the upside is that you maintain full control over the process and avoid the costs associated with cloud-based services. Keep in mind the following:

  • GPU memory: Larger models or more complex operations will require more VRAM. If you find that you're hitting your GPU's limits, consider using lower resolution settings or experimenting with lighter models.
  • Batch processing: Automatic1111 allows for batch image generation, letting you generate multiple images in one go, but this too increases memory load.

For a great community website for anyone looking to learn more or to find LoRAs and other models to get started, visit Civitai which has many examples of generated images, detailing the parameters, models and LoRAs used to generate them, which can be a great help when getting started.

Running Stable Diffusion locally with Automatic1111 gives you unmatched flexibility and control over the image generation process. With powerful features like LoRA for fine-tuning and ADetailer for enhancing details, you can push the boundaries of what AI-generated art can achieve.