Stability AI has rolled out the latest in their lineup of image-generating AI, with the Stable Diffusion 3.5 family introducing three new models designed to offer improved flexibility and faster performance. Aimed at different user groups, these models address some of the concerns users had with the previous version while keeping the door open for developers to customize them as needed.
Stable Diffusion 3.5 Large Improves Quality
At the top of the release, Stable Diffusion 3.5 Large, an 8-billion parameter model, stands out for users looking for high-quality images that adhere closely to prompts. Stability AI has described this model as well-suited for professional use, particularly when it comes to generating sharp visuals for marketing or enterprise needs, where resolution and accuracy are key.
A variation of this is the Stable Diffusion 3.5 Large Turbo model, which keeps the same image quality, but adds the ability to generate those images faster. Stability claims that Turbo requires only four steps, making it one of the speediest AI models in its class while staying competitive on both image quality and prompt accuracy.
The third addition, Stable Diffusion 3.5 Medium, is aimed more at the individual creator or smaller teams, offering a model that requires fewer resources to run. Unlike its larger siblings, it operates on 2.6 billion parameters and focuses on producing a good balance between quality and accessibility, making it suitable for a variety of devices. Stable Diffusion 3.5 Medium will be released on October 29th.
Tech Tweaks and New Features
One of the updates that sets these models apart from earlier versions is the integration of Query-Key Normalization (QK-Norm). This tweak is designed to make the model easier to fine-tune for developers while stabilizing training. It also improves the way the AI responds to user prompts, which should lead to more accurate image generation, especially when prompts are specific.
However, Stability AI has acknowledged that there are trade-offs involved with this approach. For instance, the broader variability in image output when prompts are vague is intentional, allowing the model to generate a wide range of styles. But this means that less specific prompts could result in less predictable image quality, something Stability has been upfront about.
For the Medium model in particular, the company has also enhanced the architecture to better handle multiple image resolutions. This tweak, tied into their MMDiT-X architecture, allows for multi-resolution generation, meaning the model can output images with consistent quality across different sizes.
Stability AI also says that the models “excel in creating images representative of the world, not just one type of person, with different skin tones and features, without the need for extensive prompting“, as can be seen in some sample images they provided.
Faster Generation with Prompt Precision
What Stability AI emphasizes with this release is the improved prompt adherence in the new models. This means the AI is better at sticking to user inputs, producing images that closely match what was requested. This improvement was made possible by refining the dataset and using more advanced training protocols.
According to Hanno Basse, CTO of Stability AI, the new models’ prompt adherence represents a major leap forward, particularly for the Stable Diffusion 3.5 Large model, which is said to lead the market in accurately interpreting prompts.
Open Licensing and Self-Hosting Options
As with previous versions, the Stable Diffusion 3.5 models will be made available under the company’s Stability AI Community License, allowing users to access them for free for non-commercial use. Companies making under $1 million a year in revenue can also use the models commercially without cost, but larger companies will need to opt for an enterprise license.
The new models are available on multiple platforms, including Hugging Face, where the model weights can be downloaded for self-hosting. They’re also accessible through Stability AI’s API and other services like Replicate, Fireworks, and ComfyUI.
Addressing Legal and Ethical Concerns
Like most AI companies, Stability AI relies on publicly available datasets for training its models, which has raised questions about copyright. The company maintains that its use of such data falls under fair use, though this hasn’t prevented lawsuits from cropping up.
Stability AI allows creators to request that their data be removed from training sets. As of early 2023, artists had removed about 80 million images from Stability’s training data. But the company leaves it up to its customers to handle any legal claims related to the AI-generated content they produce.
Beyond copyright, Stability AI also touches on the topic of AI misuse, stating it has taken measures to prevent its models from being used for misleading purposes, though specifics on these safety features were not provided.
Commercial Use and Ownership Rights
As with prior releases, users retain ownership of the media they generate with Stable Diffusion 3.5 models. The company encourages those who monetize their creations to include attribution, requiring the use of its community license to avoid any legal hurdles.
These rights make Stability AI’s models particularly appealing to small startups and individual creators looking for low-cost ways to generate high-quality media without having to navigate restrictive licensing terms.
Looking ahead, Stability AI has plans to release additional tools for fine-tuning the models, including ControlNets, which will provide even more control over image generation, adding to the models’ utility across professional and commercial applications.
Last Updated on November 7, 2024 2:23 pm CET
