馃帹 Stable Diffusion with Laravel

馃帹 Stable Diffusion with Laravel

A package with a Prompt builder and image generation capabilities through Replicate's API.

3 min read

Yeah, I know. Sloths are cool, right? They're even cooler when they ride on a skate in New York.

I am sure most of you already know about Stable Diffusion, so I will not spend too much time explaining how it works. Basically, it's an Open Source AI that generates images from text input. Any image. It's crazy!

There's an API where you can test it, but it can be a bit difficult to get nice results. There's a nice guide on how to create prompts in this page.

screenshot of the website showing the prompts guide

Today, I've built a Laravel package around this Stable Diffusion that makes it super easy to generate images using Replicate's Stable Diffusion API.

How to install it

First of all, sign up in Replicate and copy your API token from your profile. You will need it in a moment.

Then, go to your Laravel project and install the package with the following command:

composer require rulilg/laravel-stable-diffusion

After the package is installed, make sure to publish and run the migrations:

php artisan vendor:publish --tag="stable-diffusion-migrations"
php artisan migrate

Then, create a variable in your .env called REPLICATE_TOKEN and paste your token in there:

REPLICATE_TOKEN=token

And that's it! Now you're setup to generate images using Laravel and Stable Diffusion 馃殌

How to generate images using Laravel Stable Diffusion

It's pretty easy. First of all, it's important to understand that the image generation process is async. We need first to send a request to Replicate and, after a few seconds, we can fetch the results back.

Right now they have no webhooks, so we will need to do a bit of polling depending on your use case.

Now that this is explained, this is the code we need to generate an image:

use RuliLG\StableDiffusion\StableDiffusion;
use RuliLG\StableDiffusion\Prompt;

$result = StableDiffusion::make()
    ->withPrompt(
        Prompt::make()
            ->with('a panda sitting on the streets of New York after a long day of walking')
            ->photograph()
            ->resolution8k()
            ->trendingOnArtStation()
            ->highlyDetailed()
            ->dramaticLighting()
            ->octaneRender()
    )
    ->generate(4);

The magic happens with the Prompt::make() instruction, which lets us dynamically generate a nice prompt with some predefined templates. You can see the list of the available methods by clicking here.

For example, let's try to generate the image from this post using the library:

use RuliLG\StableDiffusion\StableDiffusion;
use RuliLG\StableDiffusion\Prompt;

$result = StableDiffusion::make()
    ->withPrompt(
        Prompt::make()
            ->with('a sloth riding on a skate in the streets of New York')
            ->photograph()
            鈫抙yperrealistic()
            ->resolution8k()
            ->highlyDetailed()
            ->dramaticLighting()
            ->octaneRender()
    )
    ->generate(); // if we don't pass a number, it will generate 1 picture

If we run this, we might get an image like this:

a sloth riding on a skate in the streets of New York

After the 鈫抔enerate() method is called, an API request to Replicate is sent so they can start processing the prompt. This will generate a record in our database with a status=starting value, and the ID returned by Replicate.

To retrieve the results back from Replicate, we need to run the following code:

use RuliLG\StableDiffusion\StableDiffusion;

// we get the Replicate ID and fetch the results back.
$freshResults = StableDiffusion::get($result鈫抮eplicate_id);

If the results were already fetched, then no API request will be made, so we can safely call this method every time.

Now, to get the images:

if ($freshResults鈫抜s_successful) {
  dd($freshResults->output); // List of URLs with the images
}

Summary

We've seen how easy it is to create prompts and generate images using Stable Diffusion thanks to this Laravel package. Please, feel free to suggest any changes or make PRs so I can review them anytime soon :-)

Thanks for reading. Ra煤l