Flux.1 + LoRA Tutorial: The duo that could replace Midjourney (Prompting guide included)

Published in

Future of Work & Society — Towards the quaternary economy

8 min readAug 12, 2024

Hi! We’ve been seeing this new AI Flux.ai on social networks, which is apparently surpassing Midjourney in terms of realism. What’s the real story, and how can we easily create ultra-realistic images of people ourselves?

Today, I’m going to guide you through creating ultra-realistic images of people using Flux, the revolutionary new AI from Black Forest Labs, combined with the power of LoRA.

Here are some examples of what you’ll be able to create by the end of this tutorial (Tomorrow Theory is my company, by the way, check it out!):

Context

Flux, a serious competitor to Midjourney, the AI image generation tool considered the most powerful to date, offers impressive results while being open-source. Open-source means you can download a Flux model directly to your personal PC or in your company and keep 100% control over your data, ideal for any organization!

A word on ethics (because it’s important!)

Before diving into the subject, let’s take a second to talk about ethics. Yes, I know it’s not the most fun, but it’s crucial!

With the power to create ultra-realistic images comes great responsibility (thanks, Uncle Ben). Here are some points to keep in mind:

Consent is key: Avoid creating images of real people without their permission. It’s not cool and potentially illegal.
Fake news, no thanks: Don’t use these images to create or spread false information. We already have enough chaos as it is!
Respect, always: Avoid creating offensive, discriminatory, or inappropriate content. Be creative, not problematic!
Transparency: If you share these images, be clear that they are AI-generated. No deception, let’s play with our cards on the table.
Copyright: Be careful with references to protected brands or characters. It’s better to stay original!

By keeping these principles in mind, you can explore all the creative potential of Flux while staying on the light side of the Force. Now that we’ve done our good deed, let’s get to the serious stuff!

For non-tech people (or lazy tech people like me!)

You’re in the right place! For simplicity and accessibility reasons, I’ve chosen to use the fal.ai platform, which will allow us to have the most realistic images simply thanks to LoRA.

In this tutorial, we’ll explore how to use the Flux and LoRA combination on fal.ai to generate stunning images. Then, we’ll push the quality of our creations even further by using the upscaling tool (realism improvement) from rendernet.ai.

For tech people

For the more tech-savvy and resourceful among you, I invite you to visit the Black Forest Labs website, where you’ll find an API endpoint, the Flux Github link, and a more advanced tutorial for installing the model locally.

What is LoRA?

Just for your general knowledge: LoRA, or Low-Rank Adaptation, is a technique that allows AI models to be fine-tuned for specific styles or subjects. It’s like adding a creative filter to our AI, allowing it to specialize in certain visual aspects without having to retrain the entire model. Here, it will allow us to obtain even more realistic images than with the base model.

How to prompt on Flux?

To get the best results with Flux, it’s crucial to prompt in English and structure your prompts effectively. Here’s a structure you can follow to create detailed and coherent prompts:

Example prompt : “A photo of a beautiful smiling brunette, with long hair in a messy ponytail wearing a colorful dress with a blue lanyard around her neck holding a sign saying “I love Tomorrow Theory” at a TedX stage. She is smiling while talking to the audience. There is a white wall behind her and there is a yellow Google logo on it. She has a nice body shape. Her eyes look like they have something interesting or smart ideas inside them. The overall atmosphere conveys confidence and determination as she shares knowledge about AI tech with people from all walks of life, in the style of a TedX talk.”

Description of the main subject: Start by describing the physical characteristics and clothing of the subject.
Example: “A beautiful smiling brunette, with long hair in a messy ponytail wearing a colorful dress”
Action/Pose: Describe what the subject is doing and their posture.
Example: “holding a sign, smiling while talking to the audience”
Context/Location: Specify the environment where the subject is.
Example: “at a TedX stage, with a white wall behind and a yellow Google logo on it”
Additional details on appearance: Add specific details to refine the image.
Example: “She has a nice body shape, her eyes look like they have interesting ideas inside them”
Mood/Atmosphere: Describe the overall atmosphere you want to convey.
Example: “The overall atmosphere conveys confidence and determination”
Activity/Theme: Explain what’s happening in the scene or the main theme.
Example: “she shares knowledge about AI tech with people from all walks of life”
Style/Visual reference: Indicate a specific style or visual reference.
Example: “in the style of a TedX talk”

By following this structure, you’ll create prompts rich in details that will guide Flux to generate precise and captivating images. Don’t hesitate to experiment and adjust your prompt based on the results obtained!

Now that you know how to structure your prompts, let’s move on to the practical steps for using Flux on fal.ai.

Let’s go!

Step 1: Create an account on Fal.ai

Start by creating an account on fal.ai. You’ll need a Github account (easy to create).

You might need to purchase a few credits, free credits were offered at the time of writing.

Step 2: Select the AI model

Select the “FLUX Realism LoRA” model, the Flux model specially optimized for creating realistic portraits.

Step 3: Click on “more”

Once you’re on the model page, you’re almost there! Now click on the “more” button to display the options.

Step 4: Configuration

Just 3 small steps left!

First, write your prompt in English (see 1 in the image above). For example:

Prompt : “A photo of a beautiful smiling brunette, with long hair in a messy ponytail wearing a colorful dress with a blue lanyard around her neck holding a sign saying “I love Tomorrow Theory” at a TedX stage. She is smiling while talking to the audience. There is a white wall behind her and there is a yellow Google logo on it. She has a nice body shape. Her eyes look like they have something interesting or smart ideas inside them. The overall atmosphere conveys confidence and determination as she shares knowledge about AI tech with people from all walks of life, in the style of a TedX talk.”

Once the prompt is written, change the “Num inference Steps” from 28 to 48, this parameter determines the number of iterations before your image is ready, here we want many iterations to have the best possible quality. (see 2 in the image above).
Finally, click on “Run” (see 3 in the image above).