• Post comments:1 Comment

AI Art – illustrator’s point of view

Ai Art – introduction

If you are passionate about painting and creating, I assume that the topic is not unknown to you. Or maybe you haven’t yet missed it? I envy you then!

I’ll try to describe what’s behind the term AI Art and give my thoughts on the topic that is firing up online forums and YouTube comment sections.

I don’t know much about programming, so I will describe the performance of AI generators in a brief way. I will focus on my opinion of the impact of the new technology on work in the illustration profession. And I make no secret of the fact that I have more fears. Fears need to be domesticated instead of cultivated, so let’s dive in, shall we?

What is AI Art

In short: an amazing technology has been created that we never dreamed of, but the way it is used and introduced is horrible I believe that sooner or later ethical models will be created with a public domain database and artists who will allow such use of their works. Using technology created in this way should no longer raise moral objections. Whether we want to use it at all is another matter.

The longer version:

Progress

I began to be interested in the topic of image generators in March, but the work had been going on for years. The idea of describing an idea in words and getting the result in the form of an image was exciting!

Generated in Dream by Wombo

After playing with Dall-E Mini and the Dream app from Wombo, I also took to testing the newly-emerging options: Disco Diffusion, Dall-E2 and MidJourney. The first generated images had a lot of anatomical errors, artifacts, but even then their colors, textures were impressive. In July, when the quality of the results improved significantly, and the work generated in MidJourney won the competition, it became clear to me that the future of illustrators may not present as colorful and luminous as on the winning work.

Ghost in the shell or the theft of the century? How generators work

So-called AI models have no intelligence or consciousness. They are neural network models. You call them what you want, but the fact is that their capabilities are astounding!

You type in a description (prompt) and puff! Magic happens and an illustration pops up that you never even dreamed of, not to mention of painting it yourself. Sometimes the description needs polishing and several rounds of generation to get a satisfying result. Often it is not possible to get an exact reproduction of the idea at all. Still, it’s comparing a couple of minutes of generating and tweaking descriptions with hours of painting and years of study to achieve a similar level of refinement of the illustration. That’s impressive, isn’t it?

It’s not magic that makes the pixels line up in a certain pattern. It’s an algorithm trained on huge databases containing indexes of countless images from the Internet with their descriptions. The most popular such database (Stable Diffusion uses it, for example) is Laion-5B. It contains 5 billion indexed images. These range from those available in the public domain to copyrighted images to cases of medical records. Naturally, no one asked the owners for permission to put this data in the index and use it to train so-called AI models.

The database was created for research purposes, so it does not violate copyright as it stands today, but is it perfectly legal to use this data to train models? Can it be determined if today’s copyright laws did not consider such use at all?

The models themselves do not generate results by creating collages from the works in the databases, although the effect can sometimes indicate this, and it is possible to generate an image very similar to the style of the artist included in the description. Their method of operation is more complicated. Saying that generators “steal” is, in my opinion, a big oversimplification.

Can we talk about abuse and exploitation of artists’ works? The quality of the results depends on the quality and quantity of illustrations and photos indexed in the database, and many times users include artists’ names in their descriptions to represent their style. Even knowing that style is not subject to copyright protection, I think it we can call it that way.

The battle over copyright

AI Art enthusiasts point out that models create just like humans: they look at indexed illustrations and images on the Internet, use them as references and “learn” to reproduce them being careful not to duplicate them 1:1 (overfitting mechanism). With the differences that they do it faster, they have an infinitely greater ability to remember and reproduce once “seen” patterns. And they never forget. Therefore, there is nothing to be gained by removing images from the database after the model has already been trained. It would be necessary to delete the entire model.

I think this is what is behind the release of the most popular one – Stable Diffusion as open source. It is on millions of users’ computers today and cannot be erased. This is my conspiracy theory.

At the same time, the “overfitting” is supposed to prevent in the creation of a result too similar to the input material (“learned” in training), although this sometimes happens.

Going back to the pattern of learning by models and artists. You can search for many similarities and differences, but I think it’s completely pointless. A model is simply not a human being, period.

That’s why I think that the legality of “training” models on copyrighted material is also heavily sketchy. All laws were created for humans, considering their capabilities and limitations. I hope that law will soon follow, allowing people to regain control over what their works are used for. A lawsuit has just been filed in the US against Stable Diffusion, MidJourney and. Deviantart (a popular site for artists and creators).

Also, the popular stock site Getty Images has filed a lawsuit against Stability AI (the company behind Stable Diffusion).

Ai art – dangers

Copyrights

We have roughly addressed the most controversial topic, that is, the presence of copyrighted materials without the authors’ permission in the training data without the possibility of removing them from the models created.

Secondly – who will be responsible for copyright infringement, in the event that a user uses a model to generate work that approaches plagiarism and starts to make money from it. He himself? The creators of the model?

Models have no consciousness of their own, and do not make any conscious decisions, generating based on user descriptions. In March, a US court refused to grant copyright to a generated image (link)

Stephen Thaler’s AI creation, A Recent Entrance to Paradise

It is also difficult to say who is the author in the case where the user provides a description (acting as client/art director/visionary, but not creator/artist), the generator results are algorithm-based and largely random. On the terms and conditions pages of, for example, MidJourney, we find a statement that we have full commercial rights to the results created by the algorithm, which contradicts a court decision in the US. Amusingly, a few lines further down MidJourney shifts all legal responsibility for possible copyright infringement problems to the user. I think this looks suspicious to say the least.

Users can also train the model on their own database. Everything is fine when you create it based on your own or public domain material. However, it is common to create on illustrators’ images downloaded from the Internet without their permission in order to reproduce their style. The most nasty examples are the model reproducing Kim Jung Gi’s style right after his death….

… Or a contest for the model that most closely reproduces the Sam Does Art style.

AI will try to steal your job

We can expect that in many companies the bosses will think more than once whether they really need a graphic design studio, illustrators, photographers, if pretty good results can be achieved by a person without much artistic knowledge, but who can enter a description into a generator. Or even better -with artsitic knowledge, but still just one person. In turn, the generator will spit out dozens of variations for a small subscription (MidJourney) or for free (Stable Diffusion).

This is already happening. People are losing their jobs. Artists are not the first and will not be the last victims of automation. However, I don’t know of a more cheeky example of removing people from their profession while (in my opinion, illegally) using their own works against themselves, on a massive scale….

It will quickly become clear that generators do a great job with general themes when there is no detailed vision of the image. In situations where we know exactly what we want, they often produce nightmares. Editing these results will often take more time than having a professional do it from the beginning. That’s the way it is at this point. But model development and improvement is super fast, and today we may laugh at anatomically twisted figures with too many fingers and teeth, but that may change sooner than we suspect.

Creating vs editing

I do not agree with the marketing tricks of the companies behind the main models, presenting them as just another tool in the artistic palette. Yes, they can be used that way. However, more often I see them being used as a replacement for the artist. They remove the entire creative process, and with it the human being and most of his decisions. They replace the image created with the intention behind every artistic choice; with a description and a rather random effect.

Many say that artists will still be needed, even more than ever, to edit the effects of generation. Just…I don’t know about you, but I’ve never dreamed of a job that involves editing illustrations created by someone else. No difference if we are talking about another artist or neural networks.

Though competition

This type of customer can be expected to welcome the opportunity to pay a much smaller sum for the images generated. Reducing the number of bids will probably raise the bar for entry into the profession. It is possible that it will come to combining a full-time job with an “after-hours” art job.  This has not been uncommon before, and has the advantage of allowing you to be a bit more picky in choosing your assignments. Instead of taking on mediocre quality projects, this allows you to choose only those you really want to be involved in and show in your portfolio. Assuming one has the energy to paint after work.

Models produce countless paintings in a short period of time, resulting in a flood of social media and even online galleries (DeviantArt, ArtStation, Behance). It’s harder than ever to break through with your own work. It’s also a problem for those looking for artists to collaborate with. The additional task now is to check whether the desired candidate actually has the skills they need, or merely can type descriptions well into a generator.

In the case of well-known artists, there is another problem. Typing Gregory Rutkowski’s name into Google will show a lot of images, and only some are actually by him. Many of them are the results of prompts containing “by Gregor Rutkowski”, “in the style of…”. This certainly hits his personal brand.

Another issue is social media. Algorithms have been making it difficult for artists to break out for years, preferring the frequency of uploaded content over its quality. Generated images can be dropped into posts even if only every 5 minutes. We are unable to compete for attention in this category with AI creations.

What prompt did you use?

Mixing AI Art with works created without their use creates big confusion. Many times people, looking at a beautiful painting, assume in advance that it is the work of, for example, MidJourney, and inquire about the details of the description, which is quite amusing.

Less funny is when artists accuse each other of using a model, when this was not the case, because the style is too reminiscent of AI Art (if you look at these types of paintings a bit, you are able to catch common, characteristic features). I can only guess how pissed off must be those whose style is similar to that preferred by model users (that is, those whose works trend on ArtStation…).

It is completely not funny cases when people sell generated images for big money, claiming that they created them themselves from scratch. Such situations occur, for example, on Fiverr or Etsy.

And sure, a significant number of threats are not the fault of the technology per se, but the way people use it. Unfortunately, without clear laws regulating generators and clear condemnation of such activities, this is how the landscape presents itself. Messy.

Fighting with windmills

I don’t mean fighting for new models trained on a “pure” database. I believe that this battle is worth to fight, even if “unethical” models will continue to hulk around the network. It’s important to call wrong what is wrong and to create a legal framework for the technology to work, also to enjoy its possibilities.

Even so, models will be a major threat to the work of illustrators as we know it today. After all, we will be competing with masters of the Middle Ages, Renaissance, Baroque…. and ever-evolving technology that removes the barrier of time and skill.

And although this sounds pessimistic, I think that in the long run, it may work out for us. About that, I hope, in the next episode of the saga “AI Art and What’s Next?” 

Now to the benefits! Because there are some.

Ai art- benefits

For a moment, let’s assume that generators trained on databases containing only public domain images and works by artists who have given their consent are now available, and the ethical problem goes away. Then the real fun begins!

Ideas

You feel that you want to draw, but you don’t know what? You spend hours flipping through the web and nothing sparks your interest. The sheet or canvas in the graphics program still remains blank. Meanwhile, all it takes is a bunch of words and a few repetitions in generation for the model to spit out dozens of ideas in the indicated direction.

The less detailed the images in the results, the better, more room for your imagination! It’s easier to see something interesting in textures, colors and shapes and use that seed from which your idea will germinate.

If the results are more defined, it may be tempting to use them and just paint on them, edit and transform them. That will certainly be interesting too, especially as an exercise. However, personally, I will always be more interested in the images that the artist painted from scratch than in the transformations.

For me, creating has always been more attractive than recreating. It’s possible that’s why I’ve sometimes had trouble getting the nature exercises right 🙂

Exercise materials

Much can be said about AI Art, but one thing is undeniable: it can often make an impression. Even if, upon closer inspection, it turns out to be completely nonsensical and, more often than not, the images lack a message. Yet they do catch the eye. I think an interesting exercise is to think about why we like some of them. Is it the play of colors, the contrast, the light, the composition, the shapes? Making quick studies from images that catch our eye can always teach us something new.

Although, studying stills from movies or paintings by masters has the great advantage that they are generally recognized as works of art for a reason, so you learn from examples that have been successful and have won public acclaim.

Color palettes

This is the first thing that caught my eye when browsing the MidJourney gallery – beautiful color combinations. I even created myself some palettes for Procreate based on the generated images. Seeing how they work out in different combinations is quite an exercise and fun.

Inspiration boards

It will certainly be very popular to use exorbitant images for inspiration, to create an idea of some place or character in a book or comic book we are creating. Just like using inspiration from the works of other artists, as long as we create something new, our own, different there is nothing wrong with such use.

As for using AI Art as reference images – it will certainly work well for designing locations, characters, costumes. However, I would be cautious about mapping especially character anatomy or perspective. These topics are not yet handled well by the models.

Your personal AI Art model

This option is very interesting to me. The ability to train your own model based on a database containing only your images, which can reproduce your style quite faithfully. This could greatly speed up work on your own illustrations while maintaining stylistic consistency.

However, I think of such use more as a curiosity than a tool in my work as an illustrator. It is very important for me to have full control over the resulting image and to work on my own ideas. I also find the greatest joy in the creative process. Reproducing and editing is boring. Still, I’m always curious to see how a given image would look in different versions, and here I think, such a model can be very useful.

Although…perhaps there are cases where I could give up some of my creative power in exchange for greater efficiency, for example, creating my own comic book or 2D animation – ventures that take years to create, and thanks to this technology seem, to be quite real.

Unique authenticity

The wider the popularity of ” images from the can”, the more space they occupy on websites, the faster they will become boring. In a world that is increasingly detaching itself from the hard ground and drifting unreflectively toward an cyber-reality, the need for contact with another human being, the exchange of thoughts and emotions, is all the more vividly marked. And nothing is better able to express them than man-made art. In a painting, I am interested not only in the result, but also in the creator behind the work, the emotions he wanted to express, the story he wanted to tell, the process that accompanied it, the choices he had to make.

All this is removed or heavily flattened in the generation process. It’s like reading only the last page in a book. I know how it ended, but I don’t care at all.

I believe, that I am not the only one who feels this way.

The triumphant return of traditional techniques

In a situation where bots combing the Internet are able to index images even from personal domains and servers (this was the case with mine), the feeling that someone is heavily jerking us around (using our work and there’s nothing we can do about it) causes a lot of frustration. I’ve noticed in myself and other illustrators a return of affection for traditional techniques. It’s safer in this area. A bot won’t break into our drawer and kidnap a freshly finished watercolor or gouache (hopefully).

When creating traditionally, there is no doubt that our skills are behind the result. No magic buttons that, when pressed, create a “wow” effect. You won’t find them in Photoshop or Procreate either, but it’s amazing how many people imagine digital painting this way and don’t see much difference between digital painting and model-generated images. When it’s just you, a blank canvas, paints and brushes no one questions your authorship and right to what you create.

Domestication of fear

And this is what (from my perspective) the landscape looks like after the appearance of the so-called AI Art. There is chaos and a storm of emotions, as there always is when we enter unknown waters. And outrage, because our creations are being used as a paddles, still we have no control over the direction we take.

🎶Hoping for the best, but expecting the worst🎶.

I’m full of the worst premonitions when it comes to working in the creative professions, but divining how things will be in a year or 5 years is like a long-term weather forecast – it will rain or it won’t. We will see.

This post was created to gather facts, predictions, emotions and consider calmly what bad can happen and what good? And if it’s the worst and the creators are actually replaced by models, will you stop creating? Because of course I won’t! And you?

I’d love to know how you see it? Are there any pros and cons that I’ve overlooked? I’ll hint that there are, but this post is already way too long. What emotions does this topic evoke in you? Leave a comment, I’m very curious!