Village in Waterfalls - AI Art and Experiment

avatar
(Edited)

It has been a hobby for me to create AI arts in the past few days. I had good results and sometimes awful and just patches of images like an ugly collage. I had a love and hate relationship with AI. Mostly in awe of what illustration I produced from my prompt and AI generator. A few days ago, I wrote After the Apocalypse, where I hypothesized that AI builds a future from the past output. The discussion was interesting in that I wanted to replicate the result. Unfortunately, my attempt was not as successful as it was before. But I still find interesting findings in my last experiments.


Exploring Perspectives


I tried a new prompt scheme to explore in my AI arts. I vividly described in one sentence what I wanted to see in my illustrations. I often explored art styles, like concept art, watercolor, gauge, ink wash, and a lot more. I never tried perspectives before. In this new experiment, I tried to incorporate perspectives in my prompt, like a perspective or a birds' eye view. I also specified where my viewer is, like a person's view at multiple angles or a pathway going into a village. I did some successful runs while some just sewed images.

AI Art: Village in Waterfalls I
2i0QlG76u5oMxZMA8zlX_out (1).png


AI Art: Village in Waterfalls II
5DqTUFhqJ4Qh4tD6CfOu_out (1).png


AI Art: Village in Waterfalls III
l5dJiqAGV7Wz3AHYD1xl_out (1).png


AI Art: Village in Waterfalls IV
EPcl28pcIk8dBv7ocMni_out (1).png

The illustrations above are successful results of putting in the prompt how we should view the village. I typically included looking at a far or seeing at a distance. There are results where it generates a person watching over rather than the perspective only. I think AI tried to associate the words to their meaning and conceptualized an illustration from that. I also did prompts on facing the gate or pathway to the village, and some good results, shown in Village in Waterfalls II and III. Village in the Waterfalls IV depicts a perspective looking at the back of the villages. It is interesting to see how AI portrays the back part of the village as it is seemingly an extension of what we see at the front.

AI Art: Village in Waterfalls shown in Bird's Eye view
DBQbVigpCQ0KfRCajCEO (1).pngNPmrKtbgzCkbuoeGm8D0 (1).png

I tried bird's eye view. I was in awe of the result. It seemingly maps the village I wanted to see in my prompt, but I find no similarities in the layout of structures in my other illustrations. AI interprets the prompt differently despite the run start at the same noise (setting seed at 42). Why seed 42? No reason at all. It is just a random number I placed to start my prompt. I think with the experiment and results that I had. VQGAN+CLIP can interpret perspective when we add it in our prompt as much as it understood art styles.


Replicate and Progress


As I mentioned earlier, I was trying to replicate the result to my previous blog. Unfortunately, I can't achieve replication, and I think it is just a one-time occurrence or coincidence. But I will look into it again with the same initial noise (a seed with a positive value and the same every run). For this experiment, I want to closely monitor how VQGAN+CLIP creates art and observe any resemblance or connection that I can perceive if the notebook runs continuously without disconnection.

The experiment that I did has set control parameters. I configured seed to -1 and no initial image. I ran the same prompt about a seed that grows in an ancient cup continuously for almost 3 hours, with each round need 30 minutes, 500 iterations before starting the next. The notebook runs without disconnecting. A new image shows up every after 50 iterations, and I interrupted the iteration after reaching 500.

ex500_out_out.png
ex0.pngex50.pngex100.pngex150.pngex200.png
ex250.pngex300.pngex350.pngex400.pngex450.png

At the initial run, I find the illustration out of context as to the prompt. The seed was not in the cup, despite its growth. I think the output illustration is just one of those experiments I did that were seemingly a collage of images or out of context. Although I was not satisfied with the first run, I did observe events that transpired. I found out that objects or elements in our illustration concentrated on the darker tone in the zeroth iterations. It was all true in all runs, as shown.

ex1_500_out_out.png
ex1_0.pngex1_50.pngex1_100.pngex1_150.pngex1_200.png
ex1_250.pngex1_300.pngex1_350.pngex1_400.pngex1_450.png

The second run was a success. It depicted a seed growing in the cup, but I still find the illustration lacking in artistic elements and seems unfinished. The iterated images change from the same starting tone and converge to slightly related color tones. What I find interesting is how the image progresses from the zeroth to the 500th iteration. The illustration above seemingly grows leaves as it reaches a higher iteration number. I think the higher the iteration can make our AI art better. It also became cohesive to the context of our prompt unless some weird things happened to our AI model.

ex2_500_out_out.png
ex2_0.pngex2_50.pngex2_100.pngex2_150.pngex2_200.png
ex2_250.pngex2_300.pngex2_350.pngex2_400.pngex2_450.png

I don't think the third run was a success. It may have elements, but it is too abstract. The seed is outside the cup. But I am not sure what I perceived in the illustration. I presume the foreground is a seed while the far back is the cup, yet leaves growing. It is one of the results where we can't figure if we reach the prompt we envision, or it is just a close patch of images to our prompt.

Again, the AI art convergences to a better form as it reaches higher iterations. I think it is not evident in the third run. At 500 iterations, the output is blurring, and some parts are gone. I think the reason behind this is the third run needs more iteration to converge a better and cohesive illustration. I guess some artwork that takes more time than others. It is like how we create traditionally. Some need years to create masterpieces, while others can do it in no time. Maybe, it also concerns with how extravagant the output illustration is that it needs more time.

ex3_500_out_out.png
ex3_0.pngex3_50.pngex3_100.pngex3_150.pngex3_200.png
ex3_250.pngex3_300.pngex3_350.pngex3_400.pngex3_450.png
ex3_500.pngex3_550.pngex3_600.pngex3_650.pngex3_700.png

I think the fourth run was the best among these runs. It is somehow close to what I envision and has the elements of the prompt. When I observed the images in the iterations, I was amazed at how the cup formed as if it was from seed and slowly got the leaves and roots. The color profiles follow how the other run builds the tones.

For replicating the artwork from the same prompt, no run has close similarities with the other. I find it peculiar that each run evolves similarly. From seed and a cup, the leaves slowly grow as AI iterates the result. Figuratively, the progress tells a story of how the seeds in the cup evolve as the illustration evolves.

To sum up, what I observed, I find that AI art converges at a different time (iteration length). It seems similar to how we create art traditionally. There is artwork that we paint or sketch on the go. Some need time and even reach years to complete. Maybe, AI takes a long time or iteration when they produce masterpieces. Aside from that, I was amazed and delighted by the progress in each iteration. The iterated images tell stories by themselves. Also, I saw more elements painted by AI in the darker tones at the zeroth iteration image.

Author's Note:
All content including digital illustrations is from the author. The illustrations are generated by VQGAN+CLIP (Notebook 1, Notebook 2 and Nightcafe). For enhancing the resolution, color, and tone, the author used Real-ESGAN and Photoshop.


0
0
0.000
11 comments
avatar

How to make this po?

0
0
0.000
avatar

I'll drop the guide in the DM. Try out the VQGAN+CLIP MSE. It is better.

0
0
0.000
avatar

Awesome analysis as always!

Nothing to discuss about, as it matches with my own studies, but I'd like to add something else. Although is true that the more iterations, the more detail, there are another factors to take in consideration. One, is which notebook is being used, and the another one which references is taking the AI for that particular prompt and settings.

Most of the notebooks are based in the Original one, and the AI is an endless rush to still transforming pixels and shapes, so after 300-500 iterations rarely gonna get more detail, as the AI is busy transforming. The MSE one has some weights that decrease the "hunger" of transform of the AI, so the after 350 iterations the image get almost stabilized in terms of hardcore transforming and goes forward getting some more detail.

The second factor... there is nothing to do about, as we cannot control which references got the AI. So the AI gonna reach as max detail, the same amount of detail that comes from the references it got for the given prompt. So after the artwork reached it, let it run more iterations not gonna add any extra detail.

Regards and take this !PIZZA!

0
0
0.000
avatar

I agree with your statement. It is not necessarily more iteration better output or more details. As the AI converges, like most AI applications, it will stabilized at a given output. I do agree that what notebook we are using definately a factor of how well it converge. Despite we add more iteration, no additional feature will be added. As I tried and compare the different notebooks, I did found out Mse regulation is better in preserving transformation by not rushing in.

0
0
0.000
avatar

At end I let the stuff run endlessly and stop at the point I like or no more detail is added. Because sometimes, for whatever references it picked, the increase in the detail continues until a lot of iterations... once 1650 seemed to be the max. With the MSE notebook of course, the original based ones just flickers and flickers.

Regards!

0
0
0.000
avatar

This definitely looks like a fun run and i like them all... the successes and the ones you feel to be less successful. They are all art even if it doesn't result with your desire as the target. As frustrating as that is... it's pretty true to life. I rarely make art where i feel like what i ended up with is Exactly what i wanted. So, it shouldn't be much different until we can optimize and get more consistency out of the A.I. Keep up the great work. I think a seed in a cup was a very interesting prompt.

0
0
0.000
avatar

As frustrating as that is... it's pretty true to life.

Indeed. We will always have outputs that we literally don't like or doesn't fit to how we envision it, whether AI or traditional means. Although we don't like how some artwork turn out, we can still label it as art.

So, it shouldn't be much different until we can optimize and get more consistency out of the A.I.

Yes, it shouldn't be much different. When we do it consistently, we can have better result. Thanks for appreciating my work. Enjoy a slice of !PIZZA.

0
0
0.000
avatar

Thank you for the Pizza and your response. I mostly just didn't want you to be too hard on yourself (or the A.I.) hahaha. I think the key to success is having fun along the way. I'm glad we are all having fun!
Cheers

0
0
0.000