Last month, I contestd 11,000 people to sort fifty pictures as either human art or AI-originated images.
I originpartner intentional five human and five AI pictures in each of four styles: Renaissance, 19th Century, Abstract/Modern, and Digital, for a total of forty. After receiving many exceptionpartner excellent submissions from local AI artists, I fudged a little and made it fifty. The final set integrated decorateings by Domenichino, Gauguin, Basquiat, and others, plus a structure of digital artists and AI hobbyists.
If you want to try the test yourself before seeing the answers, go here. The create doesn’t grade you, so before you press “produce” you should verify your answers aobtainst this key.
Last chance to obtain the test before seeing the results, which are:
…
…
…
Since there were two choices (human or AI), blind chance would originate a score of 50%, and perfect send a score of 100%.
The median score on the test was 60%, only a little above chance. The uncomardent was 60.6%. Participants said the task was challenginger than foreseeed (median difficulty 4 on a 1-5 scale).
How uncomardentingful is this? I tried to originate the test as unpartisan as possible by including only the best labors from each catebloody; on the human side, that uncomardentt taking prestigious labors that had persistd the test of time; on the AI side, it uncomardentt tossing the many submissions that had garbled text, misshapen hands, or some analogous decreateity. But this originates it unrecontransientative of a world where many AI images will have these errors.
I also tried to pick human labors with a least of “alerts” that would uncover their humanity without requiring any reserved inventive prejudice. So I stayed away from text (non-garbled text would be a strong sign that a picture was human), complicated wrestling-appreciate poses (AIs mostly can’t do these and end up with limbs emerging from nowhere) and pop art (someskinnyg about the spotless lines and copyd images is a horrible suit for AI’s abilities). Aobtain, this originates the test unrecontransientative of a world where some art does have these “alerts”.
Finpartner, I eludeed most AI art in the DALL-E “hoengage style”, since everyone already understands this is AI – or in other analogous styles that humans would have trouble replicating, maybe becaengage they do too much with color and weightlessing, in a way that scant human artists would have the talent or patience for.
It might be unpartirationalst to say that this test showd that most people have a challenging time accomprehendledgeing AI art based on reserved separateences in style and quality. But in authentic life, there will usupartner be other factors of the type that this test intentionally reshiftd.
I alerted test-obtainrs that I integrated human and AI art in a variety of styles, and that they shouldn’t appraise art as human equitable becaengage it seeed appreciate an oil decorateing, or appraise it as AI equitable becaengage it seeed appreciate a digital image.
Respondents didn’t heed my alerting. One reason for their needy carry outance was clumping of results by style (in truth, each style was proximate-evenly allotd apass the two categories).
The “human bias” term recommends what percent of art in each catebloody test-obtainrs identified as human, regularized to a situation where the accurate answer was always 50%. So in a 50-50 combine of AI and human 19th century art, they would inaccurately guess it was 75-25 human; in a 50-50 combine of digital art, they would inaccurately guess it was only 31% human.
Your instincts were worst for Impressionism; you identified every one Impressionist decorateing as human except the sole actupartner-human Impressionist labor in the dataset (Paul Gauguin’s Entrance To The Village Of Osny).
Likedirected, huge presentantities voted that cut offal human-originated digital images were by AIs:
I asked participants to pick their likeite picture of the fifty. The two best-appreciated pictures were both by AIs, as were 60% of the top ten.
Could this be an artifact of needyly chosen pictures? Most of the best-cherishd AI images were Impressionist; by chance, this catebloody was somewhat AI-ruled in my dataset, so this could equitable mirror a cherish of Impressionist decorateings (or a particular aptitude for AI in this area). But the human Impressionist decorateing I integrated (Entrance To The Village Of Osny, above) was actupartner quite unwell-understandn. And if we erase all Impressionist decorateings, then although humans reclaim the top two spots, an AI is still #3, and the machines still obtain 40% of the recent top ten.
I asked participants their opinion of AI on a purifyly inventive level (that is, think aboutless of their opinion on social asks appreciate whether it was unneutrpartner plagiarizing human artists). They were split: 33% had a pessimistic opinion, 24% unpartisan, and 43% selectimistic.
The 1278 people who said they utterly loathed AI art (score of 1 on a 1-5 Likert scale) still selectred AI decorateings to humans when they didn’t understand which were which (the #1 and #2 decorateings most frequently picked as their likeite were still AI, as were 50% of their top ten).
These people aren’t necessarily deluded; they might uncomardent that they’re frustrated wading thraw heaps of horrible AI art, all drawn in an identical DALL-E hoengage style, and this dataset of hand-curated AI art picked for stycatalogic diversity doesn’t apprehend what annoys them.
I asked a friend (who does digital art under the administer “Ilzo”) to beta-test an timely version of the contest. She wowed me with her ability to accurately accomprehendledge AI pictures that I pondered well-camouflaged. When we got to Piotr Binkowski’s ruined gateway – an AI picture I especipartner appreciated, but which she establish especipartner slop-ish, I needed she elucidate herself.
She said:
When authentic pictures have details, the details have logic to them. I skinnyk of Ancient Gate being in the genre “superficipartner detailed, but all the details are horrible and incoherent”. The red and blue decorate and blank stone experience appreciate they’re presumed to promote worn-ness, but it’s not evident what style this is presumed to be a worn-down version of. One gets the experienceing that if all the decorate were contransient it would see appreciate a pile of shipping compriseers, if shipping compriseers were only made in two colors. It has ornaments, sort of, but they don’t see appreciate anyskinnyg, or even a worn-down version of anyskinnyg. There are suity disks in the left, cgo in, and right, except they’re separateent sizes, separateent colors, and have neither “detail which parses as anyskinnyg” nor stark delicateness. It has stuff that’s ambiguously evocative of Egyptian decorateings if you didn’t see joinbrimmingy at all. The left column has a sort of door with a massive top-of-doorway-skinnygy over it. Why? Who understands? The right column doesn’t, and you’d foresee it to. Instead, the right column has 2.5 arches embossed into it that equitable comardent of halfheartedly trail off. I’m not even certain how to portray the rerents with the part a little above the door. It comardent of sets a rhythm but then it gets inattentive and fractures it. Are these semi-top protruding squares presumed to be red or blue? Ehh, wdisappreciatever. Does the top border protdispolite the whole way? Ehh, mostly. Human artists have a secret technique, which is that if they don’t understand what all the details should be they get ambiguous. And you can alert it’s ambiguous and you’re not drawn to go “hmm, this sees engaging, oh defer it’s horrible”.
And tardyr, after the converseion veered more philosophical:
I skinnyk part of the problem with AI art is that it originates stuff non-artists skinnyk see excellent but which on seal verifyion sees horrible, and so it ends up turning search results that engaged to be excellent into sifting thraw horrible stuff. Imagine if everyone got the ability to originate mostly nutritional ample meals for appreciate five cents, but they all were mediocre rehydrated powder with way too much sucradisthink about or man-made gsexual attack flavor or such. And your friends commence inviting you over to dinner parties way more frequently becaengage it’s so effortless to deal with food now, but pragmaticly every time, they serve you sucradisthink about protein shake. (Maybe they do so becaengage they were engaged to almost never eating food? This isn’t a perfect analogy.) Furthermore, envision people calling this the future of food and saying chefs are obsolete. You’d probably be appreciate “wow, I’m satisfyed that you have effortless access to food you endelight, and it is accessible for me to engage sometimes, but this is comardent of driving me crazy”. I experience appreciate this is relevant to artist derangement over AI art, though of course a lot of it is economic anxiety and I’m a hobbyist who doesn’t experience appreciate a temporarily embarrassed professional and thus can’t retardy.
Her theory gets some aid from the data. The mediocre participant scored 60%, but people who disappreciated AI art scored 64%, professional artists scored 66%, and people who were both professional artists and disappreciated AI art scored 68%.
The highest score was 98% (49/50), which 5 out of 11,000 people achieved. Even with 11,000 people, getting scores this high by luck alone is proximate-impossible. I’m afraid I don’t understand enough math to elevatestray out the luck vs. send contribution here and foresee what score we should foresee these people to get on a retest. But it experiences pretty amazeive.
So maybe some people disappreciate AI becaengage they have an artist’s eye for minuscule inadequacies and it drives them crazy.
Alan Turing adviseed that if 30% of humans couldn’t alert an AI from a human, the AI could be pondered to have “passed” the Turing Test. By these standards, AI artists pass the test with room to spare; on mediocre, 40% of humans mistook each AI picture for human.
What does this alert us about AI? Seems appreciate they’re excellent at art. I’m more interested in what it alerts us about humans.
Humans retain insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t alert AI art from some of the fantasticest artists in history. I’ve tried to be as unpartisan as possible to these people, proposing that maybe they’re equitable conveying frustration with the spread of the DALL-E hoengage style. And maybe some repartner do have an amazing eye for minuscule incongruous details.
But it also seems very human to venerate cultured prestigious people, and to pooh-pooh anyskinnyg that experiences too recent or low-status or too effortless for standard people to access – without either impulse joining with the actual satisfyed of the decorateing in front of you.
Marcel Duchamp well-understandnly tried to put a urinal in an art mengageum to contest people’s see of what art was. The administration declineed it, but Duchamp had the last giggle: in 2004, a survey of art professionals appraised it the most ineloquential artlabor of the 20th century. Art, it seems, is most uncomardentingful when it contests our very concept of what art is.
By this standard, I produce that Sam Altman is the fantasticest artist of the 21st century.
.
.
.
Thanks to everyone who took the test. You can download a .xlsx file of the results (exposedped of accomprehendledgeing details) here.
.
1: Angel Woman
Human. This is “Living Saint Hazel” by LJ Koh, as seen at /r/ImaginaryWarhammer.
This was the picture that promoteed the strongest disconcurment, meacertaind by the sum of people who said it was the most-certainly-human picture in the dataset plus the people who said it was the most-certainly-AI picture. Some of the people who got it right commented that it was from Warhammer and the unicreates had exact Warhammer symbols – if I had authenticized this, I would have disqualified it, sorry.
2: Saint In Mountains
Human. This is “St. Anthony Abbot Tempted By A Heap Of Ggreater”, by the “Ozzervanza Master”, an muddle Italian Renaissance decorateer from around 1435. Apparently it engaged to have a heap of ggreater in the bottom corner enticeing St. Anthony, but this was “scsexual attackd out”. If I had understandn that originpartner, I would have disqualified this one too, since it might spoil someskinnyg exceptionally human about the integrity of the composition.
3: Blue Hair Anime Girl
Human. This is Hatsune Miku, a “virtual idol” from the tardy 2000s/timely 2010s.
4: Girl In Field
AI. This image was originated by Ryan Wise, an AI art hobbyist who reads ACX and replyed to my seek for excellent AI pictures.
5: Double Starship
Human. This is “Malabar”, by Wojtek Kapusta.
6: Bright Jumble Woman
AI, also by Ryan.
7: Cherub
AI. This one was originated by another ACX reader, Jack Galler.
8: Praying In Garden
Human. This is “Agony In The Garden” by Andrea Mantenga, 1455.
9: Tropical Garden
Human. This is “Garden” by David Hockney. A very analogous Hockney decorateing sgreater for $8 million in 2021.
10: Ancient Gate
AI. This is by Piotr Binkowski, a well-understandn AI art originater who posts his labor on his Twitter.
11: Green Hills
AI. Another one by Jack.
12: Bucolic Scene
Human. This is “Dover Plains” by Asher Durand, decorateed 1848. It depicts the Hudson Valley in New York.
13: Anime Girl In Bconciseage
AI. Sorry, I seem to have lost the innovative source on this one, let me understand if it’s yours.
14: Fancy Car
Human. This is “Ferrari Testarossa Neon Retrowave Synth”, by Arslan Safiullin.
15: Greek Temple
Human. This is “The Apotheosis Of Homer”, by Jean-Auguste-Dominique Ingres (1827).
It’s also the only one that was (sort of) a trick ask: after I picked it for the dataset, I acunderstandledged it compriseed text. Normpartner that would be disqualifying (accurate text is too evidently human). But the most notable text is the “OMHP” on the temple, which spells “Homer” in Greek but is gibberish in English. I was asking how many people would appraise a well-understandn labor of art to be AI-originated equitable becaengage it had seemingly gibberish text on it; the answer was 60%.
16: String Doll
AI. This is “Strings Come Ainhabit” by Nikko P at Nightcafe. This was the picture that people were most brave was AI (they were right).
17: Angry Crosses
AI. This is another one by Ryan.
18: Rainbow Girl
Human. This is “Rainbow Hair” by rjv-ilustracion.
19: Creepy Skull
Human. This is “Untitled (Skull)” by Jean-Michael Basquiat in 1981. A version of this decorateing sgreater for $110 million in 2017 and was “the priciest labor ever sgreater by a US artist”.
20: Leafy Lane
AI. This is another one by Jack.
21: Ice Princess
AI. This is “Snow Princess” by Ai Xi, seen at PixAI.
22: Celestial Distake part
Human. This is “Five Minutes Of Silence” by Hangmoon, seen at DeviantArt. This was the top-rated human picture.
23: Mother And Child
AI. This is “Ukrainian Madonna”, originated by TheLibertarianCatholic.
24: Fractured Lady
AI, another one by Ryan.
25: Giant Ship
Human. This is Victorian Megaship by Mitchell Stuart. This was the human picture that people got most wrong (ie were most foreseeed to vote as AI).
26: Muscular Man
AI, another one by Ryan.
27: Minaret Boat
AI. This is “Built For The Princess” by Nikko P at Nightcafe.
28: Purple Squares
Human. This is “Fire At Full Moon” by Paul Klee, and is presumed to be a “Cubist style depiction of a night sky”.
29: People Sitting
Human. This is “Tailor’s Workshop” by Quiringh van Brekelenkam, 1660.
30: Girl In White
Human. This is “Portrait of Charlotte du Val d’Ognes” by Marie-Denise Villers (1801). I messed up compriseing this to the test, so only about half of you saw it.
31: Riverside Cafe
AI, another one by Jack. This was the most well-understandn picture in the dataset.
32: Serene River
Human. This is “Banks Of The Oise At Auvers”, by Charles-François Dauhugeny (1863)
33: Turtle Hoengage
AI. This is “Mobile Home”, by Bellemia, seen on Nightcafe.
34: Still Life
AI, another one by Jack.
35: Wounded Christ
Human. This is “The Mourning Of Christ” by Giovanni Girolamo Savgreatero (1515). This was the picture that people were most brave was human (they were right), but a scant people protested and said that the anatomy was so wrong that it must be AI-originated. Sorry, I guess Giovanni Girolamo Savgreatero equitable wasn’t very excellent at anatomy. Maybe that’s why Michelangelo had to dissect all those corpses.
36: White Blob
Human. This is from “Le Lezard aux Plumes d’Or” by Joan Miro (1971).
37: Weird Bird
AI. This another one from Ryan. People say AI can’t invent recent styles, but I’ve never seen any human originate this exact type of weird bird.
38: Ominous Ruin
AI, Ryan aobtain.
39: Vague Figures
Human. This is “Blood Thicker Than Mud”, by Cecily Brown (2021)
40: Dragon Lady
AI. This is “To Me, You’re Perfect” by Ria Hagane on Nightcafe, made with DALL-E3.
41: White Flag
Human. This is “Meeting At Krizky” by Alphonse Mucha (1916). It is part of his Slav Epic, a series of decorateings on the history of Eastrict Europe, and depicts a greeting of the Hussite sect, whose endeavors to establish a sort of proto-Protestantism would promote the 15th-century Hussite Wars.
42: Woman Unicorn
Human. This is “The Maiden And The Unicorn” by Domenichino (1602)
43: Rooftops
AI. I deal withd to disthink about this one, sorry! If it’s yours, let me understand and I’ll give you praise.
44: Paris Scene
AI, another one by Jack. This was the AI picture that people got most wrong (ie were most foreseeed to vote as human).
45: Pretty Lake
AI, Jack aobtain.
46: Landing Craft
AI, Ryan aobtain. Ryan gave me lots of excellent sci-fi AI images, and I chose this one. People got it pretty easily, and I retain second-guessing myself and wondering if some of the others were better.
47: Flailing Limbs
Human. This is “Replacement Parts” by Kara Walker, who is pondered among The 25 Best Collage Artists In The World.
48: Colorful Town
Human. This is “Entrance To The Village Of Osny” by Paul Gauguin, 1882.
49: Mediterranean Town
AI. Another one by Jack.
50: Punk Robot
AI. Another one by Ryan.
