Can an AI illustrate the work of our NCCR researchers?
A picture is worth a thousand words… even if generated by artificial intelligence (AI)? This is what researchers from the National Centre of Competence in Research (NCCR) Evolving Language tried to find out with the project “My work pictured by AI”. The journey wasn’t as easy as it seems, since some words are too technical for the bot, while others are biased towards one definition. But, after many (many) prompt iterations, the results are out! You can find them here.
Illustration of the “My work pictured by AI” project done by the MidJourney bot, in the style of Pablo Picasso.
The NCCR Evolving Language asked the MidJourney bot – an AI capable of generating images based on a text – to illustrate the work of about 30 of its researchers. “We provide a prompt – in other words an instruction – to the bot, based on the title and abstract provided by the researcher as well as an artistic style, and then select the best results generated by the AI,” explains Celia Lazzarotto from the NCCR Communication Office, in charge of the project. The researchers had their say in the matter, and could choose the one they liked best within the propositions. The final results are available as of today on the NCCR’s website, each accompanied by a text describing the researcher’s project. Would you say the AI did a good job capturing what their work is?
The fine art of making your research accessible
Explaining one’s research project can be hard, especially when confronted to the lay public or a colleague from another field. With this project, the NCCR Evolving Language hopes to make its research known to a large community. What better way to show the diversity of this national consortium? “What our researchers do should be better understood by the lay public, and launching such ideas is an innovative and creative way of introducing science to the lay public,” says Emilie Wyss, head of communication of the NCCR Evolving Language. “Furthermore, the researchers had to provide a simple text presenting their researchers, which is a great exercise for them.”
Here’s a fun game for you to try: can you guess from only the picture what the research is about?
Mastering the art of prompting
“Though the Midjourney bot is user-friendly, it wasn’t always a piece of cake to make the AI create “art” that was both aesthetically pleasing and representative of the researcher’s work and needs,” Celia recounts. Indeed, the bot can have unexpected responses to some prompts.
First, some words, such as fetus are banned, so the title of certain papers had to be modified with a synonym. “It makes the illustration not exactly accurate to the work of the researcher, but there is no other way.” In this example, the word “fetus” had to be replaced with “embryo”, which doesn’t reflect the same stage of growth and therefore is not entirely accurate.
Another problem is that the AI can’t picture certain words – like bilingual, vocal learning, semantics or brain networks – very well, so they’re often missing from the creations. “There are few visual representations of these concepts, so it would also be difficult for humans to represent them. Here, we need to help the bot a little,” Celia reveals. To be represented, they have to be artificially added, by putting in the prompt words like international, countries…
On the other hand, some words are extremely biased and lead to pictures looking similar: having the term “relationship” in a prompt – even if it’s not referring to a couple – gives out 2 protagonists facing each other in most cases. It’s best to rephrase to avoid its use, if more diversity in the pictures is wanted. The phrase “in the style of Rene Magritte” is also very hard to control and almost always produces a picture of a person with a bird on their face, no matter what the rest of the prompt is. “Adding a picture of another less famous painting by the artist to the prompt, or first generating the content of the image and then applying an artstyle are solutions I used to bypass this,” Celia shares.
Starting on a blank canvas
The “My work pictured by AI” project is adapted from a project from the Swiss Center for Affective Sciences (CISA). The idea started in July 2022 with Dr. Ben Meuleman, permanent statistician of the center with a background in psychology and statistics. “At first, it was a side project. A couple of people came to me to illustrate their thesis with the Midjourney bot”, he recalls. Then the project took on a larger scale under the input of Carole Varone, head of communication of the CISA. Ben highlights the impact of the person prompting on the final result: “We choose the key terms, specific style to follow, what looks good… In that sense, it’s not only what the A.I. understands from the work, but also the prompter.” He tried to minimize his influence on the pictures by keeping the title intact and rarely forcing elements to appear.
The art works will be displayed in various locations.
