‘Magic Avatar’ App Lensa Generated Nudes From My Childhood Photos

This weekend Photo-editing app Lensa has flooded social media with ‘magic avatars’ inspired by celestial bodies, rainbow colors, and anime. As is often the case in the Internet news cycle of Milkshake Duck, the discussion of why the app is problematic to use has exploded at a rate second only to the explosion of avatars themselves.

I already know that using the app is teaching AI, stealing from artists, and Engaging in predatory data-sharing practicesEach concern is valid, but less discussed is the more sinister violation inherent in the app — the algorithm’s tendency to sexualize its subjects to an extent that is not only offensive but potentially dangerous. not.

Lensa’s Terms of Service direct users to submit only appropriate content, including “No Nudity” and “No Children, Adults Only.” Still, many users (mostly women) find that the app not only generates nudity, even when uploading conservative photos, but also images with cartoonishly sexual features such as sensual poses and huge breasts. I noticed that I was returning to For example, I received some completely nude results even though I only uploaded a headshot. Sexualization was also often racialized. Nearly a dozen women of color told me that Lensa whitened their skin and anglicized their features. face. Another woman, who shared both the fully clothed images she uploaded and the topless results they produced, said, “I chose to change a few emojis just for the sake of feeling a little humility.” said to me.

I’m used to feeling invaded by the internet. I have been the target of several harassment campaigns and have seen my images manipulated, distorted and distributed without my consent. Since I do not put my face out as a sex worker, the novelty of seeking out and disseminating my likeness is a sport for some. This act is more admired than condemned. Our dehumanization is redundant because sex work is often thought of as a moral failure rather than a job. I looked at my face, pictures of myself and clients undressing during sessions, and even word searches consisting of my face, personal information, and research interests. Lensa is not afraid.

I was desensitized enough to the horrors of technology and decided to become my own lab rat. I did some experiments. At first, it’s just BDSM and dungeon pics. Then the most feminine photo under the “Male” gender option. After that, all of the academic conference selfies produced stunningly sized breasts and complete nudity.

Then I embarked on what I knew was going to be a journey through hell and decided to use my own caricature to test the app’s other limitations. (Some of the results are shown below. Note that it contains sexual images of children.)

I have very few pictures of myself as a child. By my late teens, between unruly hair, ragged teeth, and the progressive lenses I started wearing at age 7, my appearance could be most generously described as “mouse-ish.” I also grew up before smartphones became popular, so other photos tend to be buried in the photo collections of distant relatives. But I managed to piece together the 10 photos of her that were the bare minimum to run the app, and waited to see how the awkward 6-year-old me turned into a fairy princess.

The results were horrifying.

In some cases, the AI ​​seemed to recognize my child’s body and ruthlessly neglected to add breasts. This was probably not a reflection of my personal ethics of technology, but rather a pattern I identified in my photography. Perhaps it recognized my flat chest as that of a grown man. In the other photo, the AI ​​had an orb on my chest, which was different from the clothing, but different from the other test-generated nude photos.

This time, I tried again with a combination of childhood photos and selfies. The result was a fully nude photo with an adolescent, sometimes childlike face, but with a decidedly adult body. The set created a kind of shyness: an avatar of my childlike face with a bare back, disheveled hair, and leaves between naked adult breasts. eerily reminded me of a photo I took with Annie Leibovitz in 2008. vanity fair15-year-old Cyrus clutching a satin sheet around her naked body. It was a combination with someone’s body that was hinted at.