Seeing Silicon | How I was scammed by an AI on LinkedIn

A few weeks ago, a message popped into my LinkedIn inbox. I read it, got interested in what it said, and clicked to read the attached document. The document didn’t open. I went back to the message, and this time, opened the profile of the person who had sent it to me. The first flag – the photo of the woman in the profile was too perfect. Attractive with clear skin, clean straight hair, bright eyes, and a kind smile.

FILE PHOTO: A keyboard is placed in front of a displayed LinkedIn logo in this illustration taken February 21, 2023. REUTERS/Dado Ruvic/Illustration/File Photo(REUTERS)

The woman’s designation was a generic CXO, but the profile description felt like it was generated with all the right keywords that were related to publishing and just didn’t make complete sense. As did her education chronology. As a fiction writer, I could see it as a character created by an AI, with confusing gaps in its synthetic past. It was then that I knew I had been scammed by an AI. The document in question, a virus in most probability, had been downloaded onto my iPhone.

I felt cheated and frustrated. Not because of the suspect virus, but because a random AI had read my LinkedIn profile, found my secret desires and triggers — things I didn’t even know I needed — and managed to decode me and make me act recklessly.

Within 20 minutes, the duration in which I decided to write about synthetic profiles on LinkedIn and opened my laptop to take screenshots, the profile, message and document had all been deleted from my account. The synthetic user that an AI had created, which knew my triggers so well, ceased to exist.

A representative set of matched real and synthetic faces. (Sophie J. Nightingale H. Farid)
A representative set of matched real and synthetic faces. (Sophie J. Nightingale H. Farid)

As early as 2022 when the pandemic raged, AI-generated fake profiles were becoming a problem for LinkedIn, the world’s largest social platform for professionals. These accounts used computer-generated profile photos which felt more real and more trusting than photos of real humans according to a 2021 study by researchers at Lancaster University and University of California, Berkeley. The researchers found that faces made by AI are “indistinguishable” from real faces. People have just a 50% chance of guessing them correctly. Look at the photo of synthetic faces (S) below that the study developed. Do you find them more real than the real human faces (R)?]

The most (Top and Upper Middle) and least (Bottom and Lower Middle) accurately classified real (R) and synthetic (S) faces. (Sophie J. Nightingale H. Farid)
The most (Top and Upper Middle) and least (Bottom and Lower Middle) accurately classified real (R) and synthetic (S) faces. (Sophie J. Nightingale H. Farid)

Multiple aims


In 2022, researchers at Stanford Internet Observatory uncovered more than 1,000 LinkedIn fake profiles using synthetic faces and written details created by AI. The aim of these profiles was several, according to the researchers. Virus was one. Then there was disinformation. But there was also marketing. Marketeers ramped up synthetic profiles on the platform to drive engagement, improve the SEO of articles, or drum up sales for their clients. It was a telemarketing strategy for the digital world, explained the study. If someone took the bait, they were quickly connected to a real salesperson within the client’s company.

Soon after, LinkedIn ramped up its fake profile-detecting filters and launched new features and systems to make sure its members interact with real humans. The company’s Anti-Abuse AI team placed systems to automate fake account detection, especially fake AI-generated profile images, which create signals. Last year, they announced a new AI image detector to catch fake profiles. “Our new deep-learning-based model proactively checks profile photo uploads to determine if the image is AI-generated using cutting-edge technology designed to detect subtle image artefacts associated with the AI-based synthetic image generation process without performing facial recognition or biometric analyses,” wrote Oscar Rodriguez, vice president, product management in the LinkedIn blog.

According to the company’s most recent transparency report, in the first half of 2023, the company removed more than 15.1 million fake accounts before members reported them and about 200,000 accounts after they were flagged by members in the first six months of 2023. A whopping 104 million instances of scam and spam content were detected and removed in the first half of 2023. Three months ago, LinkedIn even won a legal case against two marketing companies, TopSocial and SocialBD24, which had built their businesses by creating fake LinkedIn profiles to generate marketing leads.

This year, it’s gone further with real-life verifications. You can add a verification tick mark to your profile with a government-issued ID for 115+ countries (India’s yet to be included in the list), and verify your workplace with work email and educational institute via education email. “Verified profiles get 60 percent more profile views, 50 percent more comments and reactions on their posts, and 30 percent more messages,” explains Rodriguez on another blog, as if to urge users to sign up.

LinkedIn also launched a feature called ‘About This Profile’ that shows you when a profile was created and last updated along with the member’s verified phone and work email.

However, with improving AI capabilities, it’s become a cat-and-mouse game. While automated AI software keeps creating fake profiles, LinkedIn’s anti-abuse AI software keeps culling them out, with a few, like in my case, slipping through the cracks for a few hours. Enough for me to fall for it, get convinced and click.

Now cautious of adding people to my network, a wiser me has reset my LinkedIn password and has been closely observing my iPhone for tell-tale signs of covert software running in the background — battery discharge, overheating or apps behaving weirdly. They haven’t yet, but I might still reset my iPhone.

Shweta Taneja is an author and journalist based in the Bay Area. Her fortnightly column will reflect on how emerging tech and science are reshaping society in Silicon Valley and beyond. Find her online with @shwetawrites. The views expressed are personal.

Leave a Comment