Signs You’re Following A Fake Twitter Account…

The challenge of dealing with fake accounts that spread disinformation is probably one of the greatest challenges facing the large social media companies.

Most methods used to detect fake Twitter accounts (like the excellent Botometer) assume that they are working at scale and are automated, and for large bot operations this is almost always the case. But what about fake accounts that that have a real person behind them? Not every fake account is a bot, but there are still plenty of accounts that purport to be real people in order to add credibility to their message.

Earlier this week The Third Man asked about a Twitter account that he was slightly suspicious of. The account purports to be that of Max Steinberg, a Mancunian (i.e. from Manchester, UK) lawyer who now lives in Brooklyn, New York. He’s passionate about left-wing political causes and spends an awful lot of time amplifying content from his favourite political accounts – but is he even a real person? In the rest of the post I’ll look at a few indicators that suggest he probably isn’t.

Profile Picture – This Person Does Not Exist

This Person Does Not Exist is a website that uses AI to generate random but realistic looking faces. It’s a great tool and has become a popular way of generating fake profiles for sock puppet accounts, but it is not without its limitations. There are a number common flaws and features in TPDNE-generated images that means it’s possible to spot them. I’m fairly confident that “Max Steinberg” has a few of these flaws too, so let’s see if we can prove this is the case.

Eye and Mouth Alignment

A common feature of TPDNE images is that the eyes and mouth of the person are always in exactly the same place in the picture. The eyes are always the same distance apart and centred in the same place. The mouth is always about one quarter of the way up from the bottom of the image and is also always centred. This occurs regardless of the angle of their head and can sometimes make for quite unusual looking faces. Let’s see if Max’s face follows the same pattern:

Interesting! By itself this may not be conclusive, but there are some other features that might indicate that Max’s origins are in an AI program, not Manchester.

Ear Asymmetry

We’re all unique and special snowflakes, and none of us have perfectly aligned facial features – but there’s still something odd about Max’s ears. In the photo he is staring at us directly but his ears are not even remotely symmetrical:

From this angle Max’s ears should look roughly the same size and shape but his right ear is a very different shape and size to his left one. Perhaps I’m being too harsh, and this is just a genetic trait – but if it is, then ear asymmetry is a genetic trait that a lot of the TPDNE family all have in common:

Hmmm. On the balance of probabilities I’m going to say that Max’s ears look like that because he’s AI generated. (I’m sure he will correct me if I’m wrong.)

Unusual Eyes And Teeth

There’s also something odd about his eyes. Look at his right pupil:

His left pupil looks normal, but the pupil in his right eye is nearly twice as wide as his left one. Another example of genetic bad luck? It seems unlikely. TPDNE has improved over times and it now renders eyes and mouths in a much more realistic way, but sometimes there are still tiny flaws that betray the origins of the image.

You can also see a similar problem with his teeth:

His two front teeth (red box) are different shapes and sizes. The teeth on the right side of his mouth have not been rendered properly at all. An analysis of Max’s facial features suggests he has more in common with an AI-generated fake than with a real person.

There are other giveaways too. TPDNE only creates a single image of a person, so if the person truly does not exist, we should never be able to find any image of them other than the fake one where they are staring directly at the camera. Sure enough, there are no other photos of Max in his Twitter account, no reverse image hits for his face, and no matches when searching for images of lawyers called Max Steinberg.

Twitter Habits

Max’s profile picture is likely fake, and it isn’t the only thing that suggests his account isn’t quite what it claims to be. There’s a lot of detail in his bio that we can try and verify. Let’s start with the location:

Although originally from Manchester, he claims to be based in Brooklyn, New York. If this is right then we should be able to see this reflected in Max’s Twitter activity. Foller.me offers some insights into a user’s tweeting habits that might tell us a little more about Max’s origins.

The first sign is the age of the account and the volume of activity. This is the account data as it appeared on the evening of 11th March:

The account less than a week old but has already posted 766 tweets – that’s a huge volume in such a short period of time. It’s also evident from the content that this nearly-new Twitter account is dedicated entirely to amplifying political messages:

A new account with a fake profile picture that shares huge amounts of political content in a short period? Hmmm.

There’s also something odd about the times that Max tweets at. This is the overview of his activity times as of  10th March 2020:

These times are UTC, so this shows that Max usually tweets at either end of the working day (with a little activity at lunchtime) – but these times only make sense if Max is based in Europe (which varies between UTC and UTC +2). If Max really was based in New York (which is at UTC -4) then his Twitter activity would peak between 4am and 6am, and again between 1pm and 3pm, which would be unusual for someone who lived there. This is not conclusive, but it does indicate that Max is still probably a lot closer to Manchester than he is to New York.

As I was writing this up I noticed that in the 24 hours since I started there had been a big spike in Max’s activity late at night:

This seemed a little odd but a quick overview of his timeline provided some explanation. The spike coincided with the close of polls for the 10th March Democratic primaries, which prompted a flurry of activity from Max.

No Backstory

I’m still not convinced on Max’s bio details. Fortunately he’s a lawyer, so he’ll be listed in one of the public registries for qualified lawyers in either the UK or in New York, right? In the UK, the Solictor’s Regulation Authority maintains a list of registered lawyers. Here’s the data they held on Max:

 

So maybe he’s registered in New York? We can verify whether he is or not because fortunately New York maintains a public database of registered attorneys too. They have not one but two Max Steinbergs registered – perhaps I’ve judged Max a little too harshly?

Nope. The register also tells us that both Max Steinbergs suffer from a serious medical condition that means they can no longer practise law, let alone run a Twitter account.

We’ve seen that Max is almost certainly not a real person. He has a fake profile picture, a fake back story, probably doesn’t live where he says he does, and seems to have been created solely for the purpose of creating and amplifying political Twitter content.

 

 

15 thoughts on “Signs You’re Following A Fake Twitter Account…”

  1. This is rather useless information for two primary reasons; 1) it provides a good case study for how to make “fake” accounts less detectable, and 2) it is biased in a way that will produce many false positives of “real” accounts that are not fake at all. Some people have sleep issues that keeps them up or wakes them up and has them getting on the internet instead of trying to address their sleep issues; others use VPNs combined with sleep issues that makes their usage patterns no fit into the above described acceptable pattern; and yet other uses may actually schedule their posts or intentionally use composite images to hide or even misdirect away from their true identity, while still being very much “real”.

    This is actually more of a rather good example of the failure of the naive and supremacist mindset of technocrats, which are not at all distant from the evil mindset of corporatists and communist/socialists thinking they are able to know and control and manage sufficient number of things to bring about a variant of a utopian system; in this example, keeping “fake” accounts away.

    1. It must be nice for you dan, to start out by calling someones else’s work “rather useless” and then have still have the confidence to accuse them of having a “naive and supremacist mindset” – what a strange an interesting world you must live in, maybe one where irony has yet to be discovered

  2. I stand behind nixintel. This post shows he’s put a lot of time and effort to write this up. His methods are thorough and intelligent. I didn’t know about foller.me, which looks very useful.

    It’s true that this info may result in better fake accounts, but that’s the nature of this good guy/bad guy dynamic.

    I believe that nixintel has done a good thing. Good on you!

  3. You haven’t really made a compelling argument that the account itself is fake; only that the purported identity of the account is fake.

    > If Max really was based in New York (which is at UTC -4) then his Twitter activity would peak between 4am and 6am, and again between 1pm and 3pm, which would be unusual for someone who lived there.

    This is a bizarre argument. Do I really need to explain to you that not everyone works 9-5 and that these times are not unusual for someone working in EST? The only thing that’s unusual is that you seem to not know anyone who works second shift.

    Your tracking of the times he tweets is bordering schizophrenic. You do realize that people who do shift work get assigned different shifts… right? Why was none of this considered in your analysis?

    As to the account behavior. Do an experiment. Follow 200 people. Start retweeting as many tweets as you can. You can easily retweet 100 tweets in 10 minutes. In seven days, you two can have “700 tweets” on your profile.

    So is the purported identity of the account fake? Yes. Is the person behind the account fake? Very unlikely, at least not with the analysis you’ve done. I can infer that someone has created a fake identity to talk about politics. Not unreasonable in the current political climate.

    Here’s an example: @ThirdMan__ just retweeted an arrest record of @CANCEL_SAM. Not endorsing any tweets from @CANCEL_SAM, but people posting other tweeters’ arrest records is enough of a deterrent to use a fake identity. So yeah – going on Twitter pseudonymously is reasonable.

    It’s also perfectly acceptable to “amplify” political messages that you agree with on Twitter (or rather people dedicate their use of Twitter to this). I’ve been retweeting and liking tweets I agree with for months. I mostly only use my account for politics. I don’t use my true identity. God forbid my odd work hours set off your alarms.

    If you have evidence that the account is a bot or apart of some network where the same person is controlling multiple accounts to amplify political messages, then I think you would have a more compelling argument. If you can show actual misinformation or some kind of coordination in the way the account retweets, then that might be compelling.

    All I’m saying is that you’re going to catch a ton of false positives with the methods you’ve use to analyze. Using a thispersondoesnotexist pseudoavatar with a pseudoidentity from fakenamegenerator.com is an actual strategy I developed along with fellow leftists to hide our identities. Why? Because there are bad actors in organizations like the Democratic Socialists of America who use various methods to reveal the true identities and wreak havoc in their real lives. This person your analyzing could very well be using this method.

    1. To clarify, I don’t think your analysis reveals that the person behind the account has nefarious intentions. It is rather someone trying to obfuscate their identity while expressing their political beliefs or honing their rhetoric.

      1. I never said anything about the account’s intentions, and neither did I argue it was part of a botnet. He might be a shift worker – but he claims to be a New York-based lawyer. It’s not impossible that he works odd hours, but the activity pattern better fits someone in Europe rather than NY, no?

        It’s a brand new fake account that seems to exist only to repost political content, not the account of a real person or a real lawyer – yes?

  4. Thanks for this article but I think disinformation goes way deeper than just knowing fake sources and real sources.

    I just showed a friend of mine this article and before she finished she asked “Why does it matter if the account is real or fake?” And then said she wasn’t interested.

    People want confirmation of their political beliefs, not the truth. That’s what these social media platforms exploit…

    Thanks for the in depth analysis though. I found it to be quite interesting and valuable.

  5. There was an OSINT tool discussed on one of the OSINT Curious webcasts which was I believe a Chrome extension that allows you to locate the original source of a tweet. I can’t find it for the life of me now. Any idea what that was?

Leave a Reply to BLCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.