← All videosTHE SHORT VERSION
More like this ↗

data analysis / fact check / ai

How I Fact-Checked Ilyas Abdulin With AI

Part 2 of my Ilyas Abdulin fact-check: exactly how I did it. I downloaded all 464 of his public videos and sent every one to the OpenAI Whisper API, about 20 hours of video. Then three independent models from three providers, OpenAI's GPT-6 Luna, DeepSeek V4.1 Flash and Claude Code, categorized every transcript using his own 7 categories. The final category is the majority vote: all three agreed on 374 videos, two of three on 80, and 10 three-way splits were reviewed. One model alone makes mistakes; GPT-6 Luna labeled his video on whether being gay is normal as a claim about women, and the other two outvoted it. All of the data is available to anyone who wants to audit it.

Transcript

0:00This guy never speaks up against misogyny He only defends men and never defends women it's also not true that

0:05I never defend women I do and at least in this example claims about women is the second biggest category on my channel

0:10A lot of people were wondering exactly how I was able to fact check this creator using

0:14AI This video will be a detailed explanation on exactly how I did that

0:19So just to recap in my last video I debunked Ilyas claim that his second most popular types of videos that he posts are videos defending women

0:28I showed that he did not use a random sample and in actuality videos defending women are his least posted category

0:35He most often posts content that is miscellaneous like dating and relationship advice and he's second most often posts videos defending men

0:43I also showed that videos defending men demand around half of his total views and likes

0:50So because of this it makes sense that people accuse him of only making content that defends men

0:56Here's how I did the analysis Step 1 I downloaded every single one of his videos from

1:02TikTok I have an Hermes agent and custom code that can work a browser

1:08It's logged into TikTok and I can use it to download every single one of his videos

1:13Step 2 I don't want to work with the raw videos directly

1:17While I can using models like Gemini 4 Argon Gemini 4 Hydrogen

1:22Gemini 4 Carbon and even other non Gemini models like Muse Spark

1:27I still didn't really want to work with videos It's just a little bit expensive

1:31So instead I took all the videos and I fed it to the

1:34OpenAI Whisper API to get a transcription of every single one of his videos

1:40Then from these transcriptions we can categorize it using the exact same categories that he mentions in his original video

1:47Now we could do it naively We could just use one single language model have the categories and get the response and say this was the category of

1:55videos But language models oftentimes make mistakes Oftentimes language models are wrong

2:01We don't want to rely on just one non deterministic faulty language model

2:05So instead of using one language model here's what I did instead

2:09I used three different independent language models from three different providers OpenAI's

2:14GPT-6 Luna DeepSeek V4.1 Flash and I use Claude Code Each of these models independently categorize the videos using the same labels that

2:24Ilyas mentions and then we do an ensemble We basically say we take the majority vote of all of the categories and that's the final category

2:34And we look at some examples that we see how this actually makes sense

2:39One language model makes mistakes sometimes but it's very very rare that three will make

2:45categorization mistakes like this Particularly these types of models they're considered fairly good for this simple task

2:52And again if someone wants to correct me or audit the results

2:55I have it all on my computer I'm happy to share all of my data because

3:00I have nothing to hide I have no ulterior motive My only motive was to correct the facts

3:05And the fact states this Ilyas does not spend most of his time making videos that defend women

3:12Let me know if there's any other content creator that needs to be fact checked

3:15I'll send my agents to them as well

Join the conversation

Loading conversation…