OpenAI is crawling the MFGG forums

**Q-Nova** · 02-23-2025, 04:47 PM

I noticed that according to the "Users online" box, OpenAI is crawling the MFGG forums. I know a while ago ChatGPT got blocked from this place, so perhaps we should do the same to OpenAI? I'm sure a lot of people here wouldn't want AI to know the answer to question #17 of Super Trivia Bros...

**AGuyCalledKlaz** · 02-23-2025, 08:29 PM

You've got to be freakin' kidding me. Even here?Â This is starting to feel shameless on the part of... Whoever is even doing this.

**VinnyVideo** · 02-24-2025, 11:12 PM

Yeah, these LLM bots can be pretty thirsty for bandwidth. I don't think OpenAI's crawler is adding much value for us, so let's see if a robots.txt update will be enough to get it to pack its bags.

Course clear! You got a card.

[Image: CourseClear.gif]

**Q-Nova** · 03-05-2025, 01:38 AM

I just noticed that ChatGPT (along with OpenAI) is also crawling the forums. I thought that bot got blocked off? The main site has been getting slow sometimes, and I suspect these bots are the cause of it.

Quillax ShipÂ - Where you can download all of my cool, keen mods and super-marvelous fangames!

**DJ Coco** · 03-05-2025, 05:55 AM

On the bright (or not so bright) side, Grok already knows all about me.

**VinnyVideo** · 03-08-2025, 09:57 AM

Actually, there's now at least three different bots related to ChatGPT. Several other companies have their own bots for training or operating LLMs. Many of these LLM bots are extremely resource-intensive, and stopping/throttling them is a constant game of Whack-a-Bot.

Course clear! You got a card.

[Image: CourseClear.gif]

**Vert** · 03-10-2025, 03:26 PM

Maybe someone ought to just put in some bot protection. Something simple like detecting if the limit rate is above what's humanly possible?

I make games with my friend DJ Coco sometimes. Check them out here: https://cliax-games.com/
Or check out my blog: https://vertette.github.io/

**VinnyVideo** · 03-10-2025, 09:16 PM

@Vert I've tried to use robots.txt to throttle specific problematic user agents, though not all bots respect robots.txt. Maybe I could beg R II to spring for Cloudflare...

Course clear! You got a card.

[Image: CourseClear.gif]