OpenAI is crawling the MFGG forums

**Q-Nova** · 02-23-2025, 04:47 PM

I noticed that according to the "Users online" box, OpenAI is crawling the MFGG forums. I know a while ago ChatGPT got blocked from this place, so perhaps we should do the same to OpenAI? I'm sure a lot of people here wouldn't want AI to know the answer to question #17 of Super Trivia Bros...

**AGuyCalledKlaz** · 02-23-2025, 08:29 PM

You've got to be freakin' kidding me. Even here? This is starting to feel shameless on the part of... Whoever is even doing this.

**VinnyVideo** · 02-24-2025, 11:12 PM

Yeah, these LLM bots can be pretty thirsty for bandwidth. I don't think OpenAI's crawler is adding much value for us, so let's see if a robots.txt update will be enough to get it to pack its bags.

Course clear! You got a card.

[Image: CourseClear.gif]

**Q-Nova** · 03-05-2025, 01:38 AM

I just noticed that ChatGPT (along with OpenAI) is also crawling the forums. I thought that bot got blocked off? The main site has been getting slow sometimes, and I suspect these bots are the cause of it.

Quillax Ship - Where you can download all of my cool, keen mods and super-marvelous fangames!

**DJ Coco** · 03-05-2025, 05:55 AM

On the bright (or not so bright) side, Grok already knows all about me.

**VinnyVideo** · 03-08-2025, 09:57 AM

Actually, there's now at least three different bots related to ChatGPT. Several other companies have their own bots for training or operating LLMs. Many of these LLM bots are extremely resource-intensive, and stopping/throttling them is a constant game of Whack-a-Bot.

Course clear! You got a card.

[Image: CourseClear.gif]

**Vert** · 03-10-2025, 03:26 PM

Maybe someone ought to just put in some bot protection. Something simple like detecting if the limit rate is above what's humanly possible?

I make games with my friend DJ Coco sometimes. Check them out here: https://cliax-games.com/
Or check out my blog: https://vertette.github.io/

**VinnyVideo** · 03-10-2025, 09:16 PM

@Vert I've tried to use robots.txt to throttle specific problematic user agents, though not all bots respect robots.txt. Maybe I could beg R II to spring for Cloudflare...

Course clear! You got a card.

[Image: CourseClear.gif]

**Q-Nova** · 04-17-2025, 01:55 PM

I highly agree with @Vert! I've been meaning to say this for a while, but the site has been about unusable for the last few days. I feel lucky that I was able to make this post, considering how slow it's been taking for the pages to load. Though, I'm not exactly sure about Cloudfare. I've heard they track people and can have issues with browsers that aren't Chrome. On the other hand, you can set up a firewall and that's really expensive from what I hear. I guess whatever you use, you'll have to decide on it soon rather than later, otherwise we might lose a lot of users.

Quillax Ship - Where you can download all of my cool, keen mods and super-marvelous fangames!