11-06-2023, 09:13 PM
@KrystalPhantasm Yep - there's obviously a bunch of reasons why we might not want ChatGPT constantly crawling MFGG.
@OssieTheOstrich Not all bots/spiders are bad; in fact, the modern Internet would be unusable without some of them. For example, search engines like Google and Bing need to be able to crawl sites regularly so that content can show up in search results. In the case of ChatGPT, the GPTBot crawls various Web sites to build large language models, and I believe most MFGGers wouldn't be too thrilled about their posts being used to build commercial products without their knowledge, consent, or attribution.
I added GPTBot to the blocklist in robots.txt for both the current forums and the phpBB archive. If the GPTBot keeps hanging out, I can play hardball and block it in .htaccess as well.
@OssieTheOstrich Not all bots/spiders are bad; in fact, the modern Internet would be unusable without some of them. For example, search engines like Google and Bing need to be able to crawl sites regularly so that content can show up in search results. In the case of ChatGPT, the GPTBot crawls various Web sites to build large language models, and I believe most MFGGers wouldn't be too thrilled about their posts being used to build commercial products without their knowledge, consent, or attribution.
I added GPTBot to the blocklist in robots.txt for both the current forums and the phpBB archive. If the GPTBot keeps hanging out, I can play hardball and block it in .htaccess as well.
Course clear! You got a card.
![[Image: CourseClear.gif]](https://dl.dropbox.com/s/d5mcpm4nmt0gd14/CourseClear.gif)
![[-]](https://forums.mfgg.net/images/emerald/collapse.png)