Why is there a crawling bot called Majestic-12?

[13th_Wolf]

Has anyone else seen that at the bottom of the forum? Its a bot like Google [bot] but it says Majestic-12? Whenever I see it I keep freaking out lol.

Its been there for a while and this post is the first one now about it.

 

[Henu the Great]

Crawlers collect data and index it. Nothing to freak out about. The only way to restrict them from collecting data is to restrict access (private forums) which is not practical or desired by those above us as it has been stated many times that this platform has to stay open for the public to access.

 

[AFODO]

Damn it's in this thread too lol :lol: it's watching what you say about it so be careful.
I also wondered why we have these bots but I guess we have to have them like Henu said.
Also, These hundreds of guests, they can't be real.

 

[Desdemona_666]

Has anyone else seen that at the bottom of the forum? Its a bot like Google [bot] but it says Majestic-12? Whenever I see it I keep freaking out lol.

Its been there for a while and this post is the first one now about it.

Because this forum is a public website, and therefore it gets all sorts of traffic from humans and bots alike.

https://www.mj12bot.com/index.html

I believe it could be blocked in robots.txt (some bots don't obey that, but this one says it does), however forum admins probably don't think it's doing any harm so they didn't block it.

 

[Ol argedco luciftias]

All this does is index all pages on the website, and connect them with search engines. So if you do a search on a regular public search engine for a topic we have discussed, it will place our website in the results. And bring this information to many new people.