Forum Overview :: Site Comments
 
I made some adjustments that I think finally took. by Ice Cream Jonsey 09/09/2025, 10:48am PDT
We were getting hammered by all manner of "bots," in so much as I think they are web crawlers and bots that Amazon is apparently using for an AI tool. We'll see if it took and if so, I will turn the search function back on. There were other web crawlers that I also blocked. I updated robots.txt, which I don't really think Amazon respected. I've heard that some crawlers will pretend to be AmazonBot, so maybe that was it.

If it was 100% the real Amazon, I think it's really irresponsible of them to unleash their site-crushing content harvesters on a small site hosted by Dreamhost of all places without contact or compensation. I didn't wake up as a newborn hating corporations, but I have probably spent 2 hours on this over the last week trying to fix things. Bezos is out this week bragging to everyone about his jewel encrusted yacht that he could still afford after a divorce where he lost billions. Ole Pissjugs didn't even bother offering me free year of Amazon Prime in exchange for taking my time and destroying our site. If you guys were seeing the 503s, then they were coming back to their crawler. No adjustments made, if anything they started scanning the site more. What a bunch of pigs.

But again, maybe it's not genuinely them, I didn't bother doing a reverse lookup on the addresses identifying themselves as Amazon. There were several other web crawlers I hadn't heard of that were here every hour, ripping away.
PREVIOUS NEXT REPLY QUOTE
 
Caltrops server to Caltrops visitors: NT by STOP POSTING!!! 09/07/2025, 7:40am PDT NEW
    On it. Ticket submitted. by Ice Cream Jonsey 09/07/2025, 8:52am PDT NEW
        That's literally what my screen said when I tried to post. NT by Mischief Maker 09/07/2025, 9:19am PDT NEW
            I made some adjustments that I think finally took. by Ice Cream Jonsey 09/09/2025, 10:48am PDT NEW
                Caltrops is now being used to train AIs by Blackwater 09/09/2025, 11:27am PDT NEW
                    Grok is going from MechaHitler to ThailandSexTourist5000 NT by Mischief Maker 09/09/2025, 12:36pm PDT NEW
                Not totally solved - will keep working at it. NT by Ice Cream Jonsey 09/09/2025, 12:22pm PDT NEW
                    Four hours and no issues.......... fixed? Are your pets fixed? NT by Carl "Raper of Babies" Higgentotham 09/09/2025, 3:09pm PDT NEW
                        This is why I come here! by A FRIEND 09/09/2025, 4:34pm PDT NEW
    I've been getting this off and on for weeks by Rafiki 09/10/2025, 4:36pm PDT NEW
        Are you by chance going to http://www.caltrops.com without the s? NT by superadmin 09/10/2025, 8:23pm PDT NEW
            Huh by Rafiki 09/10/2025, 8:57pm PDT NEW
 
powered by pointy