-
OrIdow6So Aimix-Z shut down
-
OrIdow6I regret that I did not put the time into it to get the backeed-based general crawl solution working
-
OrIdow6But hopefully this may be able to serve as a basis for more generalizable approaches to similar sites in the future
-
arkiverOrIdow6: these things happen unfortunately
-
arkiverlets just move on to the next site! and keep working on the ones we're getting now
-
RyzYeah, I'm driven more and more by websites that I checked that are just rotted out :c
-
RyzDon't be discouraged OrIdow6
-
OrIdow6Thanks arkiver and Ryz
-
OrIdow6I'll at least try to get future ones going earlier, to try to prevent this from reoccurring
-
mgrandiWhat is the thing you were working on @OrIdow6
-
OrIdow6mgrandi: Aimix-Z
-
mgrandiSounds fancy, what is it
-
mgrandiOh that's the site you were talking about
-
mgrandiWhat did your crawl thingy do
-
OrIdow6AKA Aimix-BBS
-
OrIdow6The site blocked everyone after a few hundred requests
-
OrIdow6So I would have had each item be a single URL and crawled it like a distributed Archivebot
-
OrIdow6But because I was blocked, it was logistically difficult to develop and test