-
CookMePlox
Hey gang! Apologies if this is not quite the right place, but is there anything I can do to get the Internet Archive's scraper (specifically Zeno) to be a bit smarter/gentler with what it archives? It's been hitting my site at a peak of 200 QPS and causing performance issues. I wouldn't mind it so much, except the actual URLs it's hitting are on
-
CookMePlox
the very low end of what would be useful to archive
-
CookMePlox
I would prefer not to just user-agent block it because I am obviously a big fan of the service overall. Until I checked the IP block of the scraper I assumed it was just some badly-behaved random person or SEO scraper
-
yosh
has zippyshare shutting down been noted here yet?
-
pokechu22
CookMePlox: You might want to email info⊙ao I'm not super familiar with their scraper though
-
CookMePlox
Alright, thanks - I've had fairly limited success contacting that email in the past (about much happier things) but I spose I'll give it a try
-
JAA
CookMePlox: Someone else also asked about this a while ago. You'd need to contact IA about this. info⊙ao (And I'd love to hear what that is about, if they tell you anything about it, since it sounds weird for IA's normal MO.)
-
JAA
Ah yes, I got sniped. :-)
-
pokechu22
Looks like Zeno specifically is
github.com/CorentinB/Zeno
-
CookMePlox
Yeah, it's especially weird since the user-agent doesn't even identify it as associated with IA
-
CookMePlox
I tried searching around for `Zeno user agent`, etc and didn't even find that repo
-
pokechu22
github.com/CorentinB was the 8th google result for `zeno scraper archive.org` for me - definitely not easy to find
-
h2ibot
JustAnotherArchivist edited Deathwatch (+115, /* 2023 */ Add Zippyshare):
wiki.archiveteam.org/?diff=49569&oldid=49568
-
yosh
well, that now answers my questions :-)
-
JAA
Cc arkiver, this is a big one. ^
-
audrooku|m
Very big
-
CookMePlox
Thanks for your help! I'll let you know if I hear anything back about the scraper
-
arkiver
oof
-
arkiver
let's make a zippyshare channel. anyone have ideas?
-
voltagex|m
zippyshart
-
datechnoman
^^zippyshart^^ xD
-
datechnoman
I like it
-
arkiver
lol
-
arkiver
maybe
-
datechnoman
zipoff
-
datechnoman
zipshut
-
voltagex|m
unzippy
-
JAA
slippyshare
-
audrooku|m
zippyunfair
-
audrooku|m
Zippyshart is good tho
-
audrooku|m
voltagex:
-
audrooku|m
> Jesus, that's not enough time to archive ZippyShare
-
audrooku|m
Maybe it is, afaik we cant enumerate their files, so if we just grab the urls we can scrape it's probably doable
-
JAA
zippyscare
-
h2ibot
Usernam created Zippyshare.com (+1271, Created page with "{{Infobox project | title =…):
wiki.archiveteam.org/?title=Zippyshare.com
-
tech234a
zipdespair
-
voltagex|m
zippyeet
-
tech234a
ziprepair
-
JAA
It's more of a demolition than a repair.
-
voltagex|m
zipC4share
-
voltagex|m
#takingsemtextomillionsoffiles
-
voltagex|m
Ahem
-
evilhaven
Hi
-
myself
zipdespair
-
myself
zippysnare
-
DigitalDragon
rezippy
-
sepro
Something to note is that Zippyshare blocks access from Germany, Spain and the UK (maybe others as well).
-
sepro
Servers and Warriors from these locations could likely not be used.
-
arkiver
let's do #zippyshart
-
Krum110487
Hello All, is anyone working on a service to send zippyshare links to be archived?
-
Krum110487
I plan on searching for Zippyshare links myself.
-
myself
--> #zippyshart
-
Krum110487
haha, sorry, I didn't mean to ping you.
-
Krum110487
also, thanks! I didn't see that channel on the wiki
-
h2ibot
Switchnode edited Zippyshare.com (+19, add irc channel):
wiki.archiveteam.org/?diff=49571&oldid=49570
-
Ryz
-
Ryz
Gamur Group, which owns Destructoid, Siliconera, Twinfinite, and apparently a bunch of other gaming and media websites has laid of a bunch of people Z:
-
Ryz
Siiiiiigh, proactive time to archive their stuff and/or Twitter accounts