-
datechnoman!a help
-
h2ibotdatechnoman: Not a transfer.archivete.am URL.
-
datechnoman
-
JAAdatechnoman: I don't think the bot supports decompressing.
-
JAA(But transfer does.)
-
datechnomanI submitted the link and was like oops....
-
datechnomanWas waiting for it to yell at me
-
datechnoman
-
JAAarkiver: Maybe throw an error on \.zst$ URLs?
-
datechnomanSorry all. Just end user testing :P
-
h2ibotdatechnoman: Skipped 14319277 bad URLs: transfer.archivete.am/ncIXz/goo-gl.…02-05-14-17-02.txt.zst.bad-urls.txt
-
JAAI ... think I'll skip throwing these into AB.
-
h2ibotdatechnoman: Fixed 11493254 unprintable URLs: transfer.archivete.am/nHCvu/goo-gl.…-14-17-02.txt.zst.not-printable.txt
-
h2ibotdatechnoman: Deduplicating and queuing 0 items.
-
h2ibotdatechnoman: Deduplicated and queued 0 items.
-
datechnomanJAA This is data that was taken from our WARC's on Archive.org so no point keeping them as we will be double copying them
-
JAAdatechnoman: 'Was taken from' is vague enough that I like to still keep the actual list. :-)
-
JAAOr is it from the URLTeam files?
-
datechnomanYup straight from URLTeam files
-
datechnomanI grab the warc, process it (grab google links for example) compress and tell #// to queue basically
-
JAAThere are no WARCs on URLTeam though.
-
datechnomanzip files sorry
-
datechnomanIm out of practice. Been a month >.<
-
JAARight, yeah, then I guess it's unnecessary.
-
datechnoman
-
JAAThe previous one is still ingesting.
-
JAAThose messages were for the .zst.
-
h2ibotdatechnoman: Skipped 26220 bad URLs: transfer.archivete.am/13mVPF/goo-gl…023-02-07-20-17-02.txt.bad-urls.txt
-
datechnomanMagic thanks for that. Was downloading it and trying to figure out why it failed. Makes sense
-
h2ibotdatechnoman: Deduplicating and queuing 8676692 items.
-
h2ibotdatechnoman: Deduplicated and queued 8676692 items.
-
arkiversomething is going on here
-
nstrom_I am seeing a bunch of Server returned 0 (HEOF).
-
nstrom_but maybe it's my VM. the URLs seem to load from home
-
arkiverwell we have another annoying loop... blegh
-
arkiverpausing for a few hours until I can fix this
-
datechnomanStupid loops! >:(