03:01:57 <datechnoman> !a help
03:01:58 <h2ibot> datechnoman: Not a transfer.archivete.am URL.
03:02:03 <datechnoman> !a https://transfer.archivete.am/qM02g/goo-gl.2023-02-05-14-17-02.txt.zst
03:05:38 <JAA> datechnoman: I don't think the bot supports decompressing.
03:06:02 <JAA> (But transfer does.)
03:06:12 <datechnoman> I submitted the link and was like oops....
03:06:20 <datechnoman> Was waiting for it to yell at me
03:06:51 <datechnoman> !a https://transfer.archivete.am/qM02g/goo-gl.2023-02-05-14-17-02.txt
03:06:52 <JAA> arkiver: Maybe throw an error on \.zst$ URLs?
03:07:06 <datechnoman> Sorry all. Just end user testing :P
03:10:28 <h2ibot> datechnoman: Skipped 14319277 bad URLs: https://transfer.archivete.am/ncIXz/goo-gl.2023-02-05-14-17-02.txt.zst.bad-urls.txt
03:11:08 <JAA> I ... think I'll skip throwing these into AB.
03:11:38 <h2ibot> datechnoman: Fixed 11493254 unprintable URLs: https://transfer.archivete.am/nHCvu/goo-gl.2023-02-05-14-17-02.txt.zst.not-printable.txt
03:11:39 <h2ibot> datechnoman: Deduplicating and queuing 0 items.
03:11:40 <h2ibot> datechnoman: Deduplicated and queued 0 items.
03:14:42 <datechnoman> JAA This is data that was taken from our WARC's on Archive.org so no point keeping them as we will be double copying them
03:15:28 <JAA> datechnoman: 'Was taken from' is vague enough that I like to still keep the actual list. :-)
03:15:34 <JAA> Or is it from the URLTeam files?
03:15:49 <datechnoman> Yup straight from URLTeam files
03:16:15 <datechnoman> I grab the warc, process it (grab google links for example) compress and tell #// to queue basically
03:16:30 <JAA> There are no WARCs on URLTeam though.
03:17:00 <datechnoman> zip files sorry
03:17:10 <datechnoman> Im out of practice. Been a month >.<
03:17:57 <JAA> Right, yeah, then I guess it's unnecessary.
03:21:32 <datechnoman> !a https://transfer.archivete.am/eYvIO/goo-gl.2023-02-07-20-17-02.txt
03:26:36 <JAA> The previous one is still ingesting.
03:26:44 <JAA> Those messages were for the .zst.
03:28:14 <h2ibot> datechnoman: Skipped 26220 bad URLs: https://transfer.archivete.am/13mVPF/goo-gl.2023-02-07-20-17-02.txt.bad-urls.txt
03:28:18 <datechnoman> Magic thanks for that. Was downloading it and trying to figure out why it failed. Makes sense
03:28:48 <h2ibot> datechnoman: Deduplicating and queuing 8676692 items.
03:31:44 <h2ibot> datechnoman: Deduplicated and queued 8676692 items.
17:40:41 <arkiver> something is going on here
18:04:04 <nstrom_> I am seeing a bunch of Server returned 0 (HEOF).
18:04:14 <nstrom_> but maybe it's my VM. the URLs seem to load from home
18:05:21 <arkiver> well we have another annoying loop... blegh
18:06:03 <arkiver> pausing for a few hours until I can fix this
21:08:38 <datechnoman> Stupid loops! >:(