-
arkiverdatechnoman: on that question - "it's complicated"
-
arkiverthe queue is not ordered, so in the queue they'll be 'somewhere'
-
arkivernot beginning not end per se
-
arkiverand, if the queue if very large we offload batches of items to external storage, to be loaded in again when the queue becomes smaller
-
arkiverwhich items are chosen to be offloaded... happen by chance
-
datechnomanAll good. By the sounds its more of a bucket of items
-
datechnomanVery interesting
-
datechnomanThanks for sharing
-
arkiveryeah, exactly!
-
datechnomanSpinning down till those pesky spam domains are cleared out
-
arkiverdatechnoman: yeah
-
arkiver
-
arkiverdatechnoman: &
-
arkiver^*
-
h2ibotarkiver: Skipped 28945 bad URLs: transfer.archivete.am/v8m1u/goo-gl.…023-01-01-04-17-02.txt.bad-urls.txt
-
h2ibotarkiver: Deduplicating and queuing 9034161 items.
-
h2ibotarkiver: Deduplicated and queued 9034161 items.