-
datechnoman
-
h2ibotdatechnoman: Registering LfMSW0gm for '!a transfer.archivete.am/bgybv/urls_batch_0005_002.txt'
-
h2ibotdatechnoman: Skipped 808 invalid URLs: transfer.archivete.am/zdtY4/urls_batch_0005_002.txt.bad-urls.txt (LfMSW0gm)
-
h2ibotdatechnoman: Skipped 6 very long URLs: transfer.archivete.am/5VLxh/urls_batch_0005_002.txt.skipped.txt (LfMSW0gm)
-
h2ibotdatechnoman: Deduplicating and queuing 2359959 items. (LfMSW0gm)
-
h2ibotdatechnoman: Deduplicated and queued 2359959 items. (LfMSW0gm)
-
imer
-
datechnoman
-
h2ibotdatechnoman: Registering aSNW6v3h for '!a transfer.archivete.am/mP4z5/pdf_makenyanyunhappy_8.txt'
-
imerty
-
imerdatechnoman++
-
eggdrop[karma] 'datechnoman' now has 13 karma!
-
h2ibotdatechnoman: Deduplicating and queuing 9640943 items. (aSNW6v3h)
-
h2ibotdatechnoman: Deduplicated and queued 9640944 items. (aSNW6v3h)
-
datechnoman
-
h2ibotdatechnoman: Registering xVlUxrsp for '!a transfer.archivete.am/5ZuFD/pdf_makenyanyunhappy_9.txt'
-
datechnomanimer - I got you fam :)
-
datechnomanFeed the beast :P
-
datechnomanIts nearly a new month. I wonder if nyany is going to poke their head out soon haha
-
h2ibotdatechnoman: Deduplicating and queuing 9640943 items. (xVlUxrsp)
-
h2ibotdatechnoman: Deduplicated and queued 9640943 items. (xVlUxrsp)
-
datechnoman
-
h2ibotdatechnoman: Registering PBgANBoM for '!a transfer.archivete.am/Oo2j1/urls_batch_0000_000.txt'
-
VokunI have no clue how you guys go so fast here. Last i checked, last night, there were 15M in the queue, then another like hundred+ million were queued over today, and there's 27 million in the queue now
-
datechnomanWe dont muck around here ;)
-
that_lurkergotta reach that 8 PB somehow
-
h2ibotdatechnoman: Skipped 3780 invalid URLs: transfer.archivete.am/T1OdS/urls_batch_0000_000.txt.bad-urls.txt (PBgANBoM)
-
h2ibotdatechnoman: Skipped 50 very long URLs: transfer.archivete.am/OCrCT/urls_batch_0000_000.txt.skipped.txt (PBgANBoM)
-
h2ibotdatechnoman: Deduplicating and queuing 24996169 items. (PBgANBoM)
-
datechnomanthat_lurker - 8pb, that's child's play haha :P ;)
-
datechnomanWe aren't far off it though!
-
that_lurkerthat is true, but that is the next milestone :-)
-
h2ibotdatechnoman: Deduplicated and queued 24996173 items. (PBgANBoM)
-
datechnoman
-
h2ibotdatechnoman: Registering S1cu3CKI for '!a transfer.archivete.am/bHKEW/urls_batch_0000_001.txt'
-
h2ibotdatechnoman: Skipped 3925 invalid URLs: transfer.archivete.am/118EMK/urls_batch_0000_001.txt.bad-urls.txt (S1cu3CKI)
-
h2ibotdatechnoman: Skipped 102 very long URLs: transfer.archivete.am/9Qp49/urls_batch_0000_001.txt.skipped.txt (S1cu3CKI)
-
h2ibotdatechnoman: Deduplicating and queuing 24995973 items. (S1cu3CKI)
-
h2ibotdatechnoman: Deduplicated and queued 24995981 items. (S1cu3CKI)
-
datechnoman
-
h2ibotdatechnoman: Registering tEyEhj1K for '!a transfer.archivete.am/gIdoQ/urls_batch_0000_002.txt'
-
h2ibotdatechnoman: Skipped 2039 invalid URLs: transfer.archivete.am/MC9Nv/urls_batch_0000_002.txt.bad-urls.txt (tEyEhj1K)
-
h2ibotdatechnoman: Skipped 74 very long URLs: transfer.archivete.am/fWw2R/urls_batch_0000_002.txt.skipped.txt (tEyEhj1K)
-
h2ibotdatechnoman: Deduplicating and queuing 12055109 items. (tEyEhj1K)
-
h2ibotdatechnoman: Deduplicated and queued 12055111 items. (tEyEhj1K)
-
datechnoman
-
h2ibotdatechnoman: Registering zUPmiac7 for '!a transfer.archivete.am/8LZzm/urls_batch_0001_000.txt'
-
h2ibotdatechnoman: Skipped 3576 invalid URLs: transfer.archivete.am/JiDh6/urls_batch_0001_000.txt.bad-urls.txt (zUPmiac7)
-
h2ibotdatechnoman: Skipped 101 very long URLs: transfer.archivete.am/P66uR/urls_batch_0001_000.txt.skipped.txt (zUPmiac7)
-
h2ibotdatechnoman: Deduplicating and queuing 24996322 items. (zUPmiac7)
-
h2ibotdatechnoman: Deduplicated and queued 24996324 items. (zUPmiac7)
-
datechnomanWill only add one more batch in as im going to let the queue drain right back due to the start of the month sitemap queuing that will be coming in a day or two :O
-
datechnoman
-
h2ibotdatechnoman: Registering gkBbZHo8 for '!a transfer.archivete.am/yqQuS/urls_batch_0001_001.txt'
-
TheTechRoboI think I remember when we got up to 1PB here
-
TheTechRoboand I thought "damn, that's a LOT of data"
-
h2ibotdatechnoman: Skipped 3425 invalid URLs: transfer.archivete.am/L67Dt/urls_batch_0001_001.txt.bad-urls.txt (gkBbZHo8)
-
h2ibotdatechnoman: Skipped 104 very long URLs: transfer.archivete.am/TIWxT/urls_batch_0001_001.txt.skipped.txt (gkBbZHo8)
-
h2ibotdatechnoman: Deduplicating and queuing 24996471 items. (gkBbZHo8)