01:13:25 !a https://transfer.archivete.am/iHBEG/edu_urls_batch_0004_000.txt 01:13:26 datechnoman: Registering VYYDxysX for '!a https://transfer.archivete.am/iHBEG/edu_urls_batch_0004_000.txt' 01:52:53 datechnoman: Skipped 7740 invalid URLs: https://transfer.archivete.am/1m9fb/edu_urls_batch_0004_000.txt.bad-urls.txt (VYYDxysX) 01:52:55 datechnoman: Skipped 75 very long URLs: https://transfer.archivete.am/kCT9i/edu_urls_batch_0004_000.txt.skipped.txt (VYYDxysX) 01:52:56 datechnoman: Deduplicating and queuing 24992184 items. (VYYDxysX) 02:20:34 datechnoman: Deduplicated and queued 24992185 items. (VYYDxysX) 02:45:15 !a https://transfer.archivete.am/fT2x0/edu_urls_batch_0004_001.txt 02:45:16 datechnoman: Registering Hmkr1c4D for '!a https://transfer.archivete.am/fT2x0/edu_urls_batch_0004_001.txt' 03:26:08 datechnoman: Skipped 7600 invalid URLs: https://transfer.archivete.am/144e6G/edu_urls_batch_0004_001.txt.bad-urls.txt (Hmkr1c4D) 03:26:11 datechnoman: Skipped 108 very long URLs: https://transfer.archivete.am/137s97/edu_urls_batch_0004_001.txt.skipped.txt (Hmkr1c4D) 03:26:12 datechnoman: Deduplicating and queuing 24992292 items. (Hmkr1c4D) 03:52:08 !a https://transfer.archivete.am/Lq29R/edu_urls_batch_0004_002.txt 03:52:09 datechnoman: Registering rm9HzLCw for '!a https://transfer.archivete.am/Lq29R/edu_urls_batch_0004_002.txt' 03:54:50 datechnoman: Skipped 382 invalid URLs: https://transfer.archivete.am/JWcjC/edu_urls_batch_0004_002.txt.bad-urls.txt (rm9HzLCw) 03:54:52 datechnoman: Skipped 9 very long URLs: https://transfer.archivete.am/7dO7a/edu_urls_batch_0004_002.txt.skipped.txt (rm9HzLCw) 03:54:53 datechnoman: Deduplicating and queuing 1150291 items. (rm9HzLCw) 03:55:32 datechnoman: Deduplicated and queued 24992296 items. (Hmkr1c4D) 03:56:16 datechnoman: Deduplicated and queued 1150291 items. (rm9HzLCw) 04:50:39 !a https://transfer.archivete.am/JbbTl/urls_batch_0000_000.txt 04:50:39 datechnoman: Registering ijPh0Pjl for '!a https://transfer.archivete.am/JbbTl/urls_batch_0000_000.txt' 05:33:10 datechnoman: Skipped 5400 invalid URLs: https://transfer.archivete.am/aZ1H6/urls_batch_0000_000.txt.bad-urls.txt (ijPh0Pjl) 05:33:13 datechnoman: Skipped 76 very long URLs: https://transfer.archivete.am/o3VZs/urls_batch_0000_000.txt.skipped.txt (ijPh0Pjl) 05:33:14 datechnoman: Deduplicating and queuing 24994523 items. (ijPh0Pjl) 05:39:11 !a https://transfer.archivete.am/16eyKE/urls_batch_0000_001.txt 05:39:11 datechnoman: Registering ywGrkPvo for '!a https://transfer.archivete.am/16eyKE/urls_batch_0000_001.txt' 06:12:22 datechnoman: Deduplicated and queued 24994782 items. (ijPh0Pjl) 06:28:39 datechnoman: Skipped 5737 invalid URLs: https://transfer.archivete.am/I3svX/urls_batch_0000_001.txt.bad-urls.txt (ywGrkPvo) 06:28:42 datechnoman: Skipped 158 very long URLs: https://transfer.archivete.am/RcUM/urls_batch_0000_001.txt.skipped.txt (ywGrkPvo) 06:28:43 datechnoman: Deduplicating and queuing 24994105 items. (ywGrkPvo) 06:55:17 datechnoman: Deduplicated and queued 24994394 items. (ywGrkPvo) 07:38:10 !a https://transfer.archivete.am/v3FqT/urls_batch_0000_000.txt 07:38:12 datechnoman: Registering s9cPq6OO for '!a https://transfer.archivete.am/v3FqT/urls_batch_0000_000.txt' 08:20:51 datechnoman: Skipped 4 invalid URLs: https://transfer.archivete.am/12l9bz/urls_batch_0000_000.txt.bad-urls.txt (s9cPq6OO) 08:20:52 datechnoman: Deduplicating and queuing 24999996 items. (s9cPq6OO) 08:41:34 !a https://transfer.archivete.am/gEZLq/urls_batch_0000_001.txt 08:41:34 datechnoman: Registering 4NpeBtys for '!a https://transfer.archivete.am/gEZLq/urls_batch_0000_001.txt' 08:46:05 datechnoman: Deduplicated and queued 24999996 items. (s9cPq6OO) 09:19:20 datechnoman: Skipped 1 invalid URLs: https://transfer.archivete.am/D1dZr/urls_batch_0000_001.txt.bad-urls.txt (4NpeBtys) 09:19:21 datechnoman: Deduplicating and queuing 24999999 items. (4NpeBtys) 09:41:31 datechnoman: Deduplicated and queued 24999999 items. (4NpeBtys) 09:47:58 !status 09:47:58 datechnoman: Jobs running: 0, jobs waiting for a slot: 0. 09:53:46 !a https://transfer.archivete.am/AeorY/urls_batch_0000_002.txt 09:53:46 datechnoman: Registering cIaRKKuC for '!a https://transfer.archivete.am/AeorY/urls_batch_0000_002.txt' 10:30:20 datechnoman: Deduplicating and queuing 25000000 items. (cIaRKKuC) 10:53:54 datechnoman: Deduplicated and queued 25000000 items. (cIaRKKuC) 11:19:06 datechnoman: did you see the pdf links i posted earlier? :) 12:23:39 !a https://transfer.archivete.am/68MoE/urls_batch_0000_003.txt 12:23:39 datechnoman: Registering s3taIZpa for '!a https://transfer.archivete.am/68MoE/urls_batch_0000_003.txt' 12:25:08 imer - Sorry mate, didn't see them. Tag me next time if you need. Ill throw those 2 in first thing tomorrow morning if thats ok? :) 12:25:18 sure thing 12:25:26 and no worries 12:27:28 I'm all over the place and posting things when i'm at work, home, with the kids etc so can easily miss a message or two 12:27:35 More than happy to get them in the queue 12:28:10 !a https://transfer.archivete.am/dG2Cz/pdf_CC-MAIN-2024-30.txt 12:28:13 datechnoman: Registering WqPjilSw for '!a https://transfer.archivete.am/dG2Cz/pdf_CC-MAIN-2024-30.txt' 12:28:33 ^^^ imer - will let that one queue and then throw the other one in tomorrow ^^^ 12:31:59 rewby rewby|backup - We "may" have an issue with some items not uploading correctly causing the drive on the target to fill? https://transfer.archivete.am/ckl3c/rsync_errors.txt 12:31:59 inline (for browser viewing): https://transfer.archivete.am/inline/ckl3c/rsync_errors.txt 12:33:34 thanks! 12:33:42 datechnoman++ 12:33:42 -eggdrop- [karma] 'datechnoman' now has 21 karma! 12:37:23 There appears to be no -1 or 500 connection throttling messages. Only seeing those rsync errors 12:47:42 datechnoman: Deduplicating and queuing 16408545 items. (s3taIZpa) 13:10:49 datechnoman: Deduplicated and queued 16408545 items. (s3taIZpa) 13:34:15 datechnoman: Deduplicating and queuing 25039453 items. (WqPjilSw) 13:45:16 I saw -1s earlier, I think we just overloaded things 13:51:10 getting -1s now from urls, youtube, and telegram so guessing disks are full or other general unhappiness on target 13:51:50 some things getting through sporadically though 13:58:43 datechnoman: Deduplicated and queued 25039453 items. (WqPjilSw) 18:12:44 datechnoman: Here are the edu urls and pdf urls I found from wikiteam if you could put them in at some point too. https://transfer.archivete.am/iXGwv/edu_urls.txt 18:12:44 inline (for browser viewing): https://transfer.archivete.am/inline/iXGwv/edu_urls.txt 18:12:44 https://transfer.archivete.am/9NCj8/pdf_urls.txt 21:04:29 Vokun - Cheers mate. Will throw those in too :) 21:38:58 !a https://transfer.archivete.am/iXGwv/edu_urls.txt 21:38:59 datechnoman: Registering Ry1oXLig for '!a https://transfer.archivete.am/iXGwv/edu_urls.txt' 21:39:16 ^^^ Vokun - Queuing your lists now ^^^ 21:41:53 datechnoman: Skipped 5145 invalid URLs: https://transfer.archivete.am/QvaJn/edu_urls.txt.bad-urls.txt (Ry1oXLig) 21:41:56 datechnoman: Skipped 39 very long URLs: https://transfer.archivete.am/zymTQ/edu_urls.txt.skipped.txt (Ry1oXLig) 21:41:57 datechnoman: Deduplicating and queuing 1733208 items. (Ry1oXLig) 21:43:38 datechnoman: Deduplicated and queued 1733209 items. (Ry1oXLig) 21:48:25 !a https://transfer.archivete.am/9NCj8/pdf_urls.txt 21:48:26 datechnoman: Registering 6HlvoQP7 for '!a https://transfer.archivete.am/9NCj8/pdf_urls.txt' 21:53:11 datechnoman: Skipped 1237 invalid URLs: https://transfer.archivete.am/5Jh90/pdf_urls.txt.bad-urls.txt (6HlvoQP7) 21:53:16 datechnoman: Skipped 2 very long URLs: https://transfer.archivete.am/BwHGk/pdf_urls.txt.skipped.txt (6HlvoQP7) 21:53:17 datechnoman: Deduplicating and queuing 2544292 items. (6HlvoQP7) 21:55:38 datechnoman: Deduplicated and queued 2544292 items. (6HlvoQP7) 22:13:49 ^^^ Vokun - All queued up ^^^ 22:14:12 !a https://transfer.archivete.am/wSXMu/pdf_CC-MAIN-2024-33.txt 22:14:13 datechnoman: Registering 8rtSGhC3 for '!a https://transfer.archivete.am/wSXMu/pdf_CC-MAIN-2024-33.txt' 22:53:07 datechnoman: Deduplicating and queuing 22486600 items. (8rtSGhC3) 23:16:22 datechnoman: Deduplicated and queued 22486600 items. (8rtSGhC3) 23:16:42 ^^^ imer - both your lists have been queued ^^^ 23:16:53 Now we just have to wait for the target to be fixed 23:18:04 datechnoman: Thanks! 23:28:42 thanks! 23:32:05 np all :)