01:21:15 arkiver: https://transfer.archivete.am/uAr3v/discord-krita <- Krita's unofficial Discord 01:24:22 !a https://transfer.archivete.am/uAr3v/discord-krita 01:24:25 TheTechRobo: ^ 01:24:27 arkiver: Deduplicating and queuing 127527 items. 01:24:30 arkiver: thanks! 01:24:32 arkiver: Deduplicated and queued 127527 items. 01:25:11 arkiver: I assume this goes to todo:backfeed? 01:25:20 (Since the todo:backfeed is what just blew up) 01:25:24 yeah 01:25:27 all done already 01:25:33 :) 01:25:41 this project is stupidly fast LOL 01:25:52 haha lol 01:25:54 was gonna say 01:25:58 annnnnnnnnd it gone :P 01:29:33 This project eats a million URL's each bite for breakfast 01:34:17 Have we considered doing all the URLteam urls through this? 01:36:39 makes sense considering we are finding the redirect but the link is unless if the site is no longer online 01:38:15 We are finding the redirects but as far as I know we arent actually archiving the found stuff 01:39:17 useless* 01:39:21 yeah exactly 02:53:46 Thanks for the info, arkiver. 02:54:17 And you didn't have to run that test list through, but thank you for doing so. :-) 04:23:38 Is data saved through here accessible in WARC format afterwards? 04:23:54 Yes, but it's mixed in with everything else going through this project. 11:38:21 I wonder if Hetzner support ever gets sick of "Abuse Messages" 11:38:31 I get 1 every 2-3 days 11:42:49 vortico: at this point they may know about us and not care anymore :P 11:42:56 guess everyone here gives the same reply every time 11:42:58 Haha don't think they do. 2 months ago I was averaging 10-20 a day requiring replys 11:43:06 geez that's a lot 11:43:17 datechnoman: interesting, not anymore? 11:43:21 When we did the Russian and Ukrainian bulk 11:43:38 ah right 11:43:43 Oh yeah Ukraine website make sense since we're hitting the same IP more frequently 11:44:09 Now its super quiet. Like 1 every 2-3 day requiring no replies. Nothing to worry about 11:45:05 I have copy and paste replies at the ready :P 11:45:12 Haha same 11:46:02 We're doing important work. Might not seem like it now but in the future when sites are missing 11:46:34 i'm just in it for the high-score 11:46:52 (just kidding, it's cool that we can participate in the WBM) 11:47:18 Haha the leader boards are just an insensitive 11:47:44 But if they get people donating more conpute that's a bonus 11:47:56 Compute** 11:48:19 I've wondered what the economics of storage is, at IA. 11:48:56 If a 16TB HDD costs $300 but each Hetzner server costs $20/yr, they could easily budget for what we're doing. 11:48:57 I've head 1TB of data costs like 1000 bucks to store and backup etc over its lifetime 11:49:25 Hmm, sounds about right. Maybe that's out of date, but maybe considering building/staff costs it's easily $1000 11:57:24 Some cool photos of the Petabox here. Would love lots of high-res photos of the new racks. https://archive.org/web/petabox.php 11:58:18 Funny that they only had 200TB or so in 2004. I have 160TB in my basement! 16:04:54 vortico: You lost 40TB? :P 16:05:05 I think you meant 200GB. 18:05:49 systwi: They meant that IA had 200 TB of storage in 2004. 18:06:01 So they're about 20 years behind IA now. 18:08:46 The $1000/TB number is probably still reasonably accurate. Hardware prices did decrease but not too much over the last few years. I first heard that sort of figure in 2017 during the SoundButt project, by the way, where it was slightly higher ($1500 to $2000): https://twitter.com/textfiles/status/885527796583284741 18:11:42 So something like $1000 to $1500 now seems reasonable to me. 18:30:09 Ohhh, gotcha, I didn't read it close enough. 18:30:47 I guess I interpreted it originally as, "funny that I only had 200TB or so in 2004. I have 160TB in my basement now!" 20:09:37 !a https://transfer.archivete.am/oxgLb/appStoreLinks.txt 20:09:56 lennier1: ^ your original appstore list 20:11:42 arkiver: Deduplicating and queuing 13492858 items. 20:18:32 arkiver: Deduplicated and queued 13492858 items. 20:20:28 hmm this may be larger in size than I expected 20:21:08 took it out for now 20:33:47 arkiver: How large? Was that getting the screenshots and artwork? If so, a lot of those will be duplicates since that file includes links to the same app listed in multiple regions. I also uploaded separate lists of screenshots and artwork. 20:37:51 lennier1: i see, i put it backj 20:39:12 Cool. Yeah, I found about 1.6M unique apps.