00:01:12 <nicolas17> JAA: argenteam subtitles https://transfer.archivete.am/4mmNR/argenteam.net_subtitles.txt.zst
00:01:31 <nicolas17> 88755 URLs, I estimate 1.7GB total (but with such small files I'm sure header overhead will take over)
00:01:58 <nicolas17> this is the way they are linked from the API and website, but they all redirect to another subdomain
00:06:46 <h2ibot> Nicolas17v2 edited ARGENTeaM (+20, /* Archival */ webpages now being archived): https://wiki.archiveteam.org/?diff=51263&oldid=51258
00:20:47 <flashfire42> power out at IA one minute ago
00:20:54 <nicolas17> yikes
00:21:12 <flashfire42> https://x.com/textfiles/status/1732192819098292677?s=20
00:21:12 <eggdrop> nitter: https://nitter.net/textfiles/status/1732192819098292677
00:21:16 * nicolas17 wonders if we should pause things
00:21:35 <flashfire42> at least pause queuebot probably cc JAA
00:21:42 <JAA> Why?
00:21:59 <flashfire42> IA outage?
00:22:02 <JAA> So?
00:22:36 <flashfire42>  /shrug
00:23:00 <nicolas17> oh dictionary API is down, guess things will stall even if we don't fill targets...
00:23:13 <JAA> Means the rsync target will accumulate data for now, and then the pipelines will accumulate data, and *then* AB will be in trouble.
00:24:06 <nicolas17> do archivebot pipelines still upload to "fos"? (that's what the wiki said but it may be outdated)
00:24:14 <JAA> No, not for years.
00:24:24 <JAA> And even if they did, they wouldn't crash immediately.
00:24:49 <JAA> We have 1-2 days worth of storage between the pipelines.
00:25:00 <nicolas17> oh dat's a lot
00:25:20 <JAA> AB isn't producing a lot of data in the grand scheme of things.
00:25:24 <JAA> It's only 2 TB/d or so.
00:26:17 <JAA> One of my uploads still hasn't noticed that IA is dead.
00:29:41 <project10> https://irc.project10.net/uploads/90b7450d24da0789/Chicken_no_head.gif
00:36:29 <JAA> Looks sad: https://monitor.archive.org/weathermap/weathermap.html
00:36:39 <JAA> But it seems to be returning now, slowly.
00:41:55 <fireonlive> power pls
00:44:39 <JAA> Also, no, the dict API isn't immediately a problem.
00:45:02 <fireonlive> ✨caching✨
00:48:34 <JAA> :-)
00:49:47 <fireonlive> =]
01:07:22 <thuban> well, i broke my grab-site :/
01:07:40 <fireonlive> oh no
01:09:23 <thuban> installation via pyenv+pip doesn't work because fb-re2 sets `extra-compile-args=['-std=c++11']` but current re2 actually requires c++14
01:12:11 <thuban> (https://github.com/facebook/pyre2/blob/053612cd79dab923444454d0035835422e99a632/setup.py, https://github.com/google/re2/blob/7e0c1a9e2417e70e5f0efc323267ac71d1fa0685/re2/re2.h#L222, https://github.com/abseil/abseil-cpp/blob/5dc2cc1a6a992e09d9cf930cdce7984640e2b7e0/absl/base/policy_checks.h#L68-L81)
01:12:59 <thuban> can't work around with CFLAGS because setuptools puts extra_compile_args at the end, which takes precedence
01:15:08 <thuban> is there a particular reason we're still using https://pypi.org/project/fb-re2/ instead of https://pypi.org/project/google-re2/?
01:15:36 <nicolas17> in the blogger project we got a sudden spike in completed items and then it died again huh
01:31:16 <nicolas17> I wonder what's the problem with network cost in korea
01:31:22 <nicolas17> ingress or egress?
01:31:32 <nulldata> 08:29 PM <nulldata> Twitch is shutting down in Korea on February 27, 2024 KST -- https://blog.twitch.tv/en/2023/12/05/an-update-on-twitch-in-korea/
01:31:33 <nicolas17> will they block streamers in korea or viewers in korea or both?
01:31:37 <nulldata> For those not watching main channel
01:32:54 <flashfire42> Oh dear. #burnthetwitch will be getting a workout
01:34:26 <nulldata> I wonder what the bandwidth costs are for those using an AWS Korea POP compared to other regions?
01:38:08 <nicolas17> "though SK has laws that makes the content provider pay the bandwidth cost iirc"
01:38:09 <nicolas17> oh
01:38:18 <nicolas17> nulldata: "Hong Kong, Indonesia,  Philippines, Singapore, South Korea, Taiwan, Thailand, Malaysia, and Vietnam" is the most expensive category in AWS CloudFront
01:39:23 <missaustraliana> hey! fullpwnmedia here. long time no see. It has come to my attention that an australian breakfast tv network will cease in 20ish days. their network is known to delete everything that is tied to a show if it no longer exists. the show that is ending is Studio 10 and is run by the 10 Network.
01:39:43 <flashfire42> Wait studio 10 is being axed?
01:39:46 <missaustraliana> not sure if you will be able to access this link but this is their page
01:39:46 <missaustraliana> https://10play.com.au/studio-10
01:39:51 <missaustraliana> yeah on december 22
01:39:58 <missaustraliana> it will cease to exist
01:40:43 <missaustraliana> https://www.youtube.com/channel/UCLCUbvRrQnApZldY6CTquTA thats their YouTube
01:40:54 <missaustraliana> https://twitter.com/studio10au?lang=en Twitter
01:40:54 <eggdrop> nitter: https://nitter.net/studio10au
01:41:27 <missaustraliana> https://www.instagram.com/studio10au/?hl=en Instagram
01:41:39 <flashfire42> Just threw the youtube channel into down the tube
01:42:00 <flashfire42> 10play will be behind geoblock tho I spose if you wanted to incur the wratch of IA you could use tubeup
01:42:09 <missaustraliana> I need all hands on deck to get this all archived. Especially the YouTube as it contains full length episodes
01:42:52 <flashfire42> Youtube will be done via downthetube missaustraliana as soon as the bot decides to wake up. Instagram you will have to do manually. Twitter is a pain in the ass
01:43:37 <missaustraliana> I'm not so worried about 10play episodes as they also need an account. But I would like for the page to be atleast archived
01:44:33 <missaustraliana> Thanks flashfire42
01:46:51 <missaustraliana> I'm manually archiving each show until cut off through OTA (DTV)
01:47:22 <missaustraliana> I have todays show getting uploaded to my IA account
01:50:08 <flashfire42> missaustraliana how are you uploading that? Tubeup?
01:51:44 <missaustraliana> What do you mean? I recorded todays recording with my set top box now uploading to IA
01:52:11 <flashfire42> AH now I get you
01:53:20 <missaustraliana> now having struggles uploading as IA is throwing a fit and spitting out "there was a network problem"
01:53:42 <flashfire42> oh yeah they have a power outage
01:54:29 <missaustraliana> It could be that OR i just realised that im throwing it the files that are not stored locally but on my intranet
01:57:03 <missaustraliana> Okay yeah nah its the power outage
02:00:48 <missaustraliana> flashfire42 is there another way i can upload or do i just have to wait
02:01:09 <flashfire42> Not to IA
02:01:22 <missaustraliana> Shiz..
02:03:49 <fireonlive> "Citrix pulls the plug on its User Group Community" https://www.theregister.com/2023/12/05/citrix_user_group_community_ends/
02:57:55 <cas> TheTechRobo about this video https://web.archive.org/web/20220103091055/https://www.youtube.com/watch?v=OQktVBtbygI So does that mean I can't extract the video in wayback? Is there any other ways I can extract the video's content?
03:04:49 <arkiver> (me and Sanqui are talking further over PM)
03:06:32 <h2ibot> Tech234a edited Deathwatch (+121, /* 2023 */ Add Discord attachment links date): https://wiki.archiveteam.org/?diff=51265&oldid=51261
03:07:45 <nicolas17> cas: it's likely the video wasn't archived
03:09:02 <cas> rip, that would be unfortunate to hear
03:10:33 <h2ibot> Nicolas17v2 edited ARGENTeaM (+46, Add archivebot job ID for subtitles): https://wiki.archiveteam.org/?diff=51266&oldid=51263
03:11:43 <cas> It's a tall order, but I can't help but wonder if somewhere like common crawl might have it archived
03:12:17 <JAA> I don't think Common Crawl is doing anything with YouTube videos. And they have long needed special treatment to archive.
03:12:21 <nicolas17> well generic crawlers won't get youtube videos for sure
03:12:44 <arkiver> i don't think common crawl is doing video at all?
03:12:48 <nicolas17> you need complex purpose-specific code to get the URL of the actual video file (like what youtube-dl does)
03:13:33 <h2ibot> Nulldata edited Deathwatch (+434, Added 'In The Know' and Twitch Korea.): https://wiki.archiveteam.org/?diff=51267&oldid=51265
03:13:34 <h2ibot> Inti83 edited Argentina (+282, /* List of At-Risk Websites */  - updating with…): https://wiki.archiveteam.org/?diff=51268&oldid=51252
03:13:36 <nicolas17> arkiver: even if it did, it would only work with like plain <video src="foo.mp4">, not youtube's rube goldberg.js machine
03:14:33 <h2ibot> House edited Talk:Main Page (+406, /* Instructions please */ new section): https://wiki.archiveteam.org/?diff=51269&oldid=50704
03:14:51 <nicolas17> omfg
03:15:02 <nicolas17> JAA: check that edit and tell me if it sounds familiar
03:15:19 <JAA> nicolas17: I just approved it, so yes.
03:16:15 <arkiver> nicolas17: yeah
03:16:33 <cas> saddening, guess that means I can't get the video then
03:16:47 <missaustraliana> can someone add Studio 10 December 22 AEDT
03:16:57 <cas> thanks for answering my questions folks
03:17:02 <JAA> missaustraliana: Anyone can edit the wiki. :-)
03:17:21 <missaustraliana> Without or with an account? JAA
03:17:42 <JAA> missaustraliana: With
03:17:49 <missaustraliana> mk
03:18:11 <nicolas17> missaustraliana: I can add it for you but I won't write the site description for you, give me something I can blindly copypaste :P
03:18:28 <nulldata> If you sign up right now, the account is free!
03:18:32 * fireonlive 👀 Talk:
03:18:47 <arkiver> let's see what shutdowns we have coming up
03:19:01 <JAA> We should probably have a warning that nobody really pays attention to Talk:*.
03:19:01 <missaustraliana> I'm signing up now as I'll probably be making changes in the future
03:19:08 <arkiver> JAA: yeah :P
03:19:19 <arkiver> missaustraliana: sounds good :)
03:19:23 <arkiver> welcome to the wiki
03:19:27 <missaustraliana> tsym
03:19:33 <fireonlive> wikipals!
03:19:52 <missaustraliana> okay but what in the world is that captcha
03:20:04 <missaustraliana> convert to ascii?
03:20:26 <nicolas17> yeah
03:20:42 <missaustraliana> incorrect captcha..
03:20:44 <nicolas17> the captcha used to be "ask for the secret word in #archiveteam-bs"
03:21:02 <nicolas17> guess they changed it to something more automated :P
03:21:25 <nulldata> hunter2
03:21:57 <missaustraliana> can someone convert this to ascii? the page doesnt seem to like what i input
03:22:02 <missaustraliana> 4D6F757365
03:22:42 <nicolas17> maybe you're a robot >.>
03:22:43 <fireonlive> Mouse
03:22:46 <missaustraliana> oh..
03:22:52 <nulldata> How do we know you're not a bot?! :P
03:23:05 <fireonlive> oh no!
03:23:33 <missaustraliana> Jokes on you lot i am a robot8-|
03:24:09 <nicolas17> oh no we just helped chatgpt sign up on the wiki
03:24:12 <missaustraliana> i will now fill the whole wiki with false information<3
03:24:29 <nulldata> fireonlive what have you done?!
03:24:36 <fireonlive> oh nooooo
03:24:42 <arkiver> looks like 3dtotal is done with archivebot
03:25:21 <arkiver> will post on discord in #discard
03:25:40 <missaustraliana> wait theres a discord too?
03:25:49 <missaustraliana> i thought thwere was only irc
03:25:55 <nulldata> Only IRC
03:26:03 <missaustraliana> oh:'(
03:26:19 <missaustraliana> my robot ass rlly thought
03:26:20 <nulldata> arkive.r is referring to the Discord archive project
03:26:28 <missaustraliana> ohh
03:26:36 <h2ibot> FireonLive edited Talk:Main Page (+165, beware the talk pages): https://wiki.archiveteam.org/?diff=51270&oldid=51269
03:26:48 <arkiver> larm.fm is pretty huge
03:27:09 <arkiver> JAA: do you know if we did anything for larm.fm?
03:27:37 <h2ibot> FireonLive edited Talk:Main Page (+2, ?): https://wiki.archiveteam.org/?diff=51271&oldid=51270
03:27:39 <flashfire42> So what you are saying we better start now?
03:27:50 <JAA> arkiver: Nothing yet I believe.
03:28:28 <arkiver> i wonder if they have dumps - a dump would be appropriate for this kind of data
03:28:53 <missaustraliana> do we have any updates on studio 10? someone said that they chucked their youtube into down thr tube
03:29:26 <flashfire42> I tried. When the bot is back it will go through
03:29:40 <arkiver> apo.org.au seems to be saved with AB
03:29:43 <missaustraliana> ah
03:29:55 <JAA> arkiver: Negative, APO banned all attempts.
03:29:59 <arkiver> ah
03:30:02 <arkiver> fun
03:30:18 <arkiver> i see sequential identifiers
03:30:25 <nicolas17> missaustraliana: the bot that handles the "add to queue" commands is also hosted at archive.org, and it hasn't recovered after the power outage yet
03:30:37 <arkiver> nicolas17: i'm working on it
03:31:02 <nicolas17> arkiver: oh I know :) just giving status info
03:31:15 <arkiver> JAA: does looks like they have nice sequential identifiers everywhere
03:31:52 <missaustraliana> oh shit there was an actual outage at IA? flash did mention that but i thought that was just speculation. i didnt get any sort of notification on mastodon that it was down.
03:31:58 <JAA> arkiver: Yeah, regular Drupal.
03:32:08 <arkiver> missaustraliana: it's no very rare unfortunately
03:32:18 <nicolas17> missaustraliana: https://twitter.com/textfiles/status/1732192819098292677
03:32:19 <eggdrop> nitter: https://nitter.net/textfiles/status/1732192819098292677
03:32:24 <arkiver> :P IA mastodong *was down too*
03:32:32 <nicolas17> x_x
03:32:41 <JAA> > mastodong
03:32:42 <arkiver> mastodon*
03:32:43 <JAA> :-)
03:33:05 <JAA> (preemptive bonk)
03:33:06 <missaustraliana> judging the comments it seems like they didnt have enough funds for power
03:33:09 <arkiver> hahaha looks like i made up a good one :P
03:33:34 <JAA> missaustraliana: Rule 1 about Twitter: do not believe anything in comments.
03:33:35 <missaustraliana> https://twitter.com/textfiles/status/1732193921839538439?s=61&t=8c_vSorlRfNMzyUIfsljuw
03:33:35 <eggdrop> nitter: https://nitter.net/textfiles/status/1732193921839538439
03:33:35 <arkiver> missaustraliana: which comments?
03:33:52 <JAA> 99% of it is rubbish.
03:34:00 <arkiver> oh
03:34:06 <missaustraliana> real. bring back the old twitter
03:34:08 <arkiver> no IA pays their power bill
03:34:20 <missaustraliana> old twitter was the best
03:34:29 <JAA> Meh
03:34:38 <arkiver> but IA does make maximum use of their hardware, which could leave room for an extra failure every now and then compared to other services
03:34:39 <JAA> It wasn't the hellhole it is now at least, I guess.
03:34:54 <missaustraliana> when there was no blue checkmark, no priority in comments, no rate limits
03:34:55 <nicolas17> missaustraliana: that's just textfiles telling the whining people to donate
03:36:27 <missaustraliana> is textfiles apart of ia
03:37:13 <fireonlive> he's a staffer there yeah
03:37:29 <missaustraliana> ah
03:37:48 <nicolas17> telegram and blogger projects are starting to upload now
03:38:23 <missaustraliana> im waiting for any updates before continuing to upload as i dont want to flood the server
03:38:34 <nicolas17> either our target-master-ewby added some miraculous amount of buffer space, or IA started taking uploads again
03:39:22 <fireonlive> it's probably getting rammed pretty hard as it works its way back on its feet
03:39:24 <missaustraliana> ill try once but if it fails im waiting
03:40:13 <missaustraliana> i can tell you now the upload is alot slower
03:40:55 <missaustraliana> which could indicate that it might be working again
03:42:36 <fireonlive> that would be a fun rule of tumb
03:42:38 <fireonlive> thumb
03:43:35 <missaustraliana> on my end uploads are working
03:44:41 <fireonlive> nice
03:45:06 <Vokun> Just looking at the 20k+ items in the item deriver queue is haunting
03:45:28 <arkiver> the bot to queue to projects is back!
03:46:03 <Vokun> 🎉
03:48:49 <JAA> Vokun: Those built up earlier in the day though.
03:49:02 <JAA> Starting around 10 hours ago.
03:49:19 <Vokun> Oh
03:50:41 <missaustraliana> ohh so it was restored from a backup
03:51:06 <JAA> Uh, what?
03:52:19 <missaustraliana> dw i might just be dumb
03:58:13 <nicolas17> (since I keep forgetting + docs universally suck) does archivebot do body deduplication across different URLs?
03:58:26 <JAA> No
04:01:10 <nicolas17> so if I give archivebot a list with URLs A and B, and A redirects to B, that will unfortunately request B twice (issue 431)... but it will also *store* the body of B twice? that's annoying :/
04:01:25 <nicolas17> okay that's the same URL but
04:02:05 <JAA> Yes
04:04:10 <fireonlive> won't the .gz help out a bit with that
04:04:25 <JAA> No, each record is compressed separately.
04:04:32 <fireonlive> ahh
04:06:13 <nicolas17> argenteam added info and subtitles for like 5 more TV show episodes since I scraped everything... I thought after announcing they were shutting down, they would stop adding anything :|
04:08:24 <fireonlive> you'd think...
04:35:49 <h2ibot> JustAnotherArchivist edited ARGENTeaM (-10, Use Job template): https://wiki.archiveteam.org/?diff=51272&oldid=51266
04:38:55 <nicolas17> oh neat
04:54:39 <fireonlive> <@arkiver> :P IA mastodong *was down too* < something on your mind, arkiver? :3
05:16:04 <JAA> <@JAA> (preemptive bonk)
05:16:17 <fireonlive> :D
05:24:59 <h2ibot> JustAnotherArchivist edited Argentina (+83, Fix URLs): https://wiki.archiveteam.org/?diff=51273&oldid=51268
05:36:01 <h2ibot> JustAnotherArchivist edited Argentina (+49, More URL fixes): https://wiki.archiveteam.org/?diff=51274&oldid=51273
05:37:52 <JAA> Everything from [[Argentina]] has been thrown into AB now (except for the two that are geofenced).
05:42:13 <JAA> Nearly everything in the 'Gender and Equality' section is JS hell, so no idea how well that will work.
06:58:23 <h2ibot> Switchnode edited Argentina (+2069, /* List of At-Risk Websites */ add archivebot +…): https://wiki.archiveteam.org/?diff=51275&oldid=51274
07:04:23 <thuban> !tell Inti83 all of the sites on the argentina wiki page have been submitted to archivebot; you can monitor running jobs at http://archivebot.com/ and retrieve finished ones from https://archive.fart.website/archivebot/viewer/
07:04:24 -eggdrop- [tell] ok, I'll tell Inti83 when they join next
07:04:29 <thuban> !tell Inti83 note that a job succeeding does not necessarily mean the site was adequately captured (if eg there is heavy use of javascript)
07:04:30 -eggdrop- [tell] ok, I'll tell Inti83 when they join next
17:47:34 <h2ibot> Switchnode edited Argentina (+36, /* Memory and Human Rights */ add archivebot…): https://wiki.archiveteam.org/?diff=51276&oldid=51275
18:43:29 <fireonlive> -+rss- W3C Leaves Twitter: https://w3c.social/@w3c/111534700276754588 https://news.ycombinator.com/item?id=38547203
18:43:39 <fireonlive> unfortunately the account is protected https://twitter.com/w3c
18:43:40 <eggdrop> nitter: https://nitter.net/w3c
18:44:18 <JAA> It was accessible 13 hours ago. :-|
18:44:28 <fireonlive> :\
18:52:19 <JAA> > have directed all our followers here to Mastodon
18:52:27 <JAA> Well, except nobody can read the tweets anymore...
19:09:41 <fireonlive> indeed, no one will see the 'please follow us over here' unless they specifically visit the profile which.. they wont lol
19:12:37 <JAA> I've been informed that their existing followers should still be able to see the tweets.
19:13:36 <fireonlive> oh! right
19:13:51 <fireonlive> i'm a bit OOTL on twitter it seems
19:14:21 <fireonlive> sadly... i don't seem to follow them
19:29:00 <h2ibot> Tech234a edited Chrome Web Store (+470, Update manifest v2 timeline, app deprecation,…): https://wiki.archiveteam.org/?diff=51277&oldid=49044
21:00:18 <JAA> https://www.intheknow.com/ is being annoying with a consent redirect. I'm grabbing it with grab-site from an unaffected IP.
21:06:38 <that_lurker> "
21:06:38 <that_lurker> We'll be
21:06:38 <that_lurker> right back..." nice site :P
21:08:39 <JAA> Yeah, I got that one as well with curl.
21:09:04 <JAA> < HTTP/1.1 400 Direct self loop detected