00:00:00 <pabs> thanks
00:27:50 <tech234a> Said Twitter policy has been reversed and Paul Graham has been unsuspended. Musk is now running a poll asking if he should step down.
00:36:39 <pabs> wha?!
00:36:43 <pabs> link?
00:42:30 <qwertyasdfuiopghjkl> https://twitter.com/elonmusk/status/1604616863673208832 https://twitter.com/elonmusk/status/1604617643973124097
01:04:10 <OrIdow6> This general tumultuousness seems to me increase the chance that Twitter-owned mailing list thing shuts down early, though I still think that chance is small
01:06:05 <JAA> Revue?
03:20:04 <AlsoTheTechRobo> For someone that has voted in the poll - what are the current results (re Musk's poll)? (I'll check logs later - webirc is awfully unstable and can't currently access my main computer because someone is isolating thanks to COVID)
03:21:35 <AlsoTheTechRobo> There's probably an overwhelming "don't step down", but in case it isn't :-)
03:29:52 <pabs> TheTechRobo: ~56% yes I think
03:45:55 <JAA> 5536745 aye, 4284211 nay as of right now
03:46:12 <ivan> nitter will show you poll results
03:46:41 <JAA> As will snscrape :-)
03:51:01 <pabs> https://www.gamingonlinux.com/2022/12/moving-on-from-twitter/
03:51:09 <pabs> (just did snscrape)
04:44:12 <Ryz> arkiver, any updates on contacting https://geolog.mydns.jp/ or attempts to archive the website in general?
05:47:00 <OrIdow6> JAA: Yeah
08:35:04 <pabs> Freed-ora Linux distro is being shut down https://lists.gnu.org/archive/html/info-gnu/2022-12/msg00006.html https://www.phoronix.com/news/Freed-ora-No-More https://www.fsfla.org/ikiwiki/selibre/linux-libre/freed-ora.en.html
17:21:22 <AlsoTheTechRobo> pabs, JAA: Thanks
17:21:54 <AlsoTheTechRobo> Btw, what's the best way to archive a protected account's tweets? (I'm already following that person so I can see their tweets, but I don't know what scraping tool(s) support logging in.)
17:30:45 <AlsoTheTechRobo> Might use https://github.com/n0madic/twitter-scraper
17:33:17 <AlsoTheTechRobo> Nvm, I don't know Go. :-/
17:33:18 <schwarzkatz|m> are there even any good news regarding that site lately
17:33:19 <theblazehen|m> I wonder if snscrape with passing in your cookies may work
17:33:19 <schwarzkatz|m> why is it so awfully quiet here currently, where is everybody :c
17:33:42 <AlsoTheTechRobo> theblazehen|m: I was thinking of trying that, but there might be tokens and stuff?
17:33:47 <AlsoTheTechRobo> No harm in trying ig
17:34:06 <AlsoTheTechRobo> wait, was Matrix fixed?
17:34:12 <schwarzkatz|m> yes
17:34:18 <theblazehen|m> Looks like it I guess
17:49:23 <schwarzkatz|m> arkiver or JAA, can we reduce the load on uploadir? it's resulting in high cpu usage
17:50:25 <schwarzkatz|m> and I saw that here is an error in the script, so we probably didn't archive anything yet :/ https://github.com/ArchiveTeam/uploadir-grab/issues/1
18:00:44 <schwarzkatz|m> maybe we should stop the job until it's fixed
18:01:21 <AlsoTheTechRobo> Looks like adding headers to snscrape won't work; before, snscrape wasn't outputting anything (probably because no auth) but now everything is on fire
18:01:30 <AlsoTheTechRobo> Maybe the API endpoints are different?
18:01:39 <AlsoTheTechRobo> Or maybe you aren't supposed to use a guest token.
18:02:10 <AlsoTheTechRobo> Anyway, now I've got this:
18:02:11 <AlsoTheTechRobo> snscrape.base.ScraperException: 4 requests to https://twitter.com/i/api/graphql/8svRea_Lc0_mdhwP6dqe0Q/TweetDetail?variables=%7B%22focalTweetId%22%3A%221604568489104412672%22%2C%22with_rux_injections%22%3Afalse%2C%22includePromotedContent%22%3Atrue%2C%22withCommunity%22%3Atrue%2C%22withQuickPromoteEligibilityTweetFields%22%3Atrue%2C%22withTweetQuot
18:02:11 <AlsoTheTechRobo> eCount%22%3Atrue%2C%22withBirdwatchNotes%22%3Atrue%2C%22withSuperFollowsUserFields%22%3Atrue%2C%22withBirdwatchPivots%22%3Afalse%2C%22withDownvotePerspective%22%3Afalse%2C%22withReactionsMetadata%22%3Afalse%2C%22withReactionsPerspective%22%3Afalse%2C%22withSuperFollowsTweetFields%22%3Atrue%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Afalse%7D
18:02:12 <AlsoTheTechRobo> failed, giving up.
18:03:55 <schwarzkatz|m> rewby: could you pause the uploadir task please? I hope you don't mind me pinging you.
18:07:05 <rewby> schwarzkatz|m: What do you mean pause the task?
18:08:26 <schwarzkatz|m> the grab lua has a line with an error in it, causing the warriors to just reuqest the links over and over again without doing anything with the results
18:09:34 <nepeat> pulled my logs, can confirm
18:09:35 <schwarzkatz|m> I'm sorry if you were the wrong person to ask about this, but the admin asked me to try and reduce the load a bit
18:09:38 <nepeat> 2022-12-19 10:09:09	 Lua runtime error: uploadir.lua:248: bad argument #1 to 'match' (string expected, got table).
18:09:45 <nepeat> do we have a channel for uploadir yet?
18:09:58 <schwarzkatz|m> nepeat: don't think so
18:11:02 <rewby> I'll shut off uploadir
18:11:37 <schwarzkatz|m> thanks
18:13:25 <rewby> I've set limit to 0. So no new jobs. Can't control stuff that's already running.
18:13:28 <rewby> Cc arkiver
18:28:53 <arkiver> thanks
18:33:34 <arkiver> fixed
18:33:38 <arkiver> resumed at 60 items/min
18:33:40 <arkiver> (1/sec)
18:33:52 <arkiver> when we have a target for uploadir we can start
18:53:32 <AlsoTheTechRobo> Just to clarify, Fortress of Solitude is NOT used anymore?
19:02:06 <h2ibot> TheTechRobo edited Iwata Asks (-590, Update): https://wiki.archiveteam.org/?diff=49250&oldid=47253
19:05:36 <arkiver> AlsoTheTechRobo: not for warrior projects
19:05:46 <arkiver> maybe still for AB but not sure (CC JAA )
19:05:56 <AlsoTheTechRobo> I think JAA recently said it was no longer used for AB
19:06:48 <arkiver> then only for misc stuff I guess
19:06:49 <arkiver> no idea
19:07:36 <michaelblob> is the uploadir project running? i haven't gotten any items on my node
19:08:48 <schwarzkatz|m> getting warnings as well
19:08:48 <schwarzkatz|m> No item received. There aren't any items available for this project at the moment. Try again later. Retrying after XX seconds...
19:34:25 <rewby> I'll do target ops after D&D
19:39:27 <JAA> arkiver, TheTechRobo: AB stopped using FOS early this year.
20:26:19 <h2ibot> TheTechRobo edited Fortress of Solitude (+472, Update for 2022): https://wiki.archiveteam.org/?diff=49251&oldid=34308
22:01:35 <h2ibot> Schwarzkatz uploaded File:Uploadir page screenshot.png: https://wiki.archiveteam.org/?title=File%3AUploadir%20page%20screenshot.png
22:22:46 <rewby> arkiver: Uploadir IA title, is it meant to be Archive Team Uploader: or did you typo it?
22:23:01 <rewby> I'm expecting the last e to be an i tbh
22:24:32 <arkiver> it's uploadir with i in this case
22:24:34 <arkiver> Uploadir
22:24:39 <arkiver> rewby: ^
22:24:39 <rewby> Excellent
22:24:51 <rewby> I was copy pasting your vars and had a "wait that's wrong" moment
22:24:53 <arkiver> just a TB or so, not much
22:24:59 <arkiver> yeah :P it's their name
22:25:03 <arkiver> thanks for checking though
22:27:08 <rewby> Uploadir is provisioning
22:27:37 <rewby> We're doing a whole 5gbps of target traffic atm
22:27:39 <rewby> Not bad
22:28:01 <rewby> Might need to chuck in some hetzner cloud depending on how the current ops turn out
22:31:41 <rewby> arkiver, schwarzkatz|m: Uploadir target online. Do whatever it is you're planning to do.
22:31:57 <schwarzkatz|m> great, thanks!
22:32:30 <rewby> I'll likely have to provision some extra compute to help out soon
22:32:39 <rewby> As in, to help target work
22:32:44 <rewby> Because you're hitting 8gbps
22:32:48 <rewby> Across 4 servers
22:33:01 <rewby> Which is getting towards the "probably not sustainable" mark
22:33:16 <rewby> But I'm gonna let this sit and chew through backlog for like 30 minutes
22:38:42 <arkiver> rewby: yeah unfortunately all end of year projects :/
22:38:52 <arkiver> so very soon most of that should go down again
22:49:10 <rewby> arkiver: Any more EOY projects coming up? / What is the list of EOY projects
22:53:10 <arkiver> rewby: i've been working through https://wiki.archiveteam.org/index.php/Deathwatch#2022
22:53:18 <arkiver> looks like we largely have everything started now
22:53:30 <arkiver> OrIdow6 is working on a clara.io project i believe
22:53:50 <arkiver> VKontakte will still start soon as well, which will give some data
22:53:57 <arkiver> and possibly that chinese site
22:54:04 <rewby> So is there anything you're currently missing targets on?
22:54:35 <arkiver> not currently i believe, but I can't promise we won't need anything else in the coming days/weeks
23:20:47 <h2ibot> Pokechu22 edited Deathwatch (+106, /* 2023 */ karayou.com): https://wiki.archiveteam.org/?diff=49253&oldid=49248
23:24:47 <h2ibot> Pokechu22 edited Deathwatch (+157, /* 2022 */ elisa pages): https://wiki.archiveteam.org/?diff=49255&oldid=49253