-
pabs
thanks
-
tech234a
Said Twitter policy has been reversed and Paul Graham has been unsuspended. Musk is now running a poll asking if he should step down.
-
pabs
wha?!
-
pabs
link?
-
qwertyasdfuiopghjkl
-
OrIdow6
This general tumultuousness seems to me increase the chance that Twitter-owned mailing list thing shuts down early, though I still think that chance is small
-
JAA
Revue?
-
AlsoTheTechRobo
For someone that has voted in the poll - what are the current results (re Musk's poll)? (I'll check logs later - webirc is awfully unstable and can't currently access my main computer because someone is isolating thanks to COVID)
-
AlsoTheTechRobo
There's probably an overwhelming "don't step down", but in case it isn't :-)
-
pabs
TheTechRobo: ~56% yes I think
-
JAA
5536745 aye, 4284211 nay as of right now
-
ivan
nitter will show you poll results
-
JAA
As will snscrape :-)
-
pabs
-
pabs
(just did snscrape)
-
Ryz
arkiver, any updates on contacting
geolog.mydns.jp or attempts to archive the website in general?
-
OrIdow6
JAA: Yeah
-
pabs
-
AlsoTheTechRobo
pabs, JAA: Thanks
-
AlsoTheTechRobo
Btw, what's the best way to archive a protected account's tweets? (I'm already following that person so I can see their tweets, but I don't know what scraping tool(s) support logging in.)
-
AlsoTheTechRobo
-
AlsoTheTechRobo
Nvm, I don't know Go. :-/
-
schwarzkatz|m
are there even any good news regarding that site lately
-
theblazehen|m
I wonder if snscrape with passing in your cookies may work
-
schwarzkatz|m
why is it so awfully quiet here currently, where is everybody :c
-
AlsoTheTechRobo
theblazehen|m: I was thinking of trying that, but there might be tokens and stuff?
-
AlsoTheTechRobo
No harm in trying ig
-
AlsoTheTechRobo
wait, was Matrix fixed?
-
schwarzkatz|m
yes
-
theblazehen|m
Looks like it I guess
-
schwarzkatz|m
arkiver or JAA, can we reduce the load on uploadir? it's resulting in high cpu usage
-
schwarzkatz|m
and I saw that here is an error in the script, so we probably didn't archive anything yet :/
ArchiveTeam/uploadir-grab #1
-
schwarzkatz|m
maybe we should stop the job until it's fixed
-
AlsoTheTechRobo
Looks like adding headers to snscrape won't work; before, snscrape wasn't outputting anything (probably because no auth) but now everything is on fire
-
AlsoTheTechRobo
Maybe the API endpoints are different?
-
AlsoTheTechRobo
Or maybe you aren't supposed to use a guest token.
-
AlsoTheTechRobo
Anyway, now I've got this:
-
AlsoTheTechRobo
-
AlsoTheTechRobo
eCount%22%3Atrue%2C%22withBirdwatchNotes%22%3Atrue%2C%22withSuperFollowsUserFields%22%3Atrue%2C%22withBirdwatchPivots%22%3Afalse%2C%22withDownvotePerspective%22%3Afalse%2C%22withReactionsMetadata%22%3Afalse%2C%22withReactionsPerspective%22%3Afalse%2C%22withSuperFollowsTweetFields%22%3Atrue%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Afalse%7D
-
AlsoTheTechRobo
failed, giving up.
-
schwarzkatz|m
rewby: could you pause the uploadir task please? I hope you don't mind me pinging you.
-
rewby
schwarzkatz|m: What do you mean pause the task?
-
schwarzkatz|m
the grab lua has a line with an error in it, causing the warriors to just reuqest the links over and over again without doing anything with the results
-
nepeat
pulled my logs, can confirm
-
schwarzkatz|m
I'm sorry if you were the wrong person to ask about this, but the admin asked me to try and reduce the load a bit
-
nepeat
2022-12-19 10:09:09 Lua runtime error: uploadir.lua:248: bad argument #1 to 'match' (string expected, got table).
-
nepeat
do we have a channel for uploadir yet?
-
schwarzkatz|m
nepeat: don't think so
-
rewby
I'll shut off uploadir
-
schwarzkatz|m
thanks
-
rewby
I've set limit to 0. So no new jobs. Can't control stuff that's already running.
-
rewby
Cc arkiver
-
arkiver
thanks
-
arkiver
fixed
-
arkiver
resumed at 60 items/min
-
arkiver
(1/sec)
-
arkiver
when we have a target for uploadir we can start
-
AlsoTheTechRobo
Just to clarify, Fortress of Solitude is NOT used anymore?
-
h2ibot
TheTechRobo edited Iwata Asks (-590, Update):
wiki.archiveteam.org/?diff=49250&oldid=47253
-
arkiver
AlsoTheTechRobo: not for warrior projects
-
arkiver
maybe still for AB but not sure (CC JAA )
-
AlsoTheTechRobo
I think JAA recently said it was no longer used for AB
-
arkiver
then only for misc stuff I guess
-
arkiver
no idea
-
michaelblob
is the uploadir project running? i haven't gotten any items on my node
-
schwarzkatz|m
getting warnings as well
-
schwarzkatz|m
No item received. There aren't any items available for this project at the moment. Try again later. Retrying after XX seconds...
-
rewby
I'll do target ops after D&D
-
JAA
arkiver, TheTechRobo: AB stopped using FOS early this year.
-
h2ibot
TheTechRobo edited Fortress of Solitude (+472, Update for 2022):
wiki.archiveteam.org/?diff=49251&oldid=34308
-
h2ibot
-
rewby
arkiver: Uploadir IA title, is it meant to be Archive Team Uploader: or did you typo it?
-
rewby
I'm expecting the last e to be an i tbh
-
arkiver
it's uploadir with i in this case
-
arkiver
Uploadir
-
arkiver
rewby: ^
-
rewby
Excellent
-
rewby
I was copy pasting your vars and had a "wait that's wrong" moment
-
arkiver
just a TB or so, not much
-
arkiver
yeah :P it's their name
-
arkiver
thanks for checking though
-
rewby
Uploadir is provisioning
-
rewby
We're doing a whole 5gbps of target traffic atm
-
rewby
Not bad
-
rewby
Might need to chuck in some hetzner cloud depending on how the current ops turn out
-
rewby
arkiver, schwarzkatz|m: Uploadir target online. Do whatever it is you're planning to do.
-
schwarzkatz|m
great, thanks!
-
rewby
I'll likely have to provision some extra compute to help out soon
-
rewby
As in, to help target work
-
rewby
Because you're hitting 8gbps
-
rewby
Across 4 servers
-
rewby
Which is getting towards the "probably not sustainable" mark
-
rewby
But I'm gonna let this sit and chew through backlog for like 30 minutes
-
arkiver
rewby: yeah unfortunately all end of year projects :/
-
arkiver
so very soon most of that should go down again
-
rewby
arkiver: Any more EOY projects coming up? / What is the list of EOY projects
-
arkiver
-
arkiver
looks like we largely have everything started now
-
arkiver
OrIdow6 is working on a clara.io project i believe
-
arkiver
VKontakte will still start soon as well, which will give some data
-
arkiver
and possibly that chinese site
-
rewby
So is there anything you're currently missing targets on?
-
arkiver
not currently i believe, but I can't promise we won't need anything else in the coming days/weeks
-
h2ibot
Pokechu22 edited Deathwatch (+106, /* 2023 */ karayou.com):
wiki.archiveteam.org/?diff=49253&oldid=49248
-
h2ibot
Pokechu22 edited Deathwatch (+157, /* 2022 */ elisa pages):
wiki.archiveteam.org/?diff=49255&oldid=49253