-
nico_32
-
» nico_32 is mirroring the playlist to IA
-
mgrandi
Archive bot request, this verified account has apparently passed away, but it is 100k tweets and I'm not sure if we are still backed up with election stuff
twitter.com/akirareiko
-
Ryz
mgrandi, do you have a source for said death?
-
mgrandi
-
Ryz
Acknowledged, it's going through socialbot/snscrape and will be processed in AB
-
mgrandi
-
mgrandi
Thanks
-
Ryz
Ran that Twitter account too because related
-
mgrandi
@ryz i'm not super experienced with snscrape, but only 600 requests for 100k tweets? for the akiraeiko job
-
Ryz
Talk to JAA about it, they'll tell you more about snscrape~
-
Ryz
From checking
transfer.notkiska.pw/tn7FK/twitter-@akirareiko - the count is around 96k amount of links
-
mgrandi
affirmative
-
kyndigs
started uploading itunesu stuff now
-
kyndigs
-
kyndigs
example
-
JAA
mgrandi: '600 requests for 100k tweets'?
-
purplebot
ITunesU edited by Kyndigs (+638, /* Archiving Status */) just now --
archiveteam.org/?diff=45767&oldid=45766
-
purplebot
Indafotó edited by Bzc6p (-3, upcoming) just now --
archiveteam.org/?diff=45768&oldid=45610
-
mgrandi
@JAA: I was curious so I checked the archivebot forget and it stuff it finished the guys profile and said that
-
JAA
mgrandi: Syntax error near 'forget'.
-
mgrandi
Channel*
-
mgrandi
/Stuff/said
-
JAA
Then that was probably the chromebot job. The AB job doesn't report the number of responses when it finishes.
-
mgrandi
Stupid phone*
-
JAA
chromebot only fetches the profile page, so 600-ish sounds right.
-
mgrandi
So snscrape uses chromebot?
-
JAA
No
-
kiska
What gave you that idea>
-
JAA
socialbot calls snscrape to scrape Twitter. It then feeds snscrape's output into ArchiveBot and additionally throws the profile page into chromebot.
-
JAA
The AB job for the account is still running, by the way.
-
mgrandi
Well that's what I meant, it pipes the result into AB/ CB
-
JAA
Yeah, but socialbot != snscrape.
-
mgrandi
Too many terms
-
JAA
snscrape = the actual scraper. socialbot = an IRC bot and wrapper around snscrape, interacting with AB and cb.
-
mgrandi
Ah ok
-
mgrandi
Dang, 87gb so far, hopefully that's the raw download size and it's not actually storing 8 million copies of the weird JS file
-
mgrandi
Of Twitter's big JS files*
-
JAA
Yes, that's download size without compression, and it includes outlinks, images, etc.
-
SketchTheCow
teamarchive1 thinks it's sending grafana data.
-
SketchTheCow
Up to Fusl to get it going.
-
SketchTheCow
Also, we should consider turning on its pipeline
-
Fusl
SketchTheCow: can you copy the telegraf config from teamarchive2 to teamarchive1 and just restart telegraf again?
-
SketchTheCow
Interesting. Where is it.
-
SketchTheCow
Found it
-
SketchTheCow
-
SketchTheCow
Now, turn on the pipeline, JAA< before you kill us all
-
JAA
Yay!
-
JAA
SketchTheCow: By the way, you said you forced another item for the remaining data from the test, right? I couldn't find that.
-
SketchTheCow
No, no
-
SketchTheCow
I mean I forced the remaining data into the item
-
SketchTheCow
Like, I took the 6gb lying over and slammed it into the 150gb item that was up there
-
SketchTheCow
So that if for some reason we all got distracted because Coup, we'd not have it sitting on teamarchive1 and people were wondering where the web saves were
-
JAA
Ah
-
FalconK
lol jaa, I think the ancient ubuntu is due to the fact that the machine is a one-off and can never, ever be rebooted. totally down to help change that.
-
FalconK
anyway the faulty RAM module has been narrowed down to 2 (I think I know which, but I'm not about to pass up their offer to replace both), and the parts are on order, after which we'll schedule the downtime
-
JAA
Ack
-
VADemon_
FalconK: if its consistent and you can run exhaustive memtests, you can exclude that RAM page from being used and keep the functioning stick of RAM
-
FalconK
I mean I could, but also the server is less than a month old and the warranty will take care of it.
-
FalconK
also it's 5 hours away by cessna and I don't really feel like going there right now :)