-
Ryz
JAA, for the logs, I suppose URLs only filter could be interesting for quickly finding stuff even faster; I rarely use it but some people might use it~
-
lennier1
tech234a: Cool site. Does crxcavator.io have a public list of all the apps/extensions/themes?
-
tech234a
lennier1: I'm not sure if they have a public list, but perhaps they could be asked for a list?
-
tech234a
api.crxcavator.io/v1/scans lists a few of the most recently scanned extension updates
-
JAA
Yeah, I was just playing around with that endpoint. Can't find a pagination parameter though.
-
JAA
The other endpoints require an extension ID.
-
tech234a
rss.crxcavator.io looks kind of S3-like
-
JAA
Well, there's a search endpoint, but it only returns 5 results.
-
tech234a
-
tech234a
RSS for extension versions
-
tech234a
If it is an S3 bucket could it be listed?
-
tech234a
They also list an email address: support⊙ci
-
JAA
Looks like they locked it down. Unless we can find the underlying bucket and that is misconfigured, unlikely.
-
tech234a
Alright
-
arkiver
JAA: any context on crxcavator?
-
arkiver
also ping tech234a ^
-
JAA
arkiver: Google nuking paid extensions in the Chrome Web Store. crxcavator.io is an index and archive of extensions or something like that.
-
JAA
Oh yeah, also nuking Chrome apps after that.
-
arkiver
ah for december 1st
-
JAA
Yeah and next year.
-
arkiver
June 2022 right
-
arkiver
I'll look into getting a project up for it
-
lennier1
June 2021 apps on Mac/Windows/Linux. June 2022 apps on Chrome OS.
-
arkiver
they did give us plenty of time
-
arkiver
so
-
arkiver
coming up are the other .ee sites
-
JAA
This is the S3 bucket that has the actual data, but also locked:
extensions.crxcavator.io
-
mgrandi
My2020Census.gov is probably changing on the 15th, probably has been picked up by way back machine a bunch but maybe do a archive bot run of it
-
arkiver
looks pretty quite for the rest of the year
-
arkiver
also reddit is still up
-
JAA
Docker Hub and Twitch Sings
-
lennier1
Might be some extensions with free trials or in-app purchases going away as soon as Demember 1. (And fully paid apps, which you could at least get metadata for.)
-
arkiver
yeah I'm not completely sure what we'll do on docker hub
-
arkiver
they're deleting PBs of data iirc
-
lennier1
There must be some process that these sites are using to scan the Chrome store, maybe an API for it?
-
JAA
Yes, 4.5 PB according to their FAQ.
-
mgrandi
Maybe start with the docker files first?
-
arkiver
yeah IA won't store that
-
arkiver
we can get metadata at least
-
arkiver
yeah and dockerfiles
-
mgrandi
Since those are the instructions to build said containers
-
arkiver
yep
-
arkiver
maybe
-
mgrandi
And you can reverse engineer after that
-
arkiver
maybe this is a good chance to get a copy of all docker metadata :)
-
mgrandi
Do we need 4PB of alpine linux
-
JAA
Maybe also the actual layers for official images and some other popular ones, though I'm not sure if those are affected at all.
-
arkiver
let's set up a channel
-
JAA
It'd be good to archive those regardless.
-
arkiver
any ideas?
-
mgrandi
Also, for twitch sings, should I look into archiving comments?
-
mgrandi
I don't know if there is a CLI way to get twitch vods and clips yet, there is a GUI program at least, but nothing for comments exists anywhere
-
mgrandi
Failwhale? Lol
-
mgrandi
A twitter-ism but docker is a whale
-
arkiver
:P failwhale sounds ok
-
lennier1
You have to set up a free Twitch developer account to use it, but there's this:
github.com/PetterKraabol/Twitch-Chat-Downloader
-
arkiver
JAA: did twitch use websockets for comments?
-
JAA
On live chat, yes. IRC through WebSockets.
-
arkiver
not archived that?
-
JAA
Not sure how it works on VOD.
-
arkiver
mgrandi: you have a twitch sings example?
-
mgrandi
Archived chat is JSON, not sure if it's web sockets or just long polling or what
-
mgrandi
-
mgrandi
Docker even has a failwhale icon just for us
-
arkiver
nice
-
mgrandi
I can get one later
-
mgrandi
But twitch sings are just normal vods and clips I think
-
JAA
#dick ?
-
arkiver
uh :P
-
arkiver
for twitch?
-
arkiver
of docker
-
arkiver
or
-
JAA
Well, apparently it's not obvious enough, so I guess not. :-P
-
JAA
I meant for Docker, via Moby Dick.
-
lennier1
Is there enough interest for a Chrome Web Store channel? #chromeweblore ?
-
lennier1
Surprised #dick wasn't already taken, lol.
-
arkiver
ah lol
-
arkiver
kinda liked failwhale
-
JAA
Sure, although yeah, it's usually associated with Twitter.
-
JAA
Not sure what happened with that purge of inactive accounts that was planned for late last year and then abandoned after Jason raised a shitstorm over the accounts of dead people that would be lost.
-
purplebot
Deathwatch edited by JustAnotherArchivist (-31, Sandboxie website to dead) just now --
archiveteam.org/?diff=45666&oldid=45664
-
benjins
mgandi: Twitch VOD chat is accessible through "api.twitch.tv/v5/videos/{VIDEO_ID}/comments?content_offset_seconds={SECONDS_SINCE_START}" but requires setting the Client-ID header to "kimne78kx3ncx6brgo4mv6wki5h1ko" which is what web clients use (it's not secret)
-
benjins
I just check the chat messages returned, take the highest offset, and use that to compute the next seconds offset to request. Not sure if there's some other way of getting all messages
-
benjins
mgrandi: ^
-
mgrandi
Cool, thanks
-
mgrandi
@JAA: they decided against it and haven't done anything further
-
mgrandi
Honestly it wouldn't be that hard to do what tumblr does and just rename the accounts and maybe keep a reference to the old username
-
mgrandi
But they gotta keep up the stereotype that they don't actually think about the stuff they are implementing lol
-
mgrandi
@benjins: is that their new api, I want to say it's called hydra or something?
-
benjins
No clue, I just poke at stuff in the network inspector until it works
-
benjins
There may have been some changes, but it's worked more or less the same way for a couple years
-
mgrandi
Ah, they are switching to a new api and apparently it's like no where near feature compatible with the old one, dunno how much that had changed
-
Fusl_
is there a channel for fotoalbum.ee?
-
Fusl_
the project doesnt have a wiki page
-
tech234a
#lookatthisfotograph
-
JAA
mgrandi: Last I heard was 'we won't be doing this until there is a way to memorialise accounts' or something like that, not that they abandoned it entirely.
-
purplebot
List of websites excluded from the Wayback Machine edited by Nikchemny (+52, Added this website) just now --
archiveteam.org/?diff=45667&oldid=45658
-
wessel1512
how long dus it takes to get the conformation email form hackint
-
JAA
Just a minute or two on the registrations I've done.
-
wessel1512
and thet it is
-
wessel1512
im registered JAA
-
purplebot
Coronavirus edited by Wessel1512 (+575, /* Information */) just now --
archiveteam.org/?diff=45672&oldid=45514