-
OrIdow6^2
So as much as I have been promoting the each-video-in-an-item option, there has been too low a number of people offering to run that for me to be confident it will be able to finish in time
-
OrIdow6^2
If @rkiver doesn't do it, I'll write a warrior script in the next 8h or so (unless something comes up), perhaps just a barebones one to get the video content
-
OrIdow6^2
To review, key thing for that is - API calls to get the video URLs and basic metadata - Full download if available - If not, the segments the m3u8 points to
-
OrIdow6^2
Don't know how many there are at this point, haven't gotten around to that
-
arkiver
hey I was pinged with a @ :)
-
arkiver
OrIdow6^2: talking about samsung right?
-
OrIdow6^2
Yes, Samsung XR/Samsung VR
-
arkiver
yep
-
arkiver
it's in progress actually
-
OrIdow6^2
Obviously meant not to ping you with that
-
OrIdow6^2
Oh
-
arkiver
no worries :)
-
OrIdow6^2
Good
-
arkiver
(and won't be using ZSTD btw due to videos)
-
OrIdow6^2
Yeah
-
arkiver
I'm off to bed now, but will try to have it running when I wake up
-
arkiver
the graphql stuff kinda sucks
-
purplebot
Elections/2020 September Swiss votes edited by JustAnotherArchivist (+139, /* Counter campaign by stark+vernetzt …) just now --
archiveteam.org/?diff=45535&oldid=45500
-
arkiver
I messaged themadpro on discord about the messages in #archiveteam
-
EggplantN_d
Understandable
-
EggplantN_d
Any update arkiver for Samsung?
-
EggplantN_d
That dies Monday? Or Wednesday?
-
arkiver
yes
-
arkiver
working hard on finishing it
-
arkiver
will start in a bit
-
EggplantN_d
Excellent
-
EggplantN_d
It's Sunday I plan on being drunk
-
EggplantN_d
Do you have a tracker page yet? With IA variables?
-
arkiver
can I get those to you in an hour?
-
arkiver
don't have a list of items yet, btw
-
» arkiver is back in 30
-
EggplantN_d
Can you do 45?
-
EggplantN_d
I plan on being in a pub in 68 minutes
-
SketchTheCow
I wrote a script called PHOTOFINISH which will, given an Archiveteam collection with no header images in the items, pull from a pool of them and fill them in.
-
EggplantN_d
arkiver: to help i've made
tracker.archiveteam.org/samsung-xr and added a target to it. Target is configured for rsync + chunking. If you have a FILE_PREFIX i can at least make it pack + even offload for now till IA is available
-
EggplantN_d
or IA vars should I say
-
arkiver
-
EggplantN_d
docker factory online
-
arkiver
perfect
-
arkiver
thanks
-
arkiver
enjoy the beers :)
-
EggplantN_d
I am doing
-
SketchTheCow
Quick question. Why did we stop adding to archiveteam_newssites
-
SketchTheCow
Maybe you know, Fusl
-
SketchTheCow
(Name is on some of the uploads). We stopped around July 4, 2019
-
Fusl
thats newsgrabber?
-
SketchTheCow
I assume so
-
SketchTheCow
I'm just doing some cleanup of the inbox, pushes me into a lot of nooks
-
Fusl
i believe the issue we had was deduplication
-
SketchTheCow
So we stopped dead?
-
SketchTheCow
I mean, OK, but just checking
-
Fusl
arkiver? JAA? jrwr?
-
Fusl
ideally we start doing this project again sometime soon when there's not as much going on
-
JAA
Yes, Newsgrabber was stopped after we uploaded like 200 TB of undeduplicated stuff.
-
JAA
Because our dedupe broke for a couple weeks without anyone noticing.
-
JAA
IA wanted us to use the WBM CDX API for deduping after that, which fell over immediately.
-
EggplantN_d
Happy to assist as always
-
SketchTheCow
Well, luckily no news of note has happened since July 4 2019
-
Fusl
blackbird would be a good candidate to host the deduplication database, there's still 142gb of memory free there and lots of nvme disk space
-
Fusl
or OSS3
-
SketchTheCow
-
SketchTheCow
This is a seriously huge amount of data
-
SketchTheCow
There's 212 items in the inbox now (down from 13,000 recently) and I'm making little collections for them now.
-
JAA
We absolutely should resume Newsgrabber with our own dedupe. Some major US news outlets are covered by IA's own crawls sometimes, but not nearly as thoroughly as NG did, and there's all the smaller/local and foreign stuff as well.
-
JAA
Well, #newsgrabber exists, so let's take that there.
-
arkiver
yeah newsgrabber was paused after uploading 300 TB of not deduplicated records
-
SketchTheCow
Fight the power!
-
purplebot
-
arkiver
samsung xr project is ready, will start when tracker is back to normal
-
HCross
oh boy
-
HCross
channel?
-
EggplantN_d
Surely we have an extinction rebellion pub?
-
arkiver
just -bs
-
EggplantN_d
*pun*
-
EggplantN_d
Because XR
-
arkiver
sure
-
arkiver
i dont :P
-
EggplantN_d
Me either
-
EggplantN_d
But surely one exists
-
JAA
#sandsung, but not sure if that's obvious enough.
-
HCross
arkiver: tracker is back
-
HCross
ready when you are
-
EggplantN_d
My target is already ready
-
EggplantN_d
Please send multiple gigabit
-
purplebot
Wikipedia edited by M.Barry (+885, /* External links */ Adding link …) just now --
archiveteam.org/?diff=45537&oldid=38527
-
EggplantN_d
Any news arkiver
-
arkiver
yeah
-
arkiver
tracker is sort of not working
-
arkiver
/admin/ page
-
EggplantN_d
I see
-
arkiver
:/
-
arkiver
yeah
-
arkiver
but
-
EggplantN_d
That is a problem.
-
arkiver
samsung xr is shutting down the 30th, not 28
-
EggplantN_d
Even better. Is the code "working"
-
arkiver
code is working
-
EggplantN_d
As in deployable awaiting for tracker?
-
EggplantN_d
Assuming Fusl is aware
-
arkiver
Only update it coming up for channels, but those will initially abort because not supported yet
-
arkiver
so one update coming up
-
arkiver
but if we start this tomorrow I think we should still be fine before the 30th, it's not too big
-
EggplantN_d
Okay that's fine
-
EggplantN_d
I'll have people deploy code ready
-
EggplantN_d
Let's get this project down
-
EggplantN_d
*done
-
arkiver
yeah one update coming up
-
EggplantN_d
Was Ever archived
-
EggplantN_d
Another one for ShutdownSeptember
-
EggplantN_d
30th
-
JAA
Ever is a personal photo storage thingy, can't be archived.
-
JAA
As I understand it, anyway.
-
JAA
Also, the site is already dead. 30 Sept is just the deadline for exporting your own pictures.
-
JAA
But yeah, don't think there ever was public content.
-
EggplantN_d
Ah okay
-
EggplantN_d
No worries
-
EggplantN_d
Just wanting to check off that list
-
JAA
:-)
-
EggplantN_d
I know HCross has done Dell
-
EggplantN_d
So Google sites and Samsung XR
-
EggplantN_d
Then twitch sings
-
EggplantN_d
Sooner those 3 are done the better
-
JAA
Yeah, there'll be a lot more shit before the end of the year.
-
EggplantN_d
Absolutely and I'm hoping we can get done before EOY shutdown
-
purplebot
Deathwatch edited by JustAnotherArchivist (+23, Move Citizendium to reprieves) just now --
archiveteam.org/?diff=45538&oldid=45524
-
Fusl
arkiver: tracker is functioning again
-
EggplantN_d
Excellent.
-
purplebot
Elections/2020 September Swiss votes edited by JustAnotherArchivist (+1078, Add federal results pages) just now --
archiveteam.org/?diff=45539&oldid=45535