-
TheTechRobo
How do I download a large mediafire folder without coughing up a premium subscription?
-
TheTechRobo
I could manually select all he files but that would take ages...
-
TheTechRobo
It seems to use javascript, and doesn't load in netsurf, so I fed it into the wayback machine with outlinks enabled, since I'm pretty sure it runs javascript...
-
TheTechRobo
The individual pages were archived, but I don't believe the actual downloads were, since it saves outlinks at a level of 1
-
TheTechRobo
iirc
-
TheTechRobo
Is there a simple way to archive it?
-
JAA
plowshare's plowlist + plowdown, perhaps? Also, #mediaonfire should gain folder support soonish, so then we can archive it properly through that.
-
TheTechRobo
Where are those on Apt?
-
TheTechRobo
Nvm - found it.
-
TheTechRobo
Huh?
-
TheTechRobo
"Skip: no module for URL (
mediafire.com)
-
TheTechRobo
That's... really confusing
-
JAA
-
pabs
does the wayback machine archive bot really run JavaScript?
-
pabs
(if so it would be awesome if it could save the DOM at the end, so you can get a non-JS version of any web page, like you can with archive.is)
-
JAA
SPN doesn't, SPN2 does. SPN is /save/URL, SPN2 is the form thingy.
-
pabs
is savepagenow⊙ao SPN or SPN2?
-
TheTechRobo
I'm still getting the issue
-
JAA
I think that got introduced with SPN2, but not sure.
-
OrIdow6
That's SPN2 IIRC
-
JAA
(This is our nomenclature, by the way, not IA's.)
-
TheTechRobo
Sorry didn't mean to interrupt, didn't see the chat moving ^^"
-
TheTechRobo
I'm getting:
-
OrIdow6
And the DOM thing is (I presume) on everyone's mind, but standards develop and change slowly
-
TheTechRobo
% $HOME/.local/bin/plowlist
mediafire.com\?bmqi2md844aycSkip: no module for URL (
mediafire.com)
-
JAA
TheTechRobo: Try using the /folder/ID URL instead.
-
TheTechRobo
$HOME/.local/bin/plowlist
mediafire.com/folder/bmqi2md844ayc results in
-
TheTechRobo
Skip: no module for URL (
mediafire.com)
-
JAA
Then that sounds like you didn't install the modules correctly.
-
pabs
will /save/ get migrated to SPN2? (I mostly use that)
-
JAA
It's been a while since I used plowshare though.
-
JAA
pabs: You mean /save/URL or plain /save/ ?
-
pabs
/save/URL
-
JAA
Unlikely since that would break all sorts of things.
-
TheTechRobo
Ah, I missed that part of the section. I stopped reading at the "Advanced users" and "BSD" sections since I assumed from then on would only be things like troubleshooting
-
JAA
When you load a snapshot and the browser tries to access an image that hasn't been archived before, that gets a redirect to /save/IMAGEURL to transparently archive it, for example.
-
TheTechRobo
Yep, it now works
-
TheTechRobo
JAA: Oh, that would explain why it sometimes takes ages to load
-
TheTechRobo
a low-popularity website
-
JAA
Well, the WBM is also slow, but yeah, that can be a reason.
-
pabs
guess I need to switch to using /save/
-
TheTechRobo
They've been getting quite a lot of queries too it seems... I should really make a donation soon. Their servers keep returning "could not save target url due to system overload"
-
TheTechRobo
I can't seem to find the equivalent of the `-i` option... Where can I insert a filelist?
-
JAA
Don't think there is one. Try xargs?
-
TheTechRobo
Now I'm getting this...
-
TheTechRobo
-
TheTechRobo
plowdown is both reading the ones with the hashtag
-
TheTechRobo
AND it's getting an error when actually downloading
-
JAA
Actually, plowdown accepts a filename as an argument as well.
-
JAA
But you need to invoke plowlist with some option to get plain links without filenames IIRC. Check its man page.
-
TheTechRobo
Still getting parse failed (sed): "/function[[:space:]]*_/ s/^.*"\(.\+\)";.*$/\1/p" (skip 1)
-
JAA
Welp, looks like the MediaFire module has been broken for a while:
mcrapet/plowshare-modules-legacy #220
-
TheTechRobo
Dang it
-
TheTechRobo
Any alternatives to plowshare?
-
JAA
JDownloader perhaps? It's been even longer since I've used that though.
-
JAA
None of these tools will properly archive it (as WARC) though. Only our project would do that.
-
TheTechRobo
Regarding properly archiving: Shoot, yeah I just realised that.
-
TheTechRobo
Leave a message on my talk page once the mediafire project is implemented with folder support
-
TheTechRobo
and i'd either submit it to the urls.ajay.app
-
TheTechRobo
or i'd "borrow" some code :P
-
JAA
Just submit it now. I can guarantee nobody will notify you. lol
-
TheTechRobo
Oh, it's back up
-
Jake
I'll 'remember'
-
TheTechRobo
Nah it's fine now
-
TheTechRobo
I thought the site was still down
-
Ajay
yea, fixed it today
-
TheTechRobo
Nice website btw
-
TheTechRobo
Wish I could design like that
-
h2ibot
Tech234a created Using Heroku (+3703, Create Heroku page):
wiki.archiveteam.org/?title=Using%20Heroku
-
jodizzle
Vimeo account including popular videos being deleted:
twitter.com/fatalfarm/status/1416117865594249216. Might be good to grab.
-
Ryz
Inb4 archiving Vimeo? s:
-
Ryz
-
jodizzle
Oh, and here's a direct link to the account:
vimeo.com/fatalfarm
-
wizards
is there any better way to archive an unmigrated gamepedia wiki besides 'wget -r'?
-
wizards
ah, nevermind. "WikiTeam" has tools available for it, and it happens they've already archived the specific wiki i was looking to back up
-
h2ibot
Tech234a edited Using Heroku (+4, /* Required files for web deployability of…):
wiki.archiveteam.org/?diff=46989&oldid=46988
-
h2ibot
Tech234a edited Using Heroku (+2708, Add 30 second warning, example repo,…):
wiki.archiveteam.org/?diff=46990&oldid=46989
-
h2ibot
Tech234a edited Using Heroku (+350, Additional suggestions):
wiki.archiveteam.org/?diff=46991&oldid=46990
-
ats
.4
-
JAA
A Reddit thread just reminded me of the Supercell forums. They were discussed here in early May, but I forgot about them. They're read-only since 30 June and will go down in August. Will set up a qwarc grab now.
-
h2ibot
JustAnotherArchivist edited Deathwatch (+369, /* 2021 */ Add Supercell forums):
wiki.archiveteam.org/?diff=46992&oldid=46982
-
h2ibot
-
Ryz
There may be another new layout on the YouTube video pages~ oo;
-
Ryz
I wish I can can share a screenshot but I tend to instinctively clear out the cookies to get back to the previous layout <#>;