-
Pedrosso
I'd like to request
replaymod.com for AB due limited IA and no AB coverage and
replaymod.com/forum being closed
-
ctag
thuban: I don't believe I'm looking for any server functionality. I just want to preserve the content and presentation style.
-
ScenarioPlanet
-
ScenarioPlanet
-
thuban
ctag: what i said about grab-site and wget should stand then, although people might have other recommendations wrt wget flags (i haven't done this with it in a while)
-
ctag
OK, thank you for the advice. Are you referring to wget or wget-at?
-
ctag
Wait, the distinction was in how they handle warcs. Nevermind.
-
pabs
Pedrosso: threw it in. any subdomains?
-
Pedrosso
not as far as I know. One problem is
replaymod.com/center becuase I have no clue how to look through that without manually doing search terms
-
pabs
Pedrosso: please send the GitHub to #gitgud
-
Pedrosso
just send the url, no context or !a ?
-
pabs
url + context
-
pabs
(no bot yet, J_A_A will queue it when around)
-
Pedrosso
gotcha
-
pabs
Pedrosso: re replays, we can just generate a URL list and !ao < all of them
-
Pedrosso
Cool, but how'd one generate a URL list for that?
-
pabs
-
pabs
or printf or a bash loop
-
pabs
for eg: echo
replaymod.com/replay{1..15567} | tr ' ' '\n' > www.replaymod.com-replay-1-to-15567.txt
-
pabs
-
eggdrop
-
pabs
found the max 15567 by binary search - put in 20000 and got a redirect to /center then halved it etc
-
Terbium
praise be to devs who use ascending integer ids
-
pabs
hmm, some of the early ones redirect too...
-
fireonlive
i wonder if there’s some json or something to download as well somewhere
-
fireonlive
*uses random IDs on everything >:D*
-
pabs
hmm, it uses custom links to download: replaymod://15567
-
fireonlive
looks like
-
fireonlive
-
Pedrosso
Oh, cool! I didn't notice it had enumeration
-
Pedrosso
I should've known
-
Pedrosso
Thank you pabs
-
fireonlive
brb submitting a patch to them to use uuidv4 for everything
-
fireonlive
with aggressive rate limits
-
Terbium
at least they don't use cloudflare which is nice
-
Pedrosso
pabs: I don't think it is abandoned considering that there are downloads for 1.20.4 up on the site
-
Pedrosso
the site however... Other than the downloads page it is possible
-
» nicolas17 stabs fireonlive
-
fireonlive
thank you
-
pabs
Pedrosso: I'm going by the forum topics about how it is discontinued
-
pabs
-
DigitalDragons
replaymod is awesome and definitely not abandoned
-
pabs
hmmm, they made a release after that
-
fireonlive
i don't quickly see a way to get the mprc files?
-
fireonlive
DigitalDragons: do you know where they're stored/anhything about that?
-
fireonlive
there's
minio.johni0702.de with an expired cert (23d ago)
-
DigitalDragons
I don't, i've only ever used it for my own replays
-
Pedrosso
and it seems like in future iterations they've completely removed these online features
-
Pedrosso
replaymod-1.12.2-2.2.0-b1 still has them
-
fireonlive
ah ok
-
ctag
Hmm. OK, wget -mpk did its best, but I think I need to do something to account for the site being PHP. Links from the index page (which looks correct!) don't work
-
ctag
I wish I could just flatten it down to HTML pages
-
ctag
No need for PHP, nothing on the site is going to change
-
pabs
just save it to web.archive.org using AB? :)
-
ctag
I'd like to learn how to do that
-
ctag
It's been saved partially and manually on there, but I'd also like to have a copy I can host
-
ctag
I'm their volunteer webmaster, and the organization was started in the 1950s, they have a lot of physical archival documents
-
ctag
Would be neat to keep this copy with that stuff
-
ctag
AB is archive bot, right?
-
fireonlive
indeed it is :)
-
fireonlive
not anyone can use it though but you can ask for someone to archive a particular site
-
ctag
Thanks
-
ctag
Oh
-
ctag
I'm guessing there isn't a way to archive it under the original domain?
-
ctag
I'm hosting the copy on a subdomain of my personal stuff
-
ctag
I should have thought about this more before we replaced the website
-
pabs
ctag: not unless the domain is publicly accessible
-
pabs
we can also just do the subdomain
-
ctag
The rehosted version is
vbas-legacy.berocs.com
-
ctag
Which was originally
vbas.org
-
pabs
do you still own the original domain?
-
ctag
-
ctag
Yes, partly
-
pabs
ah, vbas.org is now a new website
-
ctag
It's tied to a company that donates our hosting
-
ctag
Yeah :-/
-
ctag
If it'd help, I could try and schedule a time to revert the vbas.org domain to the old site temporarily.
-
ctag
I believe the files for the old drupal site are still on the server
-
ctag
No, I'm remembering now, one of the reasons we switched was the PHP version updated and broke drupal
-
pabs
www.vbas.org redirects to vbas.org, what about temporarily pointing the www subdomain to your server, running AB and then reverting?
-
ctag
Changing the PHP version back would break the new site, I think
-
ctag
Maybe? I'll look into it
-
ctag
That's a great idea though
-
pabs
alternately, add an obsolete/old/legacy subdomain to vbas.org pointing at your server, save that in AB
-
pabs
or just save the rehosted domain
-
w
hello, i wanted to download a track from the artist union archive but couldn't get past the login prompt
-
w
could i get some help?
-
ctag
I'm apparently not smart enough to get the redirect working. Will take a look at it this weekend, thank you again for the advice pabs.
-
fireonlive
ctag: shouldn't be a 301 or iframe but something like www.vbas.org points to your IP then your server is configured to serve the old content at that hostname
-
fireonlive
(in addition to (or temporarly instead of) vbas-legacy.berocs.com)
-
Pedrosso
-
Pedrosso
For downloading the replaymod files
-
Pedrosso
-
eggdrop
-
pabs
!ao < since !ao just downloads that URL
-
Pedrosso
Ah right the < for the file contents
-
pabs
running, getting some weird errors
-
Pedrosso
I see that
-
Pedrosso
Any idea of what the error means?
-
pabs
none, probably some sort of bug in the script
-
Pedrosso
Wow
-
Wickerz
Hi guys! Trying to find the channel specific for ArchiveTeam Warrior questions. Any hints on how to get there? :)
-
Vokun
#warrior
-
Vokun
Thanks for stopping by
-
Wickerz
Thank you! :)
-
pabs
Pedrosso: to be clear, I mean a bug in the PHP script running on the replaymod server. its clearly throwing PHP warnings before doing any HTTP header output, which means bugs in the script
-
Pedrosso
Ahh
-
pabs
possibly the error is triggered by AB (I can't seem to get the errors here), but its a bug in the script
-
Pedrosso
I can't get it to trigger either
-
pabs
we can rerun the weird-failure and conn closed ones after its done and see what happens
-
JAA
pabs: `printf '%s\n'
replaymod.com/replay{1..15567}` :-)
-
JAA
printf is much better than echo anyway.
-
Ryz
Is it me or did YouTube took out the ability to see YouTube accounts' subscription pages now? I started noticing it around this month or last month and thought it might be a bug
-
Ryz
To note, public pages~
-
Ryz
Either way, that's more lost metadata I fear~
-
thuban
< ctag> Hmm. OK, wget -mpk did its best, but I think I need to do something to account for the site being PHP. Links from the index page (which looks correct!) don't work
-
thuban
i think `-E` handles this
-
thuban
(sorry, this is why i was hoping someone would double-check my flags!)
-
Ryz
Yeah, it looks like it might be confirmed, can't see such a thing anymore:
old.reddit.com/r/youtube/comments/1…t_seriously_remove_the_channels_tab
-
Ryz
For reference, here's an example of a Channels/public subscriptions page:
web.archive.org/web/20210624044052/…ww.youtube.com/c/vinesauce/channels
-
cyrix
hey all, gamebattles is shutting down in a little over two weeks. There's a ton of info on tournaments, players, and matches. I've had a very slow long term scrape going way before they announced this but I fear it won't finish in time. There seems to be a pretty good rate limit in place and when you hit it they basically ban your ip. I might have
-
cyrix
to resort to getting one of those proxy services and just churning through those, but thought it could be a good warrior project too. I'm not really on here frequently but am on discord at cyrlx. Here's a tweet about the shutdown:
twitter.com/GameBattles/status/1724171598117101830
-
eggdrop
-
h2ibot
Switchnode edited Deathwatch (+0, /* 2024 */ correct gamebattles date):
wiki.archiveteam.org/?diff=51422&oldid=51394
-
thuban
js hell, this'll be fun.
-
thuban
cyrix: any information you can give us on site/api structure (including your existing scrape program, if you'd care to upload it to transfer.archivete.am) would be helpful
-
Exorcism
Can eggdrop change its nitter to this one :
nitter.mint.lgbt :3
-
VickoSaviour
i agree with cyrix and thuban, that would be a good project to work on.
-
ctag
thuban: Thank you, I'll give wget another shot this afternoon.
-
ctag
The redirect looks to be working from my end!
vbas.org
-
cyrix
@thuban I can clean up my scripts and share them there...their id systems are just incrementing integers so I largely just keep going up on certain endpoints
-
thuban
cyrix: sounds great, thank you!
-
h2ibot
JustAnotherArchivist edited Deathwatch (+279, motor-talk.de got a reprieve):
wiki.archiveteam.org/?diff=51423&oldid=51422
-
h2ibot
JustAnotherArchivist moved GuteFrage to Gutefrage (Fix capitalisation; although the German phrase…):
wiki.archiveteam.org/?title=Gutefrage
-
h2ibot
JustAnotherArchivist created Gutefrage.net (+23, Former official branding of [[gutefrage]]):
wiki.archiveteam.org/?title=Gutefrage.net
-
h2ibot
JustAnotherArchivist edited Gutefrage (+112, Fix capitalisation; it's been 'gutefrage.net'…):
wiki.archiveteam.org/?diff=51427&oldid=51424
-
h2ibot
JustAnotherArchivist edited Quora (+17, + [[Category:Q&A]]):
wiki.archiveteam.org/?diff=51428&oldid=49870
-
JAA
Exorcism: It could randomly return either that or
nitter.x86-64-unknown-linux-gnu.zip for extra nerdiness. :-)
-
cyrix
thuban just uploaded them:
-
thuban
cyrix: er, link?
-
Exorcism
Oooo :3
-
cyrix
thuban
-
cyrix
-
cyrix
parallel.py
-
eggdrop
-
cyrix
-
eggdrop
-
nicolas17
eek
-
thuban
cyrix: thanks!
-
JAA
You tried, eggdrop. You tried.
-
project10
:-)
-
JAA
-
eggdrop
-
JAA
Er
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot