-
Jonimus
There are some reports of some people still being able to access it so the server may still be up and its only the domain that has reverted.
-
nicolas17
ugh
-
JAA
Jonimus: Do you know the server's IP?
-
nicolas17
yeah if you figure out what IP address it used to have
-
JAA
Ah, it's in DNS History.
-
Jonimus
I'll check with the discord I found out about the issue from.
-
JAA
<html><head><title>rrpicturesarchives.net</title></head><body><h1>rrpicturesarchives.net</h1><p>Coming soon.</p></body></html>
-
JAA
Welp
-
nicolas17
/o\
-
Jonimus
ahh mybad
-
Jonimus
rrpicturearchives.net
-
Jonimus
My copy paste was failing on my IRC client :(
-
JAA
Ok, that looks better, and yep, still up.
-
JAA
208.69.231.186
-
JAA
Pretty slow though
-
Jonimus
From what people in the railfan discord I got the info from said is the site had rate limiting in later years due to server load.
-
nicolas17
well if there's multiple people trying to archive stuff at the same time, it's going to make things worse
-
Jonimus
I don't believe they are trying to archive it, most of the people in that discord can't access it anymore and aren't trying.
-
Jonimus
They would like to, but most of them are not tech savvy.
-
JAA
Ew, ASP.NET
-
Jonimus
also that.
-
JAA
148k albums with over 6 million pictures, apparently.
-
nicolas17
oh my
-
JAA
And wpull won't be able to handle it correctly because some links have backslashes.
-
JAA
<img src="/pictures\147919\thumbnails\IMG_9205.JPG"
-
nicolas17
Microsoft IIS moment
-
JAA
At least it looks like the ASP.NET form crap isn't actually used.
-
JAA
Oh, nevermind, it is on some pages.
-
JAA
E.g. archiveList.aspx
-
JAA
But the images can be retrieved without it, I think.
-
Jonimus
The images would be the main desireble thing I believe.
-
nicolas17
yeah and I guess we can grab archivethumbs.aspx?id=$number for all numbers without needing to discover them by crawling
-
nicolas17
easy to enumerate
-
JAA
Yeah
-
Jonimus
Is there anything else I can help with/ask of the regular users of the site? Or should I just let you see what you can do?
-
nicolas17
fullpwnmedia: do you think that dynabook ftp actually has a chance of changing soon? it was put into archivebot, and I archived it too, but keeping a *local* copy to sync and check for changes is burning a hole in my hard disk atm
-
JAA
Hmm, I see comments as well.
-
nicolas17
I think I'll delete the yahoo groups stuff too (deleting 500k files will take a while even on SSD...)
-
JAA
rrpicturearchives.net/showPicture.aspx?id=260097 has 30 comments and no pagination. Can't find a bigger one right now.
-
JAA
I see that they also had 'contributor sites' on subdomains, e.g.
railfanblaise05.rrpicturearchives.net
-
nicolas17
same IP?
-
JAA
Yeah
-
JAA
Same content too, just filtered down to uploads by that user, it seems.
-
JAA
But all served through the subdomain. :-|
-
pabs
immibis: I threw it into ArchiveBot
-
nicolas17
ok I added an 'at' job to delete my "tb2b" in 24h if nobody stops me before then :P
-
pabs
immibis: sadly the AB job completely failed, need an op to expire and retry
-
pabs
joepie91|m: ^
-
immibis
how does AB handle rate limits?
-
immibis
curseforge is very much an operation to make lots of money with ads and user data, so I doubt there are no rate limits
-
pabs
it has a concurrency setting and a request delay setting, might also handle 429, not sure
-
JAA
429s aren't handled specially.
-
pabs
do they get retried at least?
-
JAA
Yes
-
pabs
immibis: from #archivebot, it is apparently cloudflare and hard to archive. JAA rescheduled it on a pipeline where it might work.
-
JAA
Yeah, I tried to grab it in December, when they announced the deprecation and someone brought it up here, and that failed.
-
Jonimus
JAA: is that rrpicturearchives.net site a doable project or is there anything else I can do to help, the estimates I am seeing is about 10TB of photos.
-
JAA
Jonimus: I don't suppose there's any hint at how long the server will last?
-
Jake
(re: curseforge, I will try again, but I believe it didn't work last time.)
-
Jonimus
Nope, apparently the admin passed a little over a year ago.
-
Jonimus
I can see if anyone in the railfan groups I know of know the family or could reach out but I'm hesistant to do that.
-
Jonimus
I didn't personally know of the site until I saw people mentioning it as down and thought it might be good to bring it up to ya
-
Jonimus
y'all
-
JAA
Definitely, seems like a very nice resource and a shame to lose.
-
nicolas17
the images are in S3
-
JAA
Some, but not all of them.
-
nicolas17
oh yeah just saw some that aren't
-
nicolas17
guess I'll have to learn to use wget-at lua scripting
-
TheTechRobo
nicolas17: It's not too difficult, FWIW:
github.com/ArchiveTeam/wget-lua/wiki
-
pokechu22
Re curseforge it looks like
authors-old.curseforge.com/forums is also going away
-
nicolas17
TheTechRobo: JAA:
rrpicturearchives.net/archivethumbs.aspx?id=147904 this album had new pictures added *right now*
-
Jonimus
The DNS going down hasn't propogated to everyone yet.
-
Jonimus
Some users may not be aware its going down.
-
Jonimus
Also some people are still accessing via the IP trying to grab their own photos or their favorites etc.
-
nicolas17
view counters are going up on some images too
-
Jonimus
Yeah, that doesn't surprise me, there is likely a number of people trying to grab specific albums etc.
-
JAA
Yeah, you can access the site directly via IP as well, don't neven need /etc/hosts et al.
-
Jonimus
Does that IP tell you anything about how it was hosted, the sites owner worked in security, possibly cyber so for all we know its a box he had in the corner at work or similar.
-
nicolas17
oh I didn't check
-
nicolas17
the whole IP address block is registered under his name :|
-
JAA
Apparently hosted at
dartpoints.com
-
nicolas17
CIDR: 208.69.231.184/29
-
nicolas17
NetName: TIMHU001
-
nicolas17
OriginAS: AS15085
-
nicolas17
Customer: Tim Huemmer (C03350945)
-
Jonimus
The guy who passed was named mike so that must be who owns dartpoints
-
nicolas17
I was going off the page footer
-
nicolas17
"Site Design ©2001-2020 Tim Huemmer"
-
JAA
It does look like a 2001 design. :-)
-
Jonimus
-
Jonimus
Thats the obit for the guy who ran it from what I've seen.
-
nicolas17
welp
-
nicolas17
-
nicolas17
looks like we're gonna need IPs
-
nicolas17
images aren't affected, only aspx
-
myself
or can you just write to the Tim guy and explain you're trying to preserve the site and does he have any knobs to turn?
-
nicolas17
I *did* do an excessive number of requests :P
-
nicolas17
myself: I'm getting 503 Service Unavailable now
-
nicolas17
including for pictures
-
nicolas17
can you reproduce?
-
nicolas17
or did my IP get blocked at another level now?
-
nicolas17
ok tried from VPS, 503 there too... did I kill the site?
-
nicolas17
now connection refused on port 80
-
nicolas17
this looks a lot like someone actively messing with the server
-
nicolas17
sorry Tim Huemmer, I won't do that again, plz bring site back
-
nicolas17
if *my* requests caused high load and raised some alert that made someone go "oh this server is causing it, isn't this the customer that hasn't paid in a year?" and turn it off I'm going to die of guilt
-
myself
if that's all it took, nobody was gonna be able to archive it anyway
-
JAA
Oof
-
myself
"Hey if a few railfans pool a few bucks to pay this guy's hosting bills, can you stand the server back up long enough to archive it?"
-
nicolas17
myself: "nobody was gonna be able to archive it anyway" I downloaded a thousand /archivethumbs.aspx... multiple times, concurrency 10 each time, I could have certainly been more subtle about it 😓 I didn't expect anyone would be watching
-
myself
lmao
-
Terbium
it'll be funny if there was a firewall or IDS in front that treated the high level of requests as a DDOS and null routed the server
-
nicolas17
Terbium: port 80 is giving an active "connection refused", and port 443 is giving a sonicwall firewall login like it was before
-
nicolas17
also, first I got this
transfer.archivete.am/KAuSj/response.txt (almost certainly per IP and automated), then I got "503 Service Unavailable" (at home and at my VPS), *then* it escalated to "connection refused", sure looked like manual intervention
-
Jonimus
Do you think it would be better if someone from archiveteam or archive.org tried to contact Tim or if someone from the railfan community?
-
Jake
As of May 9th, that response was being returned for some people already.
webcache.googleusercontent.com/sear…008336586&cd=10&hl=en&ct=clnk&gl=us
-
JAA
I did see that 'Excessive Usage Error' on some search results as well.
-
nicolas17
yeah I'm sure that was automated and affecting my IP alone, and the limit has been there for a while
-
nicolas17
the error page had "Last-Modified: Fri, 10 Dec 2010"
-
nicolas17
but it feels like after I was already blocked, someone took manual action
-
JAA
It's timing out for me now.
-
nicolas17
I still get connection refused, but sometimes it takes several seconds
-
JAA
And now I'm able to connect, but the server doesn't respond to the HTTP request.
-
nicolas17
-
JAA
Ah there we go, got a response again.
-
Jonimus
It just worked for me, so I suspect the issue is multiple people were trying to download stuff and the firewall or manual intervention is happening.
-
nicolas17
everything works again now
-
nicolas17
I'm *not* going to do that request rate again
-
nicolas17
geez
-
nicolas17
okay WOW
-
nicolas17
JAA:
-
nicolas17
- <img src="/pictures\147904\thumbnails\050921 Perry (41).JPG" border="0" alt="UP 5488">
-
nicolas17
+ <img src="
s3.amazonaws.com/rrpa_photos/147904/thumbnails/050921 Perry (41).JPG" border="0" alt="UP 5488">
-
Jonimus
Wait is it actively being moved to s3 or something?
-
JAA
Huh
-
JAA
Yeah, looks like it.
-
Jonimus
Or maybe some sort of s3 using caching setup is being used?
-
pokechu22
Better local to s3 than the other way around :P
-
JAA
This would be a weird way of doing it but certainly not the weirdest.
-
JAA
Shedding load would be another possibility.
-
JAA
But again, weird.
-
JAA
Serving static files is one of the easiest things a web server can do.
-
JAA
Maybe IIS sucks at that though, who knows. It's Microsoft, after all.
-
nicolas17
well
-
nicolas17
the S3 image above
-
nicolas17
Date: Mon, 22 May 2023 04:12:08 GMT
-
nicolas17
Last-Modified: Mon, 22 May 2023 04:03:01 GMT
-
nicolas17
sure looks like it was recently uploaded to S3
-
Jonimus
The dartpoints or whateve that owns the IP is some cloudy "edge colocation
-
Jonimus
service, it could easly be some system they have doing the load shedding or whatever.
-
Jake
That's...w eird...
-
Jonimus
Yeah it is weird to move things from local to s3, that said the does mean the paths are now forward slashs which is I think better for your tools isn't it?
-
nicolas17
not *all* were moved
-
nicolas17
in fact I even see albums with a mix
-
JAA
The other possibility is that they're actively migrating everything to a new site or similar.
-
nicolas17
JAA: from the initial description of the situation, I didn't expect to find someone alive to do that
-
JAA
Yeah
-
Jonimus
Unless the tim guy is doing it, he may not have had access to the domain to renew it but may still be trying to keep the site up, and messing with it for that reason?
-
nicolas17
yeah probably him
-
nicolas17
if he's aware the domain expired *and* that some people are still accessing (someone uploaded new pictures a few hours ago!), he should put some notice on the front page...
-
Jonimus
You'd think he'd have an admin account since he designed it but maybe he doesn't for reasons.
-
Jonimus
Like depending on how the site was built in 2001 it may not be the easy to just update the homepage.
-
JAA
'Please install Microsoft FrontPage 2000'
-
Jonimus
Wait the main pages "updated photo albums" all list today.
-
Jonimus
-
nicolas17
Jonimus:
208.69.231.186/archivethumbs.aspx?id=147904 this album got new pictures *after* you told us about the site and I started looking into it
-
Jonimus
Yeah so either people are uploading photo's by connecting via the IP or their DNS hadn't updated.
-
Jonimus
I don't think there were any like phone apps or similar.
-
nicolas17
if there was a phone app, I'm sure it would depend on the domain working...
-
Jonimus
You'd think.
-
arkiver
nicolas17: what is tb2b?
-
arkiver
JAA: is rrpicturearchives.net somethig for archivebot?
-
nicolas17
-
arkiver
nicolas17: do you perhaps have a tl;dr of the above conversation?
-
nicolas17
regarding rrpicturearchives?
-
JAA
arkiver: rrpicturearchives.net uses backslashes in its image URLs, which fail on AB. Also, the domain expired, so DNS trickery is needed if we want it under the domain rather than the IP.
-
arkiver
yeah
-
arkiver
-
arkiver
(without altering DNS results)
-
nicolas17
yeah what JAA said
-
arkiver
nicolas17: if you have the only copy of that, please upload it ot IA
-
arkiver
on rrpicturearchives.net - what exactly did it hold?
-
JAA
Millions of train photos
-
nicolas17
arkiver: of the tb2b FTP? no, it was archivebot'd successfully
-
arkiver
nicolas17: ah good
-
JAA
I can throw
208.69.231.186 into AB and then deal with the backslashes when it finishes, I guess.
-
JAA
Unless we want to do DNS fuckery and archive it under the expired domain.
-
nicolas17
and when I remember I "rclone sync" against my local copy and I haven't seen the contents actually change
-
fireonlive
1-2 registrars allow anyone to renew any domain but sadly not godaddy it seems
-
arkiver
we _could_ archive it under the expired domain, but that will not go into the Wayback Machine
-
JAA
fireonlive: I'd like to know which ones, so I can add them to my 'never, ever use' list of registrars.
-
arkiver
JAA: nicolas17: i'd say, get
208.69.231.186
-
arkiver
as that IP, not under the old domain.
-
fireonlive
from what i quickly found it's just Hover
-
arkiver
archives of that IP (without DNS trickery) can go into the Wayback Machine
-
JAA
arkiver: There is precedence for such archives going into the WBM. But it's not ideal, sure.
-
arkiver
DNS trickery cannot
-
fireonlive
customers were like 'i'm locked out of my account' or 'person X is unavailable' so they were like 'sure we'll take the money but the owner retains control'
-
arkiver
JAA: is it small (and easy) enough for archivebot?
-
nicolas17
there may be 6 million photos in rrpicturearchives
-
JAA
5.6M is what the homepage says, yeah.
-
arkiver
oh they have the easy sequential IDs
-
JAA
The content can mostly be gotten easily with AB, yeah.
-
arkiver
can we just !a < a list of these URLs in archivebot together with the main page?
-
JAA
Some of the navigation is ASP.NET POST nonsense.
-
nicolas17
JAA: I was going off a photo ID being 6025791, but there may be gaps I guess
-
JAA
Yeah, the IDs go higher. Probably some deleted stuff etc.
-
JAA
arkiver: Yeah, will do that shortly.
-
JAA
What's the highest locomotive ID?
-
nicolas17
as in locoPicture.aspx?
-
JAA
Yeah
-
JAA
Oh nice, capitalisation bullshit from Microsoft, of course.
-
JAA
Locopicture.aspx and LocoPicture.aspx appear in links across the site.
-
nicolas17
/o\
-
arkiver
luckily the Wayback Machine handles that :P
-
» JAA slaps arkiver around a bit with a large trout
-
JAA
AB doesn't, anyway, so it will retrieve those things multiple times.
-
arkiver
maybe we should fix AB? :P
-
arkiver
(that was a joke - to be clear)
-
fishingforsoup__
I need help finding some YouTube videos.
-
fishingforsoup__
-
JAA
I mean, we should, URL rewriting has been on my wishlist for a while.
-
JAA
Looks like the highest loco ID is slightly above 265700.
-
JAA
Nope, exactly that.
-
nicolas17
I just tried 265700..265800 and 265700 was the only successful one
-
nicolas17
so yes
-
arkiver
i need to open source my ID range scanning thing some time soon
-
arkiver
it can be used for this stuff
-
nicolas17
binary search?
-
arkiver
though basically it's the same as is in the telegram-grab Lua code
-
arkiver
nicolas17: what?
-
arkiver
and the same thing that is used in #telegrab to find highest ID for channels that don't have a public index
-
nicolas17
I guess you do something like binary search to find the last valid ID?
-
JAA
I'll first do the photo pages in random order, then albums, then locomotives.
-
arkiver
nicolas17: sort of
-
arkiver
not exactly
-
JAA
nicolas17: You can't do a binary search if you don't know the possible upper end. And you can't simply start at a gazillion because it takes too long then.
-
nicolas17
yeah, if you have to cope with potential gaps it's trickier
-
JAA
Gaps as well, yeah.
-
JAA
The 'Excessive Usage Error' will be annoying since it's served with HTTP 200.
-
JAA
Anyway, AB job is started.
-
masterX244
JAA: ASP post request pagination is the pest... had that shit once, too when grabbing the tm-exchange sites. Bonus: bugged server where you get a 500 in the middle and unloadable pages + a ipban. Had to resort to TOR for bruteforcing id-s to do a crawl from a fresh IP
-
JAA
For the record: `{ echo
208.69.231.186; seq 6025791 | shuf | sed 's,^,
208.69.231.186/showPicture.aspx?id=,'; seq 147920 | shuf | sed 's,^,
208.69.231.186/archiveThumbs.aspx?id=,'; seq 265700 | shuf | sed 's,^,
208.69.231.186/locoPicture.aspx?id=,'; } | zstd -10`
-
nicolas17
does archivebot use warriors or otherwise parallel requests across IPs, or is it like 1 machine?
-
Maakuth|m
there is a handful of servers sharing the load
-
JAA
Each job is a single process on a single machine.
-
masterx244|m
the warrior is the "big gun", we only use it on major targets
-
masterx244|m
usually big sites that can bear the load
-
JAA
nicolas17: Welp, the AB job is also in 'Excessive Usage Error' hell now.
-
JAA
I wonder whether it really is a daily limit or not.
-
JAA
And now it's getting ECONNRESET.
-
masterx244|m
drats...
-
masterx244|m
sometimes sites are more triggerhappy if you poke on too many 404 or 403s
-
JAA
It happened after pretty much exactly 10k requests.
-
masterx244|m
603 chunks on showPicture if splitting it in 10k blocks; 15 chunks on archiveThumbs and 27 chunks on locoPicture; maybe someone with the crazy clusters can move the stuff around between his IPs so after each IP gets burned it gets continued at a fresh one
-
trumad|m
apologies for post non-urgent stuff in the main channel. I'm still getting used to how things work
-
h2ibot
Bzc6p edited EOldal (+353, /* Archiving */ Archives finished uploading):
wiki.archiveteam.org/?diff=49817&oldid=49430
-
h2ibot
Bzc6p edited EOldal (+72, Add link to archives to infobox):
wiki.archiveteam.org/?diff=49818&oldid=49817
-
Jonimus
JAA: apparently people got ahold of the Tim and he has taken over the site and apparently gotten ahold of the domain.
-
Jonimus
So crisis averted I guess? Though a good backup might still be worth while.
-
h2ibot
Bzc6p edited Kepfeltoltes.eu (+126, Added links to archives):
wiki.archiveteam.org/?diff=49819&oldid=49447
-
JAA
Jonimus: Nice, but yeah, agreed.
-
Jonimus
At least now you can just use the domain instead of hitting the IP directly.
-
JAA
Well, once the DNS propagates, at least.
-
JAA
The 10k reqs/day limit is still going to be a pain though.
-
h2ibot
-
pokechu22
-
rhodez
Thank you
-
nicolas17
JAA: I'm still blocked so I'm pretty sure rrpicturearchive's block really is daily
-
JAA
:-|
-
nicolas17
"It happened after pretty much exactly 10k requests." that's good to know, I thought of burning my VPS IP doing requests to figure out what the limit was, now I won't have to :P
-
JAA
Yeah, not going to happen with AB then obviously.
-
Jake
There's an impressive amount of spam on the Curseforge forums
-
Jake
-
nicolas17
(still blocked on rrpa)
-
icedice
Are shallow WARCs what you get from running !ao ?
-
JAA
Yep
-
nicolas17
still blocked on rrpa, will it reset at midnight UTC or another timezone? we'll see
-
nicolas17
10k/day means ~9 sec between requests per IP
-
nicolas17
harold-pain.png
-
JAA
Yeah, we need 650 IP-days to grab it.
-
nicolas17
(lol IP-days)
-
JAA
So if someone has a /24, it can be done in a couple days.
-
JAA
Assuming they only have bans per IP, anyway.
-
nicolas17
I suspect (but I haven't tested it) that the limit is on aspx and doesn't apply to jpg downloads
-
nicolas17
and if you archive albums alone, it's enough to get the image URLs (albums have thumbnails, but you can easily infer the full-image URL from that), so in theory we could do that to get albums and jpg files, and get showPicture.aspx later
-
JAA
Can confirm, images don't get blocked.
-
nicolas17
however I don't know how many requests are needed to get all albums; the highest album ID was 147920 yesterday, but some are paginated, so it's more than that
-
JAA
Or at least not at that draconian limit.
-
chrismeller
is there a warrior project for this yet?
-
nicolas17
chrismeller: no
-
nicolas17
we should archive it, but today it turned less urgent
-
chrismeller
less urgent?
-
nicolas17
<Jonimus> apparently people got ahold of the Tim and he has taken over the site and apparently gotten ahold of the domain.
-
chrismeller
oohhh, ok. well that's good.
-
chrismeller
i was looking at the number of posts per day and it was crazy :D
-
chrismeller
glad someone will be maintaining it
-
nicolas17
I wonder if he had to overpay for the domain...
-
chrismeller
the domain is only a piece of the puzzle, though
-
nicolas17
I think this Tim guy already had control over the server?
-
chrismeller
ah, ok. i thought the original maintainer had died
-
nicolas17
me too, until yesterday during the scraping I saw stuff that definitely looked like a human actively messing with the server
-
nicolas17
(scraping = exploration, I don't have anything of archival quality)
-
chrismeller
well i'm not a train guy, but i really found their corpus of train imagery amazing
-
nicolas17
these people obsessed with specific topics make the internet go round
-
chrismeller
"nerds on the internet" as they say :)
-
nicolas17
I say as I carefully curate the contents of
theapplewiki.com/wiki/Firmware/iPhone/16.x
-
nicolas17
~~when I should be studying~~
-
chrismeller
what are you studying for?
-
nicolas17
I signed up for an online DevOps course and I'm not giving it as much time and attention as I should
-
chrismeller
yadda yadda yadda kubernetes
-
nicolas17
yadda yadda devops is not just a set of tech tools, please get 'dev' and 'ops' people to talk to each other
-
fireonlive
instead of screaming matches
-
fireonlive
does AT have an official view of cloudflare/ddos-guard and the like :p
-
nicolas17
afaik cloudflare is well known for being a pain in the ass for archiving
-
icedice
If you have a sympathetic site admin they can whitelist ArchiveBot IPs in CloudFlare
-
icedice
One of the sites we're archiving is such a case
-
andrew
I've been running a grab-site against a Cloudflare-protected site for a few days by now, I guess a lot of it is down to how the owner configured it
-
icedice
Yeah, and probably the ASN and IP of your server/VPS is on
-
nicolas17
there's an "I'm under attack" mode in cloudflare that really tunes up the restrictions
-
icedice
If you're on a highly abused ASN like Frantech (BuyVM), OVH, or ColoCrossing, they're probably not going to be as nice
-
icedice
M247 also has a pretty bad rep, I think
-
Doranwen
nicolas17: "these people obsessed with specific topics" is like all of Yahoo Groups, lol - I've lost track of the weird and/or highly specific groups I've stumbled across so far while sorting the metadata
-
icedice
And Reddit
-
icedice
There's a subreddit for everything
-
Doranwen
I think Yahoo Groups was even more so. There are some *really* weird and esoteric groups out there. (Plus a few just plain brain-splodey "what on EARTH" types.)
-
nicolas17
someone said wikipedia is powered by neurodiversity...
-
Doranwen
(Or "were" might be the better verb tense, but since I'm looking at them all the time, it feels present.)
-
JAA
Yes, it's all about the site owner's config on Buttflare. Plenty of sites use it and cause no issues, but the ones with more aggressive configs are annoying.
-
fireonlive
ah that makes sense
-
fireonlive
a couple things i tried had them cranked to 11 i guess :/
-
JAA
There are three broad levels: anything goes, 'I'm under attack' mode (= JS challenge), and the hardcore mode with captchas. There's a very wide variety of options to finetune this though.
-
Jake
(for anyone wondering, the Minecraft section of CurseForge is running. Will run the author one afterwards. Everything seems to be going fine this time)
-
JAA
Nice!