03:32:56 <datechnoman> !a https://transfer.archivete.am/WMRvV/discord_urls.txt
03:33:06 <h2ibot> datechnoman: Skipped 2 invalid URLs: https://transfer.archivete.am/PIZ87/discord_urls.txt.bad-urls.txt (for 'https://transfer.archivete.am/WMRvV/discord_urls.txt')
03:33:07 <h2ibot> datechnoman: Deduplicating and queuing 200925 items. (for 'https://transfer.archivete.am/WMRvV/discord_urls.txt')
03:33:26 <h2ibot> datechnoman: Deduplicated and queued 200925 items. (for 'https://transfer.archivete.am/WMRvV/discord_urls.txt')
03:34:29 <arkiver> AK: checking
03:40:37 <arkiver> interesting one on the whitespaces
03:41:07 <arkiver> i would think they're probably in the URL as given in the HTML, and normally browsers trim them off, except Wget-AT
03:48:01 <nicolas17> arkiver: I have just subjected myself to the hell that is modern web specifications and yes it seems browsers trim leading and trailing whitespace from href attributes
03:50:05 <JAA> I wonder what wget does with '<a href="\nhttps://example.org/">'.
03:50:23 <JAA> Or other whitespace before the protocol.
03:55:27 <arkiver> nicolas17: yeah i think Wget-AT may not
03:55:55 <arkiver> i'll queue URLs in #// with trailing whitespace with an without that trailing whitespace
03:56:00 <arkiver> JAA: no idea...
03:59:26 <JAA> Well, I found part of the answer: https://github.com/ArchiveTeam/wget-lua/blob/01bae48b489b93efe26fee97f10f6f5b5ba4583e/src/html-parse.c#L1016-L1028
04:00:28 <arkiver> hah :)
04:00:41 <nicolas17> eugh
04:00:55 <arkiver> it could probably be handled better yeah
04:01:09 <JAA> I don't think that's what the specs say, but I don't feel like wading through that right now.
04:01:10 <arkiver> i'll create an issue and get it fixed soon
04:01:35 <arkiver> but there is probably a reason they do that in Wget-AT
04:01:43 <nicolas17> yeah, this would be a different part of the spec altogether
04:01:54 <arkiver> so we might run into further problems if we start treating this differently
04:01:58 <nicolas17> I looked at the part that says how to handle a click to <a href="  foo  ">
04:02:08 <JAA> Yeah, this, I think: https://html.spec.whatwg.org/multipage/parsing.html#attribute-value-(double-quoted)-state
04:02:14 <nicolas17> once you already have "  foo  " parsed as the value of the attribute
04:02:28 <nicolas17> urghh
04:02:34 <nicolas17> I get ANGRY every time I look at whatwg specs
04:02:37 <JAA> arkiver: My suspicion would be 'because that's how we've always been doing it' with a dash of 'the early internet was hell'.
04:02:50 <nicolas17> this is not a spec, this is a browser implementation, written in the programming language known as "English"
04:02:52 <arkiver> maybe yeah
04:03:07 <arkiver> do we know how usual it is for this to occur today?
04:03:08 <JAA> There are so many unspecified things in earlier HTML it isn't even funny.
04:03:09 <arkiver> the leadin \n
04:03:29 <JAA> I feel like the relevant question should probably be: what do browsers do?
04:03:42 <nicolas17> nowadays, browsers do what the spec does
04:03:49 <JAA> Yeah
04:03:57 <JAA> Except when they don't. :-P
04:03:59 <nicolas17> because if major browsers agree to handle broken code in the same way, they change the spec to match as well
04:04:00 <arkiver> browser probably do a lot more complex parsing than Wget-AT does
04:04:09 <arkiver> which means we could run into problems with Wget-AT
04:04:15 <JAA> Browsers implement the entire HTML standard, yes.
04:04:18 <arkiver> but will make an issue anyway and see
04:04:45 <nicolas17> add a unit test >.>
04:04:54 <JAA> Hmm: https://github.com/ArchiveTeam/wget-lua/blob/01bae48b489b93efe26fee97f10f6f5b5ba4583e/src/html-parse.c#L1042-L1043
04:06:41 <nicolas17> oh integration tests are written in perl
04:06:43 <nicolas17> pain
04:12:28 <fireonlive> whatwg loves you :3
04:12:38 <fireonlive> w3c was way too slow
04:18:10 <arkiver> AK: i don't see namesuppressed.com in recent WARCs
04:24:05 <fireonlive> those mentioned IPs don't do pipelines do they
08:52:03 <AK> Interesting, I wonder what it was then