11:34:51 <Sanqui> I have 3 million (80MB) sweb.cz URLs.  99% of them is probably 404.  Can I use the power of #// to get a list of valid ones?
11:43:12 <Sanqui> arkiver: maybe you're able to help with this?
21:26:44 <datechnoman> Not speaking on behalf of arkiver_ put we typically dont throw in bulk url's for the same websites as the power of this project would very eaisly cause a DDOS for a singular website
21:27:58 <datechnoman> Unless there is a new feature/function that allows for the URL's to be mixed in with the current URL's so they are disbursed amongst everything else so it isnt all being processed at once