-
SanquiI have 3 million (80MB) sweb.cz URLs. 99% of them is probably 404. Can I use the power of #// to get a list of valid ones?
-
Sanquiarkiver: maybe you're able to help with this?
-
datechnomanNot speaking on behalf of arkiver_ put we typically dont throw in bulk url's for the same websites as the power of this project would very eaisly cause a DDOS for a singular website
-
datechnomanUnless there is a new feature/function that allows for the URL's to be mixed in with the current URL's so they are disbursed amongst everything else so it isnt all being processed at once