02:03:17 https://location.services.mozilla.com/ to be sunset, announced on march 13th 02:03:47 rip 02:26:10 Fuck software patents. 02:31:20 ++ 03:38:26 pokechu22: I know you looked at https://www.libraw.org/ and adjusted, it finished earlier. (I had stepped away for a few hours..) Did you abort early or just adjust the crawl? 05:19:01 HP_Archivist: I got rid of something that *should* have been junk (pagination URLs where there was a second, different, ignored, pagination param), but I'm not 100% sure if it was complete or not. Those URLs did make up most of the queue. I'll double-check. 05:22:28 pokechu22: Ah alright, yeah I appreciate it. I recently learned that library is the basis to a variety of other software that processes camera RAW files. And the Libraw project is one that is based on *another* project that ceased in 2018, which I also want to make sure gets crawled properly 05:22:29 https://www.dechifro.org/dcraw/ 05:51:35 HP_Archivist: I can confirm that it successfully requested all 125 pages from https://www.libraw.org/comments/recent?page=1 to https://www.libraw.org/comments/recent?page=125. The ignore I added was for stuff like https://www.libraw.org/comments/recent?destination=comments/recent%3Fpage%3D10&page=1 to 05:51:37 https://www.libraw.org/comments/recent?destination=comments/recent%3Fpage%3D10&page=125 which are the exact same as the actual page list, but with a second page number in the middle that does nothing (so instead of 125 requests for 125 pages, it'd be 15625 requests... which is just silly). The site's complete. 05:53:12 Hm, alright. Thank you for checking! 06:38:17 Petchea created Piapro (+1301, Created page with "{{Infobox project | title =…): https://wiki.archiveteam.org/?title=Piapro 06:48:19 Petchea edited Piapro (+899): https://wiki.archiveteam.org/?diff=51888&oldid=51887 06:51:20 Petchea edited Piapro (+52): https://wiki.archiveteam.org/?diff=51889&oldid=51888 06:52:20 Petchea edited Piapro (-28): https://wiki.archiveteam.org/?diff=51890&oldid=51889 06:54:21 Petchea edited Piapro (+283): https://wiki.archiveteam.org/?diff=51891&oldid=51890 07:04:22 Petchea edited Piapro (+132, not just music): https://wiki.archiveteam.org/?diff=51892&oldid=51891 12:23:42 "On April 10th, 2024 the cell data downloads will be deleted and will no longer be available. " DELETED? (re mozilla location services) 12:52:40 imer: Ran the downloads page https://location.services.mozilla.com/downloads through IA with Save outlinks on and it actually got First archive on most links but someone should still check it since that can miss crawling some links from the page. 12:53:18 it might not grab the larger downloads properly I think? 12:53:40 Is there any surefire way to save a page like this on IA other than the basic web.archive.org/save UI? 12:54:02 archivebot, I'm sure someone will run it through 12:54:16 imer: I manually entered the 2 big downloads it missed as well, might have missed some smaller ones also 12:54:56 i've thrown https://location.services.mozilla.com/ into archivebot 12:55:10 thanks! 20:46:19 are there projects for repeatedly backing up job posting sites (and maybe marketplaces/classifieds like gumtree) since those tend to be deleted fairly quickly, not sure if archive.org is anything close to thorough at keeping copies... 20:47:18 If you have a good list of such sites/pages, we could throw them into #//'s thing. 22:59:47 JAA: they are often geographically limited, how much do I need to parse these and come up with a deduplicated, sorted list, also carving out subsections of host/domains that are job specific (i.e. the format of urls for jobs in hackernews, linkedin, etc?): https://en.wikipedia.org/wiki/List_of_employment_websites 22:59:51 https://en.wikipedia.org/wiki/.jobs 22:59:51 https://github.com/lukasz-madon/awesome-remote-job 22:59:51 https://github.com/hugo53/awesome-RemoteWork 22:59:51 https://github.com/zenika-open-source/awesome-remote-work 22:59:51 https://github.com/engineerapart/TheRemoteFreelancer 22:59:52 https://github.com/remoteintech/remote-jobs (seems like a list of employer websites, not likely frequent posted and deleted content like jobs) 22:59:53 https://github.com/lukasz-madon/awesome-remote-job?tab=readme-ov-file#job-boards-aggregators 22:59:54 https://github.com/lukasz-madon/awesome-remote-job?tab=readme-ov-file#job-boards 23:00:55 Ideally, you'd compile a list and create a PR against https://github.com/ArchiveTeam/urls-sources . 23:03:11 ok, this wont be done anytime in the next week from me, but if someone wants to start PRing I welcome it 23:04:46 I was actually surprised this hasnt been done yet by archiveteam, I was hoping to find where to find the archives! (currently jobsearching and a little in hurry to find income) 23:29:58 eightthree: why archive them? historical data analysis? 23:31:06 for actual job search you'd only care about recent posts... 23:49:33 nicolas17: so there's urgent searching and then there's "how long to wait and how infrequently does "dream job" or "rare desirable item for sale" show up in jobsites/classifieds/marketplaces. I was noticing a lot of "this post disappeared" type messages, getting confused and annoyed and feeling like this was worth publicly archiving... 23:50:57 (what's a "dream job", I don't understand) 23:51:00 (half joking)