00:12:16 <nicolas17> JAA: https://developer.apple.com/support/terms/apple-developer-program-license-agreement/ this was just updated (and WBM's last snapshot is being very useful for diffing), can you AB this new version?
00:13:14 <fireonlive> nicolas17: no recursion?
00:13:44 <nicolas17> nah, recursion there could go nuts
00:14:01 <fireonlive> added
00:15:36 <fireonlive> (to AB)
00:21:12 <Pedrosso> tech234a: Finally lol
00:23:49 <fireonlive> ikr?
00:31:20 <tech234a> nicolas17: yeah I just saw that on RSS
00:32:29 <Pedrosso> About bsky, https://bskyreader.xyz/ could prove a point of archival of these early days until bsky opens up their browsing as well, right?
01:01:17 <JAA> ETA for OneHallyu is 2023-12-25 23:00 now. Still a bit short, although the rate went up slightly to about 6.5k/h.
01:48:18 <nicolas17> interesting change in the apple developer agreement
01:48:49 <nicolas17> "Any litigation or other dispute resolution between You and Apple (other than a challenge to a patent right before a patent office) arising out of or relating to this Agreement, the Apple Software, or Your relationship with Apple will take place in the Northern District of California"
01:48:51 <nicolas17> the part between parentheses was added today
01:51:32 <pabs> https://nextcloud.com/blog/kiteworks-acquires-owncloud-dracoon/
02:07:07 <JAA> 2023-12-22 07:40:17.228Z INFO None topic:32 done: 8812 requests, 2113734 tx, 4296426806 rx
02:07:14 <JAA> This is part of why this is taking so long.
02:07:34 <JAA> Yes, that's 4 GiB of HTML for a single topic.
02:08:13 <JAA> It's the largest one completed so far, but there are others with even more pages.
02:17:36 <project10> potato server working up a sweat
04:29:31 <tech234a> nicolas17: perhaps relating to the Apple Watch patent thing?
05:35:03 <missaustraliana> is there a tool to pull a file from ftp and then upload it to the archive?
05:35:27 <pokechu22> https://web.archive.org/save/ can theoretically do FTP, but FTP is a bit jank in general so it doesn't always work
05:40:57 <missaustraliana> got it, but i would like to upload it onto a user account. theres this weather ftp server with current information and i would like to mirror that and upload it
05:41:20 <missaustraliana> it updates daily so i would like to run it daily
05:48:50 <pokechu22> I'm not sure about that unfortuantely
05:54:12 <missaustraliana> i think i will create it myself. but i need help with metadata on ia. is there any documentation on how i should format the file so it reads it correctly and sets the description and tags and stuff?
06:11:43 <DigitalDragons> IA item metadata is set separately/manually as part of the uploading process
06:12:22 <DigitalDragons> if you're using the CLI it's done like this: https://archive.org/developers/internetarchive/cli.html#upload
07:40:57 <h2ibot> Monika edited YouTube (+271, Add information about TubeStats): https://wiki.archiveteam.org/?diff=51409&oldid=51215
07:40:58 <h2ibot> JustAnotherArchivist changed the user rights of User:Monika
08:10:03 <h2ibot> Monika edited List of largest ArchiveTeam projects (+2, Project sizes as of Dec 2023): https://wiki.archiveteam.org/?diff=51410&oldid=49754
08:31:06 <h2ibot> Monika edited YouTube (+69, Update largest AT project and IA storage capacity): https://wiki.archiveteam.org/?diff=51411&oldid=51409
10:34:34 <h2ibot> Nemo bis edited YouTube (-3, add tubestats, remove random video): https://wiki.archiveteam.org/?diff=51412&oldid=51411
11:48:33 <Pedrosso> I'm getting anxious about copyright law again. Could I be referred to how archival of copyrighted services (such as YT videos) is okay under such laws?
12:37:59 <h2ibot> Nemo bis edited YouTube (-1, fix typo): https://wiki.archiveteam.org/?diff=51413&oldid=51412
14:48:27 <Pedrosso> After having read the wiki, I'll clarify I'm not saying "but copyright" to any of AT's archival projects, only my own.
17:29:06 <JAA> OneHallyu sped up a little since last night, and the ETA is now ~10:30 on the 25th (using the average over the last 3 hours). So maybe there's a chance it'll work out.
17:29:52 <fireonlive> 🤞🏻
17:32:01 <JAA> Pedrosso: In many jurisdictions, there are copyright exemptions for preservation. Distribution is a different topic.
17:33:29 <Pedrosso> Awesome about the OneHallyu grab. As for copyright, I find it a shame that (as far as I know) you can't upload things to IA privately, waiting until the copyright runs out
17:33:58 <JAA> See ya in a hundred years or so. :-|
17:35:13 <JAA> Uploading it for preservation purposes is likely also covered by those copyright exemptions. Public distribution is restricted, and that's why IA may have to take things down if they receive a complaint from the rightful copyright owner.
17:35:38 <JAA> Hence darked items, WBM exclusions, and so on.
17:36:19 <JAA> You can upload access-restricted items if necessary. It's rarely necessary.
17:40:17 <Pedrosso> darked items?
17:40:39 <nicolas17> items can be "darked" which removes them from public view, yet still exist in IA servers, so they can be restored in the future
17:40:58 <nicolas17> but the feature could use improvements
17:41:13 <JAA> Even you as an uploader can't access it.
17:41:38 <nicolas17> for a while I thought, it would be nice if I could hide an item and set a date where it would automatically un-hide, which could be copyright expiration or could be something closer
17:42:10 <nicolas17> then I realized if I dark an item, I can't un-dark it even manually, I have to email IA to do it
17:42:13 <nicolas17> -.-
17:42:31 <Pedrosso> You can manually dark an item?
17:42:38 <Pedrosso> like, as the uploader
17:43:24 <nicolas17> I requested creation of a collection, I think that gave me extra permissions for items inside the collection but also some limited extra permissions over all items I upload
17:44:41 <nicolas17> Pedrosso: https://transfer.archivete.am/inline/ccJu6/item-manager.png
17:46:09 <nicolas17> and all this "As an admin," and "You are a trusted admin of the Archive! You may click the checkbox on a task row or rows below to perform an admin function on it." stuff made me think I had incorrectly gotten global admin permissions
17:46:17 <nicolas17> but it's not global, it's just my items
17:46:54 <Pedrosso> Interesting. So as for grabs which are more likely to be copyright or get claims such as #down-the-tube, AT's idea is upload and if issues come up it just gets indefinitely darkened?
17:47:18 <nicolas17> I *think* so
17:47:48 <Pedrosso> Sounds good. So, if I were to follow those same steps I would personally probably not be in legal trouble?
17:47:50 <nicolas17> there's other stuff like commercial music CDs from the big record labels where IIRC Jason Scott said "do NOT upload that"
17:47:55 <JAA> Usually not darkened but WBM-excluded and possibly access-restricted (which is not the same as darked).
17:48:52 <Pedrosso> nicolas17: Haha. Well, thanks for the info. Is there any comprehensive list like that, or just random statements?
17:49:52 <nicolas17> I think IA has a lot of commercial music archived for preservation but not publicly accessible, because the RIAA and the big labels are not to be messed with, yet it's important to preserve
17:50:31 <nicolas17> IIRC one time a music store closed down and IA bought their entire inventory?
17:50:51 <fireonlive> nice
17:54:17 <Pedrosso> nice
18:52:22 <fireonlive> overnight test:  https://dl.fireon.live/irc/4d688082dd47872c/image.png
18:54:55 <fireonlive> (957 crawls, refreshed: 227)
22:18:30 <TheTechRobo> Would be interesting to compare it with the old version of the dashboard (pre i.van's changes)
23:06:37 <ScenarioPlanet> Hello. May I ask, is there some unified tool that can be used to save POST requests (including formdata) into .WARCs?
23:14:54 <JAA> wget(-at), wpull, qwarc can all do it. warcprox + the HTTP client of your choice, too. I guess there are more.
23:16:56 <fireonlive> i think that's about it :p
23:18:00 <JAA> It's plausible that Heritrix can be convinced into doing it, but it might be painful.
23:19:35 <TheTechRobo> warcio can probably do it too but don't use it
23:19:41 * TheTechRobo braces for the trout
23:20:22 * fireonlive slaps TheTechRobo around a bit with a large WARC standard
23:21:19 <TheTechRobo> I didn't brace for that one, ow...
23:21:27 <JAA> :-)
23:23:51 <TheTechRobo> Wget-AT would probably be the least painful
23:24:29 <fireonlive> :d
23:24:31 <JAA> Yeah
23:24:35 <JAA> Wget-AT++
23:24:36 -eggdrop- [karma] 'Wget-AT' now has 3 karma!
23:25:00 <fireonlive> :D
23:25:03 <TheTechRobo> Wpull requires dark magic to install and chants to use, qwarc is an opaque box, and warcio is... well, warcio
23:25:10 <TheTechRobo> Not sure about heritrix
23:26:02 <TheTechRobo> warcio--
23:26:03 -eggdrop- [karma] 'warcio' now has -1 karma!
23:26:12 <fireonlive> webrecorder--
23:26:12 -eggdrop- [karma] 'webrecorder' now has -3 karma!
23:26:14 <TheTechRobo> Is karma bot case sensitive?
23:26:31 <fireonlive> hm
23:26:37 <fireonlive> i think so yeah
23:26:43 <fireonlive> !kfind A
23:26:43 -eggdrop- [karma] 33 matches for 'A'. top 10 matches: JAA: 10  arkiver: 7  A: 4  nulldata: 3  Wget-AT: 3  that_lurker: 2  Gooshka: 2  kiska: 2  nicolas17: 2  mojibake: 1
23:26:46 <TheTechRobo> Webrecorder--
23:26:47 -eggdrop- [karma] 'Webrecorder' now has -4 karma!
23:26:51 <TheTechRobo> nope
23:26:52 <fireonlive> oh i guess not
23:27:00 <fireonlive> https://github.com/internetarchive/heritrix3
23:27:32 <TheTechRobo> Er, i hope that clients dont treat NOTICEs as pings
23:27:55 <TheTechRobo> Otherwise you just pinged 7 people :P
23:28:04 <fireonlive> (hi everybody!)
23:28:09 <TheTechRobo> s/treat/check/
23:28:21 <JAA> Some clients have options to not get pinged if more than N nicknames appear in the same message.
23:28:25 <fireonlive> https://dl.fireon.live/irc/bd74f363a0838b55/hieverybody.gif
23:28:35 <TheTechRobo> JAA: did you get pinged?
23:28:42 <TheTechRobo> (Apart from now :P)
23:29:02 <JAA> Nope
23:29:35 <JAA> At least I don't think so.
23:29:46 <JAA> I had the window open anyway, so...
23:30:14 <JAA> Nope, doesn't ping me.
23:31:40 <nulldata> *raises hand*
23:31:55 <fireonlive> oop
23:35:53 <nicolas17> it didn't ping me / highlight the window as having notifications
23:36:08 <nicolas17> but *when* I opened the window, it showed this channel as having notifications
23:37:18 <TheTechRobo> Interesting
23:42:34 <fireonlive> weird