-
nicolas17
JAA:
developer.apple.com/support/terms/a…developer-program-license-agreement this was just updated (and WBM's last snapshot is being very useful for diffing), can you AB this new version?
-
fireonlive
nicolas17: no recursion?
-
nicolas17
nah, recursion there could go nuts
-
fireonlive
added
-
fireonlive
(to AB)
-
Pedrosso
tech234a: Finally lol
-
fireonlive
ikr?
-
tech234a
nicolas17: yeah I just saw that on RSS
-
Pedrosso
About bsky,
bskyreader.xyz could prove a point of archival of these early days until bsky opens up their browsing as well, right?
-
JAA
ETA for OneHallyu is 2023-12-25 23:00 now. Still a bit short, although the rate went up slightly to about 6.5k/h.
-
nicolas17
interesting change in the apple developer agreement
-
nicolas17
"Any litigation or other dispute resolution between You and Apple (other than a challenge to a patent right before a patent office) arising out of or relating to this Agreement, the Apple Software, or Your relationship with Apple will take place in the Northern District of California"
-
nicolas17
the part between parentheses was added today
-
pabs
-
JAA
2023-12-22 07:40:17.228Z INFO None topic:32 done: 8812 requests, 2113734 tx, 4296426806 rx
-
JAA
This is part of why this is taking so long.
-
JAA
Yes, that's 4 GiB of HTML for a single topic.
-
JAA
It's the largest one completed so far, but there are others with even more pages.
-
project10
potato server working up a sweat
-
tech234a
nicolas17: perhaps relating to the Apple Watch patent thing?
-
missaustraliana
is there a tool to pull a file from ftp and then upload it to the archive?
-
pokechu22
web.archive.org/save can theoretically do FTP, but FTP is a bit jank in general so it doesn't always work
-
missaustraliana
got it, but i would like to upload it onto a user account. theres this weather ftp server with current information and i would like to mirror that and upload it
-
missaustraliana
it updates daily so i would like to run it daily
-
pokechu22
I'm not sure about that unfortuantely
-
missaustraliana
i think i will create it myself. but i need help with metadata on ia. is there any documentation on how i should format the file so it reads it correctly and sets the description and tags and stuff?
-
DigitalDragons
IA item metadata is set separately/manually as part of the uploading process
-
DigitalDragons
-
h2ibot
Monika edited YouTube (+271, Add information about TubeStats):
wiki.archiveteam.org/?diff=51409&oldid=51215
-
h2ibot
JustAnotherArchivist changed the user rights of User:Monika
-
h2ibot
Monika edited List of largest ArchiveTeam projects (+2, Project sizes as of Dec 2023):
wiki.archiveteam.org/?diff=51410&oldid=49754
-
h2ibot
Monika edited YouTube (+69, Update largest AT project and IA storage capacity):
wiki.archiveteam.org/?diff=51411&oldid=51409
-
h2ibot
Nemo bis edited YouTube (-3, add tubestats, remove random video):
wiki.archiveteam.org/?diff=51412&oldid=51411
-
Pedrosso
I'm getting anxious about copyright law again. Could I be referred to how archival of copyrighted services (such as YT videos) is okay under such laws?
-
h2ibot
-
Pedrosso
After having read the wiki, I'll clarify I'm not saying "but copyright" to any of AT's archival projects, only my own.
-
JAA
OneHallyu sped up a little since last night, and the ETA is now ~10:30 on the 25th (using the average over the last 3 hours). So maybe there's a chance it'll work out.
-
fireonlive
🤞🏻
-
JAA
Pedrosso: In many jurisdictions, there are copyright exemptions for preservation. Distribution is a different topic.
-
Pedrosso
Awesome about the OneHallyu grab. As for copyright, I find it a shame that (as far as I know) you can't upload things to IA privately, waiting until the copyright runs out
-
JAA
See ya in a hundred years or so. :-|
-
JAA
Uploading it for preservation purposes is likely also covered by those copyright exemptions. Public distribution is restricted, and that's why IA may have to take things down if they receive a complaint from the rightful copyright owner.
-
JAA
Hence darked items, WBM exclusions, and so on.
-
JAA
You can upload access-restricted items if necessary. It's rarely necessary.
-
Pedrosso
darked items?
-
nicolas17
items can be "darked" which removes them from public view, yet still exist in IA servers, so they can be restored in the future
-
nicolas17
but the feature could use improvements
-
JAA
Even you as an uploader can't access it.
-
nicolas17
for a while I thought, it would be nice if I could hide an item and set a date where it would automatically un-hide, which could be copyright expiration or could be something closer
-
nicolas17
then I realized if I dark an item, I can't un-dark it even manually, I have to email IA to do it
-
nicolas17
-.-
-
Pedrosso
You can manually dark an item?
-
Pedrosso
like, as the uploader
-
nicolas17
I requested creation of a collection, I think that gave me extra permissions for items inside the collection but also some limited extra permissions over all items I upload
-
nicolas17
-
nicolas17
and all this "As an admin," and "You are a trusted admin of the Archive! You may click the checkbox on a task row or rows below to perform an admin function on it." stuff made me think I had incorrectly gotten global admin permissions
-
nicolas17
but it's not global, it's just my items
-
Pedrosso
Interesting. So as for grabs which are more likely to be copyright or get claims such as #down-the-tube, AT's idea is upload and if issues come up it just gets indefinitely darkened?
-
nicolas17
I *think* so
-
Pedrosso
Sounds good. So, if I were to follow those same steps I would personally probably not be in legal trouble?
-
nicolas17
there's other stuff like commercial music CDs from the big record labels where IIRC Jason Scott said "do NOT upload that"
-
JAA
Usually not darkened but WBM-excluded and possibly access-restricted (which is not the same as darked).
-
Pedrosso
nicolas17: Haha. Well, thanks for the info. Is there any comprehensive list like that, or just random statements?
-
nicolas17
I think IA has a lot of commercial music archived for preservation but not publicly accessible, because the RIAA and the big labels are not to be messed with, yet it's important to preserve
-
nicolas17
IIRC one time a music store closed down and IA bought their entire inventory?
-
fireonlive
nice
-
Pedrosso
nice
-
fireonlive
-
fireonlive
(957 crawls, refreshed: 227)
-
TheTechRobo
Would be interesting to compare it with the old version of the dashboard (pre i.van's changes)
-
ScenarioPlanet
Hello. May I ask, is there some unified tool that can be used to save POST requests (including formdata) into .WARCs?
-
JAA
wget(-at), wpull, qwarc can all do it. warcprox + the HTTP client of your choice, too. I guess there are more.
-
fireonlive
i think that's about it :p
-
JAA
It's plausible that Heritrix can be convinced into doing it, but it might be painful.
-
TheTechRobo
warcio can probably do it too but don't use it
-
» TheTechRobo braces for the trout
-
» fireonlive slaps TheTechRobo around a bit with a large WARC standard
-
TheTechRobo
I didn't brace for that one, ow...
-
JAA
:-)
-
TheTechRobo
Wget-AT would probably be the least painful
-
fireonlive
:d
-
JAA
Yeah
-
JAA
Wget-AT++
-
eggdrop
[karma] 'Wget-AT' now has 3 karma!
-
fireonlive
:D
-
TheTechRobo
Wpull requires dark magic to install and chants to use, qwarc is an opaque box, and warcio is... well, warcio
-
TheTechRobo
Not sure about heritrix
-
TheTechRobo
warcio--
-
eggdrop
[karma] 'warcio' now has -1 karma!
-
fireonlive
webrecorder--
-
eggdrop
[karma] 'webrecorder' now has -3 karma!
-
TheTechRobo
Is karma bot case sensitive?
-
fireonlive
hm
-
fireonlive
i think so yeah
-
fireonlive
!kfind A
-
eggdrop
[karma] 33 matches for 'A'. top 10 matches: JAA: 10 arkiver: 7 A: 4 nulldata: 3 Wget-AT: 3 that_lurker: 2 Gooshka: 2 kiska: 2 nicolas17: 2 mojibake: 1
-
TheTechRobo
Webrecorder--
-
eggdrop
[karma] 'Webrecorder' now has -4 karma!
-
TheTechRobo
nope
-
fireonlive
oh i guess not
-
fireonlive
-
TheTechRobo
Er, i hope that clients dont treat NOTICEs as pings
-
TheTechRobo
Otherwise you just pinged 7 people :P
-
fireonlive
(hi everybody!)
-
TheTechRobo
s/treat/check/
-
JAA
Some clients have options to not get pinged if more than N nicknames appear in the same message.
-
fireonlive
-
TheTechRobo
JAA: did you get pinged?
-
TheTechRobo
(Apart from now :P)
-
JAA
Nope
-
JAA
At least I don't think so.
-
JAA
I had the window open anyway, so...
-
JAA
Nope, doesn't ping me.
-
nulldata
*raises hand*
-
fireonlive
oop
-
nicolas17
it didn't ping me / highlight the window as having notifications
-
nicolas17
but *when* I opened the window, it showed this channel as having notifications
-
TheTechRobo
Interesting
-
fireonlive
weird