-
pabs
qwertyasdfuiopghjkl arkiver - btw channel for that is #justsolve, JAA pinged the admin too
-
nicolas17
I agree with adding it to #nodeping, and that doesn't even need to wait for the site to be back
-
qwertyasdfuiopghjkl
nicolas17: I think I've seen notifications for it in #nodeping before, it's just not detecting the outage this time because it's still giving a 200 OK status code.
-
nicolas17
oh
-
nicolas17
oof
-
nicolas17
fucking PHP
-
nicolas17
how is "require(): Unable to allocate memory for pool" not a fatal error returning 500?
-
nicolas17
-
nicolas17
JAA: ^
-
fireonlive
thanks mediawiki for that 200 :p
-
nicolas17
good thing argenteam supports content-encoding because that was 3.14GiB of HTML
-
h2ibot
Nicolas17v2 created ARGENTeaM (+2540, Create page):
wiki.archiveteam.org/?title=ARGENTeaM
-
h2ibot
JustAnotherArchivist changed the user rights of User:Nicolas17v2
-
Vokun
Could someone could put this in AB?
everynoise.com The owner said he's not sure how much longer he's going to run the site
-
Vokun
-
pokechu22
Done but it looks like there was a job in 2022, will check what happened with that
-
fireonlive
for project channels with h2ibot or similar (blogger/pastebin/youtube/telegram/etc), would it be a good idea to have ‘the bot’ in a separate side channel (per project) while discussion for the project continues in the main channel? especially for when the bot can get somewhat "noisy" for some with a bunch of additions and some discussion may be
-
fireonlive
missed cc JAA arkiver Vokun
-
thuban
i've contemplated that myself
-
project10
my +1 on that fwiw ... but I might just be 'holding it wrong'
-
fireonlive
e.g. something like #telegrab-queue or #telegrab-bot
-
h2ibot
JustAnotherArchivist edited ARGENTeaM (+7, Datetimeify):
wiki.archiveteam.org/?diff=51256&oldid=51255
-
nicolas17
maybe the bot should reply in NOTICEs too?
-
fireonlive
perhaps… but would still clutter i suppose (and require more code changes than just moving a channel)
-
h2ibot
Nicolas17v2 created Argenteam (+23, Add redirect):
wiki.archiveteam.org/?title=Argenteam
-
JAA
Moving the channel is a one-line diff in a config file.
-
JAA
NOTICE isn't supported at all by the code currently.
-
thuban
oof
-
thuban
i filter several of the bots
-
thuban
but (a) that doesn't work for everyone
-
thuban
and (b) the ones that have h2ibot talk to itself mostly have to be stateful, so either i let them potentially interfere with one another or i make a new bit for every bot and copy/paste all the triggers
-
thuban
will probably have to write a full-on script at some point
-
fireonlive
indeed re: filters not working for everyone (so notice may not make a difference there)
-
fireonlive
even just people popping in to be a bit buried
-
project10
thuban: is this for weechat?
-
thuban
project10: yeah
-
fireonlive
ever since thuban brought it up i try to prefix everything eggdrop says haha
-
JAA
It'd be nice if we could associate responses directly with the bot command message, but that's not something hackint (or most clients) currently supports.
-
fireonlive
yeah… would be nice to leap forward ircv3 a bit
-
thuban
(if we don't move channels, it might be nice to have the bot accept a 'cc' argument to repeat back along with the name of the submitted item, then cc the original submitter's nick when it submits to itself)
-
JAA
That would be something for arkiver to implement. h2ibot just forwards messages, it doesn't do any message handling itself.
-
thuban
yes, that's what i meant by 'the bot'. sorry for unclearness
-
nicolas17
oh hey argenteam has an API...
-
fireonlive
we love to see it
-
flashfire42
That question just comes from my pastebin stuff XD
-
h2ibot
Nicolas17v2 edited ARGENTeaM (+1071, Explain API and tvshows):
wiki.archiveteam.org/?diff=51258&oldid=51256
-
» nicolas17 reads AB manual
-
nicolas17
so !ao gets *only* the given URL, it doesn't even recurse into images or stylesheets?
-
fireonlive
url + dependencies i believe
-
JAA
URL + page requisites, yes
-
nicolas17
if I "!ao < somelist" and the list has multiple URLs with the same prerequisites, I would hope it gets those prerequisites only once
-
nicolas17
but independent "!ao someurl" would get those multiple times right?
-
JAA
Correct
-
nicolas17
so I could make a huge list with URLs like
argenteam.net/movie/148927 and then !ao< the list, it won't recurse into links but it will get images and stylesheets, and it will get them only once
-
nicolas17
what about redirects? if I add A and B, and A redirects to B, will it request B only once?
-
JAA
No, redirects aren't deduped.
-
JAA
-
pokechu22
However, the no-parent rule can lead to some complications with that too: if the initial URL was
example.com/dira/url and
example.com/dira/url redirects to
example.com/dirb/url then that redirect's target *won't* be saved. This applies to !ao
example.com/dira/url or !a
example.com/dira/url or an !ao < list or an !a < list (but if it's
-
pokechu22
discovered from
example.com or a similar URL without a subdirectory, or if it's offsite, then the no-parent rule doesn't matter)
-
nicolas17
okay!
-
nicolas17
-
nicolas17
/tv/$id URLs redirect to the first episode of the TV show, so that will be fetched twice, but seems acceptable
-
tech234a
One of Mozilla's (partially) public telemetry websites is being replaced with a private version on December 15
groups.google.com/a/mozilla.org/g/firefox-dev/c/kNuk69n7nhc
-
tech234a
The attached doc says the data will be available elsewhere but I can't seem to find it in the other location yet
-
nicolas17
ok time to pass out
-
h2ibot
Tech234a edited Deathwatch (+221, /* 2023 */ Add Mozilla telemetry):
wiki.archiveteam.org/?diff=51259&oldid=51253
-
nicolas17
oh I forgot to add argenteam to deathwatch
-
JAA
nicolas17: AB job for that is running now.
-
h2ibot
FireonLive edited Mailman2 (+1224, add asterisk (most/(all?) to be discontinued…):
wiki.archiveteam.org/?diff=51260&oldid=51145
-
» pabs goes to AB the digium lists fireonlive
-
pabs
fireonlive: can you also add them to deathwatch?
-
fireonlive
ah sure :)
-
h2ibot
FireonLive edited Deathwatch (+400, add Sangoma/Digium/Asterisk):
wiki.archiveteam.org/?diff=51261&oldid=51259
-
nicolas17
I was wondering why the queue kept *growing* on that AB job... prerequisites :P
-
JAA
page requisites*
-
nicolas17
that
-
nicolas17
note that to discover these URLs I ran multiple threads as fast as they could go and reached 50 req/s
-
nicolas17
50 req/s would probably be way too much here, just saying if you want to raise it up a notch... the server can cope :P
-
fireonlive
con 99999999
-
h2ibot
PaulWise edited Mailman2 (-1233, digium lists in progress):
wiki.archiveteam.org/?diff=51262&oldid=51260
-
fireonlive
=] tks
-
JAA
nicolas17: So the /tv/ID URLs redirect to /episode/something, which means the redirect isn't followed as pokechu22 explained above. But since the latter URL is also in the list, that should be fine.
-
arkiver
JAA: did we make WARCs of uloz.to ?
-
JAA
arkiver: It didn't sound like WARCs were being made since pro accounts were involved.
-
JAA
Sanqui: ^
-
arkiver
ah i see
-
arkiver
where is the data now?
-
JAA
Transition of power in Argentina is on Sunday, so we should probably start doing something with that.
-
JAA
nicolas17: Any progress on setting something up for the geofenced sites?
-
arkiver
JAA: i guess that "something" would be archiving various government sites in #archivebot ?
-
arkiver
especially the departments he wants to cut out or greatly reduce funds for
-
JAA
arkiver: Yeah, plus special handling for sites that are being annoying, like Buttflare or geofencing to Argentina.
-
Sanqui
arkiver: we downloaded with pro accounts (otherwise the download speeds were abysmal and captchas were all over), using an off the shelf tool because there wasn't a time for a warc friendly solution. we currently have 40+ TB of data across multiple people and computers and are coordinating deduplication and concatenation using a few scripts, it's a work in progress. I'll try to get a wiki article up though
-
arkiver
Sanqui: sounds pretty good, do you have examples of the data?
-
arkiver
it sounds like you're still doing some cleanup, so examples might come later i guess
-
Sanqui
-
Sanqui
spoiler: there's a lot of warez
-
Sanqui
but also a lot of videos, music, video game saves and mods, other legal data.
-
Sanqui
page's not loading for me now, can write again when it is
-
arkiver
Sanqui: hah, i see
-
arkiver
but still good it was saved
-
Sanqui
seems to be loading now
-
arkiver
what were you further plans for this data?
-
Sanqui
good question
-
Sanqui
we (czechoslovak game archive) are considering subtly hinting that we downloaded a crapload of data from ulož.to and that if somebody is looking for something specific for research purposes to contact us
-
Sanqui
and of course, if it's suitable/desirable we would also put it on IA
-
Sanqui
would have to determine the best process+formats for that
-
cas
hello, Im trying to download this video, and I noticed that in "about this capture", it is claimed that this video is archived by archiveteam. However, the wayback machine claims that this video is not archived. I would like to ask if there's any way to fetch the full video for downloading, as im interested in its content.
-
cas
-
cas
forgot to post the video link
-
TheTechRobo
cas: Looks like that video's watch page was saved, but not the video itself.
-
Inti83
Hi, re requests for EndOfTerm, thy are already annuoncing they will stop financing a set of human rights sites, ex clandestine concentration camps which are now cultural centers for memory and justice. The sites are
exccdolimpo.org.ar and
memoriaexatletico.blogspot.com. I added them and a couple more to the Argentine Wiki page
-
Inti83
as well as some comments
-
Inti83
The new president takes office on 10th december. They also announced to stop financing the ministry for gender and lgtb+ which has a lot of resources - specifically this editorial
editorial.mingeneros.gob.ar
-
nicolas17
oh no
-
nicolas17
-
nicolas17
the official blog posts have links without the slug
-
nicolas17
er forum posts
-
nicolas17
I'm not awake yet
-
fireonlive
🐌
-
JAA
Sanqui: FYI, I just tried to archive the page with AB, and it went down again immediately. I think it's the people here using The ~~Lounge~~ Clownge with link prefetching enabled.
-
Sanqui
*shrug* it's not mine anyway, but I suppose I should've PM'd it to arkiver instead :)
-
JAA
It does seem to be hosted on a Z3 or something.
-
that_lurker
can you add it in a
transfer.archivete.am txt file so that the url is not posted in the channel?
-
JAA
I guess that would be a workaround, but ugh.
-
TheTechRobo
I can disable link prefetching
-
nicolas17
every single user in the channel using thelounge would need to disable it
-
TheTechRobo
Right
-
TheTechRobo
I wonder if ircv3 could add link embeds so this wasn't necessary
-
JAA
I'll just grab-site it.
-
fireonlive
tfw JAA disses your IRC client again
-
fireonlive
:'(
-
that_lurker
Well I tried making a feature request, but they did not get my point apparently
thelounge/thelounge #4805
-
Pedrosso
How's the argentina end-of-term project going?
-
that_lurker
At least its interesting to know that thelounge makes link preview requests as twitterbot :P
-
fireonlive
-
fireonlive
Mozilla/5.0 (compatible; The Lounge IRC Client; +
github.com/thelounge/thelounge) facebookexternalhit/1.1 Twitterbot/1.0
-
JAA
Pedrosso: It isn't yet, but I'll get started on it tonight.
-
Pedrosso
Awesome
-
that_lurker
I completly forgot how hard it is to argue the point of your feature requests in the land of github :P
-
fireonlive
(44 requests total to that url btw)
-
fireonlive
(39 from The Lounge)
-
that_lurker
we need 25 more request
-
that_lurker
s
-
nicolas17
now post it on mastodon and see how much worse it gets
-
fireonlive
that_lurker: indeed :3
-
nicolas17
-
nicolas17
-
fireonlive
oh no
-
JAA
Oof