-
JAA
-
fireonlive
akamai bad
-
marktheworst
Hi, I was the one who asked about divested.dev's exclusion a couple days ago or so
-
marktheworst
would like to note that the AB snapshot showed up for opinionplatform.org but still no snapshots for divested
-
marktheworst
on WBM
-
anarcat
JAA: do we have igsets for drupal sites?
-
JAA
anarcat: Funny timing, I just explained an hour ago in #archivebot why we don't. :-P
-
JAA
The ignore depends on the Drupal version as well as the path prefix. So an igset isn't really very possible.
-
anarcat
oic
-
anarcat
to pabs i bet :p
-
JAA
AB does actually have code to deal with that mess, but it's broken. :-/
-
fireonlive
:(
-
fireonlive
me 🤝 AB's code
-
marktheworst
Since divested.dev appears to have some sort of time exclusion but not an explicit whole site exclusion, should it go on the WBM exclusion list wiki?
-
h2ibot
PaulWise edited Deathwatch (+474, Open Collective Foundation shutting down end of…):
wiki.archiveteam.org/?diff=51805&oldid=51800
-
pabs
JAA: can you shove Mannie's list above (maybe fix the merged URL) into queueh2ibot? then I'll try to do subdomains for them after lad's ones
-
pabs
AB needs some more capacity :)
-
fireonlive
soon: waterpipe
-
fireonlive
maybe
-
fireonlive
lol
-
pabs
:)
-
fireonlive
=]
-
» pabs ponders if SWH would fund one
-
fireonlive
ooh
-
pabs
I'm guessing not, didn't talk to them about it yet
-
JAA
Seems too far away from their mission.
-
pabs
there was a comment in olasd's DebConf talk that made me think they might
-
fireonlive
oh?
-
pabs
something like we aren't working on archiving websites but we would like to suport that or something
-
JAA
Interesting
-
fireonlive
ooh
-
pabs
it was towards the end IIRC if you want the exact wording
-
JAA
pabs: So I can't easily stop what queueh2ibot is running through currently. :-/
-
pabs
ok, no worries
-
arkiver
the bot to queue items was killed due to memory usage. too much memory was consumed by other processes on the machine, not the bot
-
arkiver
the bot it back up now
-
c3manu
how much would a useful chunk of ab capacity roughly be per month?
-
c3manu
or does anyone know when the next "black friday style" offers will come around? ^^"
-
AK
OOh that's one I can answer, (I run the ak-was-here-hel3 + ak-was-here-hel4 boxes), lemme find some numbers
-
AK
(Pay for them, JA_A does all the hard work for me)
-
AK
So we've got one AX41-NVMe and one EX42 NVMe, each one is ~€45 per month at the moment
-
AK
🤔Hel3 is the EX42 which only has 8 cores vs hel4's 12
-
c3manu
thanks! i've not been good with financial decisions lately, but in principle i should be able to handle at least one of those (i'd be starting with only one either way)
-
c3manu
i'll think this through for a bit. i'm on vacation in a month from now so i would have time to set it up then.
-
c3manu
i think that's a plan :)
-
katia
fwiw, the EX44 seem to be a great deal, even better than the AX41-NVMe wrt. CPU perf
-
katia
if you can justify the setup fee
-
murb
they often have promos for free setup.
-
katia
r
-
katia
ops
-
AK
Yeah I'm tempted to look at replacing/upgrading hel3. But the drawdown time on pipelines can be up to 6 months, so I'd basically need to deploy hel8 and then wait 6 months to cancel hel3 which is quite a cost
-
imer
might be worth setting it up in a container/vm so you can just migrate it next time?
-
AK
That's probably a JAA question on how easy it would be to pause/migrate a running pipeline. I suspect the answer is "Not at all"
-
AK
I knew when I offered these servers that I've gotta keep them going for months (And now years) because some of the jobs can take months
-
imer
as in qemu does live migrates, so the vm won't even notice it got moved
-
murb
how does a pipeline deal with powerfailure?
-
murb
or a reboot for other (important) reasons?
-
kiska
It doesn';t
-
imer
oof.
-
murb
so no checkpointing / saving of state?
-
kiska
Nope
-
kiska
Here is the former kiskaVultrJP instance: Linux vultr.guest 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u5 (2019-08-11) x86_64 GNU/Linux
-
kiska
Its running Debian 9.9 and has: 15:22:36 up 328 days, 4:16, 10 users, load average: 0.32, 0.22, 0.09
-
arkiver
we now have a maximum to the number of concurrently running jobs by a queuing bot in a channel
-
arkiver
see !help for details on the maximum
-
AK
Hel3: 16:21:13 up 310 days, 22:45, 13 users, load average: 5.16, 5.51, 5.80
-
AK
Hel4: 16:21:35 up 593 days, 18:08, 17 users, load average: 8.88, 9.16, 8.89
-
AK
I win :P
-
kiska
Congrats! :D
-
nicolas17
c3manu: are you married? these servers tend to have a pretty bad Spouse Acceptance Factor
-
myself
So you're saying I'm the perfect candidate...
-
fireonlive
what’s “married”?
-
c3manu
nicolas17: no, i’m not ^^ but that also means my tax bracket is called "get fucked"
-
fireonlive
o3o
-
murb
couples are taxed at a lower rate? that seems strnage, surely they save money with economies of scale etc?
-
katia
i think different countries have different implementations of that, but it does seem weird
-
c3manu
just reading up on that a bit, apparently i have a wrong understanding on how tax brackets work here. it's different brackets, yes, but that mostly enables a couple to shift things around a bit. but it's not a difference in principle, as i thought until now.
-
c3manu
so nevermind what i said
-
pokechu22
Yeah my understanding is it's (at least in the US) roughly just letting you choose to merge your money with your spouse and scale up the values for 2 people (though you can also choose to file separately in which case the brackets are for one person), but I haven't looked at exactly how it works. Also remember that getting bumped up a tax bracket generally only means you pay
-
pokechu22
more in tax for money over that point; tax bracket wise there's no situation where earning an additional dollar will result in an increase of taxes over a dollar (i.e. if you make more money, even after taxes you'll make more money)
-
fireonlive
had we not broken up we would have become "common law" automatically here...
-
fireonlive
(but also best to continue in -ot)
-
AK
I am squarely in the "As long as my girlfriend doesn't know exactly how much I spend then I'll be okay" camp
-
fireonlive
;)
-
fireonlive
if it ever comes up just say 'hey at least it's not heroin or whatever'
-
JAA
AK: Rookie numbers
-
JAA
20:12:59 up 2443 days, 1:52
-
JAA
imer: IP will change though, unless you route everything through another box, which may cause pain.
-
fireonlive
everyone cross your fingers for firepipe not losing power or whatver x3
-
imer
JAA: yeah, that's the one downside. there is some hackyness you can do (done that before), assuming it handles the network disconnect and having another ip for outgoing stuff just fine, reverse proxy any incoming (http?) connections to the new ip until dns updates
-
JAA
imer: We're talking about AB pipelines, so the outgoing IP is what matters.
-
JAA
Reason being URLs or cookies that are IP-bound, of course.
-
imer
is it? not familiar how that stuff works, I was assuming it just does a recursive crawl and then hands it off once finished, ip change shouldn't matter for most things there unless a site restricts session to the ip?
-
imer
beat me to it.
-
imer
makes sense
-
JAA
It's basically impossible to know how much of a problem it would actually be without trying it out.
-
JAA
Also, we did experiment with containers before. I wanted to use CRIU to resume things after host maintenance, and that worked fine in tests. The resumption never happened though for unrelated reasons.
-
NickS|m
Does anyone know if archivebot has grabbed
dcist.com yet? Or how to find out?
-
JAA
-
NickS|m
Oh that's great, thank you!
-
Vokun
I know it would take quite some time to set up, (which doesn't seem like something we're ever going to have at this point) but would getting #Y running be better than another AB pipeline or two?