-
missaustraliana
im chucking studio 10 back down the tube as there was a few issues. but this time im going to upload it as an item
-
missaustraliana
890 videos i have got locally. praying network doesnt die and or i run out of storage
-
flashfire42
missaustraliana usually I dont reccomend tubeup but that maybe be your best choice for uploading those items
-
missaustraliana
oh?
-
missaustraliana
im using that py script on the wiki
-
flashfire42
If you are gonna insist on uploading as items despite the fact they are going through down the tube you may as well use tubeup to make sure they go through as proper items.
-
missaustraliana
oh bahaha im not throwing it down the tube i was just saying. im using tubeup and the py script
-
missaustraliana
sorry for not clairfying
-
missaustraliana
flashfire42 i passed it the channel, will it make seperate items for each video or will it be one big item?
-
flashfire42
I thought you meant #down-the-tube rather than using the turn of phrase. #down-the-tube will mean it is accessible through wayback
-
missaustraliana
yeah nah i dont want that
-
missaustraliana
i want it chucked into my profile
-
missaustraliana
but will it? i want it to have a seperate item for each video
-
that_lurker
-
that_lurker
Arch linux completed the bug tracker migration to gitlab. The old one will be closed
-
JAA
Looks like they already borked the bug list, so only the pages for individual bugs are remaining. Perhaps that static conversion already happened.
-
bjorn3
i've been looking for software to archive several sites for personal use. i found a bunch of options, but am not sure what would be the best.
-
JAA
bjorn3: grab-site is what I'd recommend. If the sites are heavily relying on scripting, it gets much harder.
-
bjorn3
does it support resuming an archive operation after i restart my system?
-
JAA
It does not.
-
JAA
Long-standing issue that's not easy to fix:
ArchiveTeam/grab-site #58
-
bjorn3
i've only got a laptop which i shut down every day (and loses internet access whenever i travel to uni) and i don't know if i will be able to archive one of the sites i want in a single day.
-
bjorn3
i don't need js support at all by the way. they are old school web 1.0 sites that i want to archive. some barely use css even.
-
JAA
Right. I can't recommend anything for that setup I'm afraid. It might be possible with grab-site if you can use sleep or hibernation rather than shutting down (by first setting the delay between requests to a large number and then putting it into sleep while it's waiting, ideally), but I'm not sure. All my archival stuff runs on hardware that's always on.
-
JAA
Alternatively, we could archive them through ArchiveBot and you could download a copy of the WARCs from IA to your local storage later.
-
bjorn3
i guess i will give grab-site a go.
-
bjorn3
does grab-site actually need python 3.8 as the readme seems to indicate or does 3.11 also work?
-
bjorn3
debian bookworm ships with 3.11
-
bjorn3
i think it really does need 3.8. got an exception thrown.
-
JAA
Yeah, it does require 3.7 or 3.8.
-
JAA
I made a container image a while ago, if that's of any use to you. Though it comes without the web UI.
-
thuban
JAA: idk whether you saw this when i mentioned it earlier, but grab-site's pip install is currently broken due to fb-re2
-
bjorn3
i should be able to figure it out. just tried to avoid installing pyenv if possible.
-
JAA
thuban: I saw, and my container build has further issues on top of that. :-|
-
thuban
nice
-
thuban
well, google-re2 should work as a drop-in
-
JAA
bjorn3: pyenv is your friend. :-)
-
bjorn3
rustup is my friend already :)
-
bjorn3
(if i understand correctly what pyenv is, rustup is the same except for rust)
-
» JAA is unfamiliar with the Rust ecosystem, so no idea.
-
JAA
It's equivalent to rbenv in Ruby. :-P
-
JAA
Yeah, rustup sounds very similar.
-
bjorn3
is pyenv compiling python from scratch?
-
bjorn3
rustup downloads precompiled versions.
-
JAA
Yes, it compiles them.
-
bjorn3
that would be a nightmare for rustc. especially if you aren't using the precompiled LLVM builds.
-
bjorn3
didn't have any issues with fb-re2 when installing.
-
thuban
older version of abseil, i suppose?
-
bjorn3
don't have any debian package with abseil in the name installed.
-
thuban
might be named libabsl or sth
-
bjorn3
i have libabsl20220623 as package
-
thuban
yeah, older. fb-re2 is behind as of 20220706
abseil/abseil-cpp 97ab3dc
-
bjorn3
got web-grab working
-
JAA
bjorn3: If there's something that's not just of personal interest/importance, please feel free to let us know anyway for running through AB. That means it becomes available in the Wayback Machine and is thus useful to others, too.
-
bjorn3
sure, will let you all know if i find anything that could be relevant for others in my bookmarks.
-
bjorn3
how do i add a regex to the set of ignored urls? --igsets only accepts one of the builtin ignore sets afaict from the readme.
-
pokechu22
For archivebot you can't add a single regex before the job starts; you have to add it afterwards. Not sure if grab-site is the same way or not
-
bjorn3
i see
-
thuban
bjorn3: to add ignores before starting the job, write them to a file and use --import-ignores
-
bjorn3
does that overwrite --igsets? or append to it?
-
thuban
append
-
JAA
And after the job is started, you can edit the 'ignores' file with one pattern per line.
-
bjorn3
thanks!
-
fireonlive
twitter.com/w3cdevs/status/1732669269526151418 < oh hey they also moved but didn't lock their account lol
-
eggdrop
-
pabs