-
ivan
it looks like some Google Sites are going down November 1, 2020 per
archiveteam.org/index.php?title=Google_Sites and #nearlylostmygoogles doesn't seem to have started
-
ivan
oh, only if they haven't been viewed in a while
-
Ajay
It has started
-
ivan
ah
-
ivan
JAA: #containerspill
-
JAA
Looks like we're using #failwhale despite the Twitter association.
-
thuban
don't stereotype, failwhales are v. diverse
-
tech234a
ivan: Google Sites has started, but complete discovery remains an issue. About 17 million sites have been found by searching for links on the internet as well as scraping the internal Google Sites search engine on various keywords (this is still ongoing).
-
cadence
what's the deal with the name failwhale?
-
JAA
-
cadence
I see
-
nico_32
OrIdow6: yes framabee, framadrop, framanews, framastory
-
nico_32
framadrop is still up
-
tech234a
-
tech234a
-
tech234a
Images from cadence in Invidious Matrix
-
tech234a
Glad to see the CEO of GitHub wants to see the YouTube-DL repo restored
-
tech234a
Though it seems they have been having difficulty contacting the maintainer
-
JAA
This is not the right channel for that.
-
tech234a
Ok so that is for -ot, sorry
-
nico_32
-
nico_32
i am sending the url into IA
-
nico_32
need to check after if it run from wbm
-
mgrandi
Welp, Playstation seems to have banned my IP lol
-
mgrandi
They allowed me to download all of my links and that finished more than 12 hours ago, hopefully the unban me soon
-
s-crypt
What does scripts only mean for the google sites warrior project?
-
Jake
it won't run directly on the warrior VMs, you'll have to clone the script and run it or use the docker container.
-
Arcorann
It seems like there have been quite a few of those lately
-
s-crypt
So that would have nothing to do with why there is no console output on warrior web UI for google sites projects?
-
Jake
yes, the "scripts only" one's won't work on regular warriors. It should have an error somewhere.
-
s-crypt
It does not give me an error when I select it in the warrior. I am using the VM.
-
s-crypt
According to the repo, I can just select it from the UI
-
cadence
have you seen this issue?
blackjack4494/yt-dlc #9
-
cadence
I like the idea that was posted there of moving discussion to a mailing list
-
cadence
mailing lists are not hard to use, and I think they're good because they don't require signing up for yet another account
-
cadence
of course the main advantage is that if the mailing list server gets taken offline, people can use their inboxes to reconstruct everything
-
s-crypt
Ill move to #nearlylostmygoogles
-
s-crypt
Would it be appropriate to change the 'active' projects that are not available in warrior to 'hiatus'?
-
s-crypt
-
lennier1
mgrandi Jean-Fred: The main problem I saw in WBM crawls of the Playstation store is that M-rated games got stuck on the age gate. Regarding completeness, I don't know.
-
mgrandi
Hmm, might need a cookie or smarter handling of those
-
mgrandi
I'm currently ip banned from playstation LOL, will need to do it on my DO box
-
mgrandi
Really hope this is temporary or else my ps4 is quite useless
-
mgrandi
so, hot tip, don't be lazy and run stuff on your home network
-
cadence
I'm sure it'll be temporary since home IPs typically rotate. PS wouldn't want to ban people who got rotated onto a formerly banned address. Give it a couple of days.
-
cadence
Or if your address rotates, unplug and replug the router.
-
mgrandi
yeah. was being lazy , now its running on a Digital ocean instance
-
mgrandi
the funny thing was is that it completed successfully? so like if they thought i was flooding them then they aren't doing a good job at i
-
mgrandi
the site is awful, i can't get to the other regions without going to this specific page, clicking on a link, and then copying either the cookie or referer URL and putting that in my script
-
mgrandi
wget-at is suddenly not getting stuff it was retrieving before as page prereqs =/ skipping images
-
LuMa
i installed this
hub.docker.com/r/archiveteam/warrior-dockerfile shouldnt it work with more then only URLTeam 2?
-
justcool393
"so, hot tip, don't be lazy and run stuff on your home network" 👀 i run way too many things on my home network
-
tech234a
LuMa: That is a docker for the whole warrior, you’d need to pull an image for the specific project you want to run
hub.docker.com/u/warcforceone
-
LuMa
So i install it and run it as a seperate thing?
-
purplebot
Warrior projects edited by S-crypt (+364, Added google sites, github, and …) just now --
archiveteam.org/?diff=45709&oldid=44433
-
tech234a
LuMa: yes. I think you have to pull and run the project image.
-
LuMa
I could get the whole warrior running but i don't know how to run a specific image
-
tech234a
Elsewhere someone provided this example: docker run warcforceone/google-sites-grab:latest username
-
tech234a
(replace username with your name)
-
tech234a
LuMa: ^
-
LuMa
Got it running any way to interface with it and can i run multiple different projects
-
tech234a
LuMa: not really sure... I think there is a web server that runs by default. And there are a few CLI parameters you can set like --concurrent N
-
tech234a
If you want to run multiple separate docker containers, you can run multiple projects
-
tech234a
Tagging arkiver and Orldow6 just to make sure what I said above is correct
-
LuMa
Doe you know what port it runs on?
-
tech234a
I think default is 8001 (based on looking at source code)
-
tech234a
8688 or 8681 also appear in the code but I don't think it's on those
-
LuMa
Nope doesnt work for me but i noticed it doesnt say any port on the docker like the normal warrior
-
LuMa
-
tech234a
Also pinging wessel1512 in case they know anything about this ^
-
tech234a
LuMa: here is a list of parameters for the CLI, you can try specifying a port
irccloud.com/pastebin/zbfBGBgM
-
LuMa
So say i want to specifiy the port 8002 how would i type it?
-
JAA
I think you need to expose the port through Docker as well, else it's only available in the container? I'm not very familiar with Docker though.
-
LuMa
run-pipeline3: error: the following arguments are required: PIPELINE, DOWNLOADER
-
LuMa
What should i type?
-
tech234a
On one project I just built the image with the command I needed
irccloud.com/pastebin/DJfnmyjs/Dockerfile
-
tech234a
(edited to remove timestamp)
-
tech234a
-
LuMa
Might do that later but i can see it working in the logs and i see my name leaderboard so for now i will let it be but if i start another project i will try with what you linked
-
tech234a
Alright :)
-
mgrandi
The args are the pipeline script and your username for the tracker
-
mgrandi
The only thing was that the docket image couldn't build the wget-at binary, it needs to be upgraded to a later version of ubuntu
-
Ryz
Wondering if RedBubble is worth a proactive archive because of
twitter.com/DAVID_FIRTH/status/1320570963117903878 - problem is, shopping website of the wazoo
-
arkiver
a project is coming up archiving URLs (outlinks) discovered in some projects and from other sources
-
arkiver
not sure if it was mentioned here already
-
arkiver
was talked about a bit in other channels
-
arkiver
in this case the outlinks would be archived without page requisites (at least for now), so it'll mostly be HTML, but without the images
-
arkiver
this is due to size, to keep it down, but at the same time archive valuable data
-
purplebot
-
purplebot
-
purplebot
-
purplebot
-
purplebot
-
purplebot