-
flashfire42
Looks like I managed to find one target at some point during the night but my warrior cries for the sweet release of targets with an item thats been trying to upload for nearly 2 days
-
fireonlive
Where’s the old wiki? The old wiki can still be accessed on archive.org.
-
fireonlive
uh ok
-
fireonlive
nixos.wiki/wiki/Main_Page < was that archivebotted
-
fireonlive
nixos.org/wiki*
-
fireonlive
for the old one
-
pokechu22
-
flashfire42
is there any way to check the capacity of our temp storage?
-
fireonlive
ah! awesome :)
-
pokechu22
Looks like that matches the timestamp they link to
-
pokechu22
so, should be complete
-
fireonlive
^_^
-
fireonlive
that's one way to pass it off!
-
fireonlive
glad it was saved though
-
imer
flashfire42: there is capacity, but very few servers with storage now, so similar situation as IA where data can't be offloaded fast enough :|
-
flashfire42
so is temp storage starting to be offloaded to IA now?
-
imer
not currently, no
-
imer
offloading as in data on targets -> temp storage
-
imer
and targets as in target :D
-
imer
that + some bugs in the new tech for temp storage which don't help either
-
flashfire42
GitHubElapsed: 01d 22h 43m 52s
-
flashfire42
thats all
-
flashfire42
Also a question about how duplicate warrior items are handled? do they just get deleted or are they both archived. Say I take a super long time uploading a 10GB video from downthetube and someone else gets it due to TTL and uploads it before me does my upload get discarded when it finally arrives?
-
fireonlive
-
fireonlive
if they're menitoned in there - i can't quite tell right now... might be worth a archivebot or something
-
fireonlive
-
nicolas17
one of these "claims to be yahoo videos but it's actually friendster" tarballs I'm indexing has 6 million files and counting
-
fireonlive
can we collect a list for later moving stuff around perhaps?
pad.notkiska.pw/p/YahooVideos%26Friendster
-
Rootliam
I'll try to find if there are any files in the friendster archive that are actually yahoo video I guess
-
Rootliam
aaand they're almost all compressed rip
-
jacobgkau
Howdy, I'm looking for archives of a defunct widget-hosting website called Widgetbox, referred to here:
wiki.archiveteam.org/index.php/Widgetbox
-
jacobgkau
I can't seem to find any on the Internet Archive, does anyone know where they might be?
-
pokechu22
Hmm, most archiveteam projects are on web.archive.org and the backing data is on archive.org. I'll do a quick check
-
pokechu22
-
pokechu22
I don't see any kind of organization to it, but arkiver probably knows more :)
-
pokechu22
(you might need to a wait a while for time zones to line up though)
-
pokechu22
specifically that page is on WEB-20140221120837710-00000-4480~Titan~8443.warc.gz, which just looks like a random file in that group of files
-
jacobgkau
Thanks, I'll take a look in there. I emailed Arkiver back on the 6th and haven't heard back, that's cool if he's in here sometimes.
-
project10
-
jacobgkau
project10: Lol. I can see how that would happen with the massive amount of data being saved and not having the chance to organize it a ton.
-
jacobgkau
I'm looking for some specific widgets that used to be on WidgetBox's widgetserver domain, looks like I have some digging to do.
-
flashfire42
Sanqui job has already been running for a bit
-
flashfire42
Knowing its one of my jobs at some point it probably ran wildly off course
-
Sanqui
oh, I see it going
-
Sanqui
well, I started my job with no-offsite, and it's on a different pipeline, so might as well keep both running
-
flashfire42
Agreed
-
rewby|backup
Looks like the Nixos wiki could do with an archive
nix-community/wiki #46
-
h2ibot
-
h2ibot
-
h2ibot
VoynichCr edited Svalbard Global Seed Vault (+9):
wiki.archiveteam.org/?diff=50788&oldid=50778
-
h2ibot
VoynichCr edited Svalbard Global Seed Vault (+10):
wiki.archiveteam.org/?diff=50789&oldid=50788
-
h2ibot
VoynichCr edited Template:Rescued (+270, categories by year):
wiki.archiveteam.org/?diff=50790&oldid=24057
-
h2ibot
VoynichCr created Category:Archived in 2019 (+29, Created page with "[[Category:Archives by year]]"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202019
-
h2ibot
VoynichCr created Template:Category archives by year (+251, Created page with "<includeonly>This category…):
wiki.archiveteam.org/?title=Templat…e%3ACategory%20archives%20by%20year
-
h2ibot
VoynichCr created Category:Archives by year (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchives%20by%20year
-
h2ibot
VoynichCr edited Template:Category archives by year (+405):
wiki.archiveteam.org/?diff=50794&oldid=50792
-
h2ibot
VoynichCr edited Template:Category archives by year (-24):
wiki.archiveteam.org/?diff=50795&oldid=50794
-
h2ibot
VoynichCr edited Category:Archived in 2019 (+0):
wiki.archiveteam.org/?diff=50796&oldid=50791
-
h2ibot
VoynichCr edited Template:Category archives by year (+6):
wiki.archiveteam.org/?diff=50797&oldid=50795
-
h2ibot
VoynichCr created Category:Archived in 2018 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202018
-
h2ibot
VoynichCr created Category:Archived in 2020 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202020
-
h2ibot
VoynichCr edited Long Now Foundation (+20):
wiki.archiveteam.org/?diff=50800&oldid=47956
-
h2ibot
-
h2ibot
-
h2ibot
VoynichCr edited Voyager Golden Record (+19, {{saved|date=2018}}):
wiki.archiveteam.org/?diff=50803&oldid=45225
-
h2ibot
VoynichCr edited International Internet Preservation Consortium (+19):
wiki.archiveteam.org/?diff=50804&oldid=47936
-
h2ibot
VoynichCr edited Arctic World Archive (+41, | archiving_status = {{saved|date=2020}}):
wiki.archiveteam.org/?diff=50805&oldid=47796
-
h2ibot
VoynichCr edited Arch Mission Foundation (+8):
wiki.archiveteam.org/?diff=50806&oldid=47795
-
h2ibot
-
h2ibot
VoynichCr created Category:Archived in 2021 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202021
-
h2ibot
VoynichCr created Category:Archived in 2022 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202022
-
h2ibot
VoynichCr created Category:Archived in 2023 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202023
-
h2ibot
VoynichCr edited Template:Category archives by year (+215):
wiki.archiveteam.org/?diff=50811&oldid=50797
-
h2ibot
VoynichCr created Category:Archived in 2017 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202017
-
h2ibot
VoynichCr created Category:Archived in 2016 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202016
-
h2ibot
VoynichCr created Category:Archived in 2015 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202015
-
h2ibot
VoynichCr created Category:Archived in 2014 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202014
-
h2ibot
VoynichCr created Category:Archived in 2013 (+29, Created page with "{{Category archives by year}}"):
wiki.archiveteam.org/?title=Category%3AArchived%20in%202013
-
h2ibot
VoynichCr edited Memory of Mankind (+19, {{saved|date=2020}}):
wiki.archiveteam.org/?diff=50817&oldid=48112
-
h2ibot
VoynichCr edited Rosetta Project (+19, {{saved|date=2020}}):
wiki.archiveteam.org/?diff=50818&oldid=45220
-
h2ibot
VoynichCr edited Endangered Languages Project (+38):
wiki.archiveteam.org/?diff=50819&oldid=45116
-
h2ibot
VoynichCr edited Endangered Languages Project (+226):
wiki.archiveteam.org/?diff=50820&oldid=50819
-
h2ibot
VoynichCr edited Endangered Languages Project (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50821&oldid=50820
-
h2ibot
VoynichCr edited Rosetta Project (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50822&oldid=50818
-
h2ibot
VoynichCr edited Memory of Mankind (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50823&oldid=50817
-
h2ibot
VoynichCr edited Arch Mission Foundation (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50824&oldid=50806
-
h2ibot
VoynichCr edited EcuRed (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50825&oldid=50807
-
h2ibot
VoynichCr edited Arctic World Archive (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50826&oldid=50805
-
h2ibot
VoynichCr edited International Internet Preservation Consortium (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50827&oldid=50804
-
h2ibot
VoynichCr edited Voyager Golden Record (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50828&oldid=50803
-
h2ibot
VoynichCr edited Long Now Foundation (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50829&oldid=50801
-
h2ibot
VoynichCr edited Svalbard Global Seed Vault (+30, | archiving_type = ArchiveBot):
wiki.archiveteam.org/?diff=50830&oldid=50789
-
Exorcism
the fck is going on 💀
-
flashfire42
I dont know. HOLD ME Exorcism
-
flashfire42
Also All I know is targets are taking ages to be found by my warrior
-
rewby
flashfire42, Exorcism: What issues are you running into?
-
JAA
rewby: #gitgud
-
h2ibot
-
Barto
-
h2ibot
HadeanEon created Deaths in 2023 (+2735, BOT - Updating page: {{saved}} (0),…):
wiki.archiveteam.org/?title=Deaths%20in%202023
-
h2ibot
HadeanEon created Deaths in 2023/list (+360, BOT - Updating list):
wiki.archiveteam.org/?title=Deaths%20in%202023/list
-
DigitalDragons
is HadeanEon back? :o
-
h2ibot
VoynichCr edited Template:Deathwatch (+292, 2023):
wiki.archiveteam.org/?diff=50835&oldid=38848
-
h2ibot
HadeanEon edited Deaths in 2023 (+112414, BOT - Updating page: {{saved}} (0),…):
wiki.archiveteam.org/?diff=50836&oldid=50833
-
h2ibot
HadeanEon edited Deaths in 2023/list (+19850, BOT - Updating list):
wiki.archiveteam.org/?diff=50837&oldid=50834
-
thuban
o:
-
h2ibot
HadeanEon edited List of current heads of state and government (+9434, BOT - Updating page):
wiki.archiveteam.org/?diff=50838&oldid=44351
-
h2ibot
HadeanEon edited List of current heads of state and government/websites-list (+388, BOT - Updating list):
wiki.archiveteam.org/?diff=50839&oldid=44340
-
h2ibot
HadeanEon edited List of current heads of state and government/facebook-list (+1482, BOT - Updating list):
wiki.archiveteam.org/?diff=50840&oldid=44324
-
h2ibot
HadeanEon edited List of current heads of state and government/instagram-list (+2888, BOT - Updating list):
wiki.archiveteam.org/?diff=50841&oldid=44200
-
h2ibot
HadeanEon edited List of current heads of state and government/twitter-list (+1412, BOT - Updating list):
wiki.archiveteam.org/?diff=50842&oldid=44352
-
that_lurker
Any plans for a channel for Taiwan, now that the situation with China is not looking that great.
-
h2ibot
-
Wohlstand
Hello!
-
Wohlstand
I wrote some changes to the
wiki.archiveteam.org/index.php/Coub article where I clarified about the current status why it survived. I added related reference source, and at the change notice, I added the explanation about the reason why Medium blog was abandoned after 2018.
-
fireonlive
Wohlstand: thanks! :) someone with moderation abilities will look though the queue soon
-
h2ibot
-
h2ibot
-
h2ibot
VoynichCr edited Glottolog (+16, /* Archive */):
wiki.archiveteam.org/?diff=50846&oldid=50845
-
h2ibot
VoynichCr created Wikistats (+195, Created page with "'''WikiStats''' is a site…):
wiki.archiveteam.org/?title=Wikistats
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot
VoynichCr edited Template:Rescued (+723, multiple dates with ','):
wiki.archiveteam.org/?diff=50854&oldid=50802
-
h2ibot
-
project10
#telegrab now seems to be running just as fast (in Mibps) as #shreddit was yesterday, probably clawing back any gains we get from pausing #shreddit
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot
-
h2ibot
Gabrinori edited FTP/List (+38, Add DataSUS (Brazil's NHS data division) public…):
wiki.archiveteam.org/?diff=50861&oldid=46521
-
h2ibot
Wohlstand edited Coub (+262, The Coub has been transferred to a new team and…):
wiki.archiveteam.org/?diff=50862&oldid=48730
-
h2ibot
VoynichCr moved Wikistats to WikiStats:
wiki.archiveteam.org/?title=WikiStats
-
h2ibot
Nulldata edited Deathwatch (+164, /* Pining for the Fjords (Dying) */ added…):
wiki.archiveteam.org/?diff=50865&oldid=50744
-
h2ibot
VoynichCr edited Arch Mission Foundation (+5):
wiki.archiveteam.org/?diff=50866&oldid=50824
-
h2ibot
VoynichCr edited Template:Rescued (+28, and more):
wiki.archiveteam.org/?diff=50867&oldid=50856
-
nicolas17
I think ARCHIVETEAM-YV-003100001-003199999/YV-003100001-003199999.tar is *truncated* too x_x
-
rewby
nicolas17: ?
-
nicolas17
rewby: we have been looking at the yahoo-videos stuff archived years ago, they were uploaded to IA as .tar or .tar.bz2 files
-
rewby
Ah
-
rewby
Interesting
-
rewby
Was worried I'd fucked something up
-
nicolas17
I was indexing them (first via curl <url> | tar tv > list, later with something smarter)
-
nicolas17
it turns out there's 4 tarballs that actually contain data from friendster, not yahoo videos
-
rewby
I may have filed things under the wrong collection in the past before once
-
nicolas17
now I found one of those is also truncated
-
nicolas17
rewby: this is weirder because the filename has YV-
-
rewby
It's not werid
-
rewby
*weird
-
rewby
What likely happened is that if this was a dpos project, it was sent to the wrong rsync port
-
rewby
Although that file format is very old so I don't know 100% what tech was in use back then
-
nicolas17
if it was friendster-1234.tar item uploaded to the wrong collection it would make more sense to me
-
rewby
I wouldn't put it past a manual filing oopsie of something
-
rewby
nicolas17: This can easily still happen. If I fuck up the port number in the tracker stuff will end up in the wrong projects with the wrong names
-
nicolas17
also it's a 76GB .tar
-
rewby
Yeah, it's from before my time so I dunno how they used to do it. But I can see various ways this can happen
-
nicolas17
all of which appears to be friendster, not mixed files from friendster and yv
-
nicolas17
but yeah
-
rewby
Hm
-
rewby
Weird
-
nicolas17
another concern is where are the videos 003100001-003199999
-
rewby
I do know that back in the day targetry was fully manual
-
rewby
With a lot of things just ending up in directories that would occasionally be found years later
-
rewby
It's highly automated these days, but a decent chunk of that was my doing
-
nicolas17
the tar metadata has jscott/jscott as username/group so yeah it seems manual stuff was involved :P
-
rewby
The megawarc factory itself is from before my time
-
rewby
But even that is still sensitive to manual messups
-
rewby
So I automated a lot around it
-
nicolas17
my smol VPS is taking ages to "zstd -19" this file list with 3 million lines
-
Rootliam
-
Rootliam
So there was apparently a tar containing file lists for all (presumably) of the friendster archives
-
Rootliam
I checked them and there are no flvs/mp4s and only one mention of yahoo video that is literally a blog post about it so I don't think any yv data was uploaded to the friendster collection
-
Rootliam
But there are some archives that were uploaded in later years that probably weren't in these lists
-
fireonlive
Tokyo Lab to close in November:
old.reddit.com/r/Archiveteam/comments/16kfjb3 "Sad news : japanese Tokyo Lab company, which is archiving many old animes since 1955, will close in November, and they'll have to destroy all the masters unclaimed by right holders."