-
DoranwenAh well, I can only imagine someone had a bad day and was taking it out on me. *shrug*
-
DoranwenMost of the people replying in both places were very nice and had some good suggestions so I've got a potential solution to try.
-
Jake(sorry for the relogs. Some weird issue with service workers or something...)
-
spiritis there a searchable 4chan archive?
-
spiritalso, is there a good way to archive a whole twitter thread "tree"? i am in the middle of a weird shitstorm and would like to future proof it
-
JAAspirit: snscrape can do it (as long as Twitter doesn't act up). Either via `snscrape twitter-tweet --recursive TWEETID` or through the search with `snscrape twitter-search 'conversation_id:TWEETID'` (where `TWEETID` is the ID of the first tweet in the thread). Reliability of either approach varies because Twitter's weird. The former used to be a bit more stable last time I tested it, but I know that
-
JAAtweet pages sometimes don't contain all replies, so it's still a mess. The search will obviously miss search-banned users' tweets and has a bunch of annoying bugs that also miss tweets.
-
JAAYou'll probably want `--jsonl` (global option, so before the `twitter-*` scraper name) to dump it into, well, JSONL.
-
JAAOh yeah, the recursive thing can be quite slow since it needs to retrieve the page for every tweet in the tree.
-
spiritthx!