00:10:12 <JAA> Two hours later, and it has written out about one quarter of the bundle... :-|
00:28:48 <Doranwen> aww
00:29:00 <Doranwen> I've managed to write three shell scripts to automate the *rest* of my process
00:29:06 <JAA> Nice!
00:29:16 <Doranwen> though I still need to figure out how to reduce the resulting file size
00:29:28 <Doranwen> the final file is a massive pdf, lol
00:29:33 <Doranwen> starting from some pngs
00:48:10 <Doranwen> (converting to jpgs before pdf shrinks the size nicely)
01:22:07 <JAA> This repo bundle is killing my disk. :-|
01:32:45 <JAA> git is reading ~100 MB/s and writing ~200 kB/s. Yeah, that'll take a while...
01:33:16 <JAA> On an HDD, as well.
01:33:18 <JAA> This sucks.
02:09:15 <JAA> Can't even TSTP/STOP it either because it's stuck on disk I/O.
02:09:50 <JAA> Well, good to know I guess.
02:43:31 * Doranwen frowns at puzzle she is stuck on
02:43:43 <Doranwen> I've got a script to auto-crop a whole batch of pics
02:43:47 <Doranwen> and that works fie
02:43:49 <Doranwen> *fine
02:44:18 <Doranwen> but I'm realizing the part I want to crop actually shifts position over the course of the set of pics
02:44:25 <Doranwen> about 1 pixel every 5 pics
02:45:20 <Doranwen> I'm quite happy with the shell script to crop, but I don't know how to set up in the shell script to set an iteration number to loop over for each pic it crops, and actually modify the crop parameters any time it lands on one divisible by 5
02:46:31 <Doranwen> for the last one I ran (I edit it to change the parameters for each batch I'm cropping):  https://paste.ee/p/rLKtT
02:47:43 <Doranwen> I'd basically be adjusting the 456 number upwards every 5 pics it crops
02:49:22 <JAA> Bash can do basic arithmetic. It's called 'shell arithmetic' in the official documentation I think. You'd have to implement a counter yourself though.
02:51:20 <Jake> ^ Are you talking about numbers on the image file names or just counting?
02:51:45 <Doranwen> well, there ARE numbers on the image file names, but the counting is all I mean
02:51:47 <JAA> Oh right, if the files are numbered anyway, that would work.
02:52:06 <Doranwen> the files are indeed numbered starting with 1
02:52:31 <Doranwen> well, there's textbitoffilename-1.png
02:52:43 <Doranwen> but same idea - the part before the numbers is absolutely identical and consistent
02:53:54 <Doranwen> I don't think I know what to search for to find useful results for this
02:54:16 <Doranwen> if I go the counter route, I can use that - I didn't know to call it a counter till you said that, that helped me find something
02:54:24 <Doranwen> but if it's easier to use the file names…
02:54:55 <thuban> Doranwen: `list=(*.png); len="${#list[@]}"; for (( i=0; i<$len; i++ )); do somethingwith $i; somethingwith ${list[$i]}; done`
02:55:30 * Doranwen studies command to try to understand it all
02:55:52 <thuban> https://www.gnu.org/software/bash/manual/html_node/Arrays.html
02:56:14 <JAA> If they're numbered like that without padding zeros, note that the order won't be right.
02:56:26 <Doranwen> oh, I can add padding zeros, I was doing that already with pyrenamer
02:56:36 <Doranwen> but that can happen at any stage
02:56:42 <Doranwen> so I didn't know if I should do that before or after the cropping
02:56:53 <Doranwen> (well, any stage *before* the pdf one)
02:57:08 <Doranwen> they start out without the zeros, I add them in
02:57:12 <Doranwen> so I can do that before cropping
02:57:54 <JAA> If you have padding zeros and you use the filenames for arithmetic, you need to be careful with the numeric base as numbers starting with a zero will be treated as octal by default. I got bitten by that recently when I was doing some nasty date calculation in Bash.
02:58:09 <Doranwen> yeah, I saw something about that on a Stack Overflow post just a few min ago
02:58:46 <thuban> (but the counter mechanism i describe above won't have this problem; you can safely use $i as a number)
02:59:02 <Doranwen> JAA: "If that's an issue, you can do: if (( 10#$number % 5 == 0 )) to force $number to be interpreted as base 10 (instead of base 8/octal implied by the leading zero)."
02:59:20 <JAA> Yep :-)
02:59:25 <Doranwen> smart solution, lol
02:59:52 <JAA> And yeah, the counter will not be padded.
03:00:00 <Doranwen> anyway, I need to go read that arrays link to try to understand all of this
03:00:05 <Doranwen> I only half-understand the command
03:02:33 <thuban> basically, expand the glob '*.png' and load the results into an array (this is guaranteed to be lexicographic); get the length of the array; create an index variable i and loop from i = 0 to i = length, with access in the body of the loop to both i (the index) and array[i] (the value of the array at that index)
03:03:07 <thuban> the man page i linked should cover all the relevant syntax
03:03:19 <Doranwen> it's … a bit over my head, though I'm trying valiantly ;)
03:03:47 <Doranwen> the one thing I definitely have a question about is, do I need to remove the non-numeric part of the filenames?
03:03:52 <thuban> nope
03:03:58 <Doranwen> didn't think so, from your explanation
03:04:08 <Doranwen> it's taking the filenames as the list
03:04:13 <Doranwen> and then counting how many it's got in the list
03:04:15 <Doranwen> basically
03:04:22 <thuban> exactly
03:04:22 <Doranwen> ?
03:04:27 <Doranwen> excellent, I'm glad I'm following that
03:04:50 <thuban> so as long as the full filenames sort in the order that you want, you're good
03:05:15 <Doranwen> ok, so the next challenge is - how do I *change* the parameters on the crop every 5 pics, lol
03:05:22 <Doranwen> because that's the whole idea here
03:06:11 <Doranwen> the first five pics would be cropped with 1005x827+456+166, the next five with 1005x827+457+166, the next five with 1005x827+458+166, and so on
03:07:12 <Doranwen> I had it all in `for f in *.png; do convert "$f" -crop 1005x827+456+166 +repage ../cropped/"$f"; done` which worked nicely but didn't adjust for the shifting center
03:09:05 <thuban> set a margin variable, update it in the body of the loop (using $i), and then convert with `-crop 1005x827+"$margin"+166`
03:09:35 <Doranwen> ooh, THAT's how I set it in there, ok
03:09:49 <Doranwen> it'll take me a bit to figure that out, but I think I can do it
03:11:59 <thuban> `margin=455` outside the loop, and then inside `if ! (($i % 5}}; then ((margin++)); fi`, if i recall my arithmetic syntax correctly...
03:12:37 <Doranwen> why 455?
03:12:45 <Doranwen> oh, because the first is 0
03:12:48 <Doranwen> and 0 is divisible by 5?
03:13:09 <Doranwen> so it automatically starts by adding 1
03:13:19 <thuban> yeah
03:14:46 <Doranwen> right now I'm stuck on the loops - am I not using the "for f in *.png" bit then, and if not, how am I telling it to grab each png in turn?  there's some piece of this that's not clear
03:14:53 <Doranwen> and I'll probably go "duh" when I see it
03:15:06 <thuban> ^ lol that's wrong, should be `(($i % 5))`. typo
03:15:13 <Doranwen> but right now it's that general structure that's baffling me
03:15:38 <Doranwen> (noted)
03:15:54 <thuban> `list=(*.png)` expands the glob '*.png' and stores the results in the array 'list'
03:16:23 <Doranwen> oh right, so I can call that somehow to pass it each filename?
03:17:05 <Doranwen> well, my "it" was unclear
03:17:42 <thuban> yes, `${list[$i]}` is the item at position $i in the list (so, the $ith filename)
03:18:45 <thuban> you can pass it to `convert` just as you would `$f`
03:19:29 <Doranwen> ah! ok
03:19:48 <Doranwen> (sorry, I really did try to read that page, but I was understanding very little of it)
03:27:41 <Doranwen> I *think* I have it, but I'm just reading over it all
03:27:57 <Doranwen> the only part I'm trying to understand still, how it works, are the bits inside the for (( ))
03:28:11 <Doranwen> setting the index to 0 to start, yes?
03:28:33 <Doranwen> and saying run as long as it's less than the last file?
03:28:54 <Doranwen> (though does that mean it won't run when it gets to the final file?)
03:29:30 <Doranwen> the pages on for don't have any examples with multiple things inside there like that, so I'm not sure how they all work or why
03:32:02 <Doranwen> the `for (( i=0; i<$len; i++ ))` bit
03:32:27 * Doranwen understands each of those separately but is having trouble understanding how it all fits in and functions in the overall process
03:41:42 <JAA> That is a syntax you will find in many programming languages.
03:41:52 <Doranwen> it does work, though - I tested by setting the actual convert line to echo first, just to see, and yup, it worked
03:42:00 <JAA> It's `for (( start; condition; action ))`.
03:42:10 <Doranwen> ah
03:42:11 <JAA> `start` is executed at the beginning of the loop.
03:42:21 <JAA> `action` is executed after every completion of the loop body.
03:42:25 <Doranwen> aha!
03:42:33 <Doranwen> thank you
03:42:44 <Doranwen> these little things are the things I find that are the hardest to find info on, lol
03:42:50 <JAA> And `condition` is the check that is run before the execution of the loop body to check whether it should still be looping.
03:42:54 <Doranwen> but they make so many other things make sense
03:43:02 <Doranwen> ahhh, ok
03:43:06 <Doranwen> now I understand it all
03:43:43 <JAA> So here, it starts with `i=0`, then increments the value after every iteration (`i++`), and keeps doing that until `i<$len` is no longer true, i.e. until i is equal to or greater than $len.
03:44:12 <JAA> Which means it iterates over all indices of the array.
03:46:12 <Doranwen> yep
03:46:16 <Doranwen> yay, I understand it all
03:46:30 <Doranwen> I knew if I kept hanging around you guys I'd start picking stuff up ;)
03:47:01 <Doranwen> I appreciate the help greatly - I've got so much more of a framework now to hang more stuff on
03:47:46 <JAA> Another useful thing when working with arrays is `for value in "${array[@]}"; do ... done`. That iterates over all values in the array (but doesn't get you the nice counter variable, so it's useless in this specific case).
03:48:18 * Doranwen nods
03:49:31 <JAA> But there's also ${!array[@]}, which are the indices. In this case, `for i in ${!list[@]}` would be identical to the loop we discussed above.
04:29:08 <Doranwen> JAA: is there an easy way to run a command on a range of files *within* that selection?
04:29:26 <Doranwen> the only methods I'm seeing operate on all the files in the folder
04:29:43 <Doranwen> but what if I wanted to, say, run a script on just files 177-185
04:30:17 <Doranwen> all inclusive and every number in that range counting, but not on 1-176
04:30:22 <Doranwen> I mean, I could just move out the other files, lol
04:30:25 <Doranwen> and then move them back
04:30:33 <Doranwen> but there's probably a way of specifying
04:30:44 <Doranwen> I'm not seeing anything like that in the for loop pages I'm looking at
04:30:50 <Doranwen> but I probably just don't know what to search for
04:35:16 <Doranwen> oh wait, just set i to 176, lol
04:35:20 <thuban> in this case you could do sth like `for (( i=177; i<186; i++ )) do somethingwith restoffilename-"$i".ext; done`
04:35:22 <thuban> yeh
04:35:23 <Doranwen> er, 177
04:35:30 <Doranwen> yeah, it just occurred to me
04:37:40 <Doranwen> no, 176 - it didn't apply it to 177 with that last attempt, only 178 onward, interesting
04:38:15 * Doranwen scratches head
04:38:49 <Doranwen> yeah, that definitely was it
04:38:50 <JAA> Ah yes, one of the two most common issues in computer science: naming things, caching, and off-by-one errors. :-)
04:38:56 <Doranwen> lol
04:39:05 <JAA> You said your files are numbered starting from one, but the array indices start from zero.
04:39:17 <Doranwen> RIGHT, I forgot about that
04:40:05 <Doranwen> I'd originally thought 176 and then figured 177 because thuban said it, lol - wasn't sure exactly why I'd thought 176, but now I guess I know
04:41:28 <thuban> my suggestion just substituted i directly into the filename; i thought you were doing a new thing rather than still using the array access
04:42:24 <Doranwen> ah
04:42:28 <Doranwen> I just copied the code and modified it
04:42:33 <Doranwen> from the last script
04:43:56 <thuban> (btw, for more general-purpose selections you can use `vipe`--puts an interactive editor into a pipeline, so you can e.g. pipe `ls` into it, manually delete the stuff you don't want, and pipe the result out into other commands)
04:47:48 <JAA> TIL. Does it default to vi though? Exiting that isn't straightforward. ;-)
04:48:54 <thuban> i've never gotten that joke. if it were ed...
04:49:59 <thuban> (EDITOR/VISUAL, so one presumes)
04:50:33 <JAA> Lots of people struggle with vi. It takes a bit of time to get used to how it works.
04:50:50 <JAA> And yeah, it uses /usr/bin/editor, EDITOR, and VISUAL with fallback to vi, at least on Debian.
04:51:40 <thuban> i suppose on an old enough system it could instead be ed! "Note the consistent user interface and error reportage."
05:19:26 <Doranwen> I've used vi some, but I always have to look up the commands before I do because I don't do it often enough to remember them, lol
05:20:22 <JAA> I never bothered with vi. The only thing I know is: :q!
05:20:24 <JAA> :-)
05:21:31 <JAA> For manual edits, I generally use nano. For anything else, I use awk/sed/grep/whatever as appropriate.
06:15:37 <JAA> Oh yeah, the chromium git bundling finished after almost 9 hours.
06:27:27 <Doranwen> lol
12:26:37 <Jake> 9 hours? wow
13:06:46 <rewby> The chromium repo is well known to be a... big one
13:07:01 <masterX244> How fat was the result file?
19:08:28 <programmerq> there are several ephemeral resources related to the buying and selling of cars. dealers might post a carfax for a given vin, but it'll disappear after it's sold, for example. There's limited public info for license plate->vin on sites like faxvin.com and vincheck.info but you need to search a specific plate to pull the info.
19:08:42 <programmerq> it's mostly just boring stuff, but it might be a neat thing to try to archive. Thoughts?
19:10:22 <JAA> masterX244: 22 GiB or something like that.
23:08:34 <OrIdow6> programmerq: That's -bs material