January 8th, 2012


Weather! (and perl programming)

I'm trying to kick-start a few dormant projects at the moment (why, yes, it's a new year, funny how that happens ;) ). One of these is to actually do something with the 26 & a bit years of data on weather in my parents' back garden (max/min temps, barometric pressure, humidity, rain, wind). There's also probably about 26 years worth of failed attempts to do something with the data ;) Starting with hand drawn monthly graphs back when I was a kid, through to various computer related things.

The last attempt got bogged down on designing a form for inputting the data so that the workflow actually worked for me. Pretty much everything I tried to implement just got in the way, and given the amount of data there is it has to be as easy as possible to do the data entry otherwise it won't get done. This time I'm side-stepping that altogether. Personally I find typing numbers into a spreadsheet both easy & accurate - I can just keep looking at the data and 99% of the time I type the right number without checking keys or screen, and in the vast vast vast majority of times when I make an error I know it as soon as I type it (the muscle movement "feels" wrong for the number I'm reading). And I can just work along one row & back the next (number, right arrow, number, right arrow ... number, down arrow, number, left arrow, number, left arrow ... number, down arrow, number, right arrow etc etc etc) - again a muscle feel thing for the right rhythm. But I could never get a form to quite sit as comfortably under the fingers, and not having the data just there to skim review (at the end of the sheet or when I thought I'd made a mistake) was a difficulty too.

So the brainwave on this reboot of the project was: if it's easiest to just type it into a spreadsheet, why not just type it into a spreadsheet? That's what I'm doing this time - type up a month, save it out as a CSV file with a sensible name (like Nov1985). And the data is there & manually editable etc (unlike the iteration of the project where it was in a SQLite database). I'd rather not just use a spreadsheet programme to do the data manipulation, I've tried that before and while I'm sure it's possible to make the spreadsheet dance & do tricks, I found it required too much manual intervention to get the graphs to look as I wanted them to. So that requires a bit of programming (which is the fun bit of the project :) ).

Yesterday I typed up a few months (Oct 1985 - Feb 1986), and started some work on a perl script to read in the data & draw graphs/generate statistics etc. Right now the script works to read in the data from the files & put it into an internal data structure. I also started to write something to calculate averages for the various parameters for a month or a year, but went round in circles a few times (nested loops, limited scope variables, & when to reset variables to zero are things that tend to break my head unless I pay close attention) & it still doesn't currently work.

The immediately obvious issues that need fixing when I next work on this (noted primarily for my own benefit) are:
  • Strip line end characters (\n\r) from line ends before manipulating data. If I recall right, there's an elegant way to do this I just need to look it up.

  • If there is no value for a parameter make this end up as distinct from when the value is 0, set this up while it's reading the data in.

  • Find the graph drawing modules(? extensions? I forget the name for these) that I had before and set them up again.