In defense of temporary variables

MP 175: They're quite useful in exploratory code.

I really enjoy working with local weather data, because it can give you insight into your local environment. We've been living in western North Carolina for a couple of years now, and recently I wanted to look at annual weather patterns here.

Here's a plot of the daily high and low temperatures for our new hometown:

This was fun to see. It matches my experience here the last couple years, especially when reflecting on how the seasons feel in comparison to southeast Alaska. Obviously it's warmer here, but the range of temperatures on any given day is wider, and the winters are actually quite a bit colder here.

One of the nice things about looking at this plot (while it's still almost 90F in early September) was being reminded that the daily high temperature tends to start dropping steadily once we get into September. At some point when I have a bit more time I'm going to do a long-term analysis of this pattern with rolling averages, and see how this trend plays out over multiple years.

Temporary variables

Here's the loop that reads in the data for this plot, from a CSV file:

dates, highs, lows = [], [], []
for row in reader:
    current_date = date.fromisoformat(row["DATE"])
    high = int(row["TMAX"])
    low = int(row["TMIN"])
    
    dates.append(current_date)
    highs.append(high)
    lows.append(low)

There are three temporary variables here: current_date, high, and low. When people have reviewed my code, they've often asked why I don't rewrite blocks like this without the temporary variables:

dates, highs, lows = [], [], []
for row in reader:
    dates.append(date.fromisoformat(row["DATE"]))
    highs.append(int(row["TMAX"]))
    lows.append(int(row["TMIN"]))

It's a fair question. The revised version is nice and compact, and does exactly the same thing as the longer version.

There are two main reasons I tend to use temporary variables. First, I'm often just trying to work out how to accurately extract the data I want from a source. That often looks like writing a line like this:

current_date = date.fromisoformat(row["DATE"])

and then inspecting the value of current_date. Sometimes the troubleshooting focuses on the parsing logic, sometimes it's about discovering whether a larger dataset is internally consistent or not. I do that for several pieces of data, and then I find it's easy to just keep working with those variables.

The second reason is that I'm often writing some code to demonstrate programming concepts to other people who are less experienced. When people have done a bunch of data extraction work, the more compact listing makes perfect sense. But if you've never extracted data from a CSV file before, the longer version lets you build an understanding of the extraction step in isolation, outside the append() calls.

Why keep temporary variables?

As I worked with this dataset further, I realized that I often end up using these "temporary" variables in a number of ways before packing them into a data structure. For example, one of the weather data files I was working with was missing some data, so I had to wrap the extraction lines in some exception handling code.

Consider this example of the more compact approach:

dates, highs, lows = [], [], []
for row in reader:
    try:
        dates.append(date.fromisoformat(row["DATE"]))
        highs.append(int(row["TMAX"]))
        lows.append(int(row["TMIN"]))
    except ValueError:
        print("Missing data.")

This looks good, but it has some issues. It appends every individual data point it can, until one fails. If a high temperature is missing it will store that day's DATE field, it will skip the problematic TMAX temperature, but it will also miss the valid TMIN reading if it's present.

This whole approach depends on keeping the three lists dates, highs, and lows aligned. These data points need to be processed as a group for any given day, or not at all. This approach fails at that goal.

Here's how the error handling was written with the use of those "temporary" variables:

dates, highs, lows = [], [], []
for row in reader:
    current_date = date.fromisoformat(row["DATE"])
    try:
        high = int(row["TMAX"])
        low = int(row["TMIN"])
    except ValueError:
        print(f"Missing data: {current_date}")
    else:
        dates.append(current_date)
        highs.append(high)
        lows.append(low)

Here I'm assuming that every row in the data file has a date. If either TMAX or TMIN is missing, the date is printed so I can check out that line in the data file. When the ValueError exception is raised, the high and low values that were extracted are discarded. It's harder to do that without the use of these temporary variables. Each day's data is only stored if all the data is present.

I recognize there are other ways to handle this, such as working with a more complex data structure than three independent lists. But the bigger point here is that temporary variables often prove to be not so temporary. In exploratory code, they often end up being useful to have around.

Conclusions

Temporary variables are a funny thing. They're an easy thing to recognize and point out when you see them. But as always, context is important. How stable is the code? How stable is the data that the code is processing? If either of those answers is "not entirely stable", then the temporary variables might be quite handy to have around.