Okay, let’s get to finishing off writing a function or two to generate the Plotly charts for monthly historical data (as discussed in the last post).

Updated Grouped Scatter Plots

But first, I am going to refactor the horizontal grouped scatter plot from the last post. And, then enhance (eye of the beholder, but…) the vertical version.

Horizontal

After completing the previous post and starting on this one, I realized that removing the monthly rainfall total from the horizontal grouped scatter plot did not make the chart axis boundaries change. And that was, of course, because I had manually set the values. Plotly wasn’t going to change them even when I removed the data for one of the markers. Didn’t like that. Was more impressed with what the grouped bar chart in the last post did when a column was removed.

So, a little research. Finally found cliponaxis=False. That, with some judiciously selected margins and padding, did the trick. Here’s the updated code for the horizontal grouped scatter plot.

      if do_sctr_h:
        # let's look at a horizontal version of the grouped scatter plot
        fl_pth = d_fig/f"mon_{c_mon}_sctr_h.html"
        fig = go.Figure()
        mx_mtot = mn_rf["m_tot"].max()
        for c_ndx, c_col in enumerate(["min", "max", "m_tot"]):
          fig.add_trace(go.Scatter(
            x=list(mn_rf.loc[:, c_col]),
            y=yr_vals,
            marker=dict(size=mrkr_sz[c_ndx], symbol=f"{mrkr_shp[c_ndx]}", color=f"{mrkr_clr[c_ndx]}",
                     line=dict(width=mrkr_ln_wd, color=f"{mrkr_ln_clr}")),
            mode='markers',   # allow markers to extend beyond plot area
            cliponaxis=False,
            name=c_nms[c_ndx]
          ))
        # generate a group bar chart
        fig.update_layout(scattermode="group")
        fig.update_yaxes(
          showgrid=True,  # Set to True to display grid lines
          gridwidth=grid_wd,    # Set the width of the grid lines
          gridcolor=grid_clr # Set the color of the grid lines
        )
        # horizontal legend at bottom of chart
        fig.update_layout(legend={'itemsizing': 'constant', 'orientation': 'h', 'y': -0.15, 'xanchor': 'right', 'x': 1})
        fig.update_layout(autosize=False, width=640, height=480, plot_bgcolor = "#ffffff",                          
          margin=dict(l=0, r=2, b=10, t=40, pad=12),
          title_text=f"Historical {l_mons[int(c_mon)-1]} Rainfall", title_font_size=25, title_x=0.5, title_y=0.97,
          xaxis=dict(title=dict(text="Rainfall")), yaxis=dict(title=dict(text="Year")),
        )
        fig.write_html(fl_pth)
        fig.show()

And here’s the udpated chart. Give it a go—click or double-click on items in the legend.

Vertical

And, now I am going to look at making the vertical grouped scatter plot a little more exciting. Seems only fair if I am making decisions about which one is better for use in the dashboard. Going to use the same markers as the horizontal chart and also look at placing the legend along the bottom. So, more repetitive code.

As with the vertical version in the previous post, I added grid lines and removed the manually fixed axis bounds.

      if do_sctr_v:
        # let's look at a vertical grouped scatter plot
        fl_pth = d_fig/f"mon_{c_mon}_sctr_v.html"
        fig = go.Figure()
        for c_ndx, c_col in enumerate(["min", "max", "m_tot"]):
          fig.add_trace(go.Scatter(
            x=x_pts,
            y=list(mn_rf.loc[:, c_col]),
            mode='markers',
            marker=dict(size=mrkr_sz[c_ndx], symbol=f"{mrkr_shp[c_ndx]}", color=f"{mrkr_clr[c_ndx]}",
                     line=dict(width=mrkr_ln_wd, color=f"{mrkr_ln_clr}")),
            cliponaxis=False,   # allow markers to extend beyond plot area
            name=c_nms[c_ndx]
          ))
        # generate a group bar chart
        fig.update_layout(scattermode="group")
        fig.update_xaxes(showgrid=True, gridwidth=grid_wd, gridcolor=grid_clr)
        # horizontal legend at bottom of chart
        fig.update_layout(legend={'itemsizing': 'constant', 'orientation': 'h', 'y': -0.1, 'xanchor': 'right', 'x': 1})
        # add some spacing to allow markers to display and not clash with tick labels
        fig.update_layout(autosize=False, width=480, height=600, plot_bgcolor = "#ffffff",                          
          margin=dict(l=0, r=0, b=10, t=40, pad=12),
          title_text=f"Historical {l_mons[int(c_mon)-1]} Rainfall", title_font_size=25, title_x=0.5, title_y=0.97,
          xaxis=dict(title=dict(text="Year")), yaxis=dict(title=dict(text="Rainfall")),
        )
        fig.write_html(fl_pth)
        fig.show()

And the generated chart follows. Think that is an improvement over the first version (last post).

Plotting Function

Okay, I now need to decide which plot or plots I am going to continue to use and code functions for in the db_charts module. As I expect I am going to have two or three rows of data (tables) and/or charts in the dashboard, I think a horizontal layout will be more useful. I am really not keen on the line chart. So, I am going to put the horizontal grouped scatter plot in a function to use when generatating the dashboard.

But given the similarity in the code for all the scatter plots, I am going to add code to allow generating the vertical grouped scatter plot as well. Don’t really want to have 2 separate functions. Perhaps not exactly in keeping with the single-responsibility principle, but… Had thought about also allowing generating the grouped bar chart, but that would add some unnecessary complexity to the function. Perhaps a 2nd function afterall.

A couple of if blocks, but otherwise pretty much the same code we have above for the horizontal version.

def hist_rf_mon(mn_rf, c_typ="hgsp", sv_pth=""):
  """Generate and return plotly chart showing the total monthly rainfall,
    the daily minimum and maximum rainfall for a given month for some
    number of years.

    Two chart types: horizontal grouped scatter plot (default),
      vertical grouped scatter plot.

    params:
      mn_rf: dataframe containing the data to plot
             one row for each year, columns [yr, mon, min, max, m_tot]
      c_typ: chart type, str, ["hgsp", "vgsp"]
      sv_pth: path to which to save chart html, do not save if falsy
    
    returns: plotly figure
  """
  c_nms = ["Daily Minimum", "Daily Maximum", "Month Total"]
  yr_vals = mn_rf.loc[:, 'yr']
  mrkr_shp = ["arrow-bar-up", "arrow-bar-down", "star"]
  mrkr_sz = [10, 12, 16]
  mrkr_clr = ["dodgerblue", "royalblue", "darkblue"]
  mrkr_ln_clr = "DarkSlateGrey"
  mrkr_ln_wd = 2
  grid_clr = "lightblue"
  grid_wd = 1
  # !! may need to add variables to control margins/padding for charts separately

  fig = go.Figure()
  for c_ndx, c_col in enumerate(["min", "max", "m_tot"]):
    if c_typ == "hgsp":
      x_vals = list(mn_rf.loc[:, c_col])
      y_vals = yr_vals
    else:
      y_vals = list(mn_rf.loc[:, c_col])
      x_vals = yr_vals
    fig.add_trace(go.Scatter(
      x=x_vals,
      y=y_vals,
      marker=dict(size=mrkr_sz[c_ndx], symbol=f"{mrkr_shp[c_ndx]}", color=f"{mrkr_clr[c_ndx]}",
                line=dict(width=mrkr_ln_wd, color=f"{mrkr_ln_clr}")),
      mode='markers',   # allow markers to extend beyond plot area
      cliponaxis=False,
      name=c_nms[c_ndx]
    ))
  # generate a group bar chart
  fig.update_layout(scattermode="group")
  if c_typ == "hgsp":
    fig.update_yaxes(showgrid=True, gridwidth=grid_wd, gridcolor=grid_clr)
  else:
    fig.update_xaxes(showgrid=True, gridwidth=grid_wd, gridcolor=grid_clr)
  # horizontal legend at bottom of chart
  fig.update_layout(legend={'itemsizing': 'constant', 'orientation': 'h', 'y': -0.15, 'xanchor': 'right', 'x': 1})
  fig.update_layout(autosize=False, width=640, height=480, plot_bgcolor = "#ffffff",                          
    margin=dict(l=0, r=12, b=5, t=40, pad=12),
    title_text=f"Historical {l_mons[int(c_mon)-1]} Rainfall", title_font_size=25, title_x=0.5, title_y=0.97,
    xaxis=dict(title=dict(text="Rainfall")), yaxis=dict(title=dict(text="Year")),
  )
  if sv_pth:
    fig.write_html(fl_pth)
  return fig

And some test code. Didn’t test saving the charts. Will find out down the road, I am sure, if it works.

... ...
    if do_mn_bar:
      do_gbar = False
      do_line = False
      do_sctr_v = False
      do_sctr_h = False
      do_sctr_f = True
... ...
      if do_sctr_f:
        fig = hist_rf_mon(mn_rf, c_typ="hgsp", sv_pth="")
        fig.show()

        fig = hist_rf_mon(mn_rf, c_typ="vgsp", sv_pth="")
        fig.show()

And the resulting charts, in two browser tabs, pretty much look like the versions above. Slight variations in the margins and padding as I am using the same for both chart types. Which was not the case in my developmental code. May eventually change that, but for now it is good enough.

Think that’s it for charts at the moment. Once I get to adding temperature information I expect there will be others. And perhaps the need to refactor current functions to be more generally applicable.

Before calling this post done, I thought I’d have a quick look at the structure of the daily note files.

Daily Note Files

For each day, an unordered list, I put individual notes in list items. There are a couple daily notes of interest. Each day I recorded the temperature when I got up. And at some point in the day recorded the rainfall for some 24 hour period. For sometime now, that has typically been 08:00. For the end of the month, I estimate the rainfall amount for the period 08:00-24:00 for the last day of the preceding month.

I will likely start processing these files at 2022.01.01, as we’ve already covered to the end of 2021. Here’s some sample notes for January, 2022. Very loosy-goosy.

2022.01.01

Start of a New Year, but sure looks and feels like the old one. Saw an owl sitting in the maple late afternoon (early evening?) yesterday. Is that a good sign or …
Looks to be cloudy. YVR is reporting -5.0°C.
08:00 & both gauges still frozen or covered with snow. Will try clearing e-guage today when I put out the hummingbird feeders.
07:21, hummingbird feeders out w/heaters. Feels quite cool at the moment. Left the rain gauge for later, still fairly dark out. But the hummingbirds are always here early. The first or amongst the first to show up each morning.
Put in a little time clearing the upper deck and a portion of the lower deck. Though still plenty of snow left on the lower deck. Not looking forward to this rain in the forecast for tomorrow and perhaps Monday. Given forecast for overnight low decided to leave hummingbirds outside with hummerhearths powered up.
16:21 & -1.5°C. 24hr low -8.0°C, reset.

2022.01.02

Cloudy. Some snow overnight, will measure later. YVR says 2.1°C. Hummingbird feeders stayed out all night. One on back deck looks okay.
08:00, all gauges still non-functional. Maybe 1 cm of snow overnight.
14:06 & 2.5°C. 24hr low -2.0°C, reset.
15:06, noticed that the wind has begun to pick up. Hope it doesn't get worse than forecast.

2022.01.03

Looks to be cloudy, don't think it is currently raining. Though YVR says it is raining and 3.3°C. But snow on roof definitely melting — can here it in the downspout SW corner of ensuite. E-gauge has recorded some rainfall for the past 24 hours. Though expect it missed most of it.
07:30 and seems darker than ususal?
08:00 & 1.5mm ?? rain last 24 hours, 1.5mm for the month. Tube: ~?mm.
Rain/freezing rain/snow started about 13:02. (YVR 4.3°C?) Our thermo says 2.5°C (hasn't changed since yesterday afternoon). Mostly rain now, 13:08. Expect it will go back and forth. Forecast says snow overnight.
Barbara put in to adopt two cats on-line. Got a call in minutes. We have appointment at Vancouver SPCA tomorrow at 11:30. Have been scrambling around for an hour or so washing litter boxes, trying to find trays we used to use to keep floor a little cleaner, dig out travelling cage/basket, get address (print map), etc.
??:?? & °C. 24hr low °C, high °C, reset.
??:?? & °C. ?hr low °C, high °C, reset.

Eventually the notes became a little more consistent. But still going to be difficult to parse out the rainfall amounts. I may need to go through the files and manually generate the rainfall amounts.

Done

I think that’s it for this post. Looking at that note file has been a bit of a downer. But, all in all, I am still rather enjoying this project.

Until next time, may your projects bring you as much pleasure as this one is giving me.

Too Old To Code

Dashboard IX