What’s New

v0.3.2 (unreleased)

v0.3.1 (19 November 2018)

This minor release fixes the builds of our documentation, adds continuous integration (CI) testing of those builds, introduces a how-to guide for future releases, and otherwise slightly modifies our docs and CI configurations. (PR313, PR314). These are all backend changes — for aospy users, there are no changes from v0.3.0.

v0.3 (15 November 2018)

This release adds a number of new features, fixes many bugs, and improves our documentation. It includes several deprecations, other breaking changes, and modifications to aospy’s list of required dependencies. We are grateful for three new contributors this time around and are eager to grow the user- and developer-base further moving forward! Highlights (full changelog below these):

  • Support for Python 3.7
  • Var objects can now be recursively computed in terms of other Var objects
  • Thanks to xarray.CFTimeIndex, no longer require special logic/handling of out-of-range datetimes
  • Improved handling of region longitude bounds in Region via new Longitude class

Breaking Changes

  • Drop support for Python 2.7 and 3.4, since our core upstream dependency xarray is also dropping these soon (PR255, PR280). By Spencer Hill.
  • aospy.Region no longer can be instantiated using lat_bounds and lon_bounds keywords. These have been replaced with the more explicit east_bound, west_bound, south_bound, and north_bound (PR266). By Spencer Hill.
  • Deprecate Constant class and constants.py module. Physical constants used internally by aospy are now stored in _constants.py (fixes GH50 via PR223). By Micah Kim.
  • Deprecate Units class, so now the units attribute of the Var class is a string. (fixes GH50 via PR222). By Micah Kim.
  • Deprecate CalcInterface class. Now, to instantiate a Calc object, pass it directly the parameters that previously would have been passed to CalcInterface (fixes GH249 via PR250). By Spencer Hill.
  • Deprecate utils.times.convert_scalar_to_indexable_coord, since as of xarray version 0.10.3 release, the functionality is no longer necessary (fixes GH268 via PR269. By Spencer Hill.
  • Deprecate func_input_dtype argument to Var (fixes GH281 via PR282). By Spencer Hill.

Documentation

  • Updates the documentation in the described calc_suite_specs argument to submit_mult_calcs in automate.py (fixes GH295 via PR310). By James Doss-Gollin.
  • Corrected link to documentation badge on repository main page (PR213). By DaCoEx.

Enhancements

  • Improve compatibility for data following IRIDL conventions or NOAA data formats. Specifically, several alternate names are defined in GRID_ATTRS, while there is no longer an assumption that BOUNDS_STR is a coordinate of time_weights (fixes GH293 and GH299 via PR309). By James Doss-Gollin.
  • aospy now uses Versioneer to manage its version strings. By Spencer Hill (PR311).
  • Add support for Python 3.7. (closes GH292 via PR306. By Spencer Hill.
  • Use an xarray.CFTimeIndex for dates from non-standard calendars and outside the Timestamp-valid range. This eliminates the need for the prior workaround, which shifted dates to within the range 1678 to 2262 prior to indexing (closes GH98 via PR273). By Spencer Clark.
  • Create utils.longitude module and Longitude class for representing and comparing longitudes. Used internally by aospy.Region to construct masks, but could also be useful for users outside the standard aospy workflow (PR266). By Spencer Hill.
  • Add support for Region methods mask_var, ts, av, and std for data that doesn’t conform to aospy naming conventions, making these methods now useful in more interactive contexts in addition to within the standard main script-based work flow (PR266). By Spencer Hill.
  • Raise an exception with an informative message if submit_mult_calcs (and thus the main script) generates zero calculations, which can happen if one of the parameters is accidentally set to an empty list (closes GH253 via PR254). By Spencer Hill.
  • Suppress warnings from xarray when loading data whose dates extend outside the range supported by the numpy.datetime64 datatype. aospy has its own logic to deal with these cases (closes GH221 via PR239). By Spencer Hill.
  • Add units and description from Var objects to output netcdf files (closes GH201 via PR232). By Micah Kim.
  • Remove potentially confusing attributes from example netcdf files. (closes GH214 via PR216). By Micah Kim.
  • Cleanup logic for Dataset drop on dimensions with and without coords. Use Dataset isel instead. (closes GH142 via PR241). By Micah Kim.
  • Expose data_vars and coords options to xr.open_mfdataset in DataLoaders. These options control how variables and coordinates are concatenated when loaded in from multiple files; by default aospy uses data_vars='minimal' and coords='minimal', but there could be use cases where other options are desired. See the xarray documentation for more information (closes GH236 via PR240). By Spencer Clark.
  • Allow for variables to be functions of other computed variables (closes GH3 via PR263). By Spencer Clark.
  • Add a grid_attrs argument to the Model constructor to allow the specification of custom alternative names for grid attributes like time, latitude, or longitude (closes GH182 via PR297). By Spencer Clark.

Bug Fixes

  • Use the new Longitude class to support any longitude numbering convention (e.g. -180 to 180, 0 to 360, or any other) for both defining Region objects and for input data to be masked. Fixes bug wherein a region could be silently partially clipped off when masking input data with longitudes of a different numbering convention. Fixes GH229 via PR266. By Spencer Hill.
  • Cast input DataArrays with datatype np.float32 to np.float64 as a workaround for incorrectly computed means on float32 arrays in bottleneck (see pydata/xarray#1346). If one would like to disable this behavior (i.e. restore the original behavior before this fix), one can set the upcast_float32 keyword argument in their DataLoaders to False. Fixes GH217 via PR218. By Spencer Clark.
  • Switch from using scipy to netcdf4 as the engine when writing to netCDF files to avoid bugs when using libnetcdf version 4.5.0 (PR235). By Spencer Hill.
  • CalcSuite (and thus submit_mult_calc) now skips calculations that involve time reductions of non-time-defined variables. Calc now raises a ValueError when instantiated with a non-time-defined variable but has one or more time-defined reductions. (closes GH202 via PR242). By Micah Kim.

Testing

  • Create Travis CI environment that tests against the xarray development branch. (closes GH224 via :pull: 226). By Micah Kim.
  • Use nbconvert and nbformat rather than runipy to test the tutorial Jupyter notebook, as runipy is deprecated (PR239). By Spencer Hill.
  • Add flake8 to Travis CI environment to check that new code adheres to pep8 style. Add verbose flag to pytest test suite. (closes GH234 via PR237). By Micah Kim.

Dependencies

  • aospy now requires a minimum version of distributed of 1.17.1 (fixes GH210 via PR211).
  • aospy now requires a minimum version of xarray of 0.10.6. See discussion in GH199, PR240, GH268, PR269, PR273, and PR275 for more details.

v0.2 (26 September 2017)

This release includes some new features plus several bugfixes. The bugfixes include some that previously made using aospy on pressure-interpolated data very problematic. We have also improved support for reading in data from the WRF and CAM atmospheric models.

As of this release, aospy has at least 2(!) confirmed regular users that aren’t the original aospy developers, bringing the worldwide total of users up to at least 4. The first user-generated Github Issues have now also been created. We’re a real thing!

Enhancements

  • Use dask.bag coupled with dask.distributed rather than multiprocess to parallelize computations (closes GH169 via PR172). This enables the optional use of an external distributed.Client to leverage computational resources across multiple nodes of a cluster. By Spencer Clark.
  • Improve support for WRF and NCAR CAM model data by adding the internal names they use for grid attributes to aospy’s lists of potential names to search for. By Spencer Hill.
  • Allow a user to specify a custom preprocessing function in all DataLoaders to prepare data for processing with aospy. This could be used, for example, to add a CF-compliant units attribute to the time coordinate if it is not present in a set of files. Addresses GH177 via PR180. By Spencer Clark.
  • Remove dask.async import in model.py; no longer needed, and also prevents warning message from dask regarding location of get_sync function (PR195). By Spencer Hill.

Dependencies

  • multiprocess is no longer required for submitting aospy calculations in parallel (see discussion in GH169 and pull request PR172).
  • aospy now requires an installation of dask with version greater than or equal to 0.14 (see discussion in pull request PR172).

Bug Fixes

  • Remove faulty logic for calculations with data coming from multiple runs. Eventually this feature will be properly implemented (fixes GH117 via PR178). By Spencer Hill.
  • Only run tests that require optional dependencies if those dependencies are actually installed (fixes GH167 via PR176). By Spencer Hill.
  • Remove obsolete operator.py module (fixes GH174 via PR175). By Spencer Clark.
  • Fix workaround for dates with years less than 1678 to support units attributes with a reference date years not equal to 0001 (fixes GH188 via PR189). By Spencer Clark.
  • Fix bug which would prevent users from analyzing a subset within the Timestamp-valid range from a dataset which included data from outside the Timestamp-valid range (fixed in PR189). By Spencer Clark.
  • Toggle the mask_and_scale option to True when reading in netCDF files to enable missing values encoded as floats to be converted to NaN’s (fixes GH190 via PR192). By Spencer Clark.
  • Force regional calculations to mask gridcell weights where the loaded datapoints were invalid instead of just masking points outside the desired region (fixes GH190 via PR192). By Spencer Clark.
  • Retain original input data’s mask during gridpoint-by-gridpoint temporal averages (fixes GH193 via PR196). By Spencer Hill.
  • Always write output to a tar file in serial to prevent empty header file errors (fixes GH75 via PR197). By Spencer Clark.
  • Allow aospy to use grid attributes that are only defined in Run objects. Previously if a grid attribute were defined only in a Run object and not also in the Run’s corresponding Model, an error would be raised (fixes GH187 via PR199). By Spencer Clark.
  • When input data for a calculation has a time bounds array, overwrite its time array with the average of the start and end times for each timestep. Prevents bug wherein time arrays equal to either the start or end bounds get mistakenly grouped into the wrong time interval, i.e. the wrong month or year (fixes :issue 185 via PR200). By Spencer Hill.

v0.1.2 (30 March 2017)

This release improves the process of submitting multiple calculations for automatic execution. The user interface, documentation, internal logic, and packaging all received upgrades and/or bugfixes.

We also now have a mailing list. Join it to follow and/or post your own usage questions, bug reports, suggestions, etc.

Enhancements

  • Include an example library of aospy objects that works out-of-the-box with the provided example main script (PR155). By Spencer Clark and Spencer Hill.
  • Improve Examples page of the documentation by using this new example object library (PR164). By Spencer Hill.
  • Improve readability/usability of the included example script aospy_main.py for submitting aospy calculations by moving all internal logic into new automate.py module (PR155). By Spencer Clark and Spencer Hill.
  • Enable user to specify whether or not to write output to .tar files (in addition to the standard output). Also document an error that occurs when writing output to .tar files for sufficiently old versions of tar (including the version that ships standard on MacOS), and print a warning when errors are caught during the ‘tar’ call (PR160). By Spencer Hill.

Bug fixes

  • Update packaging specifications such that the example main script and tutorial notebook actually ship with aospy as intended (fixes GH149 via PR161). By Spencer Hill.
  • Use the ‘scipy’ engine for the xarray.DataArray.to_netcdf call when writing aospy calculation outputs to disk to prevent a bug when trying to re-write to an existing netCDF file (fixes GH157 via PR160). By Spencer Hill.

v0.1.1 (2 March 2017)

This release includes fixes for a number of bugs mistakenly introduced in the refactoring of the variable loading step of calc.py (PR90), as well as support for xarray version 0.9.1.

Enhancements

  • Support for xarray version 0.9.1 and require it or a later xarray version. By Spencer Clark and Spencer Hill.
  • Better support for variable names relating to “bounds” dimension of input data files. “bnds”, “bounds”, and “nv” now all supported (PR140). By Spencer Hill.
  • When coercing dims of input data to aospy’s internal names, for scalars change only the name; for non-scalars change the name, force them to have a coord, and copy over their attrs (PR140). By Spencer Hill.

Bug fixes

  • Fix bug involving loading data that has dims that lack coords (which is possible as of xarray v0.9.0). By Spencer Hill.
  • Fix an instance where the name for pressure half levels was mistakenly replaced with the name for the pressure full levels (PR126). By Spencer Clark.
  • Prevent workaround for dates outside the pd.Timestamp valid range from being applied to dates within the pd.Timestamp valid range (PR128). By Spencer Clark.
  • Ensure that all DataArrays associated with aospy.Var objects have a time weights coordinate with CF-compliant time units. This allows them to be cast as the type np.timedelta64, and be safely converted to have units of days before taking time-weighted averages (PR128). By Spencer Clark.
  • Fix a bug where the time weights were not subset in time prior to taking a time weighted average; this caused computed seasonal averages to be too small. To prevent this from failing silently again, we now raise a ValueError if the time coordinate of the time weights is not identical to the time coordinate of the array associated with the aospy.Var (PR128). By Spencer Clark.
  • Enable calculations to be completed using data saved as a single time-slice on disk (fixes GH132 through PR135). By Spencer Clark.
  • Fix bug where workaround for dates outside the pd.Timestamp valid range caused a mismatch between the data loaded and the data requested (fixes GH138 through PR139). By Spencer Clark.

v0.1 (24 January 2017)