Adding new functionality#
or more verbosely:
Scenario 2: Adding new dimensions from alternate data streams and providing entirely new functionality#
If you want to add new dimensions to an xsnowDataset or more generally extend the xsnow functionality beyond a single method, write your own extension class. xsnow has everything prepared to make this very straightforward for you. As for a scenario-1-extension, write your class in a python module and decide whether you want to keep this module private to yourself or host it in a public repository.
Here is a cheat sheet for the steps you have to take. You will find more detailed explanations and a demonstration further below:
Recipe
Import:
from xsnow import DatasetDecoratorDefine your extension class: e.g.,
class EnsembleFX(DatasetDecorator):Define your class methods (and possibly generic functions)
Whenever a function returns an object of your new class,
_rewrap()the newly generated dataset
Regarding 1. and 2.)
It is important that you define your class as a subclass of the
DatasetDecorator. This allowsxsnowto configure your class to feel and behave like anxsnowDataset, while allowing multiple extensions to be enchained in custom order.
Regarding 3.)
Code all functionality you need and want. Prepend private methods or helper functions with an underscore (e.g.,
_my_private_helper).
Regarding 4.)
Rewrapping is important to ensure different extensions can be enchained. Use the pattern
xs_out = self._rewrap(xs_modified).
Scenario-2-extensions can look quite different. Therefore, the next two sections demonstrate two extensions that extend the xsnow functionality in their own ways.
Example: Ensemble-forecast extension—new data streams and dimensions#
The ensemble forecasts extension aims to facilitate research on the performance of forecasts with different lead times and from different model realizations such as deterministic or ensemble members. As such, it provides a special read routine that parses a defined directory structure into the dimensions realization and model run. This read routine is actually the heart of the extension, while the EnsembleFx class does not do anything except put its label onto the resulting dataset for consistent naming.
Excerpt from implementation#
from xsnow import DatasetDecorator
class EnsembleFX(DatasetDecorator):
"""
Decorator that enriches an xsnowDataset with run context and leadtimes.
Dimensions/coordinates guaranteed after ``read_ensemble_fx``:
- ``run`` (string) with attrs including optional ``timezone``.
- ``realization`` (string) describing the ensemble member label.
- ``run_start`` coordinate on ``run`` (datetime64[ns], tz in attrs; NaT if unknown).
- ``leadtime`` coordinate on ``time`` (float hours from run_start to valid time).
All existing xsnowDataset API remains available via inheritance.
"""
# note that the class is basically empty (no methods, etc)
# note that this is not a class method, but a module-level function
def read_ensemble_fx(
source: Union[str, Path],
# < more parameters >
) -> Optional[EnsembleFX]:
"""
Read a forecast collection into an ``EnsembleFX`` dataset.
Layout: ``source/{run}/{member}/{station}.{smet|pro|nc}``. Runs and members are
derived from folder names; station IDs from filenames. Only requested runs,
members, and filename bases are read to keep I/O minimal.
< ... >
Parameters
----------
< ... >
Returns
-------
EnsembleFX or None
Decorated dataset when data were found; otherwise ``None``.
"""
# < iterate through directory tree and read >
# < concatenate individual datasets >
xs_out = EnsembleFX(xs_combined)
return xs_out
Demo application#
import xsnow
from xsnow.extensions.ensemble_forecasts import read_ensemble_fx
datapath = xsnow.sample_data.snp_ensfx_dir()
Downloading file 'smets/ens-fx/2024-01-17T00Z/det/VIR1A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-17T00Z/det/VIR1A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-17T00Z/det/VIR5A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-17T00Z/det/VIR5A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-17T00Z/p01/VIR1A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-17T00Z/p01/VIR1A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-17T00Z/p01/VIR5A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-17T00Z/p01/VIR5A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-19T10Z/det/VIR1A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-19T10Z/det/VIR1A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-19T10Z/det/VIR5A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-19T10Z/det/VIR5A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-19T10Z/p01/VIR1A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-19T10Z/p01/VIR1A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-19T10Z/p01/VIR5A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-19T10Z/p01/VIR5A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-25T12Z/det/VIR1A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-25T12Z/det/VIR1A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-25T12Z/det/VIR5A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-25T12Z/det/VIR5A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-25T12Z/p01/VIR1A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-25T12Z/p01/VIR1A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/2024-01-25T12Z/p01/VIR5A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/2024-01-25T12Z/p01/VIR5A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/analysis/det/VIR1A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/analysis/det/VIR1A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Downloading file 'smets/ens-fx/analysis/det/VIR5A.smet' from 'https://gitlab.com/avacollabra/postprocessing/sample-data/-/raw/main/smets/ens-fx/analysis/det/VIR5A.smet' to '/root/.cache/xsnow-snp-ensfx'.
Data location: /root/.cache/xsnow-snp-ensfx
xsnow-snp-ensfx/
smets/
ens-fx/
2024-01-25T12Z/
det/
VIR1A.smet
VIR5A.smet
p01/
VIR1A.smet
VIR5A.smet
2024-01-19T10Z/
det/
VIR1A.smet
VIR5A.smet
p01/
VIR1A.smet
VIR5A.smet
analysis/
det/
VIR1A.smet
VIR5A.smet
2024-01-17T00Z/
det/
VIR1A.smet
VIR5A.smet
p01/
VIR1A.smet
VIR5A.smet
xs = read_ensemble_fx(f"{datapath}/smets/ens-fx/")
print(xs)
<xsnowDataset>
Locations: 2
Timestamps: 360 (2024-01-16--2024-01-31)
Profiles: 5760 total | 0 valid | unavailable with HS>0
employing the <xarray.Dataset> Size: 406kB
Dimensions: (location: 2, time: 360, slope: 1, realization: 2, run: 4)
Coordinates:
* location (location) object 16B 'VIR1A' 'VIR5A'
* time (time) datetime64[ns] 3kB 2024-01-16T03:00:00 ... 2024-01...
* slope (slope) int64 8B 0
* realization (realization) object 16B 'det' 'p01'
* run (run) object 32B '2024-01-17T00Z' ... 'analysis'
altitude (location) float64 16B 2.372e+03 2.066e+03
latitude (location) float64 16B 47.15 47.37
longitude (location) float64 16B 11.19 11.5
leadtime (time, run) float64 12kB nan nan nan nan ... nan nan nan nan
azimuth (slope) float64 8B nan
inclination (slope) float64 8B nan
run_start (run) datetime64[ns] 32B 2024-01-17 ... NaT
Data variables:
DW (location, time, slope, realization, run) float64 46kB na...
ISWR (location, time, slope, realization, run) float64 46kB na...
PSUM (location, time, slope, realization, run) float64 46kB na...
RH (location, time, slope, realization, run) float64 46kB na...
TA (location, time, slope, realization, run) float64 46kB na...
TAU_CLD (location, time, slope, realization, run) float64 46kB na...
VW (location, time, slope, realization, run) float64 46kB na...
VW_MAX (location, time, slope, realization, run) float64 46kB na...
profile_status (location, time, slope, realization, run) float32 23kB na...
Attributes:
Conventions: CF-1.8
crs: EPSG:4326
The resulting dataset is of class EnsembleFx. It has an additional dimension run with 4 entries. Two additional coordinates were added, run_start: dimension (run) and leadtime: dimension (time, run). You can now work with the dataset as you know it from an xsnowDataset.
For example, we could look into the first 100 values of 'TA' and 'leadtime' for the deterministic member of the 2024-01-17T00Z run at the first location:
sub = xs.sel(run="2024-01-17T00Z", realization='det').\
isel(location=0, time=slice(100)).squeeze()
print(sub['TA'].values)
print(sub['leadtime'].values)
[ nan nan nan nan nan nan nan nan nan nan
nan nan nan nan nan nan nan nan nan nan
nan 269.48 269.72 269.99 270.52 270.58 270.1 269.25 270.03 272.68
273.89 274.46 274.91 275.49 275.3 274.4 273.82 273.22 272.24 271.08
271.16 271.88 273.1 272.81 271.62 271.51 271.72 271.81 271.49 271.55
271.54 271.7 272.01 272.42 272.83 272.95 273.3 273.98 274.13 273.91
273.35 272.19 271.77 271.35 270.84 268.75 267.23 265.45 263.89 262.66
261.81 261.06 260.68 260.13 259.72 259.38 259.1 258.85 258.6 258.48
258.46 258.49 258.53 258.58 258.46 258.15 257.73 257.22 257.04 257.04
256.89 256.97 256.82 256.68 nan nan nan nan nan nan]
[nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan
nan nan nan 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.
15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32.
33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50.
51. 52. 53. 54. 55. 56. 57. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68.
69. 70. 71. 72. nan nan nan nan nan nan]
Example: Hazard chart extension—entirely new functionality#
Warning
Coming soon. In the meantime, you can checkout the source code of the Hazard chart extension directly.