add pism ice-sheet model simulations recipe.py and meta.yaml

[jkingslake]

This adds a simple, first version of a recipe for paleo-pism simulations - simulations of the antarctic ice sheet using a model called PISM, archived here. https://doi.pangaea.de/10.1594/PANGAEA.940149

This first version attempts to simply concatenate along a dimension defined by the ensemble member.

This is discussed here.

[pangeo-forge-bot]

It looks like your meta.yaml does not conform to the specification.

            1 validation error for MetaYaml
    pangeo_notebook_version
      field required (type=value_error.missing)

Please correct your meta.yaml and commit the corrections to this PR.

[jkingslake]

Thanks for reviewing this. I am a little confused, it says
Add more commits by pushing to the [paleo-pism](https://github.com/ldeo-glaciology/staged-recipes/tree/paleo-pism) branch on [ldeo-glaciology/staged-recipes](https://github.com/ldeo-glaciology/staged-recipes).
I added a commit, so should I expect to see these automatic checks being rerun? Or do I need to resubmit the PR in some way? If so, I cant see how to do that.
Thanks for your help!

[andersy005]

Ccing @cisaacstern

[jkingslake]

I think I see why this isnt working. The concat dimension is not a coordinate or dimension in the original netcdfs. I think I need to add a preprocessing step which adds ensemble_number as a coordinate and a dimension.

[cisaacstern]

@jkingslake, good catch re: concat dim name! However the particular blocker on @pangeo-forge-bot's response here looks like it may be an issue with our backend code. I'm patching that now and we'll see if it's fixed momentarily.

[pangeo-forge-bot]

🎉 New recipe runs created for the following recipes at sha 97a9579eafc5d74a6df8d0c0521692a1bf9b8318:

• paleo-pism-stacked: https://pangeo-forge.org/dashboard/recipe-run/1039

[cisaacstern]

☝️ Yep, looks like this is fixed now. 🎉

[pangeo-forge-bot]

🎉 New recipe runs created for the following recipes at sha d0b0040f3dd1a67498a247c289fce76030f82c2b:

• paleo-pism-stacked: https://pangeo-forge.org/dashboard/recipe-run/1040

[jkingslake]

thanks @cisaacstern for fixing that!

I have now added a preprocessing step the recipe which adds the ensemble number (or id number as I have call it now) as a coordinate and a dimension.

Now do I wait for the recipe to run?

[cisaacstern]

Awesome, @jkingslake. I'll trigger the test run now.

[cisaacstern]

/run recipe-test recipe_run_id=1040

[pangeo-forge-bot]

✨ A test of your recipe paleo-pism-stacked is now running on Pangeo Forge Cloud!

I'll notify you with a comment on this thread when this test is complete. (This could be a little while...)

In the meantime, you can follow the logs for this recipe run at https://pangeo-forge.org/dashboard/recipe-run/1040

[pangeo-forge-bot]

Pangeo Forge Cloud told me that our test of your recipe paleo-pism-stacked failed. But don't worry, I'm sure we can fix this!

To see what error caused the failure, please review the logs at https://pangeo-forge.org/dashboard/recipe-run/1040

If you haven't yet tried pruning and running your recipe locally, I suggest trying that now.

Please report back on the results of your local testing in a new comment below, and a Pangeo Forge maintainer will help you with next steps!

[cisaacstern]

@jkingslake from reviewing the logs linked in the previous comment (scroll down on the linked page to find them), it looks like the issue here is that you use np in your preprocessor function but you don't have import numpy as np in your recipe. Based on this error, I'm guessing you haven't run this recipe locally? This error would also come up in the local context. It can save us effort debugging here if your recipe has already been successfully run locally.

[pangeo-forge-bot]

🎉 New recipe runs created for the following recipes at sha bd45a2499c87fe67b7e907d3492141f67be56dc8:

• paleo-pism-stacked: https://pangeo-forge.org/dashboard/recipe-run/1041

[cisaacstern]

/run recipe-test recipe_run_id=1041

[pangeo-forge-bot]

✨ A test of your recipe paleo-pism-stacked is now running on Pangeo Forge Cloud!

I'll notify you with a comment on this thread when this test is complete. (This could be a little while...)

In the meantime, you can follow the logs for this recipe run at https://pangeo-forge.org/dashboard/recipe-run/1041

[pangeo-forge-bot]

🥳 Hooray! The test execution of your recipe paleo-pism-stacked succeeded.

Here is a static representation of the dataset built by this recipe:

            <xarray.Dataset>
    Dimensions:      (id: 2, time: 125, y: 381, x: 381, nv: 2)
    Coordinates:
      * id           (id) <U4 '6000' '6001'
      * time         (time) object -123999-01-01 00:00:00 ... 0001-01-01 00:00:00
      * x            (x) float64 -3.04e+06 -3.024e+06 ... 3.024e+06 3.04e+06
      * y            (y) float64 -3.04e+06 -3.024e+06 ... 3.024e+06 3.04e+06
    Dimensions without coordinates: nv
    Data variables:
        bmelt        (id, time, y, x) float32 dask.array<chunksize=(1, 125, 381, 381), meta=np.ndarray>
        thk          (id, time, y, x) float32 dask.array<chunksize=(1, 125, 381, 381), meta=np.ndarray>
        time_bounds  (id, time, nv) timedelta64[ns] dask.array<chunksize=(1, 125, 2), meta=np.ndarray>
        topg         (id, time, y, x) float32 dask.array<chunksize=(1, 125, 381, 381), meta=np.ndarray>
    Attributes:
        NCO:      netCDF Operators version 4.7.8 (Homepage = http://nco.sf.net, C...
        history:  Tue Oct 19 14:10:39 2021: ncatted -a history_of_appended_files,...

You can also open this dataset by running the following Python code

import fsspec
import xarray as xr

dataset_public_url = 'https://ncsa.osn.xsede.org/Pangeo/pangeo-forge-test/prod/recipe-run-1041/pangeo-forge/staged-recipes/paleo-pism-stacked.zarr'
mapper = fsspec.get_mapper(dataset_public_url)
ds = xr.open_zarr(mapper, consolidated=True)
ds

in this (or your Python interpreter of choice).

Checklist

Please copy-and-paste the list below into a new comment on this thread, and check the boxes off as you've reviewed them.

Note: This test execution is limited to two increments in the concatenation dimension, so you should expect the length of that dimension (e.g, "time" or equivalent) to be 2.

- [ ] Are the dimension lengths correct?
- [ ] Are all of the expected variables present?
- [ ] Does plotting the data produce a plot that looks like your dataset?
- [ ] Can you run a simple computation/reduction on the data and produce a plausible result?

[cisaacstern]

Woohoo! Success! @jkingslake, can you take a look at the dataset linked in the previous comment, and see if it matches your expectations? Note that the time dimension will be pruned to only two increments for this test run.

[rabernat]

ds.thk[0, 0].plot()

[jkingslake]

this is great!
I think it looks good, but I will do a proper check tomorrow.

A quick question: there are a couple of improvements I want to make to the recipe:

1. add four sets of parameter values as coordinates
2. add a post processing step that unstack's the dataset so that it gains four extra dimensions corresponding to the four parameters.

1 is more important and easier for me to do quickly than 2.

Should we go ahead with this simple version to get it working, or should I do 1, or 1 + 2, above first? In other words, how final is this recipe once we proceed from here?

[jkingslake]

• Are the dimension lengths correct?
• Are all of the expected variables present?
• Does plotting the data produce a plot that looks like your dataset?
• Can you run a simple computation/reduction on the data and produce a plausible result?

[jkingslake]

What is the next stage with this recipe? The pruned version all looks good to me.

[cisaacstern]

Thanks for checking in @jkingslake! And apologies for momentarily losing track of this. Re: your question of how "final" this is: not at all! That's the beauty of the feedstock repository model: after we have a dedicated repo for this, we can just keep iterating on the recipe via PRs there. With that in mind, the next step is to merge this! Which I'll do now.

[jkingslake]

Thanks @cisaacstern! I will get working on the new repo now.