In the cloud session, Kerchunk has been mentioned twice as a way to work with legacy formats to optimize them to work in the cloud.

#kerchunk #aas241 #cloud #TOPS #OpenScience

https://github.com/fsspec/kerchunk

For my graduate #ScientificComputing class I made this little #JupyterNotebook for my students showing how to download #CFS data from #AWS, concatenate with #xarray, and save as a #NetCDF file.

This is the data intake step in an Idaho climate forecast dashboard pipeline the class is working on. It could be automated and sped up significantly. Also need to look at #Kerchunk to see if there's a way to stream only the required data (data is GrIB) https://github.com/LejoFlores/Download_CFS_S3/blob/main/DownloadCFS_files.ipynb