Michael Sumner · @mdsumner
403 followers · 3141 posts · Server fosstodon.org

many data providers don't even know about the benefits of well structured GeoTIFF (let alone the buzz of COG, STAC, Zarr, etc)

I reach out to providers, but more often than not there's just another individual there scratching head at my crazy email (who even reads bathymetry on the fly from a url, nuts right...)

at institution I can try collectively to feedback, but the and and communities could join forces here - is there something in the zone?

#spatial #python #rstats #gdal #xarray

Last updated 1 year ago

ashwinvis :cc: · @ashwinvis
189 followers · 1311 posts · Server fediscience.org

:python: tip

scivision.dev/xarray-to-netcdf

to_netcdf() can compress data variables to save considerable disk space, but it is not trivial until your read the docs.

#python #netcdf #xarray

Last updated 1 year ago

sirhc · @sirhc
78 followers · 174 posts · Server fosstodon.org

A bunch of tools coming together... was revamping a soon-to-be released package that streamlines some data IO and analysis for datasets that contains pipelines to (and -xarray), and . The package came out of my need to extract data for some k-means analysis, but it also makes it very easy to pop data over to yt and in this case, !

#seismic #yt #xarray #Geopandas #yt_idv #python #opensource #datavisualization

Last updated 1 year ago

Verwechslungsgefährte · @dichotomiker
187 followers · 2764 posts · Server dresden.network
偉吉 · @weiji14
51 followers · 48 posts · Server mastodon.nz

☯️ v0.6.1. Quick patch release with full support for 🐍 Python 3.11! Made a couple of bug fixes 🐞 to DatashaderRasterizer (for turning points/lines/polygons to pixel masks based on a template raster). Also sneaked in a tiny enhancement ✨ to XpySTACAssetReader (which wraps around `xarray.open_dataset`) to work without xpystac 🙃. Find out more in the release notes at zen3geo.readthedocs.io/en/v0.6.

@pytorch

#zen3geo #torchdata #python #datashader #xarray #gischat #geospatial #datapipeline #computervision

Last updated 1 year ago

Benoît Bovy · @benbovy
58 followers · 11 posts · Server fosstodon.org

Preparing demos for next week at . Here is a sneak peek (turn on the audio, best with headphones). See you there!

💨💨 🔊💨💨

#jupytercon #sonification #xarray #ipyleaflet #ipytone #jupyter

Last updated 1 year ago

Jose · @jgomezdans
165 followers · 505 posts · Server fediscience.org

@conorosully @lavergnetho Also, consider getting a Microsoft Planetary Computer account. Although Google's equivalent has been around longer, allows you to do these things easily with standard Python tools and libraries (eg )

#xarray #PlanetaryComputer

Last updated 2 years ago

偉吉 · @weiji14
53 followers · 47 posts · Server mastodon.nz

☯️ v0.6.0. Scale to new heights if you dare. Open 📂 STAC Assets backed by Zarr, NetCDF or CloudOptimizedGeoTIFF files using the new XpySTACAssetReader DataPipe. Attempt super-resolution 🔭 with the walkthrough on handling multi-resolution CMIP6 climate data from CarbonPlan over the Philippines 🇵🇭 at zen3geo.readthedocs.io/en/v0.6. Clarity, seek it, give it 🥽

@zarr @pytorch

#zen3geo #STAC #netcdf #cloudoptimizedgeotiff #xarray #DataPipe #superresoluton #multiresolution #carbonplan #cmip6 #ClimateModel

Last updated 2 years ago

Henry Rodman · @hrodmn
10 followers · 18 posts · Server mas.to

STAC + stackstac + xarray is a powerful combination. I put together a short post highlighting my favorite new tools for working with satellite imagery.

hrodmn.dev/posts/stackstac/

#remotesensing #stac #dataengineering #eo #xarray #geospatial #Python

Last updated 2 years ago

sirhc · @sirhc
73 followers · 142 posts · Server fosstodon.org

yt_xarray release day! v0.1.1 now available on pypi!

yt-xarray.readthedocs.io/

yt_xarray is an xarray accessor that streamlines communication between and , allowing you to easily pull out a yt dataset from regularly gridded datasets that xarray can load, allowing you to more easily use yt's plotting and analysis methods. (1/N)

#yt #xarray #python #opensource #geoscience

Last updated 2 years ago

Blue Tyson · @bluetyson
161 followers · 2063 posts · Server dice.camp
David Montero Loaiza · @dmlmont
72 followers · 14 posts · Server fosstodon.org

Hey enthusiasts! I've also released v0.3.0 and now you have access to v0.3.0 in ! 🐍🛰️🌿

Check it here: github.com/awesome-spectral-in

You can use it for , (geo)pandas, , and ! 🚀😉

#remotesensing #spyndex #awesomespectralindices #python #numpy #xarray #dask #EarthEngine #gischat #eochat

Last updated 2 years ago

Hey package s, here's a question - is there anything you wish you knew before you switched to date based versioning?

Asking for a friend... and a bunch of other friends... And myself... And our package :D we've been discussing the switch at github.com/PlasmaPy/PlasmaPy-P . I think we've got most of the edge cases covered, following and 's example - but can't hurt to ask, right?

#python #maintainer #dask #xarray

Last updated 2 years ago

gauteh · @gauteh
11 followers · 17 posts · Server hachyderm.io

hidefix (github.com/gauteh/hidefix) can now be used as an backend: more than 6x faster on a 8 CPU machine!

This reads a full variable of about 370mb.

#xarray

Last updated 2 years ago

gauteh · @gauteh
10 followers · 12 posts · Server hachyderm.io

On a machine with 64 CPUs reading with hidefix is between 10x and 16x faster than native HDF5!!

And that is _including_ generating the index.

Going from 20 seconds to 1.8 seconds is pretty significant when reading a bunch of variables! 7s to 380ms, is the difference between waiting and immediately in processing!

This is for a file on a network disk.

The ultimate goal: Make this into a backend!

#xarray #rust #hidefix #hdf5 #data #met

Last updated 2 years ago

sirhc · @sirhc
71 followers · 125 posts · Server fosstodon.org

Today I've been continuing some work on an experimental + + pipeline. It's been interesting figuring out how to align the xarray-dask array chunking with yt's domain decomposition and indexing. This meaningless but kinda fun looking image is a plot made with yt using a ~10Gb collection of netcdf files loaded via xarray and dask. yt does some initial culling of files to load then hands off the actual read to xarray and dask.

#xarray #dask #yt #python #opensource

Last updated 2 years ago

sirhc · @sirhc
72 followers · 132 posts · Server fosstodon.org

Today I've been continuing some work on an experimental + + pipeline. It's been interesting figuring out how to align the xarray-dask array chunking with yt's domain decomposition and indexing. This meaningless but kinda fun looking image is a plot made with yt using a ~10Gb collection of netcdf files loaded via xarray and dask. yt does some initial culling of files to load then hands off the actual read to xarray and dask.

#xarray #dask #yt #python #opensource

Last updated 2 years ago

Romain Caneill · @rcaneill
17 followers · 7 posts · Server fediscience.org

Working with gcm and ?

I just released version 0.4.0 of , a package that can read nemo outputs, and adds the proper attributes (+ change dimension names) to be read by the fantastic package.

It tries to automatically discover the T, U, V, W, etc point of variables, and sets everything for you :). For more complicated cases (dataset already loaded into , dataset with no attributes, etc) you can easily set the point by hand.

github.com/rcaneill/xnemogcm/r

#xarray #xgcm #netcdf #xnemogcm #python #nemo

Last updated 2 years ago

Thomas Lavergne · @lavergnetho
444 followers · 341 posts · Server fediscience.org

@seaice @carlosmoffat i am not so sure, Lars, but it is true that this is a legacy software that does not use . At least it is 🙂

#python3 #xarray

Last updated 2 years ago

Lars Kaleschke · @seaice
366 followers · 76 posts · Server det.social

@lavergnetho @carlosmoffat With this would not have happened 😂

#xarray

Last updated 2 years ago