TOMORROW! E84's own Julia Signell will be representing our team by hosting #SciPy's first advanced #Dask tutorial.

The session will cover how to both improve existing code & generate more performant code using Dask.

More info: https://cfp.scipy.org/2023/talk/MQQJKG/ #SciPy2023

Hey #Python friends - a close buddy is asking which "multi-thread" libraries I like.

I'm not the kind of person who knows *that* answer but he knows I'll likely ask ya'll. 😆

I recommend #Dask but was curious if anyone had anything more.. actualy multithreading?

The ask was - "What python library do you like for multithreading?"

So I kind of shot in the dark. :)

Hot take! Dask and Shredder from Teenage Mutant Ninja Turtles would be so ridonkulous together! #TeenageMutantNinjaTurtles #TMNT #Dask #ThomPinto #Shredder #JamesAvery

Learn how to make your #Python code perform faster during our interactive Parallel #Programming #Workshop, and solve practical problems using #Dask, #Numba, and #Snakemake.
Register now!
https://www.esciencecenter.nl/event/parallel-programming-in-python-3/

HIRING: Senior Software Engineer / San Francisco, CA https://ai-jobs.net/J51131/ #AI #MachineLearning #DataJobs #MLjobs #bigdata #DataScience #AIjobs #AIcareers #hiringnow #SanFrancisco #Redis #APIs #Architecture #Avro #AWS #Postgres #Avro #Datawarehouse #SparkJobs #Dask #PythonJobs

Noticias sobre Python y Datos de la semana, episodio 70 🐍⚙️

En resumen: Versiones nuevas de Anaconda, PyCaret y pandera, diferencias de tablas con data-diff, mejora masiva de uso de memoria en Dask, y... el *temita* de sktime

https://astrojuanlu.substack.com/p/episodio-70

Apoya el noticiero suscribiéndote por correo 📬

#anaconda #pycaret #pandera #datadiff #dask #python #pydata

When a new #Dask release is out I'm just a few clicks away! 💪

Teach me how to make it even cooler 🙏

yt_xarray also allows you to split uniform grids into chunks -- this means that when xarray fields are #dask arrays yt can process dask chunks sequentially as needed within its pre-existing indexing framework!

Right now, dask chunks and yt grid chunks are not guaranteed to align (v0.2.0 will hopefully include this), but setting the chunksizes independently actually presents some interesting possibilities: if yt chunks contain multiple dask chunks some operations might speed up. (3/N)

OK, at the end I did it : I parallelized my highly inefficient #python code with #Joblib https://joblib.readthedocs.io/en/latest/
I discovered you can plug #dask as joblib backend too !!! https://ml.dask.org/joblib.html

This would be amazing, if I only had a cluster to play with....

Hey #RemoteSensing enthusiasts! I've also released #spyndex v0.3.0 and now you have access to #AwesomeSpectralIndices v0.3.0 in #Python! 🐍🛰️🌿

Check it here: https://github.com/awesome-spectral-indices/spyndex

You can use it for #numpy, (geo)pandas, #xarray, #dask and #EarthEngine! 🚀😉

#GISchat #EOchat

RT @MetroStarSystem
Join us for a talk from @_jacobtomlinson, who will walk through his experience using #Dask to accelerate RAPIDS workloads on #Kubernetes. RSVP is free! Join the #PyData DC meetup to register: https://mss.fyi/3XA4YVO

@oleg Mostly source code (so I can learn even more at the same time). Worked great with #ApacheSpark #DeltaLake #ApacheKafka (as they all are written in #Scala). Thinking of #Dask as it's close to Spark but written in #Python I'd like to know better. HTH

@oleg Mostly source code (so I can learn even more at the same time). Worked great with #ApacheSpark #DeltaLake #ApacheKafka (as they all are written in #Scala). Thinking of #Dask as it's close to Spark but written in #Python I'd like to know better. HTH

Hey #Python package #maintainer s, here's a question - is there anything you wish you knew before you switched to date based versioning?

Asking for a friend... and a bunch of other friends... And myself... And our package :D we've been discussing the switch at https://github.com/PlasmaPy/PlasmaPy-PLEPs/pull/39 . I think we've got most of the edge cases covered, following #Dask and #xarray's example - but can't hurt to ask, right?

Day 5 #miniconda with #dask

Still small random baby steps but getting used to #python dev tools.

👉 https://gist.github.com/jaceklaskowski/cf4e82638d2bb459d922df6889c95f8b

Me and 10 Minutes to #Dask with pyenv and jupyter notebook 💪

👉 https://docs.dask.org/en/stable/10-minutes-to-dask.html

Feeling Dask'ed 😏

Found my way into #Dask (and #Python in general) by building dask-distributed first 👍

👉 https://gist.github.com/jaceklaskowski/e22a9c195747604870255368bad614b2

Building the project went fine and really quick 💗 but testing didn't go well so reported an issue 💪

👉 https://github.com/dask/distributed/issues/7422

Can't think of a better way to learn a project! 🎉

Today I've been continuing some work on an experimental #xarray + #dask + #yt pipeline. It's been interesting figuring out how to align the xarray-dask array chunking with yt's domain decomposition and indexing. This meaningless but kinda fun looking image is a plot made with yt using a ~10Gb collection of netcdf files loaded via xarray and dask. yt does some initial culling of files to load then hands off the actual read to xarray and dask. #python #OpenSource

Today I've been continuing some work on an experimental #xarray + #dask + #yt pipeline. It's been interesting figuring out how to align the xarray-dask array chunking with yt's domain decomposition and indexing. This meaningless but kinda fun looking image is a plot made with yt using a ~10Gb collection of netcdf files loaded via xarray and dask. yt does some initial culling of files to load then hands off the actual read to xarray and dask. #python #OpenSource

When using #dask recently, I learned that #bokeh version >=3 breaks the dashboards, with the error message being that you need bokeh version >2.0.4, or so. Incredibly dumb error message. I spent like 2 hours debugging port forwarding to my HPC cluster before realizing that was not at all the issue.

https://github.com/dask/distributed/issues/7362