Day 3 of #Pearc2023 under way! Today's agenda is to attend the main plenary session and then attend various talks.
My favorite thing from yesterday was seeing the #Spack developers using #Ubuntu virtual machines on #AWS to provide attendees with environments for playing with Spack. Looking forward to the talks today.
#pearc2023 #spack #ubuntu #aws
One of my favourite slides from the #Spack workshop at #Pearc2023. This slide provides a great explanation for why HPC clusters need special-purpose package managers rather than using traditional Linux package managers like apt and dnf.
Next time I have someone ask me why I do not just use apt for installing software, I will just send them this slide 😅
Time to start #Pearc2023 day 2! Today's agenda is to attend the Managing HPC Software Complexity with #Spack workshop. I'm definitely excited to take a deep-dive into the package manager after listening to their #FOSDEM talks earlier this year.
@arichtman iirc, the stuff that seemed slow was compiling from source locally as opposed to a distro’s prebuilt binaries. Package managers like #spack for scientific software also have this problem, but there are binary caches that can be set up for instance. Honestly, the biggest thing that bugs me about brew is the installation: I hate copying a command into my shell from a webpage that executes a curl and then pipes the output into bash.
Bit o' spack wisdom from the Spack Slack
"I have installed GDAL with Spack, and I'm trying to install pip install gdal in my python project. But the python bindings don't link with rpath, and end up not finding them."
"Yeah, for that you’ll probably want to add LD_LIBRARY_PATH to your prefix inspections. Something like:
spack config add "modules:prefix_inspections:./lib:[LD_LIBRARY_PATH]"
spack config add "modules:prefix_inspections:./lib64:[LD_LIBRARY_PATH]"
ok I'm not really active here, but holy heck is having a local package manager and module system useful. I just had to check that something worked on multiple CUDA and GCC versions and with #spack I could just load them all on my desktop without any fuss.
🎁 We didn't bake you a pie today, but we did release @spackpm configs!
📈 Available now for @awscloud #ParallelCluster, they make it easy to install optimized #HPC apps on @intel @AMD & #Graviton EC2 instances.
💡Learn to use them today: https://aws.amazon.com/blogs/hpc/install-optimized-software-with-spack-configs-for-aws-parallelcluster/ #Spack
#parallelcluster #hpc #Graviton #spack
Excited to be on a plane to #fosdem again!
If you’re going to be there, ping me on here or the birdsite, and don’t miss #spack talks from me and Harmen on Sunday, in the Distros and HPC devrooms.
* https://fosdem.org/2023/schedule/event/rolling_release_spack/
* https://fosdem.org/2023/schedule/event/spack_stat_storm/
* https://fosdem.org/2023/schedule/event/spack_ci/
Great website on packaging and distributing #Python applications. It contains a lot of good insight on packages that link or wrap non-python code like #c and #cplusplus
#DataScience #pydata #conda #spack
https://pypackaging-native.github.io/
#python #c #cplusplus #DataScience #pydata #conda #spack
I’m speaking at the Exascale Computing Project annual meeting tomorrow, talking about how far #hpc and aaa #rseng have come in the last 7 years. I plan to call out scientific software packaging through #spack , automation of builds and tests, and the relationship/consensus building that has happened. Does anyone have any anecdotes they would like me to mention?
@hipsterelectron happy to discover #spack maybe it would be interesting to #quitGithub for @Codeberg or #sourcehut they are free from #gafam. Github is not
#spack #quitGithub #sourcehut #gafam
Deploying software for #HPC clusters is often a complex task. Check out how #spack and #Singularity (#Apptainer) with @opensuse Tumbleweed and Leap can make life a little easier. https://mslacken.github.io/opensuse/hpc/2021/12/09/singularity-container-with-spack.html
#hpc #spack #singularity #Apptainer