RT @ypriverol: A nice/small (10MB) mulled @BioContainers to decompress all types of files: zip, gzip, tar, etc. https://quay.io/repository/biocontainers/mulled-v2-796b0610595ad1995b121d0b85375902097b78d4?tab=tags&tag=latest Good for your @nextflow / #snakemake of batch workflows.

The analysis was performed using our SIB Software Resource #Vpipe - the #NGS #bioinformatics pipeline tool for virus #sequencing.
Detection handled by the component COJAC.

More details on our project and procedure available at https://cov-spectrum.org/stories/wastewater-in-switzerland and at https://github.com/cbg-ethz/cowwid
…

#opensource
#snakemake

Recently started using singularity on main lab server with snakemake. Did the install one day and forgot to note the install instructions.

Added golang and singularity to another lab server today. Took some bashing (haha) my head against the wall, but got there without too much trouble. But I was smarter this time about actually taking notes.

Always make notes the first time.

#Linux
#Bioinformatics
#Genomics
#SnakeMake

Been struggling with a #snakemake workflow for 3 days and couldn't fix it no matter what wacky ideas I tried. Started rewriting it from scratch, no changes, and it works. Wtf.

5 Latest Tools You Should Be Using With Python for Data Science.
🗂️ The article provides insightful details on tools like ConnectorX, DuckDB, Optimus, Polars, and Snakemake which could enhance data wrangling, querying, manipulation, and workflow automation capabilities.

1. 🧰 ConnectorX: Simplifying the Loading of Data
2. 🧰 DuckDB: Empowering Analytical Query Workloads
3. 🧰 Optimus: Streamlining Data Manipulation
4. 🧰 Polars: Accelerating DataFrames
5. 🧰 Snakemake: Automating Data Science Workflows

https://www.makeuseof.com/latest-python-data-science-tools/

#Python #DataScience #ConnectorX #DuckDB #Optimus #Polars #Snakemake #Programming #DataAnalysis #Productivity

Also, workflow engines cannot pick up modules reliably, as module naming schemes may differ from cluster to cluster. Hence, they rely on conda and add options allowing for environment modules and/or containerized software.

The HPC world needs to find an answer, as reproducible workflows carried out by workflow engines such as #snakemake or #nextflow are what data analysis communities are working with (across fields).

All analysis is #opensource BTW, often using the amazinh #snakemake tool: :snake:

The code: https://github.com/fneum/spatial-sector

The framework: https://github.com/pypsa/pypsa

The model: https://github.com/pypsa/pypsa-eur

#freethemodels #openmod #macroenergysystems

#Bioinformatics #HiveMind #SnakeMake #Docker

Uni cluster runs using Docker images, so I have snakemake setup to specify the images appropriately dispatching jobs.

But I'm switch back and forth between our Linux servers and the cluster. Aiming to run jobs on our servers using the same images to avoid version conflicts and unexpected behavior.

Anyone using snakemake with dockerhub images on a standard Linux server? The docs aren't clear on compatibility and how to set it up so far.

I wrote a thing (first blog post)
#Snakemake #git #ScientificSimulation

https://www.normalesup.org/~asimon/posts/2023-06-30-snakemake-git-simulations/

#snakemake 7.30 has been recently released. We have massively improved the profile support (Snakemake's way of defining default settings).

There are now workflow specific profiles in addition to global profiles. You can use them to e.g. modify resource usages of rules without having to touch the workflow code. Also, the syntax for such definitions in profiles has been simplified. (1/n)

#snakemake v7.29.0 is out . And that's fantastic! Why?

Check out the update on the `--workflow-profile`-directive, here: https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles . Now, configuring workflows for #HPC clusters is way simpler, no need to edit the workflow any more.

Many thanks to @johanneskoester for his tremendous effort!

#OpenScience #reproducibility #Bioinformatics

You are doing #Bioinformatics? Would you say data analysis requires interactive work, e.g. using jupyter, rstudio, etc.? Or would you prefer using a workflow system, e.g. #snakemake, #nextflow, #galaxy, etc.?

I am not talking about visualization, though. Visualization is important and can be achieved regardless of the approach.

Please boost. Thank you.

Learn how to make your #Python code perform faster during our interactive Parallel #Programming #Workshop, and solve practical problems using #Dask, #Numba, and #Snakemake.
Register now!
https://www.esciencecenter.nl/event/parallel-programming-in-python-3/

Introducing #Snakemake for #Azure Batch https://techcommunity.microsoft.com/t5/healthcare-and-life-sciences/introducing-snakemake-for-azure-batch/ba-p/3850649

I am happy to announce that I am seeking for a #postdoctoral bioinformatician or computer scientist for a full, #permanent_position as a research fellow in my lab! Co-maintain popular #opensource projects like #Bioconda, #Snakemake, and #Varlociraptor!
Develop exciting, versatile, and sustainably maintained solutions that will shape the future of precision medicine and bioinformatics in general.

https://koesterlab.github.io/#research-fellow-permanent

#snakemake 7.26 was recently released. It enables to use #yte templating in config files. While this can be very powerful, one should of course use this with care as it can hamper the readability of the config.

https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html

A good indicator is: curated support for workflow engines such as #snakemake, #nextflow, #galaxy .

Updated usage statistics for #Snakemake: we are now at >10 new citations per week on average and over 700k downloads of the #Conda package (not even counting pypi). Thanks a lot for the active use and all the feedback and contributions! https://snakemake.github.io

Blog post: "snakemake for doing bioinformatics - inputs and outputs and more!" http://ivory.idyll.org/blog/2023-snakemake-slithering-input-outputs.html #snakemake

Comments, corrections, additions, and thoughts solicited!

@khinsen described a pretty compelling use case for nested #Guix containers for workflow engines such as #Snakemake (widely used in bioinfo):
https://issues.guix.gnu.org/62411

#OpenScience