FedSearch - Federated network search engine

InfoQ · @infoq

703 followers · 316 posts · Server techhub.social

Open media

It's an open-source, Kubernetes-native MLOps platform for developing & deploying distributed ML for the most popular: TensorFlow, PyTorch, and more.

Check out more details on #InfoQ: https://bit.ly/3YXaSBR

#opensource #Kubernetes #MLOps #MachineLearning

#kubeflow #infoq #opensource #kubernetes #MLOps #machinelearning

Last updated 2 years ago

Original post

Puzzle ITC · @puzzle_itc

31 followers · 45 posts · Server mstdn.social

Open media

🚀 Neues Video «Ein Mate mit...» über Machine Learning und #MLOps.

Iwan Imsand, Software Engineer bei Puzzle, nimmt dich in die faszinierende Welt des Machine Learnings mit. Von den Feinheiten zwischen #AI, #MachineLearning und #DeepLearning bis hin zu schicken Open Source Tools, wie: #DVC, #CML, #MLflow, #Kubeflow.

Und weil wir bei Puzzle auch gerne komplexe Probleme lösen, zeigen wir dir, wie wir Unternehmen im MLOps Bereich unterstützen können. 🤖🚀💡

#kubeflow #MLflow #CML #DVC #deeplearning #MachineLearning #AI #MLops

Last updated 2 years ago

Original post

Dotan Horovits #CNCFAmbassador · @horovits

196 followers · 293 posts · Server fosstodon.org

Open media

glad to see #MLOps growing within the CNCF. the @kubeflow project has been around since 2017, offering a #cloudnative solution to deploy machine learning workloads on #kubernetes.
Congrats to the #kubeflow team👏
https://cncf.io/blog/2023/07/25/kubeflow-brings-mlops-to-the-cncf-incubator/
#MachineLearning #DevOps

#MLops #cloudnative #kubernetes #kubeflow #machinelearning #devops

Last updated 2 years ago

Original post

Diego Ciangottini · @imdciangot

45 followers · 84 posts · Server fosstodon.org

Registrations for the fifth edition of the International School on Open Science Cloud are now open #SOSC23 For more details visit: https://web.infn.it/SOSC23 #OpenScience #CloudComputing #DataScience #ArgoWorkflows #kubeflow

#sosc23 #openscience #cloudcomputing #datascience #ArgoWorkflows #kubeflow

Last updated 2 years ago

Original post

ai-jobs.net · @aijobs

41 followers · 169 posts · Server mstdn.social

ai-jobs.net - Senior Machine Learning Ops Engineer

HIRING: Senior Machine Learning Ops Engineer / Remote Spain https://ai-jobs.net/J48035/ #AI #MachineLearning #DataJobs #AIcareers #MLjobs #bigdata #DataScience #AIjobs #hiringnow #RemoteJob #Airflow #DeepLearning #DevOps #Engineering #Genetics #Kubeflow #Kubernetes #Mathematics

#mathematics #kubernetes #kubeflow #genetics #engineering #DevOps #deeplearning #airflow #remotejob #hiringnow #AIjobs #DataScience #bigdata #MLjobs #aicareers #DataJobs #MachineLearning #AI

Last updated 2 years ago

Original post

Wu Yuansheng · @admin

22 followers · 6321 posts · Server social.xinghaizhandui.com

RT igor_os777: Charmed Kubeflow 1.7 is now available http://dlvr.it/Slg6H9 #AIML #AIML #Kubeflow

#aiml #kubeflow

Last updated 2 years ago

Original post

Joe Seeder · @seeder

15 followers · 60 posts · Server mas.to

can you contain #kubeflow inside #vcluster and still have access to #gpu on the nodes?

#gpu #vcluster #kubeflow

Last updated 3 years ago

Original post

Kathy Reid · @KathyReid

2822 followers · 1062 posts · Server aus.social

Good morning folx of #Fediverse :fediverse: ! I interrupt swearing at #CUDA and #Torch to bring you today's #ConnectionList #Introduction #FollowFriday, where I help to more richly connect the Fediverse.

@terrytangyuan is a #MachineLearning #Kubernetes #OpenSource #MLOps engineer who is co-chair of #KubeFlow and maintains #TensorFlow (thank you!).

@yarnspinner is a #dialogue #dialog tool for #GamesDev, brought to you by the team @thesecretlab

@Cerchie is Lucia who is a #DevRel #DeveloperAdvocate at #Confluent, based in the 🇺🇸 👋

@ocramz is Marco and he is into #MachineLearning #ML #NLP #λ #lambda and #NLProc

@lira is into #poetry, #ComputerScience and #Philosophy

@frankjhopwood is a #PhD candidate #researcher #academic at #Groningen working on #speech technology #SpeechSynthesis #TTS for #Frisian. Fun Frisian fact: did you know that "keks" in Frisian means "cookies"?! Cakes and cookies! (Sorry I've always found that very amusing! 🍰 🍪 )

@blairpalese is Director of #Philanthropy at #EthInvest and Media Director and Editor at Climate and Capital Media. She's into #GreenFinance #ClimateRisk #Environment 🌱

Don't forget to share your own #ConnectionList so we can connect more widely ❤️

#fediverse #cuda #torch #connectionlist #introduction #followfriday #machinelearning #kubernetes #opensource #MLOps #kubeflow #tensorflow #dialogue #dialog #gamesdev #devrel #developeradvocate #confluent #ml #nlp #λ #lambda #nlproc #poetry #computerscience #philosophy #phd #researcher #academic #groningen #speech #speechsynthesis #tts #frisian #philanthropy #ethinvest #greenfinance #climaterisk #environment

Last updated 3 years ago

Original post

Holden · @holden

756 followers · 119 posts · Server tech.lgbt

YouTube - Kubeflow for Machine Learning • Holden Karau & Adi Polak • GOTO 2022

RT @GOTOcon
Another day, another 2022 GOTO YouTube video on the TOP 5📚

In Kubeflow for ML, @AdiPolak and @holdenkarau talk about the design principles of #Kubeflow & how to use it to build machine learning pipelines.

How did you enjoy this #GOTObookclub episode?

https://www.youtube.com/watch?v=dUZ5TrSReZc

#kubeflow #GOTObookclub

Last updated 3 years ago

Original post

vsoch · @vsoch

335 followers · 150 posts · Server mastodon.social

I’m going to be adding tests to our #FluxFramework #Kubernetes operator this week! I’ve seen scorecard for the operator-sdk but I’m looking for examples in the wild using best practices, and possibly not sticking to those suggested by that SDK. The e2e and unit tests in the #kubeflow MPI operator with ginkgo and gomega look nice! 👍 https://github.com/kubeflow/mpi-operator/blob/master/v2/test/e2e/mpi_job_test.go Anyone have other examples they like or experiences to share? 🤔

Dino #koobernottie

#fluxframework #kubernetes #kubeflow #koobernottie

Last updated 3 years ago

Original post

vsoch · @vsoch

383 followers · 214 posts · Server mastodon.social

GitHub - mpi-operator/mpi_job_test.go at master · kubeflow/mpi-operator

I’m going to be adding tests to our #FluxFramework #Kubernetes operator this week! I’ve seen scorecard for the operator-sdk but I’m looking for examples in the wild using best practices, and possibly not sticking to those suggested by that SDK. The e2e and unit tests in the #kubeflow MPI operator with ginkgo and gomega look nice! 👍 https://github.com/kubeflow/mpi-operator/blob/master/v2/test/e2e/mpi_job_test.go Anyone have other examples they like or experiences to share? 🤔

Dino #koobernottie

#fluxframework #kubernetes #kubeflow #koobernottie

Last updated 3 years ago

Original post

Charles U. Farley · @freakazoid

839 followers · 18541 posts · Server retro.social

I'm currently keeping the lights on for a deprecated installation of #Kubeflow Pipelines, which uses #Argo Workflows under the hood. None of the components has been updated for at least a couple years now, and we're planning on turning it off in June anyway, but for now there are some critical workflows that depend on it.

Recently the team that manages #Kubernetes updated the #OpenPolicyAgent gatekeeper and added some new policies, which dramatically increased its CPU consumption. The latency of Kubernetes API calls goes through the roof whenever QPS increases, until the Gatekeeper has time to scale out. This causes the Argo workflow controller to fall behind and fail to do whatever it needs to do to terminated pods long enough that the Kubernetes garbage collector gets to them first, which causes the workflow to end in an "error" state.

Now our namespace has been exempted from the Gatekeeper, so all the workflow controller's problems seem to be its own. It was still falling behind and missing terminated pods during periods of high load, so I tried scaling it up by increasing GOMAXPROCS, "workflow-workers", "pod-workers" (a now obsolete concept) as well as "--qps" and "--burst". The last two appear to have been the most important.

Now it's getting into a state where it's repeatedly complaining about conflicts when trying to update workflows. Once it starts happening with a given workflow it keeps happening, but the first retry always works. CPU consumption goes way up when this happens, and it starts missing terminated pods again. Most of the situations I can find like this on the 'net involve something external updating the workflow (we have nothing like that AFAIK) or a mutating admission controller. I can't find any evidence of properties that would be set by a mutating admission controller, so I'm guessing this is a workflow controller bug, probably related to concurrency.

I've reduced "workflow-workers" and "pod-workers" back to 2 and set GOMAXPROCS to 1 (which I think is the default, but I wanted to make sure). It's not happening right now, but load is also low and it seems to happen during periods of high or at least rapidly increasing load.

Thoughts?

#kubeflow #argo #kubernetes #OpenPolicyAgent

Last updated 3 years ago

Original post

Volkan Özçelik · @volkan

27 followers · 1056 posts · Server z2h.dev

GitHub - GitHub - kubeflow/kubeflow: Machine Learning Toolkit for Kubernetes

Kubeflow is a Cloud Native machine learning toolkit.

https://github.com/kubeflow/kubeflow

#AI #ML #Kubeflow #Kubernetes #tools

#ai #ml #kubeflow #kubernetes #tools

Last updated 3 years ago

Original post

Pedro · @pmadruga

55 followers · 74 posts · Server sigmoid.social

Open media

I've been looking at different orchestration tools for #MLOps and was curious about where's the community heading these days.

Of course, GitHub stars are a flawed metric for that, but still... I was curious.

Here's what I found. The main goal was that it needed to be open source. And goes without saying that these tools are not completely alike.

Let me know if there's any tool missing.

#machinelearning #datascience #kubernetes #airflow #flyte #kubeflow #mlflow #argo #polyaxon

#MLops #machinelearning #DataScience #kubernetes #airflow #flyte #kubeflow #mlflow #argo #polyaxon

Last updated 3 years ago

Original post

Lars · @lfloeer

41 followers · 56 posts · Server bonn.social

#introduction

I live in #Bonn and work in #DataScience and #MachineLearning Engineering. My projects in range from maintaining #Kubeflow installations in on-premises and cloud #Kubernetes clusters to developing #MLOps best practices for teams.

I mostly develop in #Python, #YAML, #Terraform, and am dabbling in #Rustlang.

I got my PhD in #Astronomy being a small part of the #EBHIS and #HI4PI team.

In recent years I’m having fun riding my #eMTB all across the #Eifel and #Ahrtal. #MTB

#introduction #bonn #datascience #machinelearning #kubeflow #kubernetes #MLOps #python #yaml #terraform #rustlang #astronomy #ebhis #hi4pi #emtb #eifel #ahrtal #mtb