3 Crucial DataOps Strategies For Scaling Organizations

#bigdata #dataanalytics #datamanagement #datascience #dataops
https://usnewsper.com/2023/08/3-crucial-dataops-strategies-for-scaling-organizations/?feed_id=27221&_unique_id=64e498e6b8cc8

A8. Which incorporates standardizing on and sharing 'known good' practices - encoding them in Process Automation so anyone can use analytics properly; applying approaches like #MLOps & #DataOps to reuse 'known good' algos; etc. #eweekchat

It's that times of the week for All Hands on #data 🥳

Whether you are trying to escape the summer heat, or hang out with Python's favorite bear 🐼 - this week's articles from across the data space have something for you!

Full articles 👇

(Subscribe to stay up to date 😊)

#dataorchestration #mlops #dataops #dataengineering

https://allhandsondata.substack.com/p/all-hands-on-data-61

Q. What's the best thing about open-source software?

A. You can modify it and add your own features❤️

That's exactly what Alex, a senior data engineer at domain.com.au, did with PipeRider.

Alex added features specific to his data migration use-case of comparing two datasets on the row-level, with tolerance

https://www.youtube.com/watch?v=SSoFyWTAYGw

#OpenSource #DataQuality #DataReliability #DataLineage #DataOps #DataTools #DataViz

🔥⏲️ Fudge Sunday "Shock the Chaos Monkey" A look at chaos engineering adoption within platform engineering teams

#chaos #chaosengineering #faulttolerance #faultfinding #faultinjection #securityengineering #dataops #dataoperations #ai #resiliency #devx #devops #devsecops #platformengineering #platformengineer #cloudinfrastructure #securityautomation #securitybydesign #shiftleft #shiftup #newsletter #newsletters

https://fudge.org/archive/shock-the-chaos-monkey/

Do you ever turn off access to your models / MI and dashboards to see if anyone even notices? 🚫✅

#MLops #dataops #modelmanagement #ML #AI

The latest issue of 'In the Pipeline', the PipeRider newsletter, is coming very soon.

Make sure you don't miss out by signing up!

https://app.us1.list-manage.com/subscribe?u=2b18366427f11835f05f68aeb&id=b82bfd845e

Featuring info about:

- New PipeRider features
- Data best practices
- Guest interviews
- Interesting data-related content

#DataTools. #DataOps #DataQuality #DataReliability #DataNewsletter #dbt #PipeRider

The PipeRider Community Office Hours for June 20th is now online:

https://www.youtube.com/watch?v=6scbaNMfXSc

- New features in PipeRider 0.27

- Lineage Diff in PipeRider Cloud!

- Special guest Spencer Ellinor from Sudo Labs

Timestamps and links in the video description

#DataQuality #DataReliability #PipeRider #DataViz #dbt #DataOps #LineageDiff #CodeReviewForData

PG Day France 2023 : Retour d'Expérience sur l'automatisation « IaaS » du déploiement de #PostgreSQL avec #Ansible chez la @MAIF par  Yohann MARTIN, Clément PAILLIER et Ludovic AUGEREAU

En direct sur https://live.pgday.fr
#pgdayfr #postgresql #ansible #dataops

PipeRider now has a dedicated tools channel on the dbt Slack!

Join the dbt Slack here:
https://getdbt.com/community/join-the-community/

Then, either search for the #tools-piperider channel, or follow this link:
https://getdbt.slack.com/archives/C05C28V7CPP

See you there 💪

#dbt #DataQuality #DataOps #OpenSource #DataReliability #DataTools #CodeReviewForData #PipeRider

How do you track down and explore data changes in your dbt project?

One way is to explore rows that fall outside previous boundaries

Check out this upcoming PipeRider feature that highlights a change, and shows the SQL you need to find the affected rows

#dbt #DataQuality #eda #DatatViz #sql #DataOps #DataReliability #DataEngineering #AnalyticsEngineering

#ITByte: #DataOps is a discipline that merges data engineering and data science teams to support an organization’s data needs, in a similar way to how DevOps helped scale software engineering.

https://knowledgezone.co.in/trends/explorer?topic=DataOps

You already use dbt to empower your data modelling workflow, now you need a way to understand how your code changes affect data

That's where "code review for data" comes in

Zero-config dbt integration and merge with confidence 💯

Get started today at
https://piperider.io

#dbt #DataQuality #DataReliability #OpenSource #DataOps #DataViz #DataProfile #DataTesting #PipeRider

PipeRider 0.26.0 has just been released 📢

'PipeRider compare' is a powerful command, so we've included two options to guide you through using it:

--dry-run
will show each command that will be run

--interactive
will guide you step by step

More info:
https://github.com/InfuseAI/piperider/releases/tag/v0.26.0

To update just run:
pip install -U piperider

#DataQuality #PipeRider #DataReliability #DataOps #DataDiff #dbt

Hi all!

PipeRider 0.25.0 has just been release with an increased focus on dbt integration.

You no longer need to initialize PipeRider in dbt projects (piperider init).

To get started in a new project, all you need to do is:

1. Install PipeRider 👩‍💻
2. Tag your models 🏷️
3. Run PipeRider 🏃‍♂️
4. Enjoy rich data profiling reports and improve your code review process 📊

Find out more:
https://medium.com/inthepipeline/zero-config-code-review-and-data-profiling-tool-for-dbt-projects-8b6de40964b4

#DataQuality #DataEngineering #DataOps #DataReliability #AnalyticsEngineering

Do you have a proper code review process for pushing changes to data projects?

With ELT and dbt, small changes to code can have a big impact on downstream models. The last thing you want is to push a breaking change to prod 😱

Why you need code review for you data projects
by Balu Rama Chandra

https://medium.com/inthepipeline/why-you-need-code-review-for-you-data-projects-95e039083df0

#DataEngineering #DataQuality #DataOps #CodeReview #dbt #AnalyticsEngineering

dbt users - Do you consider data warehouse costs when building and maintaining projects?

If so, what steps you taking to keep costs down, or keep an eye on costs?

#dbt #DataQuality #DataBuildTool #DataEngineering #AnalyticsEngineering #DataOps

[Part 5]
💭 If you're reading this & you've been involved with developing an ML Platform, how did you approach the "centralize vs distributed" discussion? What worked? What failed?

👇 Let me know in the comments below!

#mlops #mlplatform #ai #strategy #dataops #dataengineering #devops #platformengineering #platformdesign #mlengineer

📝 What kind of MLOps team are you? [Part 3/3]
#mlops #productionml #dataops #mlsystems

In early starts-ups & even at the Small/Med Size business, teams are often a combination of the different modes & that's totally fine!

You don't always need a specialized team!

💡What's important to recognize is to know this framework exists for organziational alignment, as well as to know when teams can be spun out.

"#DataOps: Understanding the Definition and Differentiation"

Interesting thinking from @MAslettVR of @ventanaresearch, not sure I agree with it all but important data and positive actionable considerations regardless.

HT @marksmithvr - https://mattaslett.ventanaresearch.com/dataops-understanding-the-definition-and-differentiation