Dataops, the discipline of managing data integration and cleansing operations, is becoming increasingly important for businesses as they strive to leverage analytics and gain a competitive advantage. Dataops teams can now leverage AI and machine learning techniques to improve efficiency, data quality, and reliability. By reducing data... infoworld.com/article/3706090/

#dataOps #datamanagement #airevolution #softcorpremium

Last updated 1 year ago

Stu Watts · @StuWatts
16 followers · 45 posts · Server hachyderm.io

So if you've got monitoring $thing as well as a continuously predicted value for $thing, you can get a rolling MAPE value visualised on a gauge in , just using PromQL! All it takes is literally the ugliest query I've ever seen!

#prometheus #grafana #dataOps #datascience

Last updated 1 year ago

Pascal Heus · @plgah
26 followers · 31 posts · Server hachyderm.io

This MIT Press Data Intelligence Special Issue on Metadata as Data Intelligence is a trove of knowledge around , , , ,
direct.mit.edu/dint/issue/5/1

#metadata #fair #ai #ml #dataOps

Last updated 2 years ago

Jennifer Riggins · @jkriggins
291 followers · 67 posts · Server hachyderm.io

Kicking off this year with a lot of storytelling around those next . The ones that are dedicated to taking down more silos and building on the promise of and .

This one is about the data reliability engineer or SRE for data or database reliability engineer (DBRE), on the front lines as “data is the new oil” becomes “data is a river delta.”

@thenewstack

thenewstack.io/why-a-dataops-t

#techjobs #devops #sre #dataOps

Last updated 2 years ago

Eric Schiller (he/him) · @triplee
32 followers · 71 posts · Server hachyderm.io

I'm not sure when the title of also added full on expert in but I guess I have some learning to do.

For the record, I've always taken security very seriously and understanding of general DevOps practices and philosophy are important to and but I'm having to crash course in some much deeper stuff and its... a lot to take in at once.

#dataengineer #devsecops #dataengineering #dataOps

Last updated 2 years ago

Elendol · @Elendol
53 followers · 118 posts · Server hachyderm.io

Part of me wants to look at software managing movie libraries or photo libraries but not sure how much will apply to my use case (mass spectrometry raw files).

I looked at data pantries / warehouses / lakes etc. but they seem to be dealing with much smaller data. Also I am not interested in anything cloud or internet related.

If anyone has some keywords to help my search that would be helpful.

#dataOps #programming

Last updated 2 years ago

Elendol · @Elendol
53 followers · 118 posts · Server hachyderm.io

Part of me wants to look at software managing movie libraries or photo libraries but not sure how much will apply to my use case (mass spectrometry raw files).

I looked at data pantries / warehouses / lakes etc. but they seem to be dealing with much smaller data. Also I am not interested in anything cloud or internet related.

If anyone has some keywords to help my search that would be helpful.

#dataOps #programming

Last updated 2 years ago

Elendol · @Elendol
53 followers · 118 posts · Server hachyderm.io

If someone knows they are likely to be here

I am looking for a software solution (or pointers to build a solution) able to organise and manage BIG data. Not billions of small text records, dozens of thousand of ~1Gb binary files.

Something better than a basic file system. So maybe a database for the metadata (and something to extract metadata). We want to keep the raw files as source of truth but also track files converted to friendlier formats (with loss).

#programming #devops #dataOps

Last updated 2 years ago