Just learning about #DataMesh. Yet another architecture for large-scale data analysis, by Thoughtworks member Zhamak Dehghani. From the introduction, I get ... maybe let's not put all data in one huge data lake, but treat different sources differently done by their immediate stake holders. Yes!

https://martinfowler.com/articles/data-mesh-principles.html

👋 TDWI München!
Wir sind diese Woche auf der TDWI Konferenz dabei und haben schöne Dinge zu #DataMesh und #MLOps im Gepäck. Und @simonharrer Harrer einen Vortrag zu #DataMeshGovernance. 💪 Schaut vorbei, wir freuen uns auf Euch!

Does anyone here know of some literature on how to share entity relations between domains? Let's say Lionel Messi, with id 123 in some sport system start an acting career and gets id 456 in #imdb. How would the sports department communicate this to some other, third domain so they can join and aggregate? How do you deal with deletion of an entity? Just tombstones? I'd love some research on this topic, as it seems to reinvent itself time and time again
#datamesh #dataproduct #kafka #protobuf

I wrote a post about #dataproducts and why they need a #dataplatform to manage updates over time:
https://www.estuary.dev/data-products-usability/
IDK about y'all, but I've always felt that the bits about data products were the best parts of #datamesh, or at least the most broadly applicable. Any size organization can benefit from treating their data as product, even if the rest of datamesh isn't relevant to you.

🤝 Was sind Data Contracts in #DataMesh und warum sind sie ein wichtiges Werkzeug der Federated Governance? Mehr dazu im Blog von Jochen Christ. 💡

https://www.innoq.com/de/blog/was-ist-ein-data-contract/

#DataWarehouse, #DataLake, #DataMesh, Oh My: The History of #Data #Storage. Looking to get into #DataMesh or curious if your #DataLake is already obsolete? Join us at #GIDS #DeveloperSummit #InPerson https://developersummit.com/, slides: https://robrich.org/slides/data-warehouse-mesh-lake/

Automate discovery of data relationships using ML and Amazon Neptune graph technology 👉 Create automated recommendations to join data products https://aws.amazon.com/blogs/big-data/automate-discovery-of-data-relationships-using-ml-and-amazon-neptune-graph-technology/ #AWS #Analytics #DataMesh #MachineLearning #ML #Graph

@visenger @susannebraun Right now @visenger is giving an intro to #DataMesh.

Improve your team's efficiency with #DataProducts and say goodbye to unproductive data research and gut-based decisions. Daniel shares his experiences with this approach on the blog. #DataMesh

👉 https://www.innoq.com/en/blog/warum-dein-team-data-products-braucht

What's new in the world of #data? This week, Bruno talks #entrepreneurship and invites a special guest to discuss the #datamesh. Catch up on what you've missed by watching the video here: https://venturebeat.com/data-infrastructure/this-week-in-data-lessons-in-entrepreneurship/ #press

Jean-Georges Perrin on #DataEngPodcast talking about #DataMesh in real life (not theoretical!) at PayPal https://www.dataengineeringpodcast.com/building-a-data-mesh-at-paypal-episode-364

Maggie Hays and Pete Soderling on #DataEngPodcast talking about #datacouncil and building an intentional data culture https://www.dataengineeringpodcast.com/data-council-data-culture-track-episode-365

@markrittman talking on #DrilltoDetail with Stewart Bryson, Keenan Rice, and Jake Stein about the Modern Data Stack past, present & future https://overcast.fm/+Iexdb0bH0

#datadon #podcasts #dataengineering

Intrigued by the potential of #DataMesh but uncertain how to get started? @andred outlines an approach on how to get started with your own data product on #AWS 👉 https://www.innoq.com/en/articles/2023/04/creating-data-products-with-terraform-on-aws

#Current23 is the next generation of #KafkaSummit, so expect some #apacheKafka talks. But there's room for everyone! #apacheDruid, #apacheFlink, #ETL, #dataMesh, #machineLearning, #IoT, #databases...

We have room for talks on the unique #devops challenges of monitoring and managing any of these systems; talks on #sociotechnical impact of rearchitecting your business for streaming; #streaming in academia; and just plain fun and geeky sessions.

The list goes on and on!

My article on Data Mesh: Concepts and Best Practices for Implementing a Product-Centric Data Architecture ❤️ ✅

#data #datawarehouse #database # dataengineering #dataengineer #datamesh #architecture #DataArchitecture

https://techleadcuriosity.com/data-mesh-concepts-and-best-practices-for-implementing-a-product-centric-data-architecture-50037063f58e

RT @LaurenMaffeo
Tonight's the night! I'll join @DataScienceDC at #GWU to talk all things #datagovernance, #datamesh, #federatedgovernance, & how to smash all 3 using #humancentereddesign. Join us for free in-person or online! https://www.meetup.com/data-science-dc/events/291231178/ #dctech cc @pragprog

If you talk a lot about data products, analytics, and data mesh seems to connect a lot of relevant dots for you, but the idea self-examination, introspection, and therapy are anathema, your struggle will be very, very real.

#productmanagement #dataproducts #datamesh #analytics #introspection #empathy #subconscious #needs #struggle

Data Mesh is an architecture for decentralized data storage that enables domain teams to utilize the storage technology of their choice. Microsoft provides Trino, a highly parallel and scalable query engine, as a managed service on Azure HDInsight for Data Mesh architectures. https://techcommunity.microsoft.com/t5/analytics-on-azure-blog/data-mesh-architecture-using-hdinsight-trino/ba-p/3733341 #DataMesh #Trino #AzureHDInsight

#chatgpt does such a good job of sounding smart enough about a topic to fool someone with passing familiarity to a sort-of-esoteric topic. Ultimately, though, it still reads like the result of an intern expanding bullet points from an architect into a blog post. #datadon #datamesh

Zhamak presenting “The State of Data Mesh” at #DataDayTexas2023 #datamesh #dataops #dataengineering - “What characterizes a data product?”

#Iceberg , #datamesh and #Datalakehouse in the news for 2023.
https://thenewstack.io/data-2023-revenge-of-the-sql-nerds/