Detailed, reproducible, and relatively objective analysis in a blog comparing performance, compression, and costs between #ClickHouse Cloud and #Snowflake

https://clickhouse.com/blog/clickhouse-vs-snowflake-for-real-time-analytics-benchmarks-cost-analysis

Summary:
* ClickHouse Cloud is 3-5x more cost-effective than Snowflake in production
* ClickHouse Cloud querying speeds are over 2x faster compared to Snowflake
* ClickHouse Cloud results in 38% better data compression than Snowflake

#MachineLearning directly in the database!

With #ClickHouse you can train a ML model (stochasticLinearRegression or stochasticLogisticRegression) and then use it to perform predictions!

The flow below also uses the s3 function to read the data from a bucket

Check it out with Aiven for ClickHouse

cool cool can't sleep since 4am because this is the OPTIMAL hour to have awesome tech ideas (or hate your life/job/relationship/neighbors/yourself of course) either way really good time to think of fundamental shit like "should I use #sqlite or #clickhouse oh I know I use both!1!! can't have enough databases!!"

Directly reading from and writing to AWS S3 buckets: faster time to analytics with #ClickHouse!

We just enabled it on Aiven for ClickHouse, check the doc
https://docs.aiven.io/docs/products/clickhouse/howto/run-federated-queries

I think for the smaller and mid-sized #observability "signoz" is looking good. it's a frontend bringing logs, traces, metrics together, stored in #clickhouse (which I can really recommend, I'm quite happy with it - but not yet arrived at petabyte level, can't attest to bigger setups yet)

Have I shown our #GitHub #actions statistics available with #ClickHouse public play.clickhouse.com instance?

https://pastila.nl/?01d5d14e/b4555c6cb4845860d90d2a1615a563e4.link#aqjvBN52V3CGcko2U783WA==

It's impossible to achieve with given GH UI, so it's implemented with a simple set of tools https://clickhouse.com/blog/monitor-github-action-workflow-job-queues

@akahn nice job! it might be interesting to note that cascaded (or chained) Materialized Views do work, and in fact are updated together with the original source table:

https://clickhouse.com/docs/en/guides/developer/cascading-materialized-views

https://clickhouse.com/docs/knowledgebase/are_materialized_views_inserted_asynchronously

recursive Materialized Views are probably another matter though :)
#clickhouse

Here’s an idea for a new web stack, CHL. #ClickHouse #HTMX and a #Lisp. I’d pronounce it CHilL🍹

Another round of ClickHouse Fundamentals free virtual instructor-led (live) training this week August 16 & 17 at 1 PM BST / 2 PM CEST / 5 AM US Pacific each day for 3 hours

Free enrollment here:
https://clickhouse.com/company/events/clickhouse-workshop

This course applies to both #ClickHouse #opensource or #Cloud users with lecture and hands-on labs - come join us and learn!

@julioj @Perl Thanks Julio. Looking at various 2D solutions to utilize and #clickhouse is definitely among those considered for a SQL form of solution (bonus there is a R interface - many analytics stuff are written in R).
Will take a look!

@ChristosArgyrop @Perl Hey Christos, I’m partial to #ClickHouse as you can see from my bio. In my security work I use it for network traffic metrics and observability. I mainly use JavaScript, Python, and Common Lisp to interact with ClickHouse. For R there’s https://github.com/IMSMWU/RClickHouse

Also, feel free to join our Slack where you can ask any and all questions from our community of users https://clickhousedb.slack.com

https://clickhouse.com/docs/en/intro

detailed and genuinely useful blog on how we built our internal data warehouse on #ClickHouse at ClickHouse

https://clickhouse.com/blog/building-a-data-warehouse-with-clickhouse

#ClickHouse #DataWarehouse
starring
#Superset #AppFlow #Airflow #Redis #Docker #opensource #RDS #PostgreSQL #DBT
and including data from
#Salesforce #AWS #S3 #BigQuery #M3ter #Segment #Marketo

With ClickPipes, #ClickHouse joins the Connect with
@confluentinc partner program!

Real-time data streams available directly within ClickHouse Cloud!

Hear about our integration with Confluent →

https://youtu.be/rSUHqyqdRuk

#olap #db #data #database

Learn everything you need to get started with #ClickHouse #opensource or #Cloud in this 2-day #free #training course “ClickHouse Fundamentals”

Join us on July 11 & 12 for 2 days (3 hrs daily at 8am US Pacific / 17:00 CEST) of virtual instructor-led courses with hands-on labs

Enroll here:
https://clickhouse.com/company/events/clickhouse-workshop

Due to recent overflow, we are sorry to limit number of PRs per day in the #clickhouse

Core engineers: 5
Contributors: 3
New contributors: 1

We plan to address it eventually!

@akahn I think I understand - it sounds like you have "semi-structured" data. Using a Nested data type with arbitrary key:value pairs might work for you. #ClickHouse stores Nested data types as Arrays that can be of arbitrary length.

This looks like a good blog article on this topic:
https://brunoscheufler.com/blog/2022-12-11-storing-user-defined-attributes-in-clickhouse-with-nested-types

Here are the official docs on Nested structures in ClickHouse: https://clickhouse.com/docs/en/sql-reference/data-types/nested-data-structures/nested

@akahn do you have some specific questions on metrics? I'm happy to answer questions or take a look around for something specific to your need. Metrics tend to be the most straightforward for #ClickHouse - it was originally designed for a metrics use case (web click data warehouse...thus the name)

@akahn hi! full disclosure, I work for #ClickHouse. I thought this recent blog series on building an Observability cluster for logs/metrics/traces was quite interesting and detailed - please let me know if useful or if I can try to assist further:

https://clickhouse.com/blog/storing-log-data-in-clickhouse-fluent-bit-vector-open-telemetry

https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse

Our friends at Netflix just gave us a pretty great shoutout on stage @monitorama! We're so glad to be able to help them with their log performance!

#clickhouse #monitorama #o11y

@mikemcquaid This is a huge win for user privacy.

May I also recommend one of the ClickHouse-based privacy-focused analytics services (PostHog, Plausible), or using #ClickHouse directly.