Detailed, reproducible, and relatively objective analysis in a blog comparing performance, compression, and costs between #ClickHouse Cloud and #Snowflake
https://clickhouse.com/blog/clickhouse-vs-snowflake-for-real-time-analytics-benchmarks-cost-analysis
Summary:
* ClickHouse Cloud is 3-5x more cost-effective than Snowflake in production
* ClickHouse Cloud querying speeds are over 2x faster compared to Snowflake
* ClickHouse Cloud results in 38% better data compression than Snowflake
#MachineLearning directly in the database!
With #ClickHouse you can train a ML model (stochasticLinearRegression or stochasticLogisticRegression) and then use it to perform predictions!
The flow below also uses the s3 function to read the data from a bucket
Check it out with Aiven for ClickHouse
cool cool can't sleep since 4am because this is the OPTIMAL hour to have awesome tech ideas (or hate your life/job/relationship/neighbors/yourself of course) either way really good time to think of fundamental shit like "should I use #sqlite or #clickhouse oh I know I use both!1!! can't have enough databases!!"
Directly reading from and writing to AWS S3 buckets: faster time to analytics with #ClickHouse!
We just enabled it on Aiven for ClickHouse, check the doc
https://docs.aiven.io/docs/products/clickhouse/howto/run-federated-queries
I think for the smaller and mid-sized #observability "signoz" is looking good. it's a frontend bringing logs, traces, metrics together, stored in #clickhouse (which I can really recommend, I'm quite happy with it - but not yet arrived at petabyte level, can't attest to bigger setups yet)
Have I shown our #GitHub #actions statistics available with #ClickHouse public play.clickhouse.com instance?
https://pastila.nl/?01d5d14e/b4555c6cb4845860d90d2a1615a563e4.link#aqjvBN52V3CGcko2U783WA==
It's impossible to achieve with given GH UI, so it's implemented with a simple set of tools https://clickhouse.com/blog/monitor-github-action-workflow-job-queues
@akahn nice job! it might be interesting to note that cascaded (or chained) Materialized Views do work, and in fact are updated together with the original source table:
https://clickhouse.com/docs/en/guides/developer/cascading-materialized-views
https://clickhouse.com/docs/knowledgebase/are_materialized_views_inserted_asynchronously
recursive Materialized Views are probably another matter though :)
#clickhouse
Here’s an idea for a new web stack, CHL. #ClickHouse #HTMX and a #Lisp. I’d pronounce it CHilL🍹
Another round of ClickHouse Fundamentals free virtual instructor-led (live) training this week August 16 & 17 at 1 PM BST / 2 PM CEST / 5 AM US Pacific each day for 3 hours
Free enrollment here:
https://clickhouse.com/company/events/clickhouse-workshop
This course applies to both #ClickHouse #opensource or #Cloud users with lecture and hands-on labs - come join us and learn!
#cloud #opensource #clickhouse
@julioj @Perl Thanks Julio. Looking at various 2D solutions to utilize and #clickhouse is definitely among those considered for a SQL form of solution (bonus there is a R interface - many analytics stuff are written in R).
Will take a look!
@ChristosArgyrop @Perl Hey Christos, I’m partial to #ClickHouse as you can see from my bio. In my security work I use it for network traffic metrics and observability. I mainly use JavaScript, Python, and Common Lisp to interact with ClickHouse. For R there’s https://github.com/IMSMWU/RClickHouse
Also, feel free to join our Slack where you can ask any and all questions from our community of users https://clickhousedb.slack.com
detailed and genuinely useful blog on how we built our internal data warehouse on #ClickHouse at ClickHouse
https://clickhouse.com/blog/building-a-data-warehouse-with-clickhouse
#ClickHouse #DataWarehouse
starring
#Superset #AppFlow #Airflow #Redis #Docker #opensource #RDS #PostgreSQL #DBT
and including data from
#Salesforce #AWS #S3 #BigQuery #M3ter #Segment #Marketo
#marketo #segment #m3ter #bigquery #s3 #aws #salesforce #dbt #postgresql #rds #opensource #docker #redis #airflow #appflow #superset #datawarehouse #clickhouse
With ClickPipes, #ClickHouse joins the Connect with
@confluentinc partner program!
Real-time data streams available directly within ClickHouse Cloud!
Hear about our integration with Confluent →
#clickhouse #olap #db #data #database
Learn everything you need to get started with #ClickHouse #opensource or #Cloud in this 2-day #free #training course “ClickHouse Fundamentals”
Join us on July 11 & 12 for 2 days (3 hrs daily at 8am US Pacific / 17:00 CEST) of virtual instructor-led courses with hands-on labs
Enroll here:
https://clickhouse.com/company/events/clickhouse-workshop
#training #free #cloud #opensource #clickhouse
Due to recent overflow, we are sorry to limit number of PRs per day in the #clickhouse
Core engineers: 5
Contributors: 3
New contributors: 1
We plan to address it eventually!
@akahn I think I understand - it sounds like you have "semi-structured" data. Using a Nested data type with arbitrary key:value pairs might work for you. #ClickHouse stores Nested data types as Arrays that can be of arbitrary length.
This looks like a good blog article on this topic:
https://brunoscheufler.com/blog/2022-12-11-storing-user-defined-attributes-in-clickhouse-with-nested-types
Here are the official docs on Nested structures in ClickHouse: https://clickhouse.com/docs/en/sql-reference/data-types/nested-data-structures/nested
@akahn do you have some specific questions on metrics? I'm happy to answer questions or take a look around for something specific to your need. Metrics tend to be the most straightforward for #ClickHouse - it was originally designed for a metrics use case (web click data warehouse...thus the name)
@akahn hi! full disclosure, I work for #ClickHouse. I thought this recent blog series on building an Observability cluster for logs/metrics/traces was quite interesting and detailed - please let me know if useful or if I can try to assist further:
https://clickhouse.com/blog/storing-log-data-in-clickhouse-fluent-bit-vector-open-telemetry
https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse
Our friends at Netflix just gave us a pretty great shoutout on stage @monitorama! We're so glad to be able to help them with their log performance!
@mikemcquaid This is a huge win for user privacy.
May I also recommend one of the ClickHouse-based privacy-focused analytics services (PostHog, Plausible), or using #ClickHouse directly.