Kusto (software)
Kusto is a big data analytics cloud platform optimized for interactive, ad-hoc queries over structured, semi-structured, and unstructured data, developed at Microsoft.
Kusto is the engine behind Microsoft's Azure Data Explorer service, as well as the backend of several Microsoft Azure services: Azure Log Analytics, Azure Application Insights, and Azure Advanced Thread Protection. [1]
The query language of Kusto is called KQL. [2] (Kusto Query Language).
History
Kusto began in 2014 as an internal Microsoft project to meet the needs of Azure services for fast and scalable log and telemetry analytics. In 2016, it became the backend of Application Insights Analytics. [3]
Microsoft announced a public preview of the Azure Data Explorer service, built on the Kusto engine, at the Ignite 2018 conference. [4][5]
Design
[6] Kusto is a distributed database running on a cluster of compute nodes in Microsoft Azure. Kusto exposes a relational data model. Data ingested into Kusto tables is split into multiple horizontal data shards, where each shard is implemented as a compressed column store augmented with indexes. Data and metadata are reliably stored in Microsoft Azure Storage services. Data shards are assigned to compute nodes using a consistent hashing algorithm. Each query is distributed and parallelized across compute nodes and data shards. Cluster nodes contribute their disk and memory resources to the data shards' caching.
Design trade-offs:
- Focus on interactive queries (as opposed to batch jobs)
- No global/shared data artifacts across data shards (indexes and dictionaries are per-shard)
- Optimized for append-only workloads (delete at the data shard granularity)
References
- ↑ "Azure Data Explorer – Data Exploration and Analytics Service | Microsoft Azure". azure.microsoft.com. Retrieved 2018-09-25.
- ↑ "Kusto Query Language (KQL) from Scratch". www.pluralsight.com. Retrieved 2018-09-25.
- ↑ "Introducing Application Insights Analytics". Retrieved 2018-09-25.
- ↑ Brust, Andrew. "The big data odyssey of SQL Server 2019, and more data and AI news from Microsoft Ignite | ZDNet". ZDNet. Retrieved 2018-09-25.
- ↑ "Introducing Azure Data Explorer | Blog | Microsoft Azure". azure.microsoft.com (in Nederlands). Retrieved 2018-09-25.
- ↑ Ryzhyk, Evgeney (2018-09-24). "Azure Data Explorer: a big data analytics cloud platform optimized for interactive, ad-hoc queries over structured, semi-structured and unstructured data" (PDF). azure.microsoft.com. Retrieved 2018-09-25.
This article "Kusto (software)" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Kusto (software). Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
