Kusto (software)
Kusto is a big data analytics cloud platform optimized for interactive, ad-hoc queries over structured, semi-structured and unstructured data, developed at Microsoft.
Kusto is the engine behind Microsoft's Azure Data Explorer service, as well as the backend of several Microsoft Azure services: Azure Log Analytics, Azure Application Insights, Azure Advanced Thread Protection.[1]
Query language of Kusto is called KQL[2] (Kusto Query Language).
History[edit]
Kusto started in 2014 as internal Microsoft project to address Azure services' needs for fast and scalable log and telemetry analytics. In 2016 it became a backend of Application Insights Analytics.[3]
Microsoft announced a Public Preview of the Azure Data Explorer service built on top of Kusto engine at Ignite 2018 conference.[4][5]
Design[6][edit]
Kusto is a distributed database running on a cluster of compute nodes in Microsoft Azure. Kusto exposes a relational data model. Data ingested into Kusto table is split into multiple horizontal data shards, where each shard is implemented as compressed column store augmented with indexes. Data and metadata is reliably stored in Azure Blobs. Data shards are assigned to compute nodes using consistent hashing algorithm. Each query is distributed and parallelized across compute nodes and data shards. Cluster nodes contribute their disk and memory resources to the data shards' caching.
Design trade-offs:
- Focus on the interactive query (as opposed to batch jobs)
- No global/shared data artifacts across data shards (indexes and dictionaries are per-shard)
- Optimized for append-only workloads (delete at the data shard granularity)
References[edit]
- ↑ "Azure Data Explorer – Data Exploration and Analytics Service | Microsoft Azure". azure.microsoft.com. Retrieved 2018-09-25.
- ↑ "Kusto Query Language (KQL) from Scratch". www.pluralsight.com. Retrieved 2018-09-25.
- ↑ "Introducing Application Insights Analytics". Retrieved 2018-09-25.
- ↑ Brust, Andrew. "The big data odyssey of SQL Server 2019, and more data and AI news from Microsoft Ignite | ZDNet". ZDNet. Retrieved 2018-09-25.
- ↑ "Introducing Azure Data Explorer | Blog | Microsoft Azure". azure.microsoft.com (in Nederlands). Retrieved 2018-09-25.
- ↑ Ryzhyk, Evgeney (2018-09-24). "Azure Data Explorer: a big data analytics cloud platform optimized for interactive, ad-hoc queries over structured, semi-structured and unstructured data" (PDF). azure.microsoft.com. Retrieved 2018-09-25.
This article "Kusto (software)" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Kusto (software). Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
