📜 Post with us on Reddit
📝 Create a new article
🏭 Create a company page
🇬🇧 - in English
🇫🇷 - en Français (FR)
🇵🇹 - em Português (PT)
🇩🇪 - in Deutsch (DE)
🇯🇵 - 日本語で (JA)
Sponsored articles

You can edit almost every page by Creating an account. Otherwise, see the FAQ.

Azure Event Hubs

From EverybodyWiki Bios & Wiki



Script error: No such module "Draft topics". Script error: No such module "AfC topic".

Azure Event Hubs
OwnerMicrosoft
Websiteazure.microsoft.com//products/event-hubs
CommercialYes
RegistrationRequired (included in free tier layer)
Launched2014; 10 years ago (2014)
Current statusActive

Azure Event Hubs is a real-time data streaming message broker Platform as a Service (PaaS) that aims to support large-scale event ingestion and streaming. Azure Event Hubs support streaming data using open standards such as Advanced Message Queuing Protocol(AMQP)[1][2], HTTPs and Apache Kafka[3].

History[edit]

Azure Event Hubs was developed[4] by Microsoft and launched[5] in 2014 as a PaaS to support event streaming using open standards such as AMQP and HTTPs. Support for Apache Kafka support was added[6] in 2018.

Overview[edit]

Modern software applications require analyzing streams of events[7] or data that is continuously generated by different sources in real-time and capture actionable insights. Event streaming analytics[8][9] is being heavily used to:

  • Perform web clickstream, anomaly, and fraud detection
  • Generate insights from application logs
  • Derive business insights from market data feeds
  • Process IoT sensor data
  • Process game telemetry
  • Real-time ETL[10]
  • Process change data capture feeds and more.

Azure Event Hubs aims to provide an event streaming Platform as a Service that enables client applications to ingest[11] streams of events and process with multiple downstream applications.

Architecture[edit]

Event Hubs[12] represents the "front door" for an event pipeline, and it sits between event producers and event consumers to decouple the production of an event stream from the consumption of those events. Event producers can ingest data to Event Hubs using AMQP, Apache Kafka, and HTTPS protocols. Event consumers can consume event streams using AMQP or Apache Kafka.

Events are organized into event hubs or topics (in Kafka parlance) and each event hubs can have one or more partitions. Event Hubs architecture[13][14] is based on the partitioned consumer model where users can use topic partitions to parallelize the consumption to stream large volumes of data.

On the event consumer side, consumer groups are used for event consumption. A consumer group is a view of an entire event hub. Therefore, consumer groups enable multiple consuming applications to each have a separate view of the event stream, and to read the stream independently at their own pace and with their own offsets. All event hubs or Kafka topics are grouped into a management component known as a 'namespace'. Event Hubs namespace is used to configure scaling units, security and availability event streams.

Event Hubs for Apache Kafka[edit]

Event Hubs supports[15] most of the Apache Kafka workloads so that existing Kafka clients and applications can integrate[16][17] with Event Hubs.

Event Hubs does not run or host Apache Kafka brokers. Instead, Event Hubs broker implementation supports Apache Kafka as a protocol head. Therefore, there are several Kafka features[18] that are not compatible with Event Hubs.

Known limitations.[edit]

Azure Event Hubs is available only as a Platform as a Service (PaaS). Therefore, users cannot run it on-premises or hybrid deployments. Also, Event Hubs is not 100% compatible with Apache Kafka features. The known feature differences are listed described in Event Hubs documentation[18].

Building event streaming pipelines[edit]

Using Azure Event Hubs as the data ingestion[19] layer, users can build event streaming pipelines that consists of real-time streaming application, event driven microservices,[20] or application that perform batch analytics.

Users can consume data using any generic client application that is based on either Apache Kafka Clients or Azure Event Hubs SDK. Also, data ingested to Event Hubs can be consumed by frameworks such as Apache Spark[21], Apache Flink[22][23] and other Azure services[24][25][26][27].

Schema Registry for schema validation[edit]

Client applications of Event Hubs can use[28] the built-in scheme registry to serialize and de-serialize events using schema formats such as Apache Avro.

It also provides a simple governance framework for reusable schemas and defines the relationships between schemas through a grouping construct. It's also compatible with Apache Kafka applications that use schema-driven serialization.

Client SDKs[edit]

To send and receive events from Azure Event Hubs, users can either use the native client SDK or any Kafka client SDK which is compatible with Apache Kafka API version 1.0 and above.

Using Event Hubs SDK/AMQP[edit]

Azure Event Hubs offers runtime client libraries for a wide range of programming languages.

Using Apache Kafka APIs[edit]

Azure Event Hubs is compatible with any Kafka SDK/client library. Apache Kafka producer and consumer samples can be found here for a wide range of programming languages and frameworks.

See also[edit]

Some use of "" in your query was not closed by a matching "".Some use of "" in your query was not closed by a matching "".

References[edit]

  1. ↑ "Home | AMQP". www.amqp.org. Retrieved 2023-02-04.
  2. ↑ The AMQP 1.0 Protocol - 1/6 - Introduction, retrieved 2023-02-04
  3. ↑ "Apache Kafka". Apache Kafka. Retrieved 2023-02-04.
  4. ↑ Bisson, Simon (2018-03-20). "Azure Service Fabric: What you need to know". InfoWorld. Retrieved 2023-02-04.
  5. ↑ "Announcing Azure Event Hubs General Availability". azure.microsoft.com. Retrieved 2022-12-10.
  6. ↑ "Microsoft Announces the General Availability of Azure Event Hubs for Apache Kafka". InfoQ. Retrieved 2023-02-04.
  7. ↑ "Design Patterns for Cloud Native Applications [Book]". www.oreilly.com. Retrieved 2023-02-04.
  8. ↑ Heller, Martin (2022-01-07). "What is streaming data? Event stream processing explained". InfoWorld. Retrieved 2023-02-06.
  9. ↑ Dieruf, David (2022-09-01). "Event Streaming and Event Sourcing: The Key Differences". The New Stack. Retrieved 2023-02-06.
  10. ↑ "ETL Is Dead, Long Live Streams". InfoQ. Retrieved 2023-02-06.
  11. ↑ "Microsoft Azure Event Hubs Surpasses 1 Trillion Transactions in a Single Month". InfoQ. Retrieved 2023-02-04.
  12. ↑ Vasters, Clemens (2023-01-23), Asynchronous Messaging and Eventing Resources (PDF), retrieved 2023-02-04
  13. ↑ Azure Event Hubs Deep Dive (Azure & AI Conference 2022), retrieved 2023-02-04
  14. ↑ "Azure Event Hubs - Behind the Scenes With Kasun Indrasiri | Current 2022". Confluent. Retrieved 2023-02-04.
  15. ↑ "Event-Driven Architecture with Apache Kafka for .NET Developers Part 3: Azure Event Hubs - DZone". dzone.com. Retrieved 2023-01-04.
  16. ↑ "Using Azure Event Hubs with MongoDB and the MongoDB Connector for Apache Kafka | MongoDB Blog". MongoDB. Retrieved 2023-02-04.
  17. ↑ "Azure Event Hubs". Dapr Docs. Retrieved 2023-02-04.
  18. ↑ 18.0 18.1 spelluru. "Use event hub from Apache Kafka app - Azure Event Hubs - Azure Event Hubs". learn.microsoft.com. Retrieved 2022-12-16.
  19. ↑ "Mastering Azure Analytics, 1st Edition [Book]". www.oreilly.com. Retrieved 2023-02-04.
  20. ↑ Indrasiri, Kasun; Siriwardena, Prabath (2018). Microservices for the Enterprise. doi:10.1007/978-1-4842-3858-5. ISBN 978-1-4842-3857-8. Unknown parameter |s2cid= ignored (help) Search this book on
  21. ↑ "Azure Event Hubs | Databricks on AWS". docs.databricks.com. Retrieved 2023-02-04.
  22. ↑ Singh, Keshav (2023-01-04). "Flink Stream Processing — Azure EventHub on Kafka Protocol". Medium. Retrieved 2023-02-04.
  23. ↑ "Kafka". nightlies.apache.org. Retrieved 2023-02-04.
  24. ↑ "Microsoft Releases Stream Analytics No-Code Editor into General Availability". InfoQ. Retrieved 2023-02-04.
  25. ↑ Bisson, Simon (2019-02-12). "How to use Azure Data Explorer for big data analysis". InfoWorld. Retrieved 2023-02-04.
  26. ↑ Bisson, Simon (2019-12-10). "A first look at Azure Synapse". InfoWorld. Retrieved 2023-02-04.
  27. ↑ "Stream Analytics with Microsoft Azure". Packt. Retrieved 2023-02-04.
  28. ↑ "Announcing Azure Schema Registry in Azure Event Hubs – GA". TECHCOMMUNITY.MICROSOFT.COM. 2021-11-02. Retrieved 2023-02-04.


This article "Azure Event Hubs" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Azure Event Hubs. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.


Retrieved from "https://en.everybodywiki.com/index.php?title=Azure_Event_Hubs&oldid=3382968"
License CC BY-SA 3.0
Powered by MediaWiki
Powered by Semantic MediaWiki