Azure Event Grid with Real World Example
Introduction
Azure Event Grid is an event routing service which consumes events from a list of source application/components (example, Azure Blob Storage) and routes to list of application/components (example, Azure Function App) who subscribes it.
It is running on top of Azure Fabric Service and provide easiest way to create event based piped operation/workflow creation. It is fully managed service so provides common features required around it like,
- scaling (HA) and DR
- monitoring
- integration with different technical component
- filtering of messages while routing
- Authentication & Authorization along with Network security
- Failure cases handling with retry mechanism
Terminologies:
Events: Event are the action/data/information which is generated by certain activity by user/application/system. Example, a file upload event on Blob Storage, it occurs when a file is uploaded to Azure Blob Storage. Events may have common properties like, source of event, unique event id, event data, timestamp when event occurred etc.
Event Source: Event source is origin of event and it is responsible to send event immediately to EventGrid when event occurs. It could be any user activity/application.
Publisher: Publishers are user who publish messages to Event Grid. Azure publishes events for various Azure Services and source application.
Topic: Topic is concept of messaging service where topic is endpoint/resource where message/event published by publisher and topic is subscribed by subscribers (target application). In Azure Event Grid, topics are classified into following two categories:
A. System Topic: System topics are built-in topics backed by Azure Services. In portal, built in services are not shown but when one subscribes, can see, and configure subscription. Its’ schema and predefined and handled by Azure. Some examples: Event Hubs, Blob Storage, IoT Hubs, Service Bus, Media Services etc.
B. Custom Topic: Custom topics are those created for application/third part other than Azure Services.
Event Grid Subscriber and Subscription: A subscriber is user/application who suppose to consume the event messages residing into topic. Subscription is contract between topic and the consumer who want to receive the event as event handler. When subscription create, filters can be created to specify if any specific type of events to be subscripted from the system topics.
Event Handler: An event handler is application/code block which receives events from the Event Grid topic and perform desired activity based upon the event received and configured.
Since basic terminologies explained, let us consider below diagram which depicts the Event Grid overall mechanisms:
Here, we can see list of event source can be configured which can generate event and push to Event Grid. Also, list of event handler can be configured.
By default, Event Grid resource provider is disabled in Azure which need to register using Azure Portal.
Azure offers various pricing and structures based upon different use cases. It provides pay as you go model which is widely used.
Problem Statement
Azure Event Grid can be used into different real-world scenario to develop serverless/server-based application where event-based work is being done.
To take real world example a problem statement discussed into article Azure Blob Storage and Azure Function App, let us take same problem: An application to be developed which scan documents, store scanned documents, enhance the captured document, do OCR, and read textual content and send notification to a web application for further processing. Once, it is processed by Web Application, user gets result on device.
Consider the diagram for problem statement:
Let us dig out the problem statement in details like how function it can be achieved.
Let us discuss the solution in next section.
Solution
Considering the use case discussed into the problem statement section, there may be various solution based upon different technical components.
To develop complete solution refer articles: Azure Function App with Java and Azure Event Grid with Real World Example
Let us see solution in Azure Cloud where Event Grid will be used:
Explanation of defined solution:
- User capture document image from device
- As user captured, parallelly uploaded to Azure Blob Storage (see Azure Blob Storage Reference)
- Once document uploaded, an event sends to Event Grid
- When event comes into Event Grid, the based upon subscription and filter, Function App automatically triggered
- Azure Function App performs the Image enhancement and notify to OCR Engine, after OCR data stored into database which further utilized by Server to create response to user
Here, there is a piped operation where captured image is processed by one component and notify to next component to for further processing. Consider below diagram which depicts piped operation on captured document:
Azure Event Grid — Azure Portal
Let us consider, Azure Blob Storage and Azure Function App already created into the Azure portal (please refer the articles: Azure Blob Storage with Java Function App).
To develop the solution first, we need to have Azure Account with Azure Blob Storage account and Azure Function App already created.
Need to register Event Grid resource provider first, in case it is not registered:
Open Azure Web Portal -> Search for ‘Subscriptions’ -> Select Subscription
A list of your subscription will be listed, select active one which is applicable for you:
A new screen will be opened. In selected Subscription menu list, go to Settings -> Resource Providers -> Search EventGrid -> Select and Register
Once, resource provider registration completed perform subscription steps.
Event Grid Topic Creation
Login in Azure web portal, search Event Grid Topics in search box and select Event Grid Topics icon (it will list topics if already topics are there):
To create topic, click on new. A tabular form will be opened:
Provide details like: Subscription (by default it is selected the default subscription), provide resource group name, provide topic name and select location where this topic will be available. In Advance tab, select Event Grid Schema (Default Schema by Azure for Event Grid), leave identity as-is if no additional authentication required to be implemented. On tag tab, provide tag name for this resource.
Once form completed, click on review and create. It will create topic. Take a note of topic name, resource group and location.
Azure Event Grid Subscription
Once Azure Blob Account, Azure Function App and Event Grid Topic created. Now time to create subscription to map source -> event grid -> event handler.
Open Azure Blob Account and Events:
Click on ‘+ Event Subscription’ button. It will open a tabbed form to create event subscription.
Provide Details: A unique event subscription name, select default Event Grid Schema, Topic Type Storage Account , Source resource, System topic name (the same topic name which we created in previous step). Select list of events to be send to event grid in event types field, In Endpoint type field select Azure Function.
When Azure Function selected in Endpoint Type, a Endpoint field start showing. Click on Endpoint field option ‘Select an Endpoint’, this will open one more form as:
Provide details: Resource group (name of resource group where Azure Function app created for Event Grid), Function App (name of Azure Function app created for Event Grid), keep slot production, in Function provide the name of function which is in pom.xml while creating function.
Click on confirm selection and then click on create.
Great, everything completed!
To test just upload document on selected Azure Blob Storage, you can see event pushed into topic, and consumed by Azure Function. You can check metrics of topic, azure function and event subscription to check different metrics.
Conclusion
Azure Event Grid provides an effective way to create event-based application with Azure Cloud. It reduces the complexity and provides different kind of applications from where events can be capture and appropriate actions/handlers can be defined.
It is managed service and provide various common features like failure handling, monitoring, filtering, liveness probe and different kind of matrices. The consumption-based pricing model as made it very effective cost wise also a specified number of consumptions will require no cost.
Developers can design complex workflows based upon events. And by using handlers like Azure Function App, Azure Logic App can take benefit of serverless computing.
References
- Azure Function App — https://siddhivinayak-sk.medium.com/azure-function-app-with-java-548db9447c31
- Java Azure Function App — https://github.com/siddhivinayak-sk/azure-function
- Azure Blob Storage — https://siddhivinayak-sk.medium.com/azure-blob-storage-with-java-5817347a89d0
- Microsoft Azure Event Grid — https://azure.microsoft.com/en-in/blog/introducing-azure-event-grid-an-event-service-for-modern-applications/
About the Author
Sandeep Kumar holds Master of Computer Application degree has been Java developer having 10 years of working experience. He has experience designing and development of enterprises applications into various education, content, laboratory, and banking domains, got various appreciation for providing solutions including spot appreciation for Glassfish to JBoss migration project. He secured Google Cloud Developer certificate and participated into OCI training. He is a part of HCL-ERS platform as Sr. Lead developer.
