Jay GordonAzureFunBytes is a weekly opportunity to learn more about the fundamentals and foundations that make up Azure. It's a chance for me to understand more about what people across the Azure organization do and how they do it. Every week we get together at 11 AM Pacific on Microsoft LearnTV and learn more about Azure.
Personal note
This will be my final live show of AzureFunBytes. I've really enjoyed doing this show for you every week. I appreciate you being part of my journey into the services, products, and people that make up an amazing Azure experience. Time for some new challenges.
This week on AzureFunBytes we’ll be discussing how to best get started in stream processing with Azure Stream Analytics. Azure Stream Analytics is a real-time analytics service that lets you define streaming jobs in SQL. Once deployed, these jobs will subscribe to streaming inputs like Event Hub, and filter, enrich, transform, join, or aggregate events as they come.
Stream Analytics offers multiple experiences for developers. Small work such as demos and prototypes can be quickly addressed in the Azure Portal. But most users will find that VS Code delivers the most complete developer experience for Stream Analytics via the ASA Tools extension.
To help get started understanding ASA jobs, I've reached out to Microsoft Senior Product Manager Florian Eiden.
00:00:00 - Opening
00:03:05 - A special announcement on the future of AzureFunBytes
00:04:45 - Welcome to the show Florian!
00:07:03 - So how'd you get here?
00:09:37 - Benefits of being an MVP
00:18:52 - Intro to Azure Stream Analytics
00:19:14 - Why (near) real time?
00:21:05 - Stream processing
00:30:05 - ASA: Canonical real-time data pipeline
00:30:31 - ASA: Value proposition
00:31:19 - Positioning in Azure
00:36:31 - ASA: Developer Experience
00:38:17 - Demo time, Let's build something!
Our planned agenda:
- High level intro to stream processing – what Stream Analytics does
- The Azure Stream Analytics service, when to use it vs Functions or Data Factory
- The local development experience in VS Code: writing a query, debugging and testing
Local development of Azure Stream Analytics jobs
There are many benefits to developing Stream Analytics jobs locally in VS Code.
It’s easier to get started. As we support fully local job runs, with sample/mock input files, there is no need to set up an input event hub and find a way to produce events. In this case there is no need to have an Azure subscription, and development costs are non-existent. And if a streaming source is available, we also support local runs on live inputs.
The developer feedback loop is much shorter with local development. Running a query only takes a few seconds. Debugging a query is accelerated. Stream processing is not an easy domain to grasp, having quick feedback on what works or not on a query is a must. In addition we offer unit-testing via our npm cicd package: the best way to make sure a query will behave properly once deployed.
Not to mention the obvious: local development unlocks source control, and greatly facilitates CI/CD.
So be part and let's learn about Azure Stream Analytics jobs together!
About Florian Eiden:
Florian is a Senior Product Manager on the Azure Stream Analytics team. He’s responsible for the SQL language used in Stream Analytics. He spent much of his career moving data in batches, and is now trying to speed things up. Born in France, now enjoying the Canadian Pacific Northwest with his family.
Learn about Azure fundamentals with me!
Live stream is normally found on Twitch, YouTube, and LearnTV at 11 AM PT / 2 PM ET Thursday. You can also find the recordings here as well:
AzureFunBytes on Twitch
AzureFunBytes on YouTube
Azure DevOps YouTube Channel
Follow AzureFunBytes on Twitter
Useful docs:
Get $200 in free Azure Credit
Microsoft Learn: Introduction to Azure fundamentals
Introduction to Azure Stream Analytics | Microsoft Docs
Quickstart - Create an Azure Stream Analytics job in Visual Studio Code | Microsoft Docs
Develop Azure Stream Analytics queries locally with Visual Studio Code | Microsoft Docs
Test an Azure Stream Analytics job locally with sample data using Visual Studio Code | Microsoft Docs
Input validation for better Azure Stream Analytics job resiliency | Microsoft Docs
Understand time handling in Azure Stream Analytics | Microsoft Docs
Stream Analytics Query Language Reference - Stream Analytics Query | Microsoft Docs
