Issue 447: 3rd December 2023
Let's start this week with AI; ML.NET 3.0 Boosts Deep Learning, Data Processing for .NET-Based AI Apps, there's also a nice Getting started with Azure AI Studio, and a feature I'm quite interested in trying - Write Your Git Commits with GitHub Copilot (hopefully it can deliver something more meaninful than "tweak", "change", "tweak" which too many of my commit messages seem to be).
In the data and analytics space: Semantic Link: OneLake integrated Semantic Models and Microsoft Fabric real-time analytics exploration:KQL Database mirroring and What are Kusto Query Language KQL databases in Microsoft Fabric?.
In Cloud Native Apps, a particularly good article - Getting Hands-On with Azure Functions: A Deep Dive and Secure access to your Azure App Services/Function Apps/Logic Apps using Azure AD OAuth and API.
I mentioned this topic earlier in the year, but code = energy = money = carbon. Creating highly efficient code, can have a huge positive impact on your infrastructure, your budget, and the environment. Several years ago we worked on a data and analytics project that needed to inject large amounts of real time data - marine shipping vessel locations from the entire planet. The initial implementation used a popular Python package (a wrapper over a C++ library) and that could process 50,000 AIS (Automatic Identification System) messages per second, and needed a 45 node cluster in order to ingest the data volume. We hypothesised that we could do a much better job, and re-implemented the AIS specification using C# / .NET Core 2.0 and achieved 2.6 million messages / sec / core, and allowed us to move the entire data ingestion process into a single Azure Function and reduced the ingestion lag from hours to near real time. With ever subsequent release of .NET we have seen significant performance improvements (with no additional code changes); and with .NET 8.0 was no exception; we've seen AIS.NET performance boosted by 27% which means we can now process 4.75 million messages / sec / core. Python may well be the most popular language for data science / engineering workloads, but for computationally intensive workloads, .NET is a great choice (and in my opinion needs better support in Spark & ML training environments like Microsoft Fabric).
Issue 446: 26th November 2023
A much slimmer edition this week after the tidal wave of announcement from Ignite last week. Highlights this week, include: Migrating from OpenAI to Azure OpenAI and Teach ChatGPT to Answer Questions: Using Azure Cognitive Search & Azure OpenAI Semantic Kernel. A 5-part series covering Azure SQL v/s Azure Synapse SQL Dedicated Pool, Serverless Pool, and Microsoft Fabric real-time analytics exploration: Streaming to a Lakehouse.
In the cloud native space; Dual-Stack Networking in Azure CNI Overlay for AKS: Step-by-Step Guide, Automated node pool migration to Azure Linux, and From Google Cloud App Engine and Cloud SQL to Azure App Service and PostgreSQL: Step-by-Step Guide.
Finally, Graph API: getting users Active Directory group names and ids with the C# SDK, and Microsoft Contributes Azure RTOS to Open Source.
Issue 445: 19th November 2023
Well, what an interesting week it has been. .NET 8.0 was released, there was .NET Conf 2023, then Microsoft Ignite (AKA "Copilot ALL the things!"), and then some unexpected news from OpenAI. This issue is an absolute beast. As ever the Ignite Book of News 2023 is a great place to start for all the official announcements. It's worth going through with a highlighter and marking the things that might be relevant to your organisation. I'm not going to try and summarise everything here, just pull out a few things I found interesting. YMMV!
First up .NET 8.0, the next LTS version of .NET, was released as was Visual Studio 17.8. The .NET Conf 2023 sessions are now available on demand. And a special shout out to Ian Griffiths who presented a session on Modernizing Reactive Extensions for .NET. The Azure Product Groups have put in huge effort to ensure Day 1 support for new releases of .NET; Azure Functions being one of the main use cases: GA: Azure Functions supports .NET 8 in the isolated worker model, they also announced Azure Functions Flex Consumption: sign up for the early access preview, which looks like the Consumption Plan everyone has been asking for. The .NET ❤️ AI session is worth a watch too.
There were so many announcements at Ignite, but the highlights for me were: Microsoft Fabric workloads are now generally available (we've been giving our opinions about Fabric since it was first announced, so are very excited to have reached this milestone). I wanted to give a shout out to Tom Peplow (who you may recognise from our "Decision Makers Guide to Microsoft Fabric" series), who featured in one of the Microsoft Fabric case study videos "Milliman enables self-service actuarial modeling with Microsoft Fabric".
Microsoft announced they have developed two custom chips, Maia 100 and Azure Cobalt to deal with the demands of AI workloads. More detailed covered in Microsoft Azure delivers purpose-built cloud infrastructure in the era of AI. Azure Boost (which offloads traditional CPU workloads to custom hardware) also became GA. Another long-gestating innovation has made it to Public Preview: Announcing the preview of Azure Managed Confidential Consortium Framework which is a framework for building multi-party confidential workloads (Party A brings the algorithm, Party B brings the data) that run on Azure Confidential Computing infrastructure.
The last three articles of note; Azure OpenAI Architecture Patterns and implementation steps, and two bits of architectural guidance Deepening Well-Architected guidance for workloads hosted on Azure and Azure Well-Architected Review Assessment Updates. I hope you enjoy the rest of this edition!
Issue 444: 12th November 2023
A short and sweet issue, as I expect a many announcements are being held back for the Microsoft Ignite Conference (15th-16th November) - so we're bracing for a bumper edition next week.
In the analytics space; Star Schemas are fundamental to unleashing value from data in Microsoft Fabric, a nice lab: Analyzing Wildlife Data with Microsoft Fabric: end-to-end workshop and a interesting post on Power BI Project (PBIP) and Azure DevOps CI performance tests
In the cloud native space: Achieving Distributed High Availability with AKS Hybrid : Hands-on PoC, and Azure Container Storage in AKS is now in preview, and a great overview of the new Radius project.
Finally a review of the new Microsoft Applied Skills - which offer a practical, lab-based alternative to the traditional Microsoft Certifications.
Azure Weekly Archive
If you would like to read more interesting articles from the Azure ecosystem check out our archive where you will find all of the back issues.
Contribute Content
If you would like to contribute any content to the Azure Weekly newsletter, please email azureweekly@endjin.com