Azure Weekly

Issue #489 - 20th October 2024

There seems to be a theme about data pipelines this week: Carbon Optimised Data Pipelines - minimise CO2 emissions through intelligent scheduling (Introduction), and Control Flow Restartability in Azure Data Factory, and Populate a Microsoft Fabric Lakehouse in different deployment pipeline stages.

In AI: there are two interesting articles: Common Challenges and Solutions for Azure OpenAI Adoption and a walkthrough on Function calling with voice using GPT-4o. In DevOps: Simplify Azure DevOps agent management with Managed DevOps Pools and Deploying .dacpacs to Multiple Environments via ADO Pipelines. And then a mix of other articles: Azure cache for Redis, an introduction, and Use Logic Apps To Save Money In Azure: Data Engineering in Fabric, and Adams Bridge: An Accelerator for Post-Quantum Resilient Cryptography.

Finally, an interesting article from David Heinemeier Hansson (of Ruby on Rails, Basecamp, and HEY fame) on how moving out of the cloud will save them $10m over 5 years. While there's a lot of devil in the detail (tech stack / workload type / demand modelling), needing 10 petabytes of resilient storage is never going to be cheap. A point I've been making for about a decade now, is that it's very difficult to build high density, secure, multi-tenancy architectures on public cloud infrastructure. It's awesome for Enterprise IT workloads, but if you're a SaaS provider, in order to make a profit you need low cost infrastructure, as the margin per customer / tennant also tends be low. Trillion dollar Cloud Service Providers seem to be oblivious to this fact. It's only in the last year or so, as they try and figure out how to monetize AI models, that they are hitting the same problems and need a similar solution to SaaS providers.