Azure Weekly Issue 181: 1st July 2018

Just as I was preparing this edition I received an email notifying me that my MVP award has been renewed for another year; I'm pretty pleased about that.

There are quite a few announcements this week; the main being about Azure Data Lake Storage Gen2. We're HUGE Data Lake fans; we've built a number of large Data Lake solution across different industries based on our Modern Data Platform Blueprint, but even I let out a slight groan when I read that it was a new implementation based on blob storage rather than an evolution of the existing Data Lake Storage implementation. I'm hoping there will be an automated way to migrate from Gen1 to Gen2 and assert that all the security boundaries have also been migrated correctly. I can feel the need to write a blog post on the cost of keeping up with the Azure Service Arms Race!

In this week's announcements: 

• General Availability and Sovereign Cloud Support of Managed Service Identity for App Service and Azure Functions
• General availability: Zone-redundant Azure SQL databases and elastic pools
• General availability: Azure Database for PostgreSQL 4-TB server storage
• General availability: Azure Database for MySQL 4-TB server storage
• Azure Data Factory new capabilities are now generally available
• Azure IoT Toolkit for Visual Studio Code generally available for managing Azure IoT Hub and Devices with ease
• Open Service Broker for Azure 1.0 and more community updates
• Function Proxies now Available in Functions Version 2.x
• Azure Elastic Database jobs is now in public preview
• New zone-redundant VPN and ExpressRoute gateways now in public preview
• Static website hosting for Azure Storage now in public preview
• Autocomplete in Azure Search now in public preview
• Virtual Network Service Endpoints for Key Vault preview
• Azure simplifies cloud analytics and A closer look at Azure Data Lake Storage Gen2
• Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight

Finally an interesting blog post on highligting Microsoft Research Open Data - a new data repository in the cloud dedicated to facilitating collaboration across the global research community. It offers datasets representing many years of data research which were used in published research studies.

AI + Machine Learning

Covering: Azure Batch AI, Azure Bot Service, Microsoft Genomics, Machine Learning services, Machine Learning Studio, Cognitive Services, Bing APIs, Computer Vision API, Content moderator, Custom Services, Emotion API, Face API, Language Understanding (LUIS), Linguistic Analysis API, QnA Maker API, Speaker Recognition API, Text Analytics API, Translator Speech API, Translator Text API, Web Language Model API

• On TechNet, Eduard Davidzhan talks about Microsoft AI resources
• On the Kloud blog, Darren Doc Robinson discusses Using your Voice to Search Microsoft Identity Manager – Part 1
• On MSDN, Sjoukje Zaal (@SjoukjeZaal) discusses Azure Content Spotlight – Microsoft Research Open Data

Analytics

Covering: Azure Databricks, HDInsight, Data Factory, Stream Analytics, Data Lake Analytics, Event Hubs, Power BI, Azure Analysis Services, Apache Spark for HDInsight, Apache Storm for HDInsight, R Server for HDInsight, Data Catalog

• On the Azure blog, Tad Brockway blogs about Azure simplifies cloud analytics
• Also on the Azure blog, Pranav Rastogi (@rustd) announces that Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight
• Also on the Azure blog, Mark Kromer announces that Azure Data Factory V2 new capabilities are now generally available
• Also on the Azure blog, Sandy Winarko describes Lift SQL Server Integration Services packages to Azure with Azure Data Factory
• Also on the Azure blog, Giuliano Rapoz explains how to Structured streaming with Azure Databricks into Power BI & Cosmos DB and then on TechNet, delves into a more detailed, step-by-step example
• Chris Webb (@cwebb_bi) blogs about OData Performance Improvements In The June 2018 Power BI Desktop Release
• On the MSSQL Tips site, Siddharth Mehta writes about Anomaly Detection without any coding using Power BI
• Sasindu Kavinda (@kaviliya123) explains how to Analyze Twitter tags using Sentiment Analysis and Visualize in Power BI
• Devin Knight (@knight_devin) blogs about Power BI Custom Visuals Class Module 109 – as Timeline
• Episode 235 of the Azure podcast, covers an AI, Blockchain and Big Data discussion

Compute

Covering: Virtual Machines, Functions, Batch, Service Fabric, Virtual Machine Scale Sets, Cloud Services, Linux Virtual Machines, SAP HANA on Azure Large Instances

• On MSDN, Alex Karcher talks about Function Proxies now Available in Functions Version 2.x
• Also on MSDN, 'esseemme' highlights Azure for Architects and Azure Serverless Computing Cookbook free ebooks and O'Reillys Designing Distributed Systems free ebook
• On TechNet, Moti Bani talks about Automating Azure Just In Time VM Access
• Also on MSDN, Mikkel Mrk Hegnhj covers Service Fabric 6.2 Refresh Release CU3

Containers

Covering: Container Registry, Container Instances, Azure Kubernetes Service (AKS), Web App for Containers

• On Microsoft + Open Source, Sean McKenna announces Open Service Broker for Azure 1.0 and more community updates
• Megan Meyer covers Accessing Certificates in Service Fabric Hosted Windows Containers
• On MSDN, Srikantan Sankaran talks about Implementing Service auto scaling in Azure Service Fabric
• Also on MSDN, Steve Lasker (@SteveLasker) covers Key Differences between VM and Container Vulnerability Scanning
• James van Den Berg highlights a Great Microservices E-book about Architecture for Containerized dotnet Apps Docker Kubernetes Containers
• Sameera Perera (@sameera_man) explains how to Publish a Container Image to Azure Container Registry and explains how to Create an Instance of the Container Image in ACI
• Daniel Neumann (@neumanndaniel) walks through Configuring Azure Kubernetes Service via the Terraform OSS Azure Resource Provider to use a custom DNS server for domain specific name resolution
• Zachary Sersland talks about Powering Kubernetes: Benefits and Drawbacks of Azure vs. AWS
• Scott Hanselman (@shanselman) writes about .NET Core and Docker

Databases

Covering: Azure SQL Database, Azure Cosmos DB, SQL Data Warehouse, Redis Cache, SQL Server Stretch Database, SQL Server on virtual machines, Table storage, Azure Database for PostgreSQL, Azure Database for MySQL

• On the Azure blog, Srini Acharya describes Azure Elastic Database jobs is now in public preview
• the Azure updates team writes about General availability: Zone-redundant Azure SQL databases and elastic pools, writes about General availability: Azure Database for PostgreSQL 4-TB server storage and writes about General availability: Azure Database for MySQL 4-TB server storage
• Tim Anderson (@timanderson) writes about Inside Azure Cosmos DB: Microsofts preferred database manager for its own high-scale applications
• Vincent-Philippe Lauzon (@vplauzon) covers Cosmos DB Stored Procedures – handling continuation
• On MSDN, Joon Kyu Lee describes How to create user-defined restore points on Azure SQL Data Warehouse and describes How to create user-defined restore points on Azure SQL Data Warehouse
• Also on MSDN, Jovan Popovic writes about Point-in-time restore of a database on Azure SQL Managed Instance using AzureRm.Sql PowerShell library

Developer Tools

Covering: Visual Studio, Visual Studio Code, SDKs, Developer tool integrations, CLIs, Blockchain Workbench

• On the Visual Studio blog, Visual Studio Blog writes about Visual Studio 2017 version 15.8 Preview 3 which contains a number of new Azure features.

DevOps

Covering: Visual Studio Team Services (VSTS), Azure DevTest Labs, DevOps tool integrations, Azure DevOps Projects, Azure Lab Services

• On the Daily Dot Net Tips blog, Abhijit Jana (@AbhijitJana) discusses Azure DevOps Project with Azure SQL Server Database for Web Applications, talks about Import an existing Git Repository in your VSTS, explains How to add custom HTML to VSTS dashboard and describes Creating Code Repositories from Existing one in VSTS
• James van Den Berg explains how to Build your Own Azure DevOps Project Pipeline Cloud Azure DevOps Pipeline
• On Channel 9, Kit Merker talks about Using JFrog Artifactory with Azure Kubernetes Service Azure Friday
• Peter Hauge covers how to improve Logs in Visual Studio Team System VSTS with special tags

Identity

Covering: Azure Active Directory, Multi-factor Authentication, Azure Active Directory Domain Services, Azure Active Directory B2C

• On the Azure blog, Yitao Dong covers Use Azure Active Directory with Spring Security 5.0 for OAuth 2.0
• Also on MSDN, Michael S. Hansen discusses Azure Active Directory Proxy for SMART on FHIR
• On TechNet, Aaron Guilmette explains How to find Staleish Azure B2B Guest Accounts
• On Channel 9, Arturo Lucatero joins Donovan Brown to discuss Azure AD Managed Service Identity Azure Friday

Integration

Covering: Service Bus, Logic Apps, Event Grid

• On MSDN, Ali Mazaheri describes Updating QnA Maker knowledge base via SharePoint Online using Logic Apps
• Also on MSDN, Aseem Bansal (@aseembansal) blogs about Ability to sign the service bus message in a VSTS task
• David Burg (@david_burg) covers Logic App SAP connector all the logs to capture for troubleshooting and requesting support and covers Understanding Logic App Custom Connectors configuration
• Justin Conway describes IP Filtering for Event Hubs and Service Bus and discusses Azure Event Hubs and Service Bus Virtual Network Service Endpoints in preview

Internet of Things

Covering: IoT Hub, IoT Suite, IoT Edge, IoT Central, IoT solution accelerators, Time Series Insights, Azure Maps, Azure Sphere

• On Channel 9, Olivier Bloch & Cameron Tomisser talk about Azure IoT Edge - ready for enterprise-grade, scaled deployment Internet of Things Show
• on the Azure blog, Sam George (@samjgeorge) announces that Azure IoT Edge generally available for enterprise-grade, scaled deployments
• On MSDN, formulahendry (@formulahendry) describes Azure IoT Toolkit for Visual Studio Code generally available for managing Azure IoT Hub and Devices with ease
• On the Azure blog, Chipalo Street discusses Migrating IoT Edge solutions to GA
• On Channel 9, Dmitry Lyalin talks about Windows IoT 4: Azure IoT Hub and IoT Central Integration for Windows IoT Getting Started Series Visual Studio Toolbox
• On the Azure blog, Chrissie Chi writes about Automatic device management, module identity, and module twin are now generally available
• Also on the Azure blog, Koichi Hirao covers Expanding Azure Certified for IoT program for the intelligent edge
• Mark Russinovich blogs about Microsoft from GeekWire Cloud Tech Summit: New Azure innovations will advance the intelligent cloud and intelligent edge

Management and Governance

Covering: Backup, Site Recovery, App Insights, Azure Advisor, Sceduler, Automation, Log Analytics, Azure Monitor, Security & Compliance, Protection & Recovery, Automation & Control, Insight & Analytics, Azure Service Health, Microsoft Azure portal, Azure Resource Manager, Cloud Shell, the Azure mobile app, Azure Policy, Cost Management, Azure Managed Applications

• On the Daily Dot Net Tips blog, Abhijit Jana (@AbhijitJana) describes How to add custom HTML to Azure Portal Dashboard and explains How to export the list of Azure Resources to Excel
• Peter Rombouts covers ARM Outputs in VSTS pipelines
• On MSDN, Scott Semyan (@scott_semyan) explains how to Terraform for the ARM Template Developer
• Roy Kim (@roykimyyz) talks about Monitoring Azure Virtual Machines with Log Analytics Part 3: Dashboard
• On Channel 9, Vinod Kurpad joins Lara Rubbelke to talk about Managing costs with the Azure Budgets API and Action Groups Azure Friday
• On the Daily Dot Net Tips blog, Abhijit Jana (@AbhijitJana) covers Pin Azure Resource details from Blades to Azure Portal dashboard and explains How to start or stop multiple Azure App Services together

Media

Covering: Media services, Encoding, Live and On-Demand Streaming, Azure Media Player, Content Protection, Media Analytics, Video Indexer

• On the Azure blog, Ella Ben-Tov explains how to Get video insights in even more languages

Microsoft Azure Stack

Covering: Azure Stack

• On Microsoft Cloud Show, Andrew Connell Chris Johnson talks about Azure Stack with Microsoft Technical Fellow Jeffrey Snover
• On the Azure blog, Aditi Gangwar announces that Backup your applications on Azure Stack with Azure Backup

Migration

Covering: Azure Database Migration Service, Azure Migrate, Data Box

• This week on the Azure podcast, Episode 234 looks at Azure Migrations with Bill Zack

Mobile

Covering: App Service (Mobile), Notification Hubs, Mobile apps, API apps, Visual Studio App Centre, Xamarin

• On the Xamarin blog, Jon Douglas describes Whats New In the Xamarin Android Designer
• On MSDN, James S White talks about The Ship List: Visual Studio App Center June Update

Networking

Covering: ExpressRoute, Azure DNS, Virtual Network, Traffic Manager, Load Balancer, VPN Gateway, Application Gateway, Network Watcher

• On the Azure blog, Ashish Jain writes about New zone-redundant VPN and ExpressRoute gateways now in public preview
• Bill Zack (@WilliamHZack) describes Building a Rational Microsoft Azure Network Architecture
• The Azure updates team talks about Azure Web Apps Analytics and Azure Network Security Group Analytics available and covers Azure DNS SLA: Updated to 100%
• On TechNet, Amit Bapat (@amitbapat) announces Virtual Network Service Endpoints for Key Vault preview

Security

Covering: Azure Information Protection, Key Vault, Security Center, Azure DDoS Protection, Azure Advanced Threat Protection

• On MSDN, CalvinH writes about solving the problem of Visual Studio slowdown because Virus & Threat protection Windows Defender Antivirus is scanning this file
• Also on MSDN, Olga Guzheva explains how to Updated ARM Template to deploy server with Auditing and Threat Detection turned ON

Storage

Covering: Storage, StorSimple, Data Lake Store, Blob Storage, Disk Storage, Managed Disks, Queue Storage, File Storage, Storage Explorer, Archive Storage

• On the Azure blog, Jason Hogg announces that A closer look at Azure Data Lake Storage Gen2
• On Channel 9, James Baker joins Lara Rubbelke to talk about Azure Data Lake Storage Gen2 overview Azure Friday
• James Serra (@JamesSerra) covers Azure Data Lake Store Gen2
• Also on the Azure blog, Michael Hauss announces that Static website hosting for Azure Storage now in public preview

Web

Covering: App Service (Web), API Management, Content Delivery Network, Azure Search, Web apps, Azure SignalR Service

• On the Azure blog, Upasana Dixit describes Autocomplete in Azure Search now in public preview
• On MSDN, Matthew Henderson - announces General Availability and Sovereign Cloud Support of Managed Service Identity for App Service and Azure Functions
• Also on MSDN, Prashant Pratap talks about PowerShell script to execute commands in SCM website on all instances
• Also on MSDN, Prashant Pratap talks about PowerShell script to list all running processes in your Azure AppService w/Computer name