Azure SDK 2.6 Diagnostics Improvements for Cloud Services

I haven’t blogged for a while because of being very busy at work. Things are slowing down a bit so I will try to write more frequently.

History

Azure SDK 2.5 made big changes to Azure diagnostics. It introduced Azure PaaS Diagnostic extension. Even though this was a good long term strategy the implementation was less than perfect. Here were a few issues that were introduced as a result of Azure SDK 2.5

  1. Local emulator did not support diagnostics
  2. No support for using different diagnostics storage account for different environments
  3. Manual editing required to create XML configuration file needed by Set-AzureServiceDiagnosticsConfiguration. This PowerShell cmdlet was required to deploy the diagnostic extension
  4. To make matters worse there was a bug in the PowerShell cmdlet which surfaced when you had a . in the name of the roles.

All these factors made it impossible to do continuous integration/deployment for Cloud service projects.

A few days ago Azure SDK 2.6 was released. I went through the release notes and read up the documentation. I ran tests to see if sanity has been restored. I am glad to report all the issues introduced by SDK 2.5 have been fixed. Here is a summary of improvements.

  1. Local emulator now supports diagnostics.
  2. Ability to specify different diagnostics storage account for different service configuration
  3. To simplify configuration of paas diagnostics extension the package output from Visual Studio contains the public configuration XML for the diagnostics extension for each role.
  4. PowerShell version 0.9.0 which was released along with the Azure SDK 2.6 also fixed the pesky bug that was happening when you had a . in the name of the role.

Here is a document that provides all the gory details for Azure SDK 2.6 diagnostics changes.

https://msdn.microsoft.com/en-us/library/azure/dn186185.aspx

Overview

If you are developing application and still not using continuous integration and continuous deployment  you should be learn more about it. I will use rest of this blog post to show how you can use PowerShell cmdlets to automate the installation and updating of PaaS diagnostics extension for Cloud Services built using Azure SDK 2.6.

Details

I installed Azure SDK 2.6 on my development machine. I install PowerShell cmdlets(version 0.9.0) and Azure CLI as well.

I created a simple Cloud Service Project. I added a web role and a worker role to it.

I  added one more Service Configuration called “Test” to this project.

image 

I examined the properties of the WebRole1 to see what has changed with SDK 2.6

If you select “All Configurations”  you can still enable/disable the diagnostics like you used to do in SDK 2.5

image

When I selected “Configure” button to configure the diagnostics I found that we don’t have to select the diagnostics storage account in the “General” tab like we used to do. Rest of the configuration is same.

image

Returning back to the configuration of the WebRole1 I changed the Service Configuration to “Cloud”.

In the past there was no way to configure diagnostics storage account for configuration type.

But now we can define a different diagnostics storage account for each configuration type.

image

Quick examination of the ServiceConfiguration.Cloud.cscfg confirmed that diagnostics connection string was defined in it.

This makes a lot of sense because rest of the environment specific configuration setting are also defined in the same file.

image 

I did not want to deploy this project directly from Visual Studio because most build servers do not use Visual studio to deploy applications.

First I created a deployment package by selecting the Cloud project and select Package.

image

Selected the “Cloud” Service Configuration and Press “Package” button.

image

The project was built and packaged successfully. It opened up the location where the package and related files were created.

It created a directory called app.publish in the bin\debug directory under the cloud service project.

This is not any different from the past. However there is a new directory called Extensions.

image

Extensions directory has PubConfig.xml file for each role type. You had to create this file manually from diagnostics.wadcfg in the past. These files are needed by PowerShell cmdlets that are used to deploy diagnostics extension.

image

We use AppVeyor for continuous integration and deployment. It uses msbuild to build the projects.

I ran “Developer Command Prompt for Visual Studio 2013” and used the following command to build and package the cloud project.

msbuild <ccproj_file> /t:Publish /p:PublishDir=<temp_path>

I verified that msbuild also created the package and all the related files.

PowerShell Cmdlets for Azure Diagnostics

image

For new Cloud Services there are two ways to apply diagnostics extensions.

  1. You can pass the extension configuration to New-AzureDeployment via –ExtensionConfiguration parameter.
  2. You can create the Cloud Service first and use Set-AzureServiceDiagnosticsExtension to apply the PaaS diagnostics extension.

You can learn about it here.

https://msdn.microsoft.com/en-us/library/azure/dn495270.aspx

I chose method one because it was faster than applying extension in a separate call.

Deploying PaaS Diagnostics Extension for the first time

The following script creates a new Cloud Services, creates the diagnostics configuration and deploys the package which also deploys the PaaS diagnostics extension.

I am setting the diagnostics extension for each Role separately.

At the end of this script I use Get-AzureServiceDiagnosticsExtension to verify if the diagnostics has been installed.

You can also use Visual Studio Server Explorer to view the diagnostics.

001
002
003
004
005
006
007
008
009
010
011
012
013
014
015
016
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
050
051
052
053
054
055
056
057
058
059
060
061
062
063
064
065
066
<#
.SYNOPSIS
Provisions a new cloud service with web/worker role built with SDK 2.6 and applies diagnostics extension
 
.DESCRIPTION
This script will create a new cloud service, deploy cloud service and apply azure diagnostics extension to each role type.
This cloud service has a WebRole1 and WorkerRole2
#>

$VerbosePreference = “Continue” 
$ErrorActionPreference = “Stop”

$SubscriptionName = “Your Subscription Name”
$VMStorageAccount = “storage account used during deployment”
$service_name = ‘cloud service name’
$location = “Central US”
$package = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\DiagnosticsSDK26.cspkg”
$configuration = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\ServiceConfiguration.Cloud.cscfg”
$slot = “Production”
#diagnostics storage account
$storage_name = ‘diagnostics storage account name’
#diagnostics storage account key
$key= ‘storage account key’


# SDK 2.6 tool generate these pubconfig files for each role type
$webrolediagconfig = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\Extensions\PaaSDiagnostics.WebRole1.PubConfig.xml”
$workerrolediagconfig = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\Extensions\PaaSDiagnostics.WorkerRole1.PubConfig.xml”

#Print the version of the PowerShell Cmdlets you are currently using
(Get-Module Azure).Version

# Mark the start time of the script execution
$startTime = Get-Date 

#set the default storage account for the subscription
Set-AzureSubscription -SubscriptionName $SubscriptionName -CurrentStorageAccountName $VMStorageAccount

if(Test-AzureName -Service $service_name)
{
    Write-Host “Serivice [$service_name] already exists”
}
else
{ 
    #Create new cloud service
    New-AzureService -ServiceName $service_name -Label “Raj SDK 2.6 Diagnostics Demo” -Location $location
}

#create storage context
$storageContext = New-AzureStorageContext –StorageAccountName $storage_name –StorageAccountKey $key

$workerconfig = New-AzureServiceDiagnosticsExtensionConfig -StorageContext $storageContext -DiagnosticsConfigurationPath $workerrolediagconfig -role “WorkerRole1”
$webroleconfig = New-AzureServiceDiagnosticsExtensionConfig -StorageContext $storageContext -DiagnosticsConfigurationPath $webrolediagconfig -role “WebRole1”

#deploy to the new cloud service and diagnostics extension
New-AzureDeployment -ServiceName rajsdk26diagdemo -Package $package -Configuration $configuration -Slot $slot -ExtensionConfiguration @($workerconfig,$webconfig)

# Mark the finish time of the script execution
$finishTime = Get-Date 

#Display the details of the extension
Get-AzureServiceDiagnosticsExtension -ServiceName $service_name -Slot Production

 
# Output the time consumed in seconds
$TotalTime = ($finishTime  $startTime).TotalSeconds 
Write-Output “The script completed in $TotalTime seconds.”

 

 

Update PaaS Diagnostics Extension

I wanted to see how we can update diagnostics extension so I made these changes to my project.

I added a new worker role to the same project. I also changed the the configuration of diagnostics.

Typically an extension is only deployed once. To deploy the extension again you have two option:

  1. You can either change the name of the extension
  2. You can remove the extension and install it again

I chose the second option.

Here is what this script does:

It removes the PaaS Diagnostics extension from the cloud service

It creates PaaS diagnostics configuration for each role.

It updates the Cloud Service and applies PaaS diagnostics extension to each role including the new worker role Hard.WorkerRole.

Having a . in the name used to break the Set-AzureServiceDiagnosticsExtension. It is nice to see it is working now

001
002
003
004
005
006
007
008
009
010
011
012
013
014
015
016
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
050
051
052
053
054
055
056
057
058
059
060
061
062
063
064
065
066
<#
.SYNOPSIS
Updates an existing Cloud service and applies azure diagnostics extension as well
 
.DESCRIPTION
This script removes diagnostics extension, updates cloud service, applies azure diagnostics extension to each role type.
This cloud service had a WebRole1 and WorkerRole2 initially. I added a new role called Hard.WorkerRole
I put . in the name because SDK 2.5 Set-AzureServiceDiagnosticsExtension had a bug where . in the name broke it.
#>

# Set the output level to verbose and make the script stop on error
$VerbosePreference = “Continue” 
$ErrorActionPreference = “Stop” 

$service_name = ‘Cloud service name’
$storage_name = ‘diagnostics storage account’
$key= ‘storage account key’
$package = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\DiagnosticsSDK26.cspkg”
$configuration = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\ServiceConfiguration.Cloud.cscfg”

#Print the version of the PowerShell Cmdlets you are currently using
(Get-Module Azure).Version

# Mark the start time of the script execution
$startTime = Get-Date 

#remove the old diagnostics extension
Remove-AzureServiceDiagnosticsExtension -ServiceName $service_name -Slot Production -ErrorAction SilentlyContinue -ErrorVariable errorVariable
if (!($?)) 
{ 
        Write-Error “Unable to remove diagnostics extension from Service [$service_name]. Error Detail: $errorVariable” 
        Exit
}

$storageContext = New-AzureStorageContext –StorageAccountName $storage_name –StorageAccountKey $key
$webrolediagconfig = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\Extensions\PaaSDiagnostics.WebRole1.PubConfig.xml”
$workerrolediagconfig = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\Extensions\PaaSDiagnostics.WorkerRole1.PubConfig.xml”
$hardwrkdiagconfig = “C:\Git\DiagnosticsSDK26\DiagnosticsSDK26\bin\Debug\app.publish\Extensions\PaaSDiagnostics.Hard.WorkerRole.PubConfig.xml”
 

#create extension config
$workerconfig = New-AzureServiceDiagnosticsExtensionConfig -StorageContext $storageContext -DiagnosticsConfigurationPath $workerrolediagconfig -role “WorkerRole1”
$webroleconfig = New-AzureServiceDiagnosticsExtensionConfig -StorageContext $storageContext -DiagnosticsConfigurationPath $webrolediagconfig -role “WebRole1”
$hardwrkconfig = New-AzureServiceDiagnosticsExtensionConfig -StorageContext $storageContext -DiagnosticsConfigurationPath $hardwrkdiagconfig -role “Hard.WorkerRole”

#upgrade the existing code and apply diagnostic extension at the same time
Set-AzureDeployment -Upgrade -ServiceName $service_name -Mode Auto -Package $package -Configuration $configuration  -Slot Production -ErrorAction SilentlyContinue -ErrorVariable errorVariable -ExtensionConfiguration @($workerconfig,$webconfig, $hardworkconfig)
if (!($?)) 
{ 
        Write-Error “Unable to upgrade Service [$service_name]. Error Detail: $errorVariable” 
        Exit
}

# Mark the finish time of the script execution
$finishTime = Get-Date 

#Display the details of the extension
Get-AzureServiceDiagnosticsExtension -ServiceName $service_name -Slot Production

 
# Output the time consumed in seconds
$TotalTime = ($finishTime  $startTime).TotalSeconds 
Write-Output “The script completed in $TotalTime seconds.”

 

Summary

Azure SDK 2.6 has addressed most of the issues related to deploying diagnostics to Cloud Services that were introduced by SDK 2.5. Cleanest way to update diagnostics extensions is the remove the existing diagnostics extension and setting it again during the deployment.  I tested deploying Diagnostics extension individually on each role it took 3-4 minutes to deploy each extension so if  you have a large number of roles your deployment times may increase. In my case with 3 role types it was taking 12 minutes for the script to run. When I used –ExtensionConfiguration parameter of New-AzureDeployment and Set-AzureDeployment it took only 5 minutes for the entire script to run.

This entry was posted in Automation, Azure, DevOps, PowerShell and tagged . Bookmark the permalink.
  • Gaurav Mantri

    Thank you for the detailed instructions. It is indeed very-very helpful! I have one quick question: When I was testing my cloud service (comprising of one web and one worker role) in storage emulator, I noticed that IIS logs (and other directory-based logs) are not transferred to my storage account. Did you also notice the same thing? Is there a work around for that?

    • admin

      Gaurav,

      I did not notice this issue but I was not looking for it so I may have missed it. I have a word out to Azure Diagnostic team to see if they have heard about it. I will get back to you as soon as I have more information.

      Raj

    • http://rajinders.com rajisingh

      Here is what Azure Diagnostic folks from MSFT had to say.

      You do need full IIS for the IIS
      Logs , IIS Express is not supported. Also sometimes the logs take a while to
      show up if the file handle is still open. They will only get transferred once
      the file handle is closed. They will directly follow up with you as well.

  • Andriy Pavliuchenko

    I’ve tried your approach, but when I’ve pass array of configrations into Set-AzureDeployment -ExtensionConfiguration @($workerconfig,$webconfig, $hardworkconfig)

    Powershell returns Object reference not set to an instance of an object. error.

    Is this depend on powershell cmdlets version?

    • admin

      Which version of PowerShell cmdlet are you using. I used the latest PowerShell cmdlet when I wrote the blog post. This version was released at the same time as SDK 2.6. Can you try to update to the latest powershell and let me know if it fixes your issue.

    • http://rajinders.com rajisingh

      Yes it is dependent on the latest version of Powershell cmdlet. This version was released at the same time as SDK 2.6. Can you try and let me know if updating to the latest version fixes your issue. Other thing that may be happening is that your configuration is invalid.