Extending your alerts with additional data from the Log Analytics API

I recently came accross a scenario where I needed to extend the data that was available in an alert. The alert data that was sent in the actiongroup only contained an EventId field. I was not at liberty to modify the alert, but I did notice an interesting extra property in the payload. The payload looked something like this:

{
    "schemaId": "azureMonitorCommonAlertSchema",
    "data": {
        "essentials": {
            "alertId": "/subscriptions/<subscriptionId>/providers/Microsoft.AlertsManagement/alerts/<alertId>",
            "alertRule": "<MyAlertRule>",
            (...)
        },
        "alertContext": {
            "properties": null,
            "conditionType": "LogQueryCriteria",
            "condition": {
                "windowSize": "PT5M",
                "allOf": [
                    {
                        "searchQuery": "AzureActivity | where OperationNameValue startswith \"MICROSOFT AUTHORIZATION/ROLEASSIGNMENTSCHEDULEREQUESTS\" | where ActivityStatusValue has \"Start\" | where Properties has \"requestBody\"",
                        "metricMeasureColumn": null,
                        "targetResourceTypes": null,
                        "operator": "GreaterThan",
                        "threshold": "0",
                        "timeAggregation": "Count",
                        "dimensions": [
                            {
                                "name": "EventDataId",
                                "value": "<An Id>"
                            }
                        ],
                        "metricValue": 1,
                        "failingPeriods": {
                            "numberOfEvaluationPeriods": 1,
                            "minFailingPeriodsToAlert": 1
                        },
                        "linkToSearchResultsUI": "<A link to the search results>",
                        "linkToFilteredSearchResultsUI": "<A link to the filtered search results>",
                        "linkToSearchResultsAPI": "<A link to the search results>",
                        "linkToFilteredSearchResultsAPI": "https://api.loganalytics.io/v1/workspaces/(...)"
                    }
                ],
                (...)
            }
        },
        "customProperties": null
    }
}

If I want to handle this alert, all I really get is an EventDataId. Sure, I can add some context based on the name of the alert, or the query, but what I would really like is to provide some relevant info from the logs that triggered this alert. Luckily, the payload already points me in the right direction. The linkToFilteredSearchResultsAPI property contains a link to the exact logs that triggered this alert. All I need to do is to call this API and I can get the data I need.

The challenge

The challenge is calling this API, which requires an authenticated principal. In my case, I was running in an Azure Function, and I wanted to use a Managed Identity to authenticate against the API. A similar case would be handling alerts in a logic app.

In both cases, I want the identity of the function or logic app to be used to authenticate against the API. Instead of using a service principal with a client secret, I just want to use the Managed Identity that is easily created out of the box. This is a bit more challenging than it sounds.

The solution

The link I got in the payload is pointing to https://api.loganalytics.io, which the is Azure Log Analytics API. To access this, my Managed Identity needs to be granted access to the API through Microsoft Entra ID. Besides that, my Managed Identity needs to be granted access to the Log Analytics workspace. This is done through the Azure RBAC system, using the ‘Log Analytics Reader’ role.

The latter part is relatively simple, and can be done through any conventional means, like bicep or the Azure Portal. The first part is a bit more challenging. When I looked at the Enterprise App in Entra ID, I noticed that it is not actually possible to grant access through the portal. Instead, I had to resort to PowerShell, using the Microsoft Graph module.

The Microsoft Graph module is a bit of a pain to work with, since the documentation is fairly limited. I often find pieces of information on the old Azure AD PowerShell module, which usually translate decently well to the Microsoft Graph module. After consulting the documentation to get the right signatures, and some trial and error, I managed to get the right PowerShell script to grant access to the Log Analytics API for my Managed Identity.

First, I needed to get the ID of the Log Analytics API resource in Entra ID. This is a static id, and can be retrieved through the portal. It is ca7f3f0b-7d91-482c-8e09-c5d840d0eac5. Next, I needed to get the ID of the AppRole that allows another app to access the Log Analytics API. This is a bit more challenging, but can be done through the following PowerShell command:

$logAppId = "ca7f3f0b-7d91-482c-8e09-c5d840d0eac5"
$permissionName = "Data.Read"

$logAppSpn = Get-MgServicePrincipal -Filter "appId eq '$logAppId'"
$appRole = $logAppSpn.AppRoles | Where-Object {$_.Value -eq $permissionName}

Lastly, I need the ID of my Managed Identity. Get this either through the portal, or with PowerShell:

$msi = Get-MgServicePrincipal -Filter "displayName eq 'my-app-name'"

Note that by convention, a System-Assigned Managed Identity carries the same Display Name as the resource to which it belongs. This is not the case for User-Assigned Managed Identities. Update the script accordingly.

Now that I have all the required IDs, I can grant access to the Log Analytics API for my Managed Identity:

New-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $msi.Id -ResourceId $logAppSpn.Id -AppRoleId $appRole.Id -PrincipalId $msi.Id

Calling the API

Now that my application has the right permissions, all that is left is to get an access token and make my API call. To do so, call the built-in access token endpoint, or use the Azure.Identity SDK. In my case, I had to go the manual route. My function app did grant me the token endpoint, through the environment variable $env:IDENTITY_ENDPOINT. I also needed to provide the identity header, which is $env:IDENTITY_HEADER. With these two variables, I can get an access token from the token endpoint. The following PowerShell script does just that, and returns the access token in the variable $tokenResponse.access_token.

$resource = "https://api.loganalytics.io"
$tokenEndpoint = "$($env:IDENTITY_ENDPOINT)?resource=$resource&api-version=2019-08-01"
$tokenHeaders = @{ "X-IDENTITY-HEADER" = $env:IDENTITY_HEADER }

$tokenResponse = Invoke-RestMethod -Method Get -Uri $tokenEndpoint -Headers $tokenHeaders
$accessToken = $tokenResponse.access_token

All that was left now was to call the API and get the extra information I need to generate a meaningful alert. Simply call the API with the access token in the authorization header, and you’re good to go.

$linkToFilteredSearchResultsAPI = "https://api.loganalytics.io/v1/workspaces/(...)"

$response = Invoke-RestMethod -Method Get -Uri $linkToFilteredSearchResultsAPI -Headers @ { Authorization = "Bearer $accessToken" }

Conclusion

With this info, you should now be able to easily extend your alert data with the original log rows by accessing the Log Analytics API with the Managed Identity of a Function App or Logic App. Using the Microsoft Graph module, you grant the identity the right Entra ID API permissions to access the Log Analytics API. After that, simply get an access token, and call the URI that you so graciously received from Azure Monitor.

Extending your alerts with additional data from the Log Analytics API#

The challenge#

The solution#

Calling the API#

Conclusion#

Extending your alerts with additional data from the Log Analytics API

The challenge

The solution

Calling the API

Conclusion