WAF Triage Workbook error

Question

WAF Triage Workbook error

BigRollTide opened this issue 7 months ago · comments

Describe the bug
Error received when connecting to Log Analytics Workspace. There is no policyScopeName_s table.

Reproduce
Steps to reproduce the behavior:
I deploy the Application Gateway WAF triage workbook, and chose my Subscription and Log Analytics Workspace
I then get an error in the middle (Select the scope of the WAF policy) and received the error "project-reorder: Failed to resolve attribute as a column entity: policyScopeName_s..."

Expected behavior
Expected the workbook to show some data

Screenshots

Environment- if applicable
Used the link to Deploy to Azure

Desktop (please complete the following information if applicable):

OS: Windows 10
Browser: Chrome and Edge get same results

Additional context
There is no policyScopeName_s table in Log Analytics. We are using only v2 APGs and are checking all boxes under Diagnostics Settings to export
This is a Log Analytics Workspace for most of our Azure resources - so it has diagnostic logs for both a few FD WAFs and a few AG WAFs. I also have setup the Azure Monitor Workbook for WAF with no issues pointing to the same Log Analytics Workspace.

tobi o · Answer 1 · Thu Dec 14 2023 00:00:16 GMT+0800 (China Standard Time)

Thank you for the feedback. We are aware of this update request and working on making the reference links aligned

tobi o · Answer 2 · Thu Dec 14 2023 01:05:28 GMT+0800 (China Standard Time)

@xstof is there logic to capture both CRS and DRS for the URL reference for the signatures or do we need a DRS version of the workbook entirely?

BigRollTide · Answer 3 · Thu Jan 11 2024 07:22:36 GMT+0800 (China Standard Time)

I have updated the code for the new logging. Here is a new template

{ "contentVersion": "1.0.0.0", "parameters": { "workbookDisplayName": { "type": "string", "defaultValue": "Application Gateway WAF Triage", "metadata": { "description": "The friendly name for the workbook that is used in the Gallery or Saved List. This name must be unique within a resource group." } }, "workbookSourceId": { "type": "string", "defaultValue": "", "metadata": { "description": "The id of resource instance to which the workbook will be associated" } }, "workbookId": { "type": "string", "defaultValue": "[newGuid()]", "metadata": { "description": "The unique guid for this workbook instance" } } }, "variables": { "workbookContent": { "version": "Notebook/1.0", "items": [ { "type": 1, "content": { "json": "## Application Gateway WAF triage workbook\n---\n\nThis workbook will help you triage Application Gateway WAF violations." }, "name": "text - 2" }, { "type": 9, "content": { "version": "KqlParameterItem/1.0", "parameters": [ { "id": "63ea93e5-3270-4447-9572-ad9e62095e7b", "version": "KqlParameterItem/1.0", "name": "subscription", "label": "Subscription", "type": 6, "isRequired": true, "multiSelect": true, "quote": "'", "delimiter": ",", "value": [ "/subscriptions/651dc44c-5d8e-48da-8cd3-cd79224ac290" ], "typeSettings": { "additionalResourceOptions": [], "includeAll": false } }, { "id": "12476248-81b7-46af-a9b4-790f93306442", "version": "KqlParameterItem/1.0", "name": "workspace", "label": "Workspace", "type": 5, "isRequired": true, "query": "where type =~ 'microsoft.operationalinsights/workspaces'\r\n| summarize by id, name\r\n| project id", "crossComponentResources": [ "{subscription}" ], "value": "/subscriptions/651dc44c-5d8e-48da-8cd3-cd79224ac290/resourceGroups/xstof-appgw/providers/Microsoft.OperationalInsights/workspaces/xstof-appgw", "typeSettings": { "additionalResourceOptions": [] }, "queryType": 1, "resourceType": "microsoft.resourcegraph/resources" }, { "id": "f451ddc5-00e3-4fe7-908e-22d0be9ab59c", "version": "KqlParameterItem/1.0", "name": "detectionTime", "label": "Detection Time", "type": 4, "isRequired": true, "value": { "durationMs": 14400000 }, "typeSettings": { "selectableValues": [ { "durationMs": 300000 }, { "durationMs": 900000 }, { "durationMs": 1800000 }, { "durationMs": 3600000 }, { "durationMs": 14400000 }, { "durationMs": 43200000 }, { "durationMs": 86400000 }, { "durationMs": 172800000 }, { "durationMs": 259200000 }, { "durationMs": 604800000 }, { "durationMs": 1209600000 }, { "durationMs": 2419200000 }, { "durationMs": 2592000000 }, { "durationMs": 5184000000 }, { "durationMs": 7776000000 } ] }, "timeContext": { "durationMs": 86400000 } } ], "style": "pills", "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces" }, "name": "parameters - 2" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \n| where Category == \"ApplicationGatewayFirewallLog\"\n| where TimeGenerated {detectionTime}\n| extend Id = new_guid()\n| extend policyScopeName = AdditionalFields.policyScopeName\n//| project-reorder Id, ResourceId, action_s\n| project-reorder Id, ResourceId, policyScopeName, action_s\n//| summarize count() by ResourceId, policyScopeName, action_s\n//| evaluate pivot(action_s, count(), ResourceId)\n| evaluate pivot(action_s, count(), ResourceId, policyScopeName)", "size": 1, "title": "Select the scope of the WAF policy", "exportedParameters": [ { "fieldName": "policyScopeName", "parameterName": "PolicyScope", "parameterType": 1 }, { "fieldName": "ResourceId", "parameterName": "ResourceId", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "$gen_group", "formatter": 13, "formatOptions": { "linkTarget": null, "showIcon": true } }, { "columnMatch": "ResourceId", "formatter": 5 }, { "columnMatch": "Matched", "formatter": 8, "formatOptions": { "palette": "greenRed" } }, { "columnMatch": "Blocked", "formatter": 8, "formatOptions": { "palette": "greenRed" } } ], "rowLimit": 500, "hierarchySettings": { "treeType": 1, "groupBy": [ "ResourceId" ] }, "labelSettings": [ { "columnId": "ResourceId", "label": "App Gateway" }, { "columnId": "policyScopeName", "label": "Policy Scope" } ] }, "tileSettings": { "showBorder": false } }, "name": "query-app-gateway-resources" }, { "type": 11, "content": { "version": "LinkItem/1.0", "style": "tabs", "links": [ { "id": "14e61d36-d59a-405d-8363-ac1aa8a2f491", "cellValue": "SelectedTabPage", "linkTarget": "parameter", "linkLabel": "Triage by Rule", "subTarget": "triage-by-rule", "preText": "Triage by Rule", "style": "link" }, { "id": "93464fc4-d183-4d74-9e06-111c6de081a9", "cellValue": "SelectedTabPage", "linkTarget": "parameter", "linkLabel": "Triage by URL", "subTarget": "triage-by-url", "preText": "Triage ", "style": "link" } ] }, "name": "TabNavigation" }, { "type": 12, "content": { "version": "NotebookGroup/1.0", "groupType": "editable", "items": [ { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where TimeGenerated {detectionTime}\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| extend ruleSetType_s = AdditionalFields.ruleSetType\r\n| extend ruleSetVersion_s = AdditionalFields.ruleSetVersion\r\n| extend ruleId_s = AdditionalFields.ruleId\r\n| extend hostname_s = AdditionalFields.hostname\r\n| extend details_message_s = AdditionalFields.details_message\r\n| extend details_file_s = AdditionalFields.details_file\r\n| extend details_line_s = AdditionalFields.details_line\r\n| summarize Count = count() by tostring(ruleSetType_s), tostring(ruleSetVersion_s), tostring(ruleId_s), tostring(details_file_s), tostring(details_line_s), action_s\r\n| order by Count desc\r\n| project-reorder Count\r\n| extend normalizedRuleSetVersion = case(isempty(ruleSetVersion_s), \"\",\r\n // old engine: ruleSetVersion_s looks like \"3.1.0\" while type is \"OWASP_CRS\" => leave untouched, can use this as-is\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.1.0/rules/REQUEST-913-SCANNER-DETECTION.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+.[0-9]+\", ruleSetVersion_s,\r\n // new engine: ruleSetVersion_s looks like \"3.2\" while type is \"OWASP CRS\" => need a version that looks like \"3.2.0\"\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.2.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+\", strcat(ruleSetVersion_s, \".0\"),\r\n \"\")\r\n| extend normalizedDetailsFile = case( ruleSetType_s == \"OWASP_CRS\", replace(@'rules/', '', details_file_s), // old engine: details_file_s looks like \"rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP_CRS\"\r\n ruleSetType_s == \"OWASP CRS\", details_file_s, // new engine: details_file_s looks like \"REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP CRS\"\r\n details_file_s)\r\n| extend Url = iif(isnotempty(ruleSetVersion_s) and ruleSetVersion_s !startswith \"1\", strcat('https://github.com/coreruleset/coreruleset/blob/', 'v', normalizedRuleSetVersion, '/rules/', normalizedDetailsFile, '#L', details_line_s), '')\r\n// add sparklines\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayFirewallLog\" | where TimeGenerated {detectionTime} | where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| extend ruleId_s = AdditionalFields.ruleId\r\n| make-series Trend = count() on TimeGenerated from {detectionTime:start} to {detectionTime:end} step (time({detectionTime:seconds} seconds) / 20) by tostring(ruleId_s)\r\n ) on ruleId_s\r\n| extend Id = new_guid()\r\n| project Count,ruleSetVersion_s,ruleId_s,action_s,Trend,Url,Id", "size": 0, "showAnalytics": true, "title": "Rules that got triggered", "exportedParameters": [ { "fieldName": "ruleId_s", "parameterName": "RuleId" }, { "fieldName": "Url", "parameterName": "SpiderLabsUrl", "parameterType": 1 }, { "fieldName": "ruleSetVersion_s", "parameterName": "RuleSetVersion", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "Count", "formatter": 8, "formatOptions": { "palette": "greenRed" } }, { "columnMatch": "ruleId_s", "formatter": 5 }, { "columnMatch": "Trend", "formatter": 10, "formatOptions": { "palette": "yellowOrangeRed" } }, { "columnMatch": "Url", "formatter": 7, "formatOptions": { "linkTarget": "Url", "customColumnWidthSetting": "100%" }, "numberFormat": { "unit": 0, "options": { "style": "decimal" }, "emptyValCustomText": "No link for Custom or Bot rules" } }, { "columnMatch": "Id", "formatter": 5 } ], "sortBy": [ { "itemKey": "$gen_heatmap_Count_0", "sortOrder": 2 } ], "labelSettings": [ { "columnId": "ruleSetVersion_s", "label": "Ruleset Version" }, { "columnId": "ruleId_s", "label": "Rule Id" }, { "columnId": "action_s", "label": "Action" } ] }, "sortBy": [ { "itemKey": "$gen_heatmap_Count_0", "sortOrder": 2 } ] }, "customWidth": "100", "name": "query - violated rules" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}' and AdditionalFields.ruleId == '{RuleId}' and AdditionalFields.ruleSetVersion == '{RuleSetVersion}'\r\n| where TimeGenerated {detectionTime}\r\n// at this point AdditionalFields.hostname can still contain both hostname and port; for example: www.myhost.com:80 -> we need to split this up to match on just the host later\r\n| extend hostname_s = AdditionalFields.hostname\r\n| extend splittedhost = split(hostname_s, ':')\r\n| extend hostname_s_noport = tostring(splittedhost[0]) //override AdditionalFields.hostname to just contain host; drop the port section\r\n| extend port = tostring(splittedhost[1])\r\n| distinct tostring(hostname_s), hostname_s_noport", "size": 0, "title": "Hosts Affected", "exportedParameters": [ { "fieldName": "hostname_s", "parameterName": "HostName" }, { "fieldName": "hostname_s_noport", "parameterName": "HostNameNoPort", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "hostname_s_noport", "formatter": 5 } ], "labelSettings": [ { "columnId": "hostname_s", "label": "Hostname" } ] } }, "customWidth": "15", "showPin": true, "name": "Affected Hosts" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where TimeGenerated {detectionTime}\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}' and AdditionalFields.ruleId == '{RuleId}'\r\n| where AdditionalFields.hostname == '{HostName}'\r\n| extend hostname_s = AdditionalFields.hostname\r\n| project-reorder hostname_s, transactionId_g\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayAccessLog\" | where TimeGenerated {detectionTime}) on transactionId_g\r\n| distinct host_s1, httpMethod_s1, requestUri_s1, originalRequestUriWithArgs_s1\r\n| order by host_s1, requestUri_s1, httpMethod_s1, originalRequestUriWithArgs_s1\r\n| extend id = new_guid()", "size": 0, "showAnalytics": true, "title": "Hosts and urls impacted by selected rule", "exportedParameters": [ { "fieldName": "httpMethod_s1", "parameterName": "Method", "parameterType": 1 }, { "fieldName": "originalRequestUriWithArgs_s1", "parameterName": "FullUri", "parameterType": 1 }, { "fieldName": "requestUri_s1", "parameterName": "RequestUri", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "requestUri_s", "formatter": 5 }, { "columnMatch": "encodedRequestUri", "formatter": 5 }, { "columnMatch": "id", "formatter": 5 } ], "rowLimit": 1000, "hierarchySettings": { "treeType": 1, "groupBy": [ "requestUri_s1" ], "expandTopLevel": true, "finalBy": "requestUri_s" }, "labelSettings": [ { "columnId": "host_s1", "label": "Hostname" }, { "columnId": "httpMethod_s1", "label": "HTTP Method" }, { "columnId": "requestUri_s1", "label": "Request Path" }, { "columnId": "originalRequestUriWithArgs_s1", "label": "Original Request Uri with Arguments" } ] } }, "customWidth": "60", "name": "affected urls" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| where TimeGenerated {detectionTime}\r\n| distinct transactionId_g\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayAccessLog\" | where TimeGenerated {detectionTime}) \r\n on transactionId_g\r\n| where originalHost_s == '{HostNameNoPort}' and httpMethod_s == '{Method}' and requestUri_s == ```{RequestUri}``` and originalRequestUriWithArgs_s == ```{FullUri}```\r\n| project transactionId_g\r\n| extend access_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayAccessLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")\r\n| extend firewall_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayFirewallLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")\r\n//let transIdsWithFirewallLog = AzureDiagnostics\r\n//| where Category == \"ApplicationGatewayFirewallLog\"\r\n//| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n//| where TimeGenerated {detectionTime}\r\n//| distinct transactionId_g;\r\n//AzureDiagnostics\r\n//| where Category == \"ApplicationGatewayAccessLog\"\r\n//| where TimeGenerated {detectionTime}\r\n//| where host_s == '{HostName}' and httpMethod_s == '{Method}' and requestUri_s == '{RequestUri}' and originalRequestUriWithArgs_s == '{FullUri}'\r\n//| distinct transactionId_g\r\n//| where transactionId_g in (transIdsWithFirewallLog) // TODO: how to get tenant id in here?\r\n//| extend access_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayAccessLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")\r\n//| extend firewall_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayFirewallLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")", "size": 0, "title": "Requests on selected host and url", "exportFieldName": "transactionId_g", "exportParameterName": "TransactionId", "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "transactionId_g", "formatter": 0, "formatOptions": { "customColumnWidthSetting": "40%" } }, { "columnMatch": "access_log", "formatter": 7, "formatOptions": { "linkTarget": "OpenBlade", "linkLabel": "Find in Access Log", "linkIsContextBlade": true, "bladeOpenContext": { "bladeName": "LogsBlade", "extensionName": "Microsoft_Azure_Monitoring_Logs", "bladeParameters": [ { "name": "resourceId", "source": "parameter", "value": "workspace" }, { "name": "query", "source": "cell", "value": "" } ] } } }, { "columnMatch": "firewall_log", "formatter": 7, "formatOptions": { "linkTarget": "OpenBlade", "linkLabel": "Find in Firewall Log", "linkIsContextBlade": true, "bladeOpenContext": { "bladeName": "LogsBlade", "extensionName": "Microsoft_Azure_Monitoring_Logs", "bladeParameters": [ { "name": "resourceId", "source": "parameter", "value": "workspace" }, { "name": "query", "source": "cell", "value": "" } ] } } } ], "labelSettings": [ { "columnId": "transactionId_g", "label": "TransactionId" }, { "columnId": "access_log", "label": "Access Log" }, { "columnId": "firewall_log", "label": "Firewall Log" } ] } }, "customWidth": "25", "name": "ImpactedRequestsFromSelectedUri" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where transactionId_g == '{TransactionId}'\r\n| extend Id = new_guid()\r\n| extend ruleSetType_s = AdditionalFields.ruleSetType\r\n| extend ruleSetVersion_s = AdditionalFields.ruleSetVersion\r\n| extend ruleId_s = AdditionalFields.ruleId\r\n| extend hostname_s = AdditionalFields.hostname\r\n| extend details_message_s = AdditionalFields.details_message\r\n| extend details_file_s = AdditionalFields.details_file\r\n| extend details_line_s = AdditionalFields.details_line| project-reorder Id, action_s, ruleSetType_s, ruleSetVersion_s, ruleId_s, Message, hostname_s, requestUri_s, details_message_s, details_data_s, details_file_s, details_line_s, transactionId_g\r\n| extend normalizedRuleSetVersion = case(isempty(ruleSetVersion_s), \"\",\r\n // old engine: ruleSetVersion_s looks like \"3.1.0\" while type is \"OWASP_CRS\" => leave untouched, can use this as-is\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.1.0/rules/REQUEST-913-SCANNER-DETECTION.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+.[0-9]+\", ruleSetVersion_s,\r\n // new engine: ruleSetVersion_s looks like \"3.2\" while type is \"OWASP CRS\" => need a version that looks like \"3.2.0\"\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.2.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+\", strcat(ruleSetVersion_s, \".0\"),\r\n \"\")\r\n| extend normalizedDetailsFile = case( ruleSetType_s == \"OWASP_CRS\", replace(@'rules/', '', tostring(details_file_s)), // old engine: details_file_s looks like \"rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP_CRS\"\r\n ruleSetType_s == \"OWASP CRS\", details_file_s, // new engine: details_file_s looks like \"REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP CRS\"\r\n details_file_s)\r\n| extend Url = iif(isnotempty(ruleSetVersion_s) and ruleSetVersion_s !startswith \"1\", strcat('https://github.com/coreruleset/coreruleset/blob/', 'v', normalizedRuleSetVersion, '/rules/', normalizedDetailsFile, '#L', details_line_s), '')\r\n| project action_s, ruleId_s, Message, details_message_s, details_data_s, TimeGenerated, Url", "size": 0, "title": "Rules that got triggered for selected request", "exportedParameters": [ { "fieldName": "transactionId_g", "parameterName": "TransactionId" }, { "fieldName": "Message", "parameterName": "message", "parameterType": 1 }, { "fieldName": "details_message_s", "parameterName": "details", "parameterType": 1 }, { "fieldName": "details_data_s", "parameterName": "matcheddata", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "action_s", "formatter": 18, "formatOptions": { "thresholdsOptions": "icons", "thresholdsGrid": [ { "operator": "==", "thresholdValue": "Matched", "representation": "2", "text": "{0}{1}" }, { "operator": "==", "thresholdValue": "Blocked", "representation": "4", "text": "{0}{1}" }, { "operator": "Default", "thresholdValue": null, "representation": "more", "text": "{0}{1}" } ] } }, { "columnMatch": "ruleId_s", "formatter": 18, "formatOptions": { "thresholdsOptions": "colors", "thresholdsGrid": [ { "operator": "==", "thresholdValue": "{RuleId}", "representation": "yellow", "text": "{0}{1}" }, { "operator": "Default", "thresholdValue": null, "text": "{0}{1}" } ] } }, { "columnMatch": "Url", "formatter": 7, "formatOptions": { "linkTarget": "Url", "customColumnWidthSetting": "100%" }, "numberFormat": { "unit": 0, "options": { "style": "decimal" }, "emptyValCustomText": "No link for Custom or Bot rules" } }, { "columnMatch": "statusFromAction", "formatter": 5, "numberFormat": { "unit": 0, "options": { "style": "decimal" } } } ], "labelSettings": [ { "columnId": "action_s", "label": "Action" }, { "columnId": "ruleId_s", "label": "Rule Id" }, { "columnId": "Message", "label": "Message" }, { "columnId": "details_message_s", "label": "Details" }, { "columnId": "details_data_s", "label": "Matched Data" }, { "columnId": "TimeGenerated", "label": "Time Generated" } ] } }, "name": "TriggeredRules" }, { "type": 1, "content": { "json": "## Selection Summary:\r\n\r\nYou were looking for example requests which were impacted by WAF rule with id: **{RuleId}**. \r\n\r\nMore information on the selected rule which impacted this url, can be found on the [Core Rule Set Project](https://coreruleset.org/) GitHub page, here:, here: {SpiderLabsUrl}\r\n\r\nOn the host with name \"*{HostName}*\" and url \"*{FullUri}*\" you have found an HTTP request, impacted by this rule, with transaction id equal to *{TransactionId}*. \r\n\r\nMore info on this request and the selected rule:\r\n- Message: {message}\r\n- Details: {details}\r\n- Matched Data: {matcheddata}" }, "name": "text - 7" } ], "exportParameters": true }, "conditionalVisibility": { "parameterName": "SelectedTabPage", "comparison": "isEqualTo", "value": "triage-by-rule" }, "name": "Triage by rule" }, { "type": 12, "content": { "version": "NotebookGroup/1.0", "groupType": "editable", "items": [ { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| where TimeGenerated {detectionTime}\r\n| distinct tostring(AdditionalFields.hostname)\r\n| project hostname_s = AdditionalFields_hostname", "size": 0, "title": "Hostnames with entries in Firewall Log", "exportFieldName": "hostname_s", "exportParameterName": "HostName", "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "hostname_s", "formatter": 0, "formatOptions": { "customColumnWidthSetting": "100%" } } ], "sortBy": [ { "itemKey": "hostname_s", "sortOrder": 1 } ], "labelSettings": [ { "columnId": "hostname_s", "label": "Hostname" } ] }, "sortBy": [ { "itemKey": "hostname_s", "sortOrder": 1 } ] }, "customWidth": "10", "name": "Hostnames with entries in Firewall Log" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| where AdditionalFields.hostname == '{HostName}'\r\n| where TimeGenerated {detectionTime}\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayAccessLog\" | where ResourceId == '{ResourceId}')\r\n on transactionId_g\r\n| distinct requestUri_s, httpMethod_s1, originalRequestUriWithArgs_s1, httpStatus_d1\r\n| extend Id = new_guid()", "size": 0, "showAnalytics": true, "title": "Paths which have triggered firewall rules", "exportedParameters": [ { "fieldName": "requestUri_s", "parameterName": "RequestUri" }, { "fieldName": "originalRequestUriWithArgs_s1", "parameterName": "RequestUriWithArgs", "parameterType": 1 }, { "fieldName": "httpMethod_s1", "parameterName": "HttpMethod", "parameterType": 1 }, { "fieldName": "httpStatus_d1", "parameterName": "HttpStatus", "parameterType": 1 }, { "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "encodedRequestUriWithArgs", "formatter": 5 }, { "columnMatch": "Id", "formatter": 5 } ], "filter": true, "hierarchySettings": { "treeType": 1, "groupBy": [ "requestUri_s", "httpMethod_s1" ], "finalBy": "httpStatus_d1" }, "sortBy": [ { "itemKey": "httpStatus_d1", "sortOrder": 1 } ], "labelSettings": [ { "columnId": "httpMethod_s1", "label": "HTTP Method" }, { "columnId": "originalRequestUriWithArgs_s1", "label": "Original Request Uri with Arguments" }, { "columnId": "httpStatus_d1", "label": "HTTP Status" } ] }, "sortBy": [ { "itemKey": "httpStatus_d1", "sortOrder": 1 } ] }, "customWidth": "60", "name": "Paths which have triggered firewall rules" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| where AdditionalFields.hostname == '{HostName}'\r\n| where TimeGenerated {detectionTime}\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayAccessLog\" | where ResourceId == '{ResourceId}')\r\n on transactionId_g\r\n| where originalRequestUriWithArgs_s1 == ```{RequestUriWithArgs}``` and httpMethod_s1 == '{HttpMethod}' and httpStatus_d1 == '{HttpStatus}'\r\n| distinct transactionId_g\r\n| extend access_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayAccessLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")\r\n| extend firewall_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayFirewallLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")", "size": 0, "title": "Requests on selected host and url", "exportFieldName": "transactionId_g", "exportParameterName": "TransactionId", "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "transactionId_g", "formatter": 0, "formatOptions": { "customColumnWidthSetting": "40%" } }, { "columnMatch": "access_log", "formatter": 7, "formatOptions": { "linkTarget": "OpenBlade", "linkLabel": "Find in Access Log", "linkIsContextBlade": true, "bladeOpenContext": { "bladeName": "LogsBlade", "extensionName": "Microsoft_Azure_Monitoring_Logs", "bladeParameters": [ { "name": "resourceId", "source": "parameter", "value": "workspace" }, { "name": "query", "source": "cell", "value": "" } ] } } }, { "columnMatch": "firewall_log", "formatter": 7, "formatOptions": { "linkTarget": "OpenBlade", "linkLabel": "Find in Firewall Log", "linkIsContextBlade": true, "bladeOpenContext": { "bladeName": "LogsBlade", "extensionName": "Microsoft_Azure_Monitoring_Logs", "bladeParameters": [ { "name": "resourceId", "source": "parameter", "value": "workspace" }, { "name": "query", "source": "cell", "value": "" } ] } } } ], "labelSettings": [ { "columnId": "transactionId_g", "label": "TransactionId" }, { "columnId": "access_log", "label": "Access Log" }, { "columnId": "firewall_log", "label": "Firewall Log" } ] } }, "customWidth": "25", "name": "query - 3" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n//| where TimeGenerated {detectionTime}\r\n| where transactionId_g == '{TransactionId}'\r\n| extend Id = new_guid()\r\n| extend ruleSetType_s = AdditionalFields.ruleSetType\r\n| extend ruleSetVersion_s = AdditionalFields.ruleSetVersion\r\n| extend ruleId_s = AdditionalFields.ruleId\r\n| extend hostname_s = AdditionalFields.hostname\r\n| extend details_message_s = AdditionalFields.details_message\r\n| extend details_file_s = AdditionalFields.details_file\r\n| extend details_line_s = AdditionalFields.details_line\r\n| project-reorder Id, action_s, ruleSetType_s, ruleSetVersion_s, ruleId_s, Message, hostname_s, requestUri_s, details_message_s, details_data_s, details_file_s, details_line_s, transactionId_g\r\n| extend normalizedRuleSetVersion = case(isempty(ruleSetVersion_s), \"\",\r\n // old engine: ruleSetVersion_s looks like \"3.1.0\" while type is \"OWASP_CRS\" => leave untouched, can use this as-is\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.1.0/rules/REQUEST-913-SCANNER-DETECTION.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+.[0-9]+\", ruleSetVersion_s,\r\n // new engine: ruleSetVersion_s looks like \"3.2\" while type is \"OWASP CRS\" => need a version that looks like \"3.2.0\"\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.2.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+\", strcat(ruleSetVersion_s, \".0\"),\r\n \"\")\r\n| extend normalizedDetailsFile = case( ruleSetType_s == \"OWASP_CRS\", replace(@'rules/', '', tostring(details_file_s)), // old engine: details_file_s looks like \"rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP_CRS\"\r\n ruleSetType_s == \"OWASP CRS\", details_file_s, // new engine: details_file_s looks like \"REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP CRS\"\r\n details_file_s)\r\n| extend Url = iif(isnotempty(ruleSetVersion_s) and ruleSetVersion_s !startswith \"1\", strcat('https://github.com/coreruleset/coreruleset/blob/', 'v', normalizedRuleSetVersion, '/rules/', normalizedDetailsFile, '#L', details_line_s), '')\r\n| project action_s, ruleSetType_s, ruleSetVersion_s, ruleId_s, Message, details_message_s, details_data_s, Url, TimeGenerated", "size": 0, "title": "Rules that got triggered for selected request", "exportedParameters": [ { "fieldName": "transactionId_g", "parameterName": "TransactionId" }, { "fieldName": "Message", "parameterName": "message", "parameterType": 1 }, { "fieldName": "details_message_s", "parameterName": "details", "parameterType": 1 }, { "fieldName": "details_data_s", "parameterName": "matcheddata", "parameterType": 1 }, { "fieldName": "Url", "parameterName": "SpiderLabsUrl", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "action_s", "formatter": 18, "formatOptions": { "thresholdsOptions": "icons", "thresholdsGrid": [ { "operator": "==", "thresholdValue": "Matched", "representation": "2", "text": "{0}{1}" }, { "operator": "==", "thresholdValue": "Blocked", "representation": "4", "text": "{0}{1}" }, { "operator": "Default", "thresholdValue": null, "representation": "more", "text": "{0}{1}" } ] } }, { "columnMatch": "ruleId_s", "formatter": 18, "formatOptions": { "thresholdsOptions": "colors", "thresholdsGrid": [ { "operator": "==", "thresholdValue": "{RuleId}", "representation": "yellow", "text": "{0}{1}" }, { "operator": "Default", "thresholdValue": null, "text": "{0}{1}" } ] } }, { "columnMatch": "Url", "formatter": 7, "formatOptions": { "linkTarget": "Url", "customColumnWidthSetting": "100%" }, "numberFormat": { "unit": 0, "options": { "style": "decimal" }, "emptyValCustomText": "No link for Custom or Bot rules" } }, { "columnMatch": "statusFromAction", "formatter": 5, "numberFormat": { "unit": 0, "options": { "style": "decimal" } } } ], "labelSettings": [ { "columnId": "action_s", "label": "Action" }, { "columnId": "ruleSetType_s", "label": "Rule Set Type" }, { "columnId": "ruleSetVersion_s", "label": "Rule Set Version" }, { "columnId": "ruleId_s", "label": "Rule Id" }, { "columnId": "Message", "label": "Message" }, { "columnId": "details_message_s", "label": "Details" }, { "columnId": "details_data_s", "label": "Matched Data" }, { "columnId": "TimeGenerated", "label": "Time Generated" } ] } }, "name": "TriggeredRules - By Url" }, { "type": 1, "content": { "json": "## Selection Summary:\r\n\r\nYou were looking for requests which triggered a WAF rule on url: _{RequestUri}_. \r\nYou have found an HTTP request, impacted by this rule, with transaction id equal to *{TransactionId}*. \r\n\r\nMore information on the selected rule which impacted this url, can be found on the [Core Rule Set Project](https://coreruleset.org) GitHub page, here: {SpiderLabsUrl}\r\n\r\n\r\n\r\nMore info on this request and the selected rule:\r\n- Message: {message}\r\n- Details: {details}\r\n- Matched Data: {matcheddata}" }, "name": "text - 7 - Copy" } ], "exportParameters": true }, "conditionalVisibility": { "parameterName": "SelectedTabPage", "comparison": "isEqualTo", "value": "triage-by-url" }, "name": "Triage by URL" } ], "defaultResourceIds": [ "/subscriptions/651dc44c-5d8e-48da-8cd3-cd79224ac290/resourceGroups/appgw-waf-triage/providers/Microsoft.OperationalInsights/workspaces/la-appgw-waf-triage" ], "$schema": "https://github.com/Microsoft/Application-Insights-Workbooks/blob/master/schema/workbook.json" } }, "resources": [ { "name": "[parameters('workbookId')]", "type": "microsoft.insights/workbooks", "location": "[resourceGroup().location]", "apiVersion": "2021-03-08", "dependsOn": [], "kind": "shared", "properties": { "displayName": "[parameters('workbookDisplayName')]", "serializedData": "[string(variables('workbookContent'))]", "version": "1.0", "sourceId": "[parameters('workbookSourceId')]", "category": "workbook" } } ], "outputs": { "workbookId": { "type": "string", "value": "[resourceId( 'microsoft.insights/workbooks', parameters('workbookId'))]" } }, "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#" }

shabaz-github · Answer 4 · Thu Jun 20 2024 15:30:24 GMT+0800 (China Standard Time)

This issue was resolved by the user with the following PR - #343 - However user did not provide more details to include this in the repository - so closing this issue for now.