Scripts

July 10, 2026 · View on GitHub

PowerShell scripts that drive the deploy and detection pipelines, plus one-time bootstrap and ad-hoc maintenance tooling.

ScriptPurposeDoc anchor
Setup-ServicePrincipal.ps1One-time bootstrap of all required service-principal permissions#setup-serviceprincipalps1
Deploy-SentinelContentHub.ps1Deploys Content Hub solutions and OoB content#deploy-sentinelcontenthubps1
Deploy-CustomContent.ps1Deploys repo-authored custom content#deploy-customcontentps1
Deploy-DefenderDetections.ps1Deploys Defender XDR custom detections via Graph#deploy-defenderdetectionsps1
Import-CommunityRules.ps1Imports community rule sources (Dalonso)#import-communityrulesps1
Set-PlaybookPermissions.ps1Post-deploy: grants managed-identity roles based on each playbook's actual workflow content#set-playbookpermissionsps1
Set-RunbookPermissions.ps1Post-deploy: grants the DCR-watchlist Automation Account managed identity the roles its runbook needs#set-runbookpermissionsps1
Build-DependencyManifest.ps1Auto-derives dependencies.json from KQL discovery (Generate / Verify / Update modes)#build-dependencymanifestps1
Export-SentinelWorkbooks.ps1Exports every Sentinel workbook in a workspace into the Content/Workbooks/ folder shape that Deploy-CustomWorkbooks reads back#export-sentinelworkbooksps1
Invoke-DCRWatchlistSync.ps1Rebuilds the DCR-resources Sentinel watchlist from live DCR associations (runs on the Automation Account schedule)#invoke-dcrwatchlistsyncps1
Migrate-ForkLayout.ps1One-shot fork helper: relocates stragglers left at the pre-26.06 flat layout onto the by-concern layout#migrate-forklayoutps1
Invoke-PRValidation.ps1Cross-platform PR-validation entrypoint: runs every Pester suite under Tests/ and emits an NUnit 2.5 XML reportSee Pester Tests
Test-PullRequestTemplate.ps1Validates a PR description against .github/PULL_REQUEST_TEMPLATE.md; drives the PR Template Validation workflowSee Pipelines
Test-SentinelRuleDrift.ps1Detects portal-edited rules and absorbs Custom driftSee Sentinel Drift Detection

Setup-ServicePrincipal.ps1

One-time bootstrap script that grants the service principal all required Azure, Entra ID, and Microsoft Graph permissions needed for the pipeline to operate autonomously.

Key Features

  • Automated Permission Grant: Grants Contributor, User Access Administrator (ABAC-conditioned), Security Administrator (Entra ID), and CustomDetection.ReadWrite.All (Graph) roles
  • Permission Summary: Displays full summary of permissions before requesting consent
  • User Consent: Y/N prompt with disclaimer before applying changes
  • Selective Steps: Skip optional Entra ID or Graph permissions with -SkipEntraRole and -SkipGraphPermission switches
  • ABAC-Conditioned UAA: User Access Administrator is condition-restricted so the SP can only assign 5 specific roles to other identities (such as playbook managed identities): Microsoft Sentinel Responder, Microsoft Sentinel Reader, Log Analytics Reader, Logic App Contributor, and Managed Identity Operator. Note this is what the SP may hand out, not what it holds itself: the SP's own workspace read access comes from its Contributor grant, and "Microsoft Sentinel Reader" is one of the assignable roles rather than a role granted to the SP directly
  • One-Time Setup: After running once, the pipeline is fully autonomous and requires no manual intervention

Prerequisites

  • Service Principal (app registration) already created
  • The user running the script needs Owner on the target subscription and at least Privileged Role Administrator in Entra ID to grant these permissions
  • Authenticated Azure context (Connect-AzAccount)
  • Az.Accounts, Az.Resources, and Microsoft.Graph PowerShell modules

Parameter Reference

ParameterTypeRequiredDefaultDescription
SubscriptionIdstringYes-Target Azure subscription ID
ServicePrincipalAppIdstringYes-Application (client) ID of the deployment service principal
SkipEntraRoleswitchNo$falseSkip granting the Security Administrator (Entra ID) directory role
SkipGraphPermissionswitchNo$falseSkip granting the CustomDetection.ReadWrite.All (Graph) permission

Both SubscriptionId and ServicePrincipalAppId are mandatory, so every invocation must pass them (there is no TenantId parameter; the tenant is taken from the authenticated context).

Usage Examples

Full setup (all permissions)

.\Setup-ServicePrincipal.ps1 `
    -SubscriptionId "00000000-0000-0000-0000-000000000000" `
    -ServicePrincipalAppId "your-app-id-here"

Skip Entra ID role (UEBA/Entity Analytics enabled separately)

.\Setup-ServicePrincipal.ps1 `
    -SubscriptionId "00000000-0000-0000-0000-000000000000" `
    -ServicePrincipalAppId "your-app-id-here" `
    -SkipEntraRole

Skip Graph permission (for environments without Defender XDR)

.\Setup-ServicePrincipal.ps1 `
    -SubscriptionId "00000000-0000-0000-0000-000000000000" `
    -ServicePrincipalAppId "your-app-id-here" `
    -SkipGraphPermission

Skip both optional permissions

.\Setup-ServicePrincipal.ps1 `
    -SubscriptionId "00000000-0000-0000-0000-000000000000" `
    -ServicePrincipalAppId "your-app-id-here" `
    -SkipEntraRole -SkipGraphPermission

How It Works

  1. Prompt for Confirmation: Displays a comprehensive permission summary and requests Y/N consent before proceeding
  2. Grant Contributor: Grants subscription-level Contributor role for resource group, workspace, Bicep, and content deployment. (Contributor implies Reader at the same scope, which is what ADO needs to save a workload-identity-federation service connection — see ADO OIDC Setup for context.)
  3. Grant UAA (ABAC-Conditioned): Grants User Access Administrator at subscription scope with ABAC conditions restricting assignment to 5 specific roles (Microsoft Sentinel Responder, Microsoft Sentinel Reader, Log Analytics Reader, Logic App Contributor, Managed Identity Operator)
  4. Grant Security Administrator (optional): Grants Entra ID Security Administrator role for UEBA and Entity Analytics settings
  5. Grant Graph Permission (optional): Grants CustomDetection.ReadWrite.All Graph application permission for Defender XDR custom detection rules
  6. Completion: Prints confirmation that setup is complete and the pipeline is ready to run autonomously

Deploy-SentinelContentHub.ps1

Automates the end-to-end deployment of Microsoft Sentinel Content Hub solutions and their packaged content via the Azure REST API (API version 2025-09-01).

Key Features

  • Full Content Type Support: Solutions, analytics rules, workbooks, automation rules, and hunting queries
  • Customisation Protection: Detects locally modified analytics rules and skips them with pipeline warnings
  • Disabled Rule Deployment: Deploy analytics rules in a disabled state for review before enabling
  • Dry Run Mode: WhatIf parameter to preview all changes without applying them
  • Semantic Version Comparison: Detects solutions that require updates
  • ADO Pipeline Integration: Emits pipeline warnings, section messages, and structured output
  • Azure Government Support: Targets Azure Government cloud with the -IsGov switch
  • Metadata Linking: Proper metadata association so content appears correctly in Content Hub

Prerequisites

  • Az.Accounts PowerShell module
  • Authenticated Azure context (Connect-AzAccount or Azure DevOps service connection)

Parameter Reference

ParameterTypeRequiredDefaultDescription
SubscriptionIdstringNoCurrent contextAzure Subscription ID
ResourceGroupstringYes-Resource Group containing the Sentinel workspace
WorkspacestringYes-Log Analytics workspace name
RegionstringYes-Azure region (e.g., uksouth, eastus)
Solutionsstring[]Yes-Content Hub solution names to deploy
SeveritiesToIncludestring[]NoHigh,Medium,Low,InformationalAnalytics rule severities to include
DisableRulesswitchNo$falseDeploy analytics rules as disabled
SkipSolutionDeploymentswitchNo$falseSkip deploying/updating solutions
SkipAnalyticsRulesswitchNo$falseSkip analytics rule deployment
SkipWorkbooksswitchNo$falseSkip workbook deployment
SkipAutomationRulesswitchNo$falseSkip automation rule deployment
SkipHuntingQueriesswitchNo$falseSkip hunting query deployment
ForceSolutionUpdateswitchNo$falseForce update even if version matches
ForceContentDeploymentswitchNo$falseForce redeployment of all content
ProtectCustomisedRulesswitchNo$trueSkip updating locally modified rules
IsGovswitchNo$falseTarget Azure Government cloud
WhatIfswitchNo$falseDry run (no changes applied)

Usage Examples

Basic Deployment

.\Deploy-SentinelContentHub.ps1 `
    -ResourceGroup "rg-sentinel-prod" `
    -Workspace "law-sentinel-prod" `
    -Region "uksouth" `
    -Solutions "Microsoft Defender XDR", "Azure Activity" `
    -DisableRules

Selective Content Deployment

.\Deploy-SentinelContentHub.ps1 `
    -ResourceGroup "rg-sentinel-prod" `
    -Workspace "law-sentinel-prod" `
    -Region "uksouth" `
    -Solutions "Microsoft Defender XDR" `
    -SkipWorkbooks `
    -SkipAutomationRules `
    -SeveritiesToInclude "High", "Medium"

Dry Run

.\Deploy-SentinelContentHub.ps1 `
    -ResourceGroup "rg-sentinel-prod" `
    -Workspace "law-sentinel-prod" `
    -Region "uksouth" `
    -Solutions "Microsoft 365" `
    -WhatIf

Azure Government

.\Deploy-SentinelContentHub.ps1 `
    -ResourceGroup "rg-sentinel-gov" `
    -Workspace "law-sentinel-gov" `
    -Region "USGovVirginia" `
    -Solutions "Azure Activity" `
    -IsGov

How It Works

  1. Authentication and Setup: Validates Azure context, resolves subscription ID, and configures API endpoints
  2. Solution Deployment: Retrieves Content Hub catalogue, compares installed versions, and deploys or updates solutions
  3. Analytics Rule Deployment: Fetches rule templates, filters by severity, detects customised rules, and deploys with metadata
  4. Workbook Deployment: Retrieves workbook templates and deploys with proper metadata linking
  5. Automation Rule Deployment: Discovers and deploys automation rules from solution packages
  6. Hunting Query Deployment: Discovers and deploys hunting queries from solution packages
  7. Status Reporting: Provides detailed deployment summaries with ADO pipeline integration

Tested Solutions

  • Azure Activity
  • Azure Key Vault
  • Azure Logic Apps
  • Azure Network Security Groups
  • Microsoft 365
  • Microsoft Defender for Cloud
  • Microsoft Defender for Cloud Apps
  • Microsoft Defender for Endpoint
  • Microsoft Defender for Identity
  • Microsoft Defender Threat Intelligence
  • Microsoft Defender XDR
  • Microsoft Entra ID
  • Microsoft Purview Insider Risk Management
  • Syslog
  • Threat Intelligence
  • Windows Security Events
  • Windows Server DNS

Known Limitations

  • Solutions requiring specific permissions or prerequisites may need additional configuration
  • Analytics rules referencing tables/columns not present in your environment will be skipped
  • Deprecated rules are skipped by design to prevent deploying outdated content
  • Some workbooks may have dependencies on specific data sources being configured

Deploy-CustomContent.ps1

Deploys custom content from the repository to a Microsoft Sentinel workspace: KQL parsers (YAML), analytics rules (YAML), watchlists (JSON+CSV), playbooks (ARM templates), workbooks (gallery JSON), hunting queries (YAML), automation rules (JSON), and summary rules (JSON).

Key Features

  • Smart Deployment: Opt-in via -SmartDeployment (defaults to OFF, i.e. a full deploy). When enabled, uses git diff to detect changed files and skip unchanged content; deployment-state.json tracks deployment outcomes across runs to automatically retry previously failed items
  • Dependency Graph System: Validates prerequisites per content item (tables, watchlists, functions); detections with missing dependencies deploy disabled, other content types skip
  • KQL Parser Deployment: Deploy workspace saved searches as reusable KQL functions from YAML
  • YAML Detection Rules: Author detections in YAML (Azure-Sentinel repo format), converted to REST API JSON at deploy time
  • Watchlist Management: Deploy watchlists with inline CSV upload via REST API
  • Playbook Deployment: Deploy Logic App playbooks via ARM template deployments with module-first ordering, ARM parameter auto-injection, optional separate resource group, template folder exclusion, and 64-character name truncation
  • Workbook Deployment: Deploy workbooks with stable GUIDs for idempotent updates; accepts both raw gallery/notebook template JSON and ARM deployment templates that wrap a Microsoft.Insights/workbooks resource (the inner serializedData is extracted automatically)
  • Hunting Query Deployment: Deploy YAML-based saved searches to the workspace
  • Automation Rule Deployment: Deploy JSON automation rules for incident auto-response
  • Summary Rule Deployment: Deploy JSON summary rules to aggregate verbose logs into cost-effective custom tables via the Log Analytics API
  • Selective Deployment: Skip individual content types with -Skip* switches
  • Dry Run Mode: WhatIf parameter previews all changes without applying
  • ADO Pipeline Integration: Emits pipeline warnings, section messages, and structured output
  • Azure Government Support: Targets Azure Government cloud with the -IsGov switch

Prerequisites

  • Az.Accounts PowerShell module
  • powershell-yaml module (for YAML detection parsing)
  • Authenticated Azure context (Connect-AzAccount or Azure DevOps service connection)

Parameter Reference

ParameterTypeRequiredDefaultDescription
SubscriptionIdstringNoCurrent contextAzure Subscription ID
ResourceGroupstringYes-Resource Group containing the Sentinel workspace
PlaybookResourceGroupstringNoSame as ResourceGroupResource Group for playbook (Logic App) deployments
WorkspacestringYes-Log Analytics workspace name
RegionstringYes-Azure region (e.g., uksouth, eastus)
BasePathstringNo$env:BUILD_SOURCESDIRECTORY or .Repo root path
SmartDeploymentswitchNo$falseOpt-in switch; when passed, uses git diff to detect changed files and skip unchanged content. Omitted (the default) means a full deploy of all content
SkipParsersswitchNo$falseSkip custom KQL parser deployment
SkipDetectionsswitchNo$falseSkip custom detection deployment
SkipCommunityDetectionsswitchNo$falseSkip rules under Content/AnalyticalRules/Community/** only — non-community detections still deploy. Used by the pipeline's "Skip Community Detections" toggle
SkipWatchlistsswitchNo$falseSkip custom watchlist deployment
SkipPlaybooksswitchNo$falseSkip custom playbook deployment
SkipWorkbooksswitchNo$falseSkip custom workbook deployment
SkipHuntingQueriesswitchNo$falseSkip custom hunting query deployment
SkipAutomationRulesswitchNo$falseSkip custom automation rule deployment
SkipSummaryRulesswitchNo$falseSkip custom summary rule deployment
IsGovswitchNo$falseTarget Azure Government cloud
WhatIfswitchNo$falseDry run (no changes applied)

Usage Examples

Deploy All Custom Content

.\Deploy-CustomContent.ps1 `
    -ResourceGroup "rg-sentinel-prod" `
    -Workspace "law-sentinel-prod" `
    -Region "uksouth"

Deploy Only Detections and Watchlists

.\Deploy-CustomContent.ps1 `
    -ResourceGroup "rg-sentinel-prod" `
    -Workspace "law-sentinel-prod" `
    -Region "uksouth" `
    -SkipPlaybooks `
    -SkipWorkbooks

Deploy Only Hunting Queries and Automation Rules

.\Deploy-CustomContent.ps1 `
    -ResourceGroup "rg-sentinel-prod" `
    -Workspace "law-sentinel-prod" `
    -Region "uksouth" `
    -SkipDetections `
    -SkipWatchlists `
    -SkipPlaybooks `
    -SkipWorkbooks

Deploy Playbooks to a Separate Resource Group

.\Deploy-CustomContent.ps1 `
    -ResourceGroup "rg-sentinel-prod" `
    -PlaybookResourceGroup "rg-playbooks-prod" `
    -Workspace "law-sentinel-prod" `
    -Region "uksouth"

Dry Run

.\Deploy-CustomContent.ps1 `
    -ResourceGroup "rg-sentinel-prod" `
    -Workspace "law-sentinel-prod" `
    -Region "uksouth" `
    -WhatIf

How It Works

  1. Authentication and Setup: Validates Azure context, resolves subscription ID, and configures API endpoints
  2. Smart Deployment Check: Smart deployment is off unless -SmartDeployment is passed; the default is a full deploy of every content type (the log records "Smart deployment disabled, all content will be deployed"). When enabled, uses git diff to detect changed files; unchanged files are skipped unless they were not previously deployed successfully (tracked in deployment-state.json). Failed items are automatically retried on subsequent runs. The smart-deployment skip applies to every content type, not just detections
  3. Dependency Graph Validation: Loads dependencies.json and performs pre-flight checks to bulk-fetch workspace state (tables, watchlists, functions); runs Test-ContentDependencies before each content type
  4. Parser Deployment (Stage 1): Scans Content/Parsers/ for YAML files, validates required fields including functionAlias, converts to saved search body, and deploys via PUT to the savedSearches endpoint
  5. Watchlist Deployment (Stage 2): Scans Content/Watchlists/ for subdirectories with watchlist.json + data.csv, validates metadata, and deploys via PUT with inline CSV content
  6. Detection Deployment (Stage 3): Scans Content/AnalyticalRules/ for YAML files, validates required fields, converts to REST API JSON; if dependencies are missing, deploys as disabled (not skipped) — if API rejects disabled state, gracefully skipped
  7. Hunting Query Deployment (Stage 4): Scans Content/HuntingQueries/ for YAML files, validates required fields, builds saved search body with tactics/techniques tags, and deploys via PUT to the savedSearches endpoint
  8. Playbook Deployment (Stage 5): Scans Content/Playbooks/ for subdirectories with azuredeploy.json (excludes Template/ directory), orders Module/ playbooks first with leaf modules deployed before dependent modules, auto-injects known ARM parameters (ResourceGroup, Workspace, SubscriptionId, WorkspaceId, PlaybookResourceGroup), truncates names to 64 characters, and deploys via New-AzResourceGroupDeployment to the playbook resource group (uses Test-AzResourceGroupDeployment for WhatIf)
  9. Workbook Deployment (Stage 6): Scans Content/Workbooks/ for subdirectories with workbook.json, reads optional metadata.json for stable GUIDs, and deploys via PUT to the Microsoft.Insights/workbooks endpoint. The workbook.json may be either the raw gallery/notebook template JSON, or an ARM deployment template that wraps a Microsoft.Insights/workbooks resource (for example UnifiNetworkOverview); the deployer detects the ARM shape (a $schema matching a deployment template) and extracts the inner workbook serializedData before the PUT, so both formats deploy correctly
  10. Automation Rule Deployment (Stage 7): Scans Content/AutomationRules/ for JSON files, validates required fields (automationRuleId, displayName, order, triggeringLogic, actions), and deploys via PUT to the automationRules endpoint
  11. Summary Rule Deployment (Stage 8): Scans Content/SummaryRules/ for JSON files, validates required fields (name, query, binSize, destinationTable), validates binSize against allowed values and _CL suffix, and deploys via PUT to the summarylogs endpoint using the Microsoft.OperationalInsights provider
  12. Status Reporting: Prints a summary table with deployed/skipped/failed counts per content type

Content Folder Structure and Deployment Order

Content deploys in the following order (also driven by dependencies.json):

  1. Parsers — KQL parser/function definitions
  2. Watchlists — Reusable data lists
  3. Detections — Analytics rules
  4. Hunting Queries — Saved searches
  5. Playbooks — Logic App automation
  6. Workbooks — Visualisation dashboards
  7. Automation Rules — Incident auto-response
  8. Summary Rules — Cost-optimised aggregation

Schema details for each content type:


Deploy-DefenderDetections.ps1

Deploys custom detection rules to Microsoft Defender XDR via the Microsoft Graph Security API (beta). Rules are authored as YAML files in the Content/DefenderCustomDetections/ folder and use the Advanced Hunting KQL schema.

Key Features

  • Graph Security API: Deploys rules via POST/PATCH to /beta/security/rules/detectionRules
  • Upsert Logic: Creates new rules or updates existing ones matched by displayName
  • Response Actions: Supports all Defender response actions (isolate device, force password reset, soft-delete email, etc.)
  • Pagination Handling: Fetches all existing rules with OData pagination
  • Dry Run Mode: WhatIf parameter previews all changes without applying
  • ADO Pipeline Integration: Emits pipeline warnings, section messages, and structured output
  • Azure Government Support: Targets Azure Government Graph endpoint with the -IsGov switch

Prerequisites

  • Az.Accounts PowerShell module
  • powershell-yaml module (for YAML parsing)
  • Authenticated Azure context with CustomDetection.ReadWrite.All Graph application permission
  • Admin consent granted for the Graph permission

Parameter Reference

ParameterTypeRequiredDefaultDescription
BasePathstringNoRepo root (two levels above Deploy/content/)Repo root path containing Content/DefenderCustomDetections/
IsGovswitchNo$falseTarget Azure Government cloud (graph.microsoft.us)
WhatIfswitchNo$falseDry run (no changes applied)

Usage Examples

Deploy All Defender Detections

.\Deploy-DefenderDetections.ps1

Deploy from a Specific Path

.\Deploy-DefenderDetections.ps1 -BasePath "C:\Repos\Sentinel-As-Code"

Dry Run

.\Deploy-DefenderDetections.ps1 -WhatIf

How It Works

  1. Authentication: Acquires a Microsoft Graph token via Get-AzAccessToken -ResourceUrl "https://graph.microsoft.com" using the existing Azure context
  2. Fetch Existing Rules: Queries the Graph API to list all current custom detection rules for upsert matching by displayName
  3. YAML Processing: Scans Content/DefenderCustomDetections/ for YAML files, validates required fields (displayName, queryCondition.queryText, schedule.period, detectionAction.alertTemplate)
  4. Upsert: If a rule with the same displayName exists, updates it via PATCH; otherwise creates via POST
  5. Status Reporting: Prints a summary with created/updated/skipped/failed counts

Content Folder Structure

See Defender-Custom-Detections.md for the full YAML schema, response action types, and impacted asset identifiers.


Import-CommunityRules.ps1

Imports community analytical rules from external repositories into the local codebase. Currently supports the David Alonso (Dalonso) Security repository. See Community-Rules.md for the full contribution model.

Key Features

  • External Repository Support: Currently supports David Alonso's Threat Hunting rules (111 published rules)
  • YAML Rule Import: Clones external Git repositories and copies YAML-format detection rules
  • Format Conversion: Optionally converts KQL+ARM hybrid rules to YAML format for consistency
  • Attribution Tracking: Automatically adds source attribution and creation metadata to imported rules
  • Disabled by Default: All imported rules deploy with enabled: false for review before activation
  • Manifest Generation: Creates import-manifest.json (next to the rules) with content hashes for upstream-drift detection
  • Auto-Generated Summary: Writes a Markdown summary to Docs/Content/Community/{ContributorName}.md with rule counts and per-category listings
  • Change Detection: Uses SHA-256 checksums to track which rules have changed between runs; only updates modified content
  • Idempotent: Fully re-runnable — unchanged rules are skipped, failed rules are retried on next execution
  • Dry Run Mode: DryRun parameter previews all changes without writing to disk
  • ADO Pipeline Integration: Emits pipeline warnings and structured output

Prerequisites

  • powershell-yaml module (for YAML parsing and generation)
  • Git command-line tools (for repository cloning)
  • Internet access to external repositories

Parameter Reference

ParameterTypeRequiredDefaultDescription
OutputPathstringNoContent/AnalyticalRules/Community/DalonsoTarget folder for imported rule YAMLs and import-manifest.json
DocsPathstringNoDocs/Content/Community/{ContributorName}.md (auto-derived from OutputPath leaf)Destination for the auto-generated Markdown summary
SourceRepostringNohttps://github.com/davidalonsod/Dalonso-Security-Repo.gitGit URL of source repository
SourceBranchstringNomainBranch name to clone from
IncludeKqlConversionswitchNo$falseAlso convert KQL+ARM rules to YAML format
DryRunswitchNo$falsePreview changes without writing files

Usage Examples

Import Default Community Rules

.\Import-CommunityRules.ps1

Import with KQL Conversion

.\Import-CommunityRules.ps1 -IncludeKqlConversion

Dry Run Preview

.\Import-CommunityRules.ps1 -DryRun

Onboard a New Contributor

.\Import-CommunityRules.ps1 `
    -OutputPath ./Content/AnalyticalRules/Community/NewContributor `
    -DocsPath   ./Docs/Content/Community/NewContributor.md `
    -SourceRepo "https://github.com/example/threat-rules.git"

How It Works

  1. Repository Cloning: Clones the source repository to a temporary directory using the specified branch
  2. Rule Discovery: Scans the cloned repository for YAML-format detection rules
  3. Metadata Extraction: Reads rule metadata (title, description, tactics, techniques, severity)
  4. Format Conversion (optional): If -IncludeKqlConversion is enabled, converts KQL+ARM hybrid rules to YAML
  5. Attribution: Adds source repository attribution and original author information to each rule
  6. Deployment State: Sets enabled: false on all imported rules for review before activation
  7. Change Detection: Compares SHA-256 checksums of each rule against the import-manifest.json manifest to detect changes
  8. File Writing: Writes new and updated rules to OutputPath, skips unchanged rules
  9. Manifest Creation: Updates import-manifest.json with metadata for all imported rules including source, rule count, and import timestamp
  10. Summary Generation: Writes the human-readable summary to DocsPath (under Docs/Content/Community/) with import source, total rule count, organisation, and instructions for enabling rules
  11. Cleanup: Removes temporary clone directory

Import State Tracking

The import process tracks changes using import-manifest.json (next to the rules):

  • First Run: Creates initial manifest; imports all rules
  • Subsequent Runs: Compares content hashes; only updates changed rules
  • Retries: Previously failed rules are flagged for retry on next execution
  • Idempotency: Running multiple times produces the same result with no duplicate imports

Content Folder Structure

Imported community rules are organised by source:

Content/AnalyticalRules/Community/
└── Dalonso/
    ├── *.yaml                    # Imported community detection rules (per category subfolder)
    └── import-manifest.json      # Content-hash manifest

Docs/Content/Community/
└── Dalonso.md                    # Auto-generated summary, governance doc lives at Docs/Content/Community-Rules.md

Each imported rule includes:

  • Original rule title and description
  • Tactics and techniques (MITRE ATT&CK)
  • Severity level
  • Source repository attribution
  • Original author information (where available)
  • enabled: false flag for review

Set-PlaybookPermissions.ps1

Post-deployment RBAC bootstrap for Logic App playbooks. After Deploy-CustomContent.ps1 deploys playbooks (each tagged Source: Sentinel-As-Code), this script scans them, determines what permissions each playbook actually needs based on its workflow content, and assigns the minimum required roles to each Logic App's system-assigned managed identity.

Why this script exists

Deploy-CustomContent.ps1 runs as the deployment service principal, which has Contributor on the resource group plus an ABAC-conditioned User Access Administrator role (set up by Setup-ServicePrincipal.ps1). That ABAC condition is intentionally restrictive — the deployer can only assign a fixed set of low-privilege roles. It cannot, for example, grant Microsoft Sentinel Contributor directly on the workspace.

The role-assignment work is therefore split out:

  • Deploy-CustomContent.ps1 deploys the Logic App ARM templates (the playbook resource itself), enables system-assigned managed identity, and tags it with Source: Sentinel-As-Code.
  • Set-PlaybookPermissions.ps1 runs separately as a higher-privilege identity (User Access Administrator or Owner) and grants the playbook MSIs the roles they need to actually function.

Key features

  • Tag-scoped discovery — only Logic Apps with the Source: Sentinel-As-Code tag are considered, so hand-deployed Logic Apps in the same resource group are left alone.
  • Workflow-content analysis — instead of granting every role to every playbook, the script inspects each workflow's JSON to derive the minimum role set:
    • API connectors (azuresentinel, keyvault, wdatp, azuremonitorlogs, microsoftgraphsecurity, etc.) → mapped to specific roles per connector type.
    • HTTP actions using ManagedServiceIdentity — when a playbook calls Graph, Defender, or Log Analytics REST APIs directly via the HTTP action with authentication.type: ManagedServiceIdentity, the script reads the URL prefix to grant the right role for that endpoint.
    • Sentinel-specific actions — actions that modify incidents, watchlists, or comments require Sentinel-Contributor-tier roles even when the connector is generic.
  • Idempotent — re-runs are safe; existing assignments are detected and skipped.
  • WhatIf mode — preview every assignment before applying.

Parameter reference

ParameterTypeRequiredDefaultDescription
SubscriptionIdstringNoCurrent Az contextAzure subscription containing the Logic Apps
PlaybookResourceGroupstringYes-Resource group where Logic App playbooks are deployed
SentinelResourceGroupstringNoSame as PlaybookResourceGroupResource group containing the Sentinel workspace, if different
SentinelWorkspaceNamestringYes-Log Analytics workspace name (used to scope Sentinel role assignments)
KeyVaultNamestringNo-Key Vault used by playbooks. Required only when at least one playbook references Key Vault
WhatIfswitchNo$falsePreview role assignments without applying

Usage

# Standard run — Sentinel + playbooks in the same resource group
./Set-PlaybookPermissions.ps1 `
    -PlaybookResourceGroup "rg-sentinel-prod" `
    -SentinelWorkspaceName "law-sentinel-prod"

# Sentinel and playbooks in separate resource groups, Key Vault in use
./Set-PlaybookPermissions.ps1 `
    -PlaybookResourceGroup "rg-playbooks-prod" `
    -SentinelResourceGroup "rg-sentinel-prod" `
    -SentinelWorkspaceName "law-sentinel-prod" `
    -KeyVaultName "kv-sentinel-prod"

# Dry run
./Set-PlaybookPermissions.ps1 `
    -PlaybookResourceGroup "rg-sentinel-prod" `
    -SentinelWorkspaceName "law-sentinel-prod" `
    -WhatIf

Prerequisites for the executing principal

The identity running this script needs User Access Administrator or Owner on the playbook resource group (and on the Sentinel resource group if different). The deployment SPN does not have this role by default — Setup-ServicePrincipal.ps1 only grants it under an ABAC condition that doesn't permit Sentinel-tier role assignments. Run this script under a separate elevated identity (typically a one-off run by an admin user, not the pipeline SPN).


Set-RunbookPermissions.ps1

Post-deployment RBAC bootstrap for the DCR-watchlist Automation Account. It is the Automation-runbook sibling of Set-PlaybookPermissions.ps1: after the Infra/dcr-watchlist/ stack deploys the Automation Account and its system-assigned managed identity, this script grants that identity the two roles the sync runbook (Invoke-DCRWatchlistSync.ps1) needs to operate.

Why this script exists

The pipeline service principal does not hold Microsoft.Authorization/roleAssignments/write, so it cannot assign these roles itself. RBAC for the Automation Account managed identity is therefore applied out of band by a user with Owner or User Access Administrator on the subscription. Run it once after the DCR-watchlist infrastructure is first deployed.

Roles granted

  • Monitoring Reader (subscription scope) - lets the runbook list Data Collection Rules and their associations via the ARM API.
  • Microsoft Sentinel Contributor (Sentinel resource group scope) - lets the runbook create and update the Sentinel watchlist.

Parameter reference

ParameterTypeRequiredDefaultDescription
SubscriptionIdstringYes-Target subscription ID
AutomationAccountNamestringYes-Name of the DCR-watchlist Automation Account (conventionally aa-dcr-watchlist-sync)
AutomationResourceGroupstringYes-Resource group containing the Automation Account (conventionally rg-dcr-watchlist-sync)
SentinelResourceGroupstringYes-Resource group containing the Sentinel workspace (scope for the Sentinel Contributor grant)
RemoveswitchNo$falseRemove the role assignments instead of creating them

The script uses [CmdletBinding(SupportsShouldProcess)], so -WhatIf previews every assignment before applying.

Usage

# Apply permissions
.\Set-RunbookPermissions.ps1 `
    -SubscriptionId "00000000-0000-0000-0000-000000000000" `
    -AutomationAccountName "aa-dcr-watchlist-sync" `
    -AutomationResourceGroup "rg-dcr-watchlist-sync" `
    -SentinelResourceGroup "rg-sentinel-prod"

# Remove permissions
.\Set-RunbookPermissions.ps1 `
    -SubscriptionId "00000000-0000-0000-0000-000000000000" `
    -AutomationAccountName "aa-dcr-watchlist-sync" `
    -AutomationResourceGroup "rg-dcr-watchlist-sync" `
    -SentinelResourceGroup "rg-sentinel-prod" `
    -Remove

Prerequisites for the executing principal

Owner or User Access Administrator on the subscription. As with Set-PlaybookPermissions.ps1, the deployment SPN cannot run this because it lacks role-assignment write.


Build-DependencyManifest.ps1

Walks Content/AnalyticalRules/ and Content/HuntingQueries/, parses every embedded KQL query, and emits or verifies dependencies.json — the manifest the deploy pipeline reads to drive content-ordering decisions. Replaces the previously hand-maintained workflow.

See Operations / Dependency Manifest for the full discovery model, schema, and reasoning. This section covers the script's parameter surface and operating modes.

Key features

  • Three operating modes — Generate (write the manifest from discovery), Verify (drift-check vs the on-disk manifest, used by the PR-validation gate), Update (drift-detect + write to disk for the daily auto-PR workflow).
  • Repo-driven inventory — no hard-coded table list. Functions come from Content/Parsers/**/*.yaml, watchlists from Content/Watchlists/*/watchlist.json, playbooks from Content/Playbooks/**/*.json. Tables default to "anything not classified as a function" because tables are external (data plane) and not deployable from the repo.
  • KQL pattern coverage — extracts identifiers from start of statement, after let X =, after union, inside join / lookup / materialize / view / toscalar subqueries, and from table('X') string-arg patterns (lambda-wrapper invocations).
  • Watchlist cross-validation — flags _GetWatchlist('alias') references that don't resolve to an in-repo watchlist as warnings.
  • Offline — no Azure auth required; reads YAML and JSON only.

Parameter reference

ParameterTypeRequiredDefaultDescription
Modestring (Generate | Verify | Update)Yes-Operating mode (see below)
RepoPathstringNoParent of the script folder (Deploy/ or Tools/)Repository root
ManifestPathstringNo<RepoPath>/dependencies.jsonManifest file path

Operating modes

Generate

Walks content, builds the manifest, writes dependencies.json. Authors run this locally after editing rules and commit the regenerated file alongside the rule changes.

./Tools/Build-DependencyManifest.ps1 -Mode Generate

Verify

Builds the manifest in-memory and compares against the on-disk file. Exits 0 on match, 1 on drift with a structured diff. Used by:

./Tools/Build-DependencyManifest.ps1 -Mode Verify
# exit 0 = manifest matches; exit 1 = drift (with diff printed)

Update

Like Verify, but on detected drift writes the regenerated manifest to disk and exits 0. The calling pipeline owns the commit + branch + PR step. Used by the daily auto-PR workflow (sentinel-dependency-update.yml and its ADO equivalent).

./Tools/Build-DependencyManifest.ps1 -Mode Update

Author workflow

After editing or adding a rule that references new tables, watchlists, or functions:

./Tools/Build-DependencyManifest.ps1 -Mode Generate
git add dependencies.json
git commit

If you forget, the PR-validation gate fails with a clear "out of sync" error and the regenerate command. If you forget AND the gate's path filter misses the change (rare — happens for doc-only commits that adjust an inline query), the daily workflow opens a chore PR within 24 hours.

Prerequisites

  • PowerShell 7.2+
  • powershell-yaml module (auto-installed by the script if missing)
  • Modules/Sentinel.Common (auto-imported from the script — provides Get-ContentDependencies and the KQL extractors)

Authoring with GitHub Copilot

When editing files under Deploy/**, Tools/**, or Modules/**, Copilot automatically loads .github/instructions/powershell-scripts.instructions.md. The path-scoped instructions cover the file-header convention, Sentinel.Common import patterns, and the foot-gun list ([void] Boolean leak, single-element array indexing, strict-mode property access, $script: scope rules).

Copilot tooling for scripts and modules:

  • Agent Sentinel-As-Code: PowerShell Engineer — owns Modules/Sentinel.Common end-to-end. Add functions, refactor scripts, modernise legacy patterns, harden against strict-mode failures. Knows the module-manifest discipline (version bump + ReleaseNotes + Pester tests in lockstep).
  • Agent Sentinel-As-Code: Dependencies Engineer — when a change touches the discovery extractors (Get-KqlBareIdentifiers, Get-ContentDependencies, etc.).
  • Agent Sentinel-As-Code: Security Reviewer — for review of secret-handling, log-leak surface, RBAC-impacting scripts.
  • Slash command /regenerate-deps (VS Code) — runs Build-DependencyManifest -Mode Generate.

See GitHub Copilot setup for the full layout.


Export-SentinelWorkbooks.ps1

Exports every Sentinel workbook in a workspace into the Content/Workbooks/<FolderName>/ shape that Deploy-CustomContent.ps1's Deploy-CustomWorkbooks function redeploys from. The exported tree is byte-compatible with what the deployer reads — round-trip (export → commit → redeploy) lands updates on the same Azure resource rather than spawning duplicates.

Key features

  • Bulk export — lists every Sentinel-scoped workbook in one REST call (Microsoft.Insights/workbooks filtered by category=sentinel and sourceId={workspaceResourceId}).
  • Content Hub workbooks excluded by default — only Custom workbooks land in the repo. The script enumerates the workspace's Microsoft.SecurityInsights/metadata resources to identify which workbooks were installed by a Content Hub solution (via source.kind == 'Solution'), and skips them. Override with -IncludeContentHub if you have a specific reason (e.g. forking a solution-provided workbook into your own governance — rare).
  • Folder name = PascalCase compaction of displayName — the on-disk folder is the workbook's displayName with non-alphanumeric runs (spaces, punctuation, parens) treated as word boundaries and each word TitleCased. All-upper acronyms become TitleCase (GBPGbp); user-curated camelCase brands (e.g. pfSense) are preserved. This matches the existing Content/Workbooks/* convention in the repo (e.g. MicrosoftSentinelCostGbp, MicrosoftSentinelMonitoring).
  • Workspace-name suffix stripped — Microsoft-published workbook templates that get instantiated per-workspace pick up a - <workspace-name> suffix on their displayName (e.g. Data Collection Rule Toolkit - stl-eus-siem-law). The script strips this suffix before deriving the folder name AND from the metadata.json's displayName. Sentinel re-attaches it at display time on redeploy, so the round-trip stays stable.
  • Workspace ARM ID stripped from workbook content — Sentinel bakes the source workspace's full ARM resource ID into fallbackResourceIds and sometimes inline resource references (e.g. /subscriptions/<sub>/resourceGroups/<rg>/providers/microsoft.operationalinsights/workspaces/<ws>). The script replaces every case-insensitive occurrence with the placeholder convention used by hand-curated repo workbooks (/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/your-resource-group/providers/microsoft.operationalinsights/workspaces/your-workspace), so the workbook isn't pinned to one specific workspace. The field only affects standalone Workbooks-portal viewing; opening the workbook from within a Sentinel workspace uses the workspace context regardless.
  • Workbook GUID preservation — writes the workbook's resource GUID into metadata.json as workbookId. The next deploy reads it back and hits the same Azure resource, avoiding the duplicate-workbook failure mode.
  • -OnlyMissing mode — skip workbooks that already have a folder. Useful for incremental import without overwriting in-repo customisations.
  • -Filter regex — narrow the export to a subset by displayName match.
  • -WhatIf mode — read everything, write nothing.
  • Azure Government Support-IsGov switch.
  • Existing-metadata preservation — extra keys in an existing metadata.json (tags, custom annotations) survive overwrite; only the keys this script writes are replaced.

Parameter reference

ParameterTypeRequiredDefaultDescription
SubscriptionIdstringNoCurrent Az contextAzure Subscription ID
ResourceGroupstringYes-Resource group containing the Sentinel workspace
WorkspacestringYes-Log Analytics workspace name (used to derive WorkspaceResourceId for the sourceId filter)
RegionstringYes-Azure region (passed through to Connect-AzureEnvironment)
BasePathstringNoParent of the script folder (Deploy/ or Tools/)Repo root path; output is written to <BasePath>/Content/Workbooks/
FilterstringNo'.'Regex applied to each workbook's displayName; non-matching workbooks skipped
OnlyMissingswitchNo$falseSkip workbooks that already have a folder under Content/Workbooks/
IncludeContentHubswitchNo$falseBy default, Content Hub-managed workbooks are excluded. Pass to include them (advanced; usually wrong because Content Hub will overwrite on update)
WhatIfswitchNo$falsePreview without writing
IsGovswitchNo$falseTarget Azure Government cloud

Usage examples

Export every workbook in the workspace

.\Export-SentinelWorkbooks.ps1 `
    -ResourceGroup 'rg-sentinel-prod' `
    -Workspace     'law-sentinel-prod' `
    -Region        'uksouth'

Incremental import — only workbooks not in the repo yet

.\Export-SentinelWorkbooks.ps1 `
    -ResourceGroup 'rg-sentinel-prod' `
    -Workspace     'law-sentinel-prod' `
    -Region        'uksouth' `
    -OnlyMissing

Preview without writing

.\Export-SentinelWorkbooks.ps1 `
    -ResourceGroup 'rg-sentinel-prod' `
    -Workspace     'law-sentinel-prod' `
    -Region        'uksouth' `
    -WhatIf

Export only workbooks whose display name starts with 'Identity'

.\Export-SentinelWorkbooks.ps1 `
    -ResourceGroup 'rg-sentinel-prod' `
    -Workspace     'law-sentinel-prod' `
    -Region        'uksouth' `
    -Filter        '^Identity'

How it works

  1. Authentication and Setup: Imports the Sentinel.Common module and calls Connect-AzureEnvironment to acquire an access token + workspace resource ID.
  2. Identify Content Hub workbooks: Unless -IncludeContentHub was supplied, the script first calls GET .../providers/Microsoft.SecurityInsights/metadata?api-version=2025-09-01 (with pagination) and builds a HashSet of workbook resource IDs where properties.kind == 'Workbook' AND properties.source.kind == 'Solution'. Any workbook found in this set is skipped during the export pass.
  3. List: Calls GET /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Insights/workbooks?api-version=2022-04-01&category=sentinel&sourceId={workspaceResourceId}&canFetchContent=true to enumerate Sentinel workbooks. canFetchContent=true is essential — without it the LIST API returns only resource metadata, omitting the serializedData (the gallery template content) to keep response sizes small. The flag tells ARM to include full content for workbooks the caller has read access to.
  4. Per-workbook GET fallback: If a workbook's content is still absent from the LIST response (rare — typically a Microsoft- published Content Hub workbook the workspace inherits, which the metadata-based filter in step 2 should have already excluded), the script does a per-resource GET /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Insights/workbooks/{name}?canFetchContent=true to resolve content one workbook at a time. Only workbooks where neither the list nor the detail fetch returns content get skipped (with a clear warning explaining the cause).
  5. Per-workbook export: For each result:
    • Skip if the workbook ID is in the Content Hub set (unless -IncludeContentHub was supplied)
    • Filter by regex on displayName if -Filter was supplied
    • Strip any trailing - <WorkspaceName> suffix from the displayName (Microsoft attaches this to workspace-instantiated templates; not useful for repo storage)
    • Skip if -OnlyMissing and the folder already exists
    • Reformat serializedData (a JSON string) via ConvertFrom-Json + ConvertTo-Json -Depth 32 so the on-disk file is pretty-printed
    • Replace every occurrence of the source workspace's ARM resource ID in the JSON with the placeholder /subscriptions/00000000-0000-0000-0000-000000000000/... (case-insensitive) so the workbook isn't pinned to one workspace
    • Write Content/Workbooks/<FolderName>/workbook.json and Content/Workbooks/<FolderName>/metadata.json (using the cleaned displayName for both folder name and metadata.json)
  6. Status reporting: Prints a summary table with exported / skipped / failed counts.

Symmetry contract with Deploy-CustomWorkbooks

ConcernExport sideDeploy side
API version2022-04-012022-04-01
Folder namePascalCase compaction of displayName (with workspace suffix stripped)Folder name walked as-is; deploy reads displayName from metadata.json
workbook.json contentThe full gallery templateRead into serializedData of the deploy body
metadata.json keys readdisplayName, description, category, sourceId, workbookIdSame keys consumed
Resource GUIDPreserved via workbookIdUsed as the URI's {workbookId} segment

Prerequisites

  • PowerShell 7.2+
  • Az.Accounts (auto-imported by Sentinel.Common)
  • Authenticated Azure context (Connect-AzAccount or service-principal context)
  • The deploy SP needs at least Microsoft Sentinel Reader on the workspace to list workbooks. The standard Setup-ServicePrincipal.ps1 Contributor grant covers this.

Known limitations

  • Only workbooks where category == sentinel are exported. Application Insights / general-purpose workbooks living under the same workspace are filtered out by design.
  • Content Hub-managed workbooks are excluded by default (see step 2 of How it works). If the workspace has many Content Hub solutions installed, you may see a long list of "Skipping ... — Content Hub-managed workbook" messages. This is intentional; pass -IncludeContentHub only if you have a specific reason.
  • Templates that haven't been customised in the workspace don't appear at all — they're served from the Content Hub catalogue, not the workspace's resource list, so there's nothing to export.

Invoke-DCRWatchlistSync.ps1

Rebuilds the Sentinel "Customer DCR Resources" watchlist from the live inventory of Data Collection Rules and their associations. This is the runbook that runs on the schedule set up by the Infra/dcr-watchlist/ stack (Automation Account plus runbook), authenticating as the Automation Account's system-assigned managed identity. Set-RunbookPermissions.ps1 grants that identity the roles this script needs.

What it does

  1. Authenticates via the system-assigned managed identity.
  2. Lists every Data Collection Rule in the subscription with Invoke-AzRestMethod (DCR api-version 2024-03-11) and retrieves the associations for each DCR. There is no Az.ResourceGraph dependency; it uses the same ARM REST pattern as Invoke-DCRAudit.ps1.
  3. Builds an in-memory CSV (one row per DCR).
  4. Calls the Sentinel watchlist REST API (api-version 2025-09-01) to delete and recreate the watchlist in a single full-replace operation.

Because there is one row per DCR, the watchlist's search key is DCRName (the script's default). The DCR-inventory pipelines register the runbook with SearchKey=DCRName to match the per-DCR rows.

Parameter reference

ParameterTypeRequiredDefaultDescription
SubscriptionIdstringYes-Subscription to enumerate DCRs from
WorkspaceResourceGroupstringYes-Resource group containing the Sentinel Log Analytics workspace
WorkspaceNamestringYes-Log Analytics workspace name (Sentinel)
WatchlistAliasstringYes-Alias (unique identifier) for the Sentinel watchlist
WatchlistDisplayNamestringNoCustomer DCR ResourcesHuman-readable display name shown in Sentinel
SearchKeystringNoDCRNameColumn used as the watchlist search key

Required RBAC on the managed identity

  • Monitoring Reader on the subscription (to list DCRs and their associations via ARM).
  • Microsoft Sentinel Contributor on the Sentinel resource group (watchlist write).

Both are granted by Set-RunbookPermissions.ps1.


Migrate-ForkLayout.ps1

One-shot helper for fork maintainers. The 26.06 restructure moved the repository from a flat root into grouped folders (Content/, Infra/, Deploy/, Tools/). Tracked files move automatically when you merge or rebase the restructure (git rename detection reconciles your customisations). This helper catches stragglers - untracked custom content or conflict leftovers still sitting at an old path - and moves them to their new home.

What it does

  • Moves files at the filesystem level (Move-Item), which works for both tracked and untracked files; git detects the renames for tracked content on your next commit.
  • Is idempotent: paths already at the new location are skipped. When both the old and new path exist (a partial migration), the old folder's contents are merged into the new folder and a warning is emitted for anything that would collide.
  • Does NOT rewrite file contents, regenerate dependencies.json, or commit. After running it, review git status, run ./Tools/Build-DependencyManifest.ps1 -Mode Generate, then commit.

Parameter reference

ParameterTypeRequiredDefaultDescription
RepoPathstringNoParent of the Tools/ folder this script lives inRepository root
AllowDirtyswitchNo$falseProceed even if the working tree has uncommitted changes. By default the script refuses to run on a dirty tree so the moves are easy to review and revert

The script uses [CmdletBinding(SupportsShouldProcess)], so -WhatIf previews every move without touching the tree.

Usage

# Preview every move without touching the tree
./Tools/Migrate-ForkLayout.ps1 -WhatIf

# Apply the moves
./Tools/Migrate-ForkLayout.ps1

Documenter scripts

The repository documentation generator lives under Tools/Documenter/ (Export-SentinelInventory.ps1, Convert-SentinelInventoryToMarkdown.ps1, Convert-MermaidToImage.ps1, plus the Report/ and Private/ helpers) and is documented separately. See Sentinel Documenter for the inventory export model, Markdown/Word report generation, and diagram rendering.