Skip to content

Commit

Permalink
v6_major_20220622_1
Browse files Browse the repository at this point in the history
  • Loading branch information
JulianHayward committed Jun 22, 2022
1 parent e22f11c commit eb4275f
Show file tree
Hide file tree
Showing 12 changed files with 1,288 additions and 36 deletions.
15 changes: 10 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,13 @@ Listed as [security monitoring tool](https://docs.microsoft.com/en-us/azure/arch

## Release history

__Changes__ (2022-Jun-14 / Major)
__Changes__ (2022-Jun-22 / Major)

* Fix issue #110 / handle `DisallowedProvider` errorCode (Blueprints, PolicyInsights)
* Fix issue #111 / replace .AddRange with foreach/.Add
* Use AzAPICall PowerShell module version 1.1.16
* New feature 'Orphaned Resources' - Azure Resource Graph based reporting on orphaned resources (TenantSummary, ScopeInsights, CSV export). [Azure Orphan Resources - GitHub](https://github.com/dolevshor/azure-orphan-resources) ARG queries and workbooks by Dolev Shor
* New feature 'Resource fluctuation' - Compare against Resources from previous run and output aggregated summary of the Resource fluctuation (TenantSummary, ScopeInsights, CSV export)
* Fix `/providers/Microsoft.Authorization/roleAssignmentScheduleInstances` AzAPICall errorhandling (error 400, 500)
* Optimize procedure to update the AzAPICall module
* Use AzAPICall PowerShell module version 1.1.17

Passed tests: Powershell Core 7.2.4 on Windows
Passed tests: Powershell Core 7.2.4 Azure DevOps hosted agent ubuntu-20.04
Expand Down Expand Up @@ -174,7 +176,7 @@ Short presentation on AzGovViz [[download](slides/AzGovViz_intro.pdf)]
* Hierarchy Settings | Require authorization for Management Group creation
* __Subscriptions, Resources & Defender__
* Subscription insights
* QuotaId, State, Tags, Microsoft Defender for Cloud Secure Score, Cost, Management Group path, Role assignment limit
* QuotaId, State, Tags, Microsoft Defender for Cloud Secure Score, Cost, Management Group path, Role assignment limit, enabled Preview features
* Tag Name usage
* Insights on usage of Tag Names on Subscriptions, ResourceGroups and Resources
* Resources
Expand All @@ -185,13 +187,16 @@ Short presentation on AzGovViz [[download](slides/AzGovViz_intro.pdf)]
* Explicit Resource Provider state per Subscription
* Resource Locks
* Aggregated insights for Lock and respective Lock-type usage on Subscriptions, ResourceGroups and Resources
* Orphaned Resources (ARG)
* Microsoft Defender for Cloud
* Summary of Microsoft Defender for Cloud coverage by plan (count of Subscription per plan/tier)
* Summary of Microsoft Defender for Cloud plans coverage by Subscription (plan/tier)
* Highlight the usage of deprecated Defender plans (e.g. Container Registry & Kubernetes)
* UserAssigned Managed Identities assigned to Resources / vice versa
* Summary of all UserAssigned Managed Identities assigned to Resources
* Summary of Resources that have an UserAssigned Managed Identity assigned
* PSRule for Azure
* Well-Architected Framework aligned best practice analysis for resources, including guidance for remediation
* __Diagnostics__
* Management Groups Diagnostic settings report
* Management Group, Diagnostic setting name, target type (LA, SA, EH), target Id, Log Category status
Expand Down
8 changes: 8 additions & 0 deletions history.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@

### AzGovViz version 6

__Changes__ (2022-Jun-22 / Major)

* New feature 'Orphaned Resources' - Azure Resource Graph based reporting on orphaned resources (TenantSummary, ScopeInsights, CSV export). [Azure Orphan Resources - GitHub](https://github.com/dolevshor/azure-orphan-resources) ARG queries and workbooks by Dolev Shor
* New feature 'Resource fluctuation' - Compare against Resources from previous run and output aggregated summary of the Resource fluctuation (TenantSummary, ScopeInsights, CSV export)
* Fix `/providers/Microsoft.Authorization/roleAssignmentScheduleInstances` AzAPICall errorhandling (error 400, 500)
* Optimize procedure to update the AzAPICall module
* Use AzAPICall PowerShell module version 1.1.17

__Changes__ (2022-Jun-14 / Major)

* Fix issue #110 / handle `DisallowedProvider` errorCode (Blueprints, PolicyInsights)
Expand Down
649 changes: 634 additions & 15 deletions pwsh/AzGovVizParallel.ps1

Large diffs are not rendered by default.

11 changes: 9 additions & 2 deletions pwsh/dev/devAzGovVizParallel.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -283,10 +283,10 @@ Param
$Product = 'AzGovViz',

[string]
$AzAPICallVersion = '1.1.16',
$AzAPICallVersion = '1.1.17',

[string]
$ProductVersion = 'v6_major_20220614_1',
$ProductVersion = 'v6_major_20220622_1',

[string]
$GithubRepository = 'aka.ms/AzGovViz',
Expand Down Expand Up @@ -511,6 +511,7 @@ Write-Host "Start AzGovViz $($startTime) (#$($ProductVersion))"
. ".\$($ScriptPath)\functions\processHierarchyMapOnly.ps1"
. ".\$($ScriptPath)\functions\getSubscriptions.ps1"
. ".\$($ScriptPath)\functions\detailSubscriptions.ps1"
. ".\$($ScriptPath)\functions\getOrphanedResources.ps1"
. ".\$($ScriptPath)\functions\getMDfCSecureScoreMG.ps1"
. ".\$($ScriptPath)\functions\getConsumption.ps1"
. ".\$($ScriptPath)\functions\cacheBuiltIn.ps1"
Expand Down Expand Up @@ -745,6 +746,7 @@ if ($azAPICallConf['htParameters'].HierarchyMapOnly -eq $false) {
$arrayPsRule = [System.Collections.ArrayList]::Synchronized((New-Object System.Collections.ArrayList))
$arrayPSRuleTracking = [System.Collections.ArrayList]::Synchronized((New-Object System.Collections.ArrayList))
$htClassicAdministrators = [System.Collections.Hashtable]::Synchronized((New-Object System.Collections.Hashtable)) #@{}
$arrayOrphanedResources = [System.Collections.ArrayList]::Synchronized((New-Object System.Collections.ArrayList))
}

getEntities
Expand All @@ -766,6 +768,8 @@ if ($azAPICallConf['htParameters'].HierarchyMapOnly -eq $false) {
getSubscriptions
detailSubscriptions
showMemoryUsage
getOrphanedResources
showMemoryUsage

if ($azAPICallConf['htParameters'].NoMDfCSecureScore -eq $false) {
getMDfCSecureScoreMG
Expand Down Expand Up @@ -1718,6 +1722,9 @@ if ($azAPICallConf['htParameters'].HierarchyMapOnly -eq $false) {
if ($arrayFeaturesAll.Count -gt 0) {
$script:subFeaturesGroupedBySubscription = $arrayFeaturesAll | Group-Object -property subscriptionId
}
if ($arrayOrphanedResourcesSlim.Count -gt 0) {
$arrayOrphanedResourcesGroupedBySubscription = $arrayOrphanedResourcesSlim | Group-Object subscriptionId
}
processScopeInsights -mgChild $ManagementGroupId -mgChildOf $getMgParentId
showMemoryUsage
#[System.GC]::Collect()
Expand Down
4 changes: 2 additions & 2 deletions pwsh/dev/functions/dataCollection/dataCollectionFunctions.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -2368,7 +2368,7 @@ function dataCollectionRoleAssignmentsMG {
$method = 'GET'
$roleAssignmentScheduleInstancesFromAPI = AzAPICall -AzAPICallConfiguration $azAPICallConf -uri $uri -method $method -currentTask $currentTask -caller 'CustomDataCollection'

if ($roleAssignmentScheduleInstancesFromAPI -eq 'ResourceNotOnboarded' -or $roleAssignmentScheduleInstancesFromAPI -eq 'TenantNotOnboarded' -or $roleAssignmentScheduleInstancesFromAPI -eq 'InvalidResourceType') {
if ($roleAssignmentScheduleInstancesFromAPI -eq 'ResourceNotOnboarded' -or $roleAssignmentScheduleInstancesFromAPI -eq 'TenantNotOnboarded' -or $roleAssignmentScheduleInstancesFromAPI -eq 'InvalidResourceType' -or $roleAssignmentScheduleInstancesFromAPI -eq 'RoleAssignmentScheduleInstancesError') {
#Write-Host "Scope '$($scopeDisplayName)' ('$scopeId') not onboarded in PIM"
}
else {
Expand Down Expand Up @@ -2644,7 +2644,7 @@ function dataCollectionRoleAssignmentsSub {
$method = 'GET'
$roleAssignmentScheduleInstancesFromAPI = AzAPICall -AzAPICallConfiguration $azAPICallConf -uri $uri -method $method -currentTask $currentTask -caller 'CustomDataCollection'

if ($roleAssignmentScheduleInstancesFromAPI -eq 'ResourceNotOnboarded' -or $roleAssignmentScheduleInstancesFromAPI -eq 'TenantNotOnboarded' -or $roleAssignmentScheduleInstancesFromAPI -eq 'InvalidResourceType') {
if ($roleAssignmentScheduleInstancesFromAPI -eq 'ResourceNotOnboarded' -or $roleAssignmentScheduleInstancesFromAPI -eq 'TenantNotOnboarded' -or $roleAssignmentScheduleInstancesFromAPI -eq 'InvalidResourceType' -or $roleAssignmentScheduleInstancesFromAPI -eq 'RoleAssignmentScheduleInstancesError') {
#Write-Host "Scope '$($scopeDisplayName)' ('$scopeId') not onboarded in PIM"
}
else {
Expand Down
2 changes: 1 addition & 1 deletion pwsh/dev/functions/getMDfCSecureScoreMG.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ function getMDfCSecureScoreMG {
Write-Host ' Microsoft Defender for Cloud SecureScore for Management Groups will not be available' -ForegroundColor Yellow
}
else {
foreach ($entry in $getMgAscSecureScore.data) {
foreach ($entry in $getMgAscSecureScore) {
$script:htMgASCSecureScore.($entry.mgId) = @{}
if ($entry.secureScore -eq 404) {
$script:htMgASCSecureScore.($entry.mgId).SecureScore = 'n/a'
Expand Down
104 changes: 104 additions & 0 deletions pwsh/dev/functions/getOrphanedResources.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
function getOrphanedResources {
$start = Get-Date
Write-Host 'Getting orphaned resources (ARG)'

$queries = [System.Collections.ArrayList]@()
$intent = 'clean up'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.resources/subscriptions/resourceGroups'
query = "ResourceContainers | where type =~ 'microsoft.resources/subscriptions/resourceGroups' | extend rgAndSub = strcat(resourceGroup, '--', subscriptionId) | join kind=leftouter (Resources | extend rgAndSub = strcat(resourceGroup, '--', subscriptionId) | summarize count() by rgAndSub) on rgAndSub | where isnull(count_) | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$intent = 'misconfiguration'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.network/networkSecurityGroups'
query = "Resources | where type =~ 'microsoft.network/networkSecurityGroups' and isnull(properties.networkInterfaces) and isnull(properties.subnets) | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$intent = 'misconfiguration'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.network/routeTables'
query = "resources | where type =~ 'microsoft.network/routeTables' | where isnull(properties.subnets) | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$intent = 'misconfiguration'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.network/networkInterfaces'
query = "Resources | where type =~ 'microsoft.network/networkInterfaces' | where isnull(properties.privateEndpoint) | where isnull(properties.privateLinkService) | where properties.hostedWorkloads == '[]' | where properties !has 'virtualmachine' | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$intent = 'cost savings'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.compute/disks'
query = "Resources | where type =~ 'microsoft.compute/disks' | extend diskState = tostring(properties.diskState) | where managedBy == '' | where not(name endswith '-ASRReplica' or name startswith 'ms-asr-') | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$intent = 'cost savings'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.network/publicIpAddresses'
query = "Resources | where type =~ 'microsoft.network/publicIpAddresses' | where properties.ipConfiguration == '' | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$intent = 'misconfiguration'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.compute/availabilitySets'
query = "Resources | where type =~ 'microsoft.compute/availabilitySets' | where properties.virtualMachines == '[]' | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$intent = 'misconfiguration'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.network/loadBalancers'
query = "Resources | where type =~ 'microsoft.network/loadBalancers' | where properties.backendAddressPools == '[]' | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$intent = 'clean up'
$null = $queries.Add([PSCustomObject]@{
queryName = 'microsoft.web/serverfarms'
query = "Resources | where type =~ 'microsoft.web/serverfarms' | where properties.numberOfSites == 0 | project type, subscriptionId, Resource=id, Intent='$intent'"
})

$queries | foreach-object -Parallel {
$queryDetail = $_
$arrayOrphanedResources = $using:arrayOrphanedResources
$subsToProcessInCustomDataCollection = $using:subsToProcessInCustomDataCollection
$azAPICallConf = $using:azAPICallConf

#Batching: https://docs.microsoft.com/en-us/azure/governance/resource-graph/troubleshoot/general#toomanysubscription
$counterBatch = [PSCustomObject] @{ Value = 0 }
$batchSize = 1000
$subscriptionsBatch = $subsToProcessInCustomDataCollection | Group-Object -Property { [math]::Floor($counterBatch.Value++ / $batchSize) }

$uri = "$($azAPICallConf['azAPIEndpointUrls'].ARM)/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01"
$method = "POST"
foreach ($batch in $subscriptionsBatch) {
" Getting orphaned $($queryDetail.queryName) for $($batch.Group.subscriptionId.Count) Subscriptions"
$subscriptions = '"{0}"' -f ($batch.Group.subscriptionId -join '","')
$body = @"
{
"query": "$($queryDetail.query)",
"subscriptions": [$($subscriptions)]
}
"@

$res = (AzAPICall -AzAPICallConfiguration $azAPICallConf -uri $uri -method $method -body $body -listenOn 'Content' -currentTask "Getting orphaned $($queryDetail.queryName)")
#Write-Host '$res.count:' $res.count
if ($res.count -gt 0) {
foreach ($resource in $res) {
$null = $script:arrayOrphanedResources.Add($resource)
}
}
}
} -ThrottleLimit ($queries.Count)

if ($arrayOrphanedResources.Count -gt 0) {
Write-Host " Found $($arrayOrphanedResources.Count) orphaned Resources"
Write-Host " Exporting OrphanedResources CSV '$($outputPath)$($DirectorySeparatorChar)$($fileName)_ResourcesOrphaned.csv'"
$arrayOrphanedResources | Export-Csv -Path "$($outputPath)$($DirectorySeparatorChar)$($fileName)_ResourcesOrphaned.csv" -Delimiter "$csvDelimiter" -NoTypeInformation
}
else {
Write-Host " No orphaned Resources found"
}

$end = Get-Date
Write-Host "Getting orphaned resources (ARG) processing duration: $((NEW-TIMESPAN -Start $start -End $end).TotalMinutes) minutes ($((NEW-TIMESPAN -Start $start -End $end).TotalSeconds) seconds)"
}
Loading

0 comments on commit eb4275f

Please sign in to comment.