From 63d9fc5663bcec466bd98b2f23cede1d897a1228 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Mon, 1 Jul 2024 14:02:23 -0700 Subject: [PATCH 1/4] Require PowerShell 7 + handle throttling errors --- .build/BuildHelper/Build-PsModule.ps1 | 2 +- docs/_automation/powershell/README.md | 12 +++ .../powershell/cost/Start-FinOpsCostExport.md | 38 ++++++-- docs/_resources/changelog.md | 13 +++ src/powershell/Private/Invoke-Rest.ps1 | 1 + .../Public/Start-FinOpsCostExport.ps1 | 93 +++++++++++++++--- .../Tests/Integration/CostExports.Tests.ps1 | 97 +++++++++++++++++++ .../Unit/Start-FinOpsCostExport.Tests.ps1 | 33 +++++++ 8 files changed, 269 insertions(+), 20 deletions(-) diff --git a/.build/BuildHelper/Build-PsModule.ps1 b/.build/BuildHelper/Build-PsModule.ps1 index 2bfe564e3..e07d87399 100644 --- a/.build/BuildHelper/Build-PsModule.ps1 +++ b/.build/BuildHelper/Build-PsModule.ps1 @@ -54,7 +54,7 @@ function Build-PsModule VariablesToExport = @() AliasesToExport = @() Copyright = "(c) $((Get-Date).Year) Microsoft Corporation. All rights reserved." - PowerShellVersion = '5.1' + PowerShellVersion = '7.0' RequiredModules = @( @{ ModuleName = 'Az.Accounts' diff --git a/docs/_automation/powershell/README.md b/docs/_automation/powershell/README.md index 81f4d047a..2b3f05fe3 100644 --- a/docs/_automation/powershell/README.md +++ b/docs/_automation/powershell/README.md @@ -34,10 +34,22 @@ The FinOps toolkit PowerShell module is a collection of commands to automate and ## 📥 Install the module +The FinOps toolkit module requires PowerShell 7, which is supported on all major operating systems but is not installed by default. For details, see [Install PowerShell](https://learn.microsoft.com/powershell/scripting/install/installing-powershell). Once PowerShell has been installed, run the following: + ```powershell +Install-Module -Name Az.Accounts +Install-Module -Name Az.Resources Import-Module -Name FinOpsToolkit ``` +If this is the first time using Azure PowerShell, you will also need to sign into your account and select a default subscription: + +```powershell +Connect-AzAccount +``` + +This will show a popup window to sign in to your account. If you do not see the window, it may be on a different screen. +
## ⚡ Commands diff --git a/docs/_automation/powershell/cost/Start-FinOpsCostExport.md b/docs/_automation/powershell/cost/Start-FinOpsCostExport.md index a608ef18b..f3c8ce477 100644 --- a/docs/_automation/powershell/cost/Start-FinOpsCostExport.md +++ b/docs/_automation/powershell/cost/Start-FinOpsCostExport.md @@ -42,6 +42,9 @@ This command has been tested with the following API versions: Start-FinOpsCostExport ` [-Name] ` [-Scope ] ` + [-StartDate ] ` + [-EndDate ] ` + [-Backfill ] ` [-ApiVersion ] ``` @@ -49,23 +52,42 @@ Start-FinOpsCostExport ` ## 📥 Parameters -| Name | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------ | -| `‑Name` | Optional. Name of the export. Supports wildcards. | -| `‑Scope` | Optional. Resource ID of the scope the export was created for. If empty, defaults to current subscription context. | -| `‑ApiVersion` | Optional. API version to use when calling the Cost Management exports API. Default = 2023-03-01. | +| Name | Description | +| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `‑Name` | Required. Name of the export. | +| `‑Scope` | Optional. Resource ID of the scope to export data for. If empty, defaults to current subscription context. | +| `‑StartDate` | Optional. Day to start pulling the data for. If not set, the export will use the dates defined in the export configuration. | +| `‑EndDate` | Optional. Last day to pull data for. If not set and -StartDate is set, -EndDate will use the last day of the month. If not set and -StartDate is not set, the export will use the dates defined in the export configuration. | +| `‑Backfill` | Optional. Number of months to export the data for. Make note of throttling (429) errors. This is only run once. Failed exports are not re-attempted. Default = 0. | +| `‑ApiVersion` | Optional. API version to use when calling the Cost Management Exports API. Default = 2023-07-01-preview. |
## 🌟 Examples -### Run export +### Export configured period ```powershell -Start-FinopsCostExport -Name 'July2023OneTime' +Start-FinopsCostExport -Name 'CostExport' ``` -Runs an export called 'July2023OneTime'. +Runs an export called 'CostExport' for the configured period. + +### Export specific dates + +```powershell +Start-FinopsCostExport -Name 'CostExport' -StartDate '2023-01-01' -EndDate '2023-12-31' +``` + +Runs an export called 'CostExport' for a specific date range. + +### Backfill export + +```powershell +Start-FinopsCostExport -Name 'CostExport' -Backfill 12 +``` + +Runs an export called 'CostExport' for the previous 12 months.
diff --git a/docs/_resources/changelog.md b/docs/_resources/changelog.md index c5468c9bc..b9427c357 100644 --- a/docs/_resources/changelog.md +++ b/docs/_resources/changelog.md @@ -94,6 +94,19 @@ Legend: > - Reports will still be released as PBIX files so this change should not impact end users. > - Visualizations are not being switched to [Power BI Enhanced Report (PBIR)](https://learn.microsoft.com/power-bi/developer/projects/projects-report#pbir-format) format yet due to functional limitations that would impact end users (as of June 2024). +🖥️ PowerShell +{: .fs-5 .fw-500 .mt-4 mb-0 } + +> ➕ Added: +> +> 1. Added progress tracking to [Start-FinOpsCostExport](../_automation/powershell/cost/Start-FinOpsCostExport.md) for multi-month exports. +> 2. Added a 60-second delay when Cost Management returns throttling (429) errors in [Start-FinOpsCostExport](../_automation/powershell/cost/Start-FinOpsCostExport.md). +> +> 🗑️ Removed: +> +> 1. Removed support for Windows PowerShell. +> > _We discovered errors with Windows PowerShell due to incompatibilities in Windows PowerShell and PowerShell 7. Due to our limited capacity, we decided to only support [PowerShell 7](https://learn.microsoft.com/powershell/scripting/install/installing-powershell) going forward._ + 🌐 Open data {: .fs-5 .fw-500 .mt-4 mb-0 } diff --git a/src/powershell/Private/Invoke-Rest.ps1 b/src/powershell/Private/Invoke-Rest.ps1 index 15a9fd68c..136cbe3c9 100644 --- a/src/powershell/Private/Invoke-Rest.ps1 +++ b/src/powershell/Private/Invoke-Rest.ps1 @@ -104,6 +104,7 @@ function Invoke-Rest Success = $response.StatusCode -ge 200 -and $response.StatusCode -lt 300 Failure = $response.StatusCode -ge 300 NotFound = $response.StatusCode -eq 404 -or $response.StatusCode -eq 'NotFound' + Throttled = $response.StatusCode -eq 429 -or $response.StatusCode -eq 'ResourceRequestsThrottled' Content = $content } } diff --git a/src/powershell/Public/Start-FinOpsCostExport.ps1 b/src/powershell/Public/Start-FinOpsCostExport.ps1 index c51dcb8a9..263c219b9 100644 --- a/src/powershell/Public/Start-FinOpsCostExport.ps1 +++ b/src/powershell/Public/Start-FinOpsCostExport.ps1 @@ -26,15 +26,20 @@ Optional. Last day to pull data for. If not set and -StartDate is set, -EndDate will use the last day of the month. If not set and -StartDate is not set, the export will use the dates defined in the export configuration. .PARAMETER Backfill - Optional. IndicLast day to pull data for. If not set and -StartDate is set, -EndDate will use the last day of the month. If not set and -StartDate is not set, the export will use the dates defined in the export configuration. + Optional. Number of months to export the data for. Make note of throttling (429) errors. This is only run once. Failed exports are not re-attempted. Default = 0. .PARAMETER ApiVersion Optional. API version to use when calling the Cost Management Exports API. Default = 2023-07-01-preview. .EXAMPLE - Start-FinopsCostExport -Name 'July2023OneTime' + Start-FinopsCostExport -Name 'CostExport' - Runs an export called 'July2023OneTime'. + Runs an export called 'CostExport' for the configured period. + + .EXAMPLE + Start-FinopsCostExport -Name 'CostExport' -Backfill 12 + + Runs an export called 'CostExport' for the previous 12 months. .LINK https://aka.ms/ftk/Start-FinOpsCostExport @@ -82,8 +87,22 @@ function Start-FinOpsCostExport # Set start date if using -Backfill if ($Backfill -gt 0) { - $StartDate = (Get-Date -Day 1 -Hour 0 -Minute 0 -Second 0 -Millisecond 0 -AsUTC).AddMonths($Backfill * -1) - $EndDate = $StartDate.AddMonths($Backfill).AddMilliseconds(-1) + # TODO: Consider updating this to account for one-time exports where we should copy the start date from + + # If -StartDate is not set, assume the current month + if (-not $StartDate) + { + $StartDate = (Get-Date -Day 1 -Hour 0 -Minute 0 -Second 0 -Millisecond 0 -AsUTC) + } + + # If -EndDate is not set, assume 1 month + if (-not $EndDate) + { + $EndDate = $StartDate.AddMonths(1).AddMilliseconds(-1) + } + + # Move start date to account for the backfill period + $StartDate = $StartDate.AddMonths($Backfill * -1) Write-Verbose "Backfill $Backfill months = $($StartDate.ToUniversalTime().ToString('yyyy-MM-dd"T"HH:mm:ss"Z"')) to $($EndDate.ToUniversalTime().ToString('yyyy-MM-dd"T"HH:mm:ss"Z"'))" } @@ -102,18 +121,43 @@ function Start-FinOpsCostExport Write-Verbose "Updated dates = $($StartDate.ToUniversalTime().ToString('yyyy-MM-dd"T"HH:mm:ss"Z"')) to $($EndDate.ToUniversalTime().ToString('yyyy-MM-dd"T"HH:mm:ss"Z"'))" } + # Start measuring progress + $progressActivity = "Running exports" + $startTime = [DateTime]::Now + $months = (($EndDate.Year - $StartDate.Year) * 12) + $EndDate.Month - $StartDate.Month + 1 + if ($months -lt 1) { $months = 1 } # Assume at least 1 month to avoid errors + $estimatedSecPerMonth = 6 # Estimated time to trigger a single month export accounting for throttling (10 per minute) + # Loop thru each month $monthToExport = 0 $success = $true $body = $null - $multipleMonths = $StartDate -and $StartDate.Year -ne $EndDate.Year -or $StartDate.Month -ne $EndDate.Month - Write-Verbose "Exporting $($StartDate) - $($EndDate)" + if ($StartDate) + { + Write-Verbose "Exporting dates configured on the export definition" + } + else + { + Write-Verbose "Exporting $($StartDate) - $($EndDate)" + } do { + # Report progress + if ($months -gt 1) + { + $percent = [Math]::Round((1.0 * $monthToExport / $months) * 100, 0) + $remaining = $estimatedSecPerMonth * ($months - $monthToExport) + Write-Progress ` + -Activity $progressActivity ` + -Status "$percent% complete - $monthToExport of $months months" ` + -PercentComplete $percent ` + -SecondsRemaining $remaining + } + if ($StartDate) { # If more than one month - if ($multipleMonths) + if ($months -gt 1) { $firstDay = $EndDate.AddDays(-$EndDate.Day + 1).AddMonths($monthToExport * -1) $lastDay = $firstDay.AddMonths(1).AddMilliseconds(-1) @@ -124,7 +168,7 @@ function Start-FinOpsCostExport $lastDay = $EndDate } $body = @{ timePeriod = @{ from = $firstDay.ToString("yyyy-MM-dd'T'HH:mm:ss'Z'"); to = $lastDay.ToString("yyyy-MM-dd'T'HH:mm:ss'Z'") } } - Write-Verbose "Executing $($firstDay.ToString("MMM d yyyy HH:mm:ss")) export $runpath" + Write-Verbose "Executing $($firstDay.ToString("MMM d yyyy")) export $runpath" } else { @@ -140,10 +184,37 @@ function Start-FinOpsCostExport { Write-Verbose "Export failed to execute" } - $success = $success -and $response.Success + + # If export throttled, wait 60 seconds and try again + if ($response.Throttled) + { + Write-Verbose "Export request throttled. Waiting 60 seconds and retrying." + + # Report progress + if ($months -gt 1) + { + Write-Progress ` + -Activity $progressActivity ` + -Status "$percent% complete - Throttled by Cost Management. Waiting 60 seconds." ` + } + else + { + Write-Information "Requests are being throttled by Cost Management. Waiting 60 seconds and retrying..." + } + Start-Sleep -Seconds 60 + } + else + { + # If not retrying, then track the success + $success = $success -and $response.Success + } $monthToExport += 1 - } while ($multipleMonths -and $EndDate.AddMonths($monthToExport * -1) -ge $StartDate) + } while ($months -gt 1 -and $EndDate.AddMonths($monthToExport * -1) -ge $StartDate) + if ($months -gt 1) + { + Write-Progress -Activity $progressActivity -Completed + } return $success } diff --git a/src/powershell/Tests/Integration/CostExports.Tests.ps1 b/src/powershell/Tests/Integration/CostExports.Tests.ps1 index 5c8760b37..29d595534 100644 --- a/src/powershell/Tests/Integration/CostExports.Tests.ps1 +++ b/src/powershell/Tests/Integration/CostExports.Tests.ps1 @@ -238,6 +238,103 @@ Describe 'CostExports' { } } + It 'Should handle progress and throttling for 12 month backfill' { + # Arrange + $startDate = (Get-Date -Day 1 -Hour 0 -Minute 0 -Second 0 -Millisecond 0) + + Monitor "Export throttling tests..." -Indent ' ' { + Monitor "Creating $exportName export..." { + # Act -- create + $newResult = New-FinOpsCostExport -Name $exportName -Scope $scope -StorageAccountId $storage.Id -Execute -Backfill 12 + # TODO: Run tests for all supported API versions: -ApiVersion '2023-08-01' + + # Assert + Report -Object $newResult + $newResult.Name | Should -Be $exportName + $newResult.RunHistory | Should -BeNullOrEmpty -Because "the -RunHistory option was not specified" + } + + Monitor "Getting $exportName..." { + # Act -- read + $getResult = Get-FinOpsCostExport -Name $exportName -Scope $scope -RunHistory + + # Assert + Report "Found $($getResult.Count) export(s)" + Report -Object $getResult + $getResult.Count | Should -Be 1 + $getResult.Name | Should -Be $exportName + $getResult.RunHistory.Count | Should -BeGreaterThan 0 -Because "-Execute -Backfill was specified during creation" + } + + Monitor "Deleting $exportName..." { + # Act -- delete + $deleteResult = Remove-FinOpsCostExport -Name $exportName -Scope $scope + $confirmDeleteResult = Get-FinOpsCostExport -Name $exportName -Scope $scope + + # Assert + Report $deleteResult + $deleteResult | Should -BeTrue + Report "$($getResult.Count) export(s) remaining" + $confirmDeleteResult | Should -BeNullOrEmpty + } + } + } + + Context "Long-running unit tests" { + BeforeAll { + [Diagnostics.CodeAnalysis.SuppressMessageAttribute("PSUseDeclaredVarsMoreThanAssignments", "")] + $exportName = 'ftk-test-Start-FinOpsCostExport' + + [Diagnostics.CodeAnalysis.SuppressMessageAttribute("PSUseDeclaredVarsMoreThanAssignments", "")] + $scope = "/subscriptions/$([Guid]::NewGuid())" + + [Diagnostics.CodeAnalysis.SuppressMessageAttribute("PSUseDeclaredVarsMoreThanAssignments", "")] + $mockExport = @{ + id = "$scope/providers/Microsoft.CostManagement/exports/$exportName" + name = $exportName + } + } + + It 'Should wait 60s when throttled' { + # NOTE: This is a unit test that mocks dependencies. It's run with integration tests due to how long it takes to run. + + # Arrange + $waited = $false + $testStartTime = Get-Date + function CheckDate($date) { + $monthToThrottle = (Get-Date -Day 1 -Hour 0 -Minute 0 -Second 0 -Millisecond 0 -AsUTC).AddMonths(-3).ToUniversalTime().Date.ToString("yyyy-MM-dd'T'HH:mm:ss'Z'") + if ($date -eq $monthToThrottle -and -not $waited) { + $waited = $true + return $true + } + return $false + } + Mock -ModuleName FinOpsToolkit -CommandName 'Get-FinOpsCostExport' -MockWith { $mockExport } + Mock -ModuleName FinOpsToolkit -CommandName 'Invoke-Rest' -MockWith { + if (CheckDate($Body.timePeriod.from)) + { + @{ Success = $false; Throttled = $true } + } + else + { + @{ Success = $true } + } + } + Mock -ModuleName FinOpsToolkit -CommandName 'Write-Progress' -MockWith {} + + # Act + $success = Start-FinOpsCostExport ` + -Name $exportName ` + -Scope $scope ` + -Backfill 12 + + # Assert + Assert-MockCalled -ModuleName FinOpsToolkit -CommandName 'Write-Progress' -Times 4 + $success | Should -Be $true + ((Get-Date) - $testStartTime).TotalSeconds | Should -BeGreaterThan 60 + } + } + AfterAll { # Cleanup Remove-AzStorageAccount -ResourceGroupName $rg -Name $storageName -Force diff --git a/src/powershell/Tests/Unit/Start-FinOpsCostExport.Tests.ps1 b/src/powershell/Tests/Unit/Start-FinOpsCostExport.Tests.ps1 index 650ca6354..3f4292436 100644 --- a/src/powershell/Tests/Unit/Start-FinOpsCostExport.Tests.ps1 +++ b/src/powershell/Tests/Unit/Start-FinOpsCostExport.Tests.ps1 @@ -116,4 +116,37 @@ Describe 'Start-FinOpsCostExport' { } $success | Should -Be $true } + + It 'Should report status when exporting multiple months' { + # Arrange + Mock -ModuleName FinOpsToolkit -CommandName 'Get-FinOpsCostExport' { $mockExport } + Mock -ModuleName FinOpsToolkit -CommandName 'Invoke-Rest' { @{ Success = $true } } + Mock -ModuleName FinOpsToolkit -CommandName 'Write-Progress' {} + + # Act + $success = Start-FinOpsCostExport ` + -Name $exportName ` + -Scope $scope ` + -Backfill 3 + + # Assert + Assert-MockCalled -ModuleName FinOpsToolkit -CommandName 'Write-Progress' -Times 4 + $success | Should -Be $true + } + + It 'Should not report status when exporting 1 month' { + # Arrange + Mock -ModuleName FinOpsToolkit -CommandName 'Get-FinOpsCostExport' { $mockExport } + Mock -ModuleName FinOpsToolkit -CommandName 'Invoke-Rest' { @{ Success = $true } } + Mock -CommandName 'Write-Progress' {} + + # Act + $success = Start-FinOpsCostExport ` + -Name $exportName ` + -Scope $scope + + # Assert + Assert-MockCalled -CommandName 'Write-Progress' -Times 0 + $success | Should -Be $true + } } From a747691017959b99ebde4d7bce09cef0ad70da19 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Tue, 2 Jul 2024 02:14:23 -0700 Subject: [PATCH 2/4] Handle errors in JSON response --- src/powershell/Private/Invoke-Rest.ps1 | 7 ++++++- src/powershell/Public/Start-FinOpsCostExport.ps1 | 7 ++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/src/powershell/Private/Invoke-Rest.ps1 b/src/powershell/Private/Invoke-Rest.ps1 index 136cbe3c9..64ee109f0 100644 --- a/src/powershell/Private/Invoke-Rest.ps1 +++ b/src/powershell/Private/Invoke-Rest.ps1 @@ -86,7 +86,12 @@ function Invoke-Rest catch { $response = $_.Exception.Response - $content = $_.ErrorDetails.Message | ConvertFrom-Json -Depth 10 + try + { + $content = $_.ErrorDetails.Message | ConvertFrom-Json -Depth 10 + } + catch {} + if ($content.error) { $errorCode = $content.error.code diff --git a/src/powershell/Public/Start-FinOpsCostExport.ps1 b/src/powershell/Public/Start-FinOpsCostExport.ps1 index 263c219b9..1c4c35b47 100644 --- a/src/powershell/Public/Start-FinOpsCostExport.ps1 +++ b/src/powershell/Public/Start-FinOpsCostExport.ps1 @@ -36,11 +36,16 @@ Runs an export called 'CostExport' for the configured period. + .EXAMPLE + Start-FinopsCostExport -Name 'CostExport' -StartDate '2023-01-01' -EndDate '2023-12-31' + + Runs an export called 'CostExport' for a specific date range. + .EXAMPLE Start-FinopsCostExport -Name 'CostExport' -Backfill 12 Runs an export called 'CostExport' for the previous 12 months. - + .LINK https://aka.ms/ftk/Start-FinOpsCostExport #> From cd6fcb12b7137edd1e5e0ea0bcbb909df55d9601 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Mon, 8 Jul 2024 03:32:38 -0700 Subject: [PATCH 3/4] Apply suggestions from code review --- docs/_automation/powershell/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_automation/powershell/README.md b/docs/_automation/powershell/README.md index 2b3f05fe3..f35084cac 100644 --- a/docs/_automation/powershell/README.md +++ b/docs/_automation/powershell/README.md @@ -39,7 +39,7 @@ The FinOps toolkit module requires PowerShell 7, which is supported on all major ```powershell Install-Module -Name Az.Accounts Install-Module -Name Az.Resources -Import-Module -Name FinOpsToolkit +Install-Module -Name FinOpsToolkit ``` If this is the first time using Azure PowerShell, you will also need to sign into your account and select a default subscription: From 5986ac12315d830546480940e3b2c05d213d3af1 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Thu, 11 Jul 2024 01:19:59 -0700 Subject: [PATCH 4/4] Apply suggestions from code review --- docs/_automation/powershell/README.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/_automation/powershell/README.md b/docs/_automation/powershell/README.md index f35084cac..d71d1920f 100644 --- a/docs/_automation/powershell/README.md +++ b/docs/_automation/powershell/README.md @@ -34,11 +34,18 @@ The FinOps toolkit PowerShell module is a collection of commands to automate and ## 📥 Install the module -The FinOps toolkit module requires PowerShell 7, which is supported on all major operating systems but is not installed by default. For details, see [Install PowerShell](https://learn.microsoft.com/powershell/scripting/install/installing-powershell). Once PowerShell has been installed, run the following: +The FinOps toolkit module requires PowerShell 7, which is built into [Azure Cloud Shell](https://portal.azure.com/#cloudshell) and supported on all major operating systems. + +Azure Cloud Shell comes with PowerShell 7 and Azure PowerShell pre-installed. If you are not using Azure Cloud Shell, you will need to [Install PowerShell](https://learn.microsoft.com/powershell/scripting/install/installing-powershell) first and then run the following commands to install Azure PowerShell: ```powershell Install-Module -Name Az.Accounts Install-Module -Name Az.Resources +``` + +To install the FinOps toolkit module, run the following in either Azure Cloud Shell or a PowerShell client: + +```powershell Install-Module -Name FinOpsToolkit ```