Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update regions for Aug 2024 #925

Merged
merged 3 commits into from
Aug 23, 2024
Merged

Update regions for Aug 2024 #925

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bb7a5af
Update regions for Aug 2024
flanakin Aug 23, 2024
2091704
Update KQL query
flanakin Aug 23, 2024
a242a0b
Apply suggestions from code review
flanakin Aug 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/_resources/changelog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,8 @@ Legend:
>
> - [Pricing units](../_reporting/data/README.md#-pricing-units)
> 1. Added handling for the following new UnitOfMeasure values: "1 /Minute", "10 PiB/Hour", "100000 /Month", "Text".
> - [Regions](../_reporting/data/README.md#️-Regions)
flanakin marked this conversation as resolved.
Show resolved Hide resolved
> 1. Added the following new region values: "asiapacific", "australia", azure "stack", "eastsu2", "gbs", germany west "central", "japan", sweden "central", "unitedstates", us dod "central", us dod "east", us gov "iowa", us gov "virginia", "us2", "usa", "usv".
>
> ✏️ Changed
>
Expand Down
38 changes: 35 additions & 3 deletions src/open-data/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,15 +99,47 @@ Meters
## 🗺️ Regions

<sup>
📅 Updated: Jun 1, 2024<br>
📅 Updated: Aug 22, 2024<br>
➡️ Source: Commerce Platform Data Model team<br>
</sup>

<br>

The [Regions.csv](./Regions.csv) file contains data from several internal sources. We shouldn't need to update this file as Cost Management data is standardizing on Azure regions.
The [Regions.csv](./Regions.csv) file contains the list of all unique `ResourceLocation` and `ResourceLocationNormalized` values. This data will need to be updated periodically as new regions are added.

Use the following query to update the data:

```kql
let oldValues = externaldata(OriginalValue:string, RegionId:string, RegionName:string,) [@"https://raw.githubusercontent.com/microsoft/finops-toolkit/dev/src/open-data/Regions.csv"] with (format="csv", ignoreFirstRecord=true);
let newValues = union cluster('<cluster>.kusto.windows.net').database('<shard>*').<table> | where ResourceType != 'Microsoft.Security/securityConnectors' | distinct ResourceLocation, ResourceLocationNormalized;
newValues | project OriginalValue = tolower(ResourceLocation)
| union (newValues | project OriginalValue = tolower(ResourceLocationNormalized))
| where isnotempty(OriginalValue) and OriginalValue !in ('null', 'true', 'false', 'test', 'unknown', 'zone1', 'zone 1')
| distinct OriginalValue
| where OriginalValue !in ((oldValues | distinct OriginalValue))
| union (oldValues)
| as reg
| extend regionWithoutLetter = extract(@'^([a-z ]+[1-2])[a-c]$', 1, OriginalValue)
| extend regionWithoutNumber = extract(@'^([a-z]+( [a-z]+)*)[1-3][a-c]?$', 1, OriginalValue)
| extend regionWithoutSpace = case(
OriginalValue contains ' ', replace_string(OriginalValue, ' ', ''),
OriginalValue == 'eastsu2', 'eastus2',
OriginalValue == 'gbs', 'uksouth',
OriginalValue == 'usa', 'usgovarizona',
OriginalValue == 'usv', 'usgovvirginia',
'')
// DEBUG: | where isempty(regionWithoutLetter) or regionWithoutLetter !in ((reg | distinct OriginalValue))
// DEBUG: | where isempty(regionWithoutNumber) or regionWithoutNumber !in ((reg | distinct OriginalValue))
| join kind=leftouter (reg) on $left.regionWithoutLetter == $right.OriginalValue
| join kind=leftouter (reg) on $left.regionWithoutNumber == $right.OriginalValue
| join kind=leftouter (reg) on $left.regionWithoutSpace == $right.OriginalValue
| extend RegionId = case(isnotempty(RegionId), RegionId, isnotempty(RegionId1), RegionId1, isnotempty(RegionId2), RegionId2, isnotempty(RegionId3), RegionId3, RegionId)
| extend RegionName = case(isnotempty(RegionName), RegionName, isnotempty(RegionName1), RegionName1, isnotempty(RegionName2), RegionName2, isnotempty(RegionName3), RegionName3, RegionName)
| project OriginalValue, RegionId, RegionName
| order by OriginalValue asc
```

> ℹ️ _Internal only: Contact the CPDM PM team for any updates._
After updating the list of available original values, other columns must be manually populated.

<br>

Expand Down
17 changes: 16 additions & 1 deletion src/open-data/Regions.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ ap southeast,southeastasia,Southeast Asia
apeast,eastasia,East Asia
apsoutheast,southeastasia,Southeast Asia
asiaeast,eastasia,East Asia
asiapacific,asiapacific,Asia Pacific
asiasoutheast,southeastasia,Southeast Asia
at east,austriaeast,Austria East
ate,austriaeast,Austria East
Expand All @@ -23,6 +24,7 @@ aucentral2,australiacentral2,Australia Central 2
aueast,australiaeast,Australia East
auh,uaecentral,UAE Central
ausoutheast,australiasoutheast,Australia Southeast
australia,australia,Australia
australia east,australiaeast,Australia East
australia southeast,australiasoutheast,Australia Southeast
australia_east,australiaeast,Australia East
Expand All @@ -34,6 +36,7 @@ australiaeast,australiaeast,Australia East
australiasoutheast,australiasoutheast,Australia Southeast
austriae,austriaeast,Austria East
austriaeast,austriaeast,Austria East
azure stack,azurestack,Azure Stack
azurestack,azurestack,Azure Stack
bd,usdodeast,USDoD East
be central,belgiumcentral,Belgium Central
Expand Down Expand Up @@ -140,6 +143,7 @@ east_asia,eastasia,East Asia
eastasia,eastasia,East Asia
eastcanada,canadaeast,Canada East
eastjapan,japaneast,Japan East
eastsu2,eastus2,East US 2
eastus,eastus,East US
eastus2,eastus2,East US 2
eastus2euap,eastus2euap,East US 2 EUAP
Expand Down Expand Up @@ -173,6 +177,8 @@ francesouth,francesouth,France South
frc,francecentral,France Central
frcentral,francecentral,France Central
frsouth,francesouth,France South
gbs,uksouth,UK South
germany west central,germanywestcentral,Germany West Central
germanycentral,germanycentral,Germany Central
germanyn,germanynorth,Germany North
germanynorth,germanynorth,Germany North
Expand Down Expand Up @@ -220,6 +226,7 @@ itn,italynorth,Italy North
ja east,japaneast,Japan East
ja west,japanwest,Japan West
jaeast,japaneast,Japan East
japan,japan,Japan
japan east,japaneast,Japan East
japan west,japanwest,Japan West
japan_east,japaneast,Japan East
Expand Down Expand Up @@ -353,6 +360,7 @@ southindia,southindia,South India
southsoutheastasia,southeastasia,Southeast Asia
spainc,spaincentral,Spain Central
spaincentral,spaincentral,Spain Central
sweden central,swedencentral,Sweden Central
swedenc,swedencentral,Sweden Central
swedencentral,swedencentral,Sweden Central
swedens,swedensouth,Sweden South
Expand Down Expand Up @@ -388,12 +396,17 @@ ukwest,ukwest,UK West
unassigned,global,Global
unitedkingdomsouth,uksouth,UK South
unitedkingdomsouth2,uksouth,UK South
unitedstates,unitedstates,United States
us central,centralus,Central US
us dod central,usdodcentral,USDoD Central
us dod east,usdodeast,USDoD East
us east,eastus,East US
us east 2,eastus2,East US 2
us east 3,eastus3,East US 3
us gov az,usgovarizona,USGov Arizona
us gov iowa,usgoviowa,USGov Iowa
us gov tx,usgovtexas,USGov Texas
us gov virginia,usgovvirginia,USGov Virginia
us north central,northcentralus,North Central US
us sec east,usseceast,USSec East
us sec west,ussecwest,USSec West
Expand All @@ -403,9 +416,10 @@ us west,westus,West US
us west 2,westus2,West US 2
us west 3,westus3,West US 3
us west central,westcentralus,West Central US
us2,eastus2,East US 2
us_east,eastus,East US
us_west,westus,West US
us2,eastus2,East US 2
usa,usgovarizona,USGov Arizona
uscentral,centralus,Central US
usdodcentral,usdodcentral,USDoD Central
usdodeast,usdodeast,USDoD East
Expand Down Expand Up @@ -446,6 +460,7 @@ usstagesc,southcentralusstg,South Central US STG
ussw,ussecwest,USSec West
usswc,ussecwestcentral,USSec West Central
ustsc,southcentralusstg,South Central US STG
usv,usgovvirginia,USGov Virginia
usw3,westus3,West US 3
uswest,westus,West US
uswest2,westus2,West US 2
Expand Down
17 changes: 16 additions & 1 deletion src/powershell/Private/Get-OpenDataRegion.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'apeast'; RegionId = 'eastasia'; RegionName = 'East Asia'; }
,[PSCustomObject]@{ OriginalValue = 'apsoutheast'; RegionId = 'southeastasia'; RegionName = 'Southeast Asia'; }
,[PSCustomObject]@{ OriginalValue = 'asiaeast'; RegionId = 'eastasia'; RegionName = 'East Asia'; }
,[PSCustomObject]@{ OriginalValue = 'asiapacific'; RegionId = 'asiapacific'; RegionName = 'Asia Pacific'; }
,[PSCustomObject]@{ OriginalValue = 'asiasoutheast'; RegionId = 'southeastasia'; RegionName = 'Southeast Asia'; }
,[PSCustomObject]@{ OriginalValue = 'at east'; RegionId = 'austriaeast'; RegionName = 'Austria East'; }
,[PSCustomObject]@{ OriginalValue = 'ate'; RegionId = 'austriaeast'; RegionName = 'Austria East'; }
Expand All @@ -29,6 +30,7 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'aueast'; RegionId = 'australiaeast'; RegionName = 'Australia East'; }
,[PSCustomObject]@{ OriginalValue = 'auh'; RegionId = 'uaecentral'; RegionName = 'UAE Central'; }
,[PSCustomObject]@{ OriginalValue = 'ausoutheast'; RegionId = 'australiasoutheast'; RegionName = 'Australia Southeast'; }
,[PSCustomObject]@{ OriginalValue = 'australia'; RegionId = 'australia'; RegionName = 'Australia'; }
,[PSCustomObject]@{ OriginalValue = 'australia east'; RegionId = 'australiaeast'; RegionName = 'Australia East'; }
,[PSCustomObject]@{ OriginalValue = 'australia southeast'; RegionId = 'australiasoutheast'; RegionName = 'Australia Southeast'; }
,[PSCustomObject]@{ OriginalValue = 'australia_east'; RegionId = 'australiaeast'; RegionName = 'Australia East'; }
Expand All @@ -40,6 +42,7 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'australiasoutheast'; RegionId = 'australiasoutheast'; RegionName = 'Australia Southeast'; }
,[PSCustomObject]@{ OriginalValue = 'austriae'; RegionId = 'austriaeast'; RegionName = 'Austria East'; }
,[PSCustomObject]@{ OriginalValue = 'austriaeast'; RegionId = 'austriaeast'; RegionName = 'Austria East'; }
,[PSCustomObject]@{ OriginalValue = 'azure stack'; RegionId = 'azurestack'; RegionName = 'Azure Stack'; }
,[PSCustomObject]@{ OriginalValue = 'azurestack'; RegionId = 'azurestack'; RegionName = 'Azure Stack'; }
,[PSCustomObject]@{ OriginalValue = 'bd'; RegionId = 'usdodeast'; RegionName = 'USDoD East'; }
,[PSCustomObject]@{ OriginalValue = 'be central'; RegionId = 'belgiumcentral'; RegionName = 'Belgium Central'; }
Expand Down Expand Up @@ -146,6 +149,7 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'eastasia'; RegionId = 'eastasia'; RegionName = 'East Asia'; }
,[PSCustomObject]@{ OriginalValue = 'eastcanada'; RegionId = 'canadaeast'; RegionName = 'Canada East'; }
,[PSCustomObject]@{ OriginalValue = 'eastjapan'; RegionId = 'japaneast'; RegionName = 'Japan East'; }
,[PSCustomObject]@{ OriginalValue = 'eastsu2'; RegionId = 'eastus2'; RegionName = 'East US 2'; }
,[PSCustomObject]@{ OriginalValue = 'eastus'; RegionId = 'eastus'; RegionName = 'East US'; }
,[PSCustomObject]@{ OriginalValue = 'eastus2'; RegionId = 'eastus2'; RegionName = 'East US 2'; }
,[PSCustomObject]@{ OriginalValue = 'eastus2euap'; RegionId = 'eastus2euap'; RegionName = 'East US 2 EUAP'; }
Expand Down Expand Up @@ -179,6 +183,8 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'frc'; RegionId = 'francecentral'; RegionName = 'France Central'; }
,[PSCustomObject]@{ OriginalValue = 'frcentral'; RegionId = 'francecentral'; RegionName = 'France Central'; }
,[PSCustomObject]@{ OriginalValue = 'frsouth'; RegionId = 'francesouth'; RegionName = 'France South'; }
,[PSCustomObject]@{ OriginalValue = 'gbs'; RegionId = 'uksouth'; RegionName = 'UK South'; }
,[PSCustomObject]@{ OriginalValue = 'germany west central'; RegionId = 'germanywestcentral'; RegionName = 'Germany West Central'; }
,[PSCustomObject]@{ OriginalValue = 'germanycentral'; RegionId = 'germanycentral'; RegionName = 'Germany Central'; }
,[PSCustomObject]@{ OriginalValue = 'germanyn'; RegionId = 'germanynorth'; RegionName = 'Germany North'; }
,[PSCustomObject]@{ OriginalValue = 'germanynorth'; RegionId = 'germanynorth'; RegionName = 'Germany North'; }
Expand Down Expand Up @@ -226,6 +232,7 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'ja east'; RegionId = 'japaneast'; RegionName = 'Japan East'; }
,[PSCustomObject]@{ OriginalValue = 'ja west'; RegionId = 'japanwest'; RegionName = 'Japan West'; }
,[PSCustomObject]@{ OriginalValue = 'jaeast'; RegionId = 'japaneast'; RegionName = 'Japan East'; }
,[PSCustomObject]@{ OriginalValue = 'japan'; RegionId = 'japan'; RegionName = 'Japan'; }
,[PSCustomObject]@{ OriginalValue = 'japan east'; RegionId = 'japaneast'; RegionName = 'Japan East'; }
,[PSCustomObject]@{ OriginalValue = 'japan west'; RegionId = 'japanwest'; RegionName = 'Japan West'; }
,[PSCustomObject]@{ OriginalValue = 'japan_east'; RegionId = 'japaneast'; RegionName = 'Japan East'; }
Expand Down Expand Up @@ -359,6 +366,7 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'southsoutheastasia'; RegionId = 'southeastasia'; RegionName = 'Southeast Asia'; }
,[PSCustomObject]@{ OriginalValue = 'spainc'; RegionId = 'spaincentral'; RegionName = 'Spain Central'; }
,[PSCustomObject]@{ OriginalValue = 'spaincentral'; RegionId = 'spaincentral'; RegionName = 'Spain Central'; }
,[PSCustomObject]@{ OriginalValue = 'sweden central'; RegionId = 'swedencentral'; RegionName = 'Sweden Central'; }
,[PSCustomObject]@{ OriginalValue = 'swedenc'; RegionId = 'swedencentral'; RegionName = 'Sweden Central'; }
,[PSCustomObject]@{ OriginalValue = 'swedencentral'; RegionId = 'swedencentral'; RegionName = 'Sweden Central'; }
,[PSCustomObject]@{ OriginalValue = 'swedens'; RegionId = 'swedensouth'; RegionName = 'Sweden South'; }
Expand Down Expand Up @@ -394,12 +402,17 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'unassigned'; RegionId = 'global'; RegionName = 'Global'; }
,[PSCustomObject]@{ OriginalValue = 'unitedkingdomsouth'; RegionId = 'uksouth'; RegionName = 'UK South'; }
,[PSCustomObject]@{ OriginalValue = 'unitedkingdomsouth2'; RegionId = 'uksouth'; RegionName = 'UK South'; }
,[PSCustomObject]@{ OriginalValue = 'unitedstates'; RegionId = 'unitedstates'; RegionName = 'United States'; }
,[PSCustomObject]@{ OriginalValue = 'us central'; RegionId = 'centralus'; RegionName = 'Central US'; }
,[PSCustomObject]@{ OriginalValue = 'us dod central'; RegionId = 'usdodcentral'; RegionName = 'USDoD Central'; }
,[PSCustomObject]@{ OriginalValue = 'us dod east'; RegionId = 'usdodeast'; RegionName = 'USDoD East'; }
,[PSCustomObject]@{ OriginalValue = 'us east'; RegionId = 'eastus'; RegionName = 'East US'; }
,[PSCustomObject]@{ OriginalValue = 'us east 2'; RegionId = 'eastus2'; RegionName = 'East US 2'; }
,[PSCustomObject]@{ OriginalValue = 'us east 3'; RegionId = 'eastus3'; RegionName = 'East US 3'; }
,[PSCustomObject]@{ OriginalValue = 'us gov az'; RegionId = 'usgovarizona'; RegionName = 'USGov Arizona'; }
,[PSCustomObject]@{ OriginalValue = 'us gov iowa'; RegionId = 'usgoviowa'; RegionName = 'USGov Iowa'; }
,[PSCustomObject]@{ OriginalValue = 'us gov tx'; RegionId = 'usgovtexas'; RegionName = 'USGov Texas'; }
,[PSCustomObject]@{ OriginalValue = 'us gov virginia'; RegionId = 'usgovvirginia'; RegionName = 'USGov Virginia'; }
,[PSCustomObject]@{ OriginalValue = 'us north central'; RegionId = 'northcentralus'; RegionName = 'North Central US'; }
,[PSCustomObject]@{ OriginalValue = 'us sec east'; RegionId = 'usseceast'; RegionName = 'USSec East'; }
,[PSCustomObject]@{ OriginalValue = 'us sec west'; RegionId = 'ussecwest'; RegionName = 'USSec West'; }
Expand All @@ -409,9 +422,10 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'us west 2'; RegionId = 'westus2'; RegionName = 'West US 2'; }
,[PSCustomObject]@{ OriginalValue = 'us west 3'; RegionId = 'westus3'; RegionName = 'West US 3'; }
,[PSCustomObject]@{ OriginalValue = 'us west central'; RegionId = 'westcentralus'; RegionName = 'West Central US'; }
,[PSCustomObject]@{ OriginalValue = 'us2'; RegionId = 'eastus2'; RegionName = 'East US 2'; }
,[PSCustomObject]@{ OriginalValue = 'us_east'; RegionId = 'eastus'; RegionName = 'East US'; }
,[PSCustomObject]@{ OriginalValue = 'us_west'; RegionId = 'westus'; RegionName = 'West US'; }
,[PSCustomObject]@{ OriginalValue = 'us2'; RegionId = 'eastus2'; RegionName = 'East US 2'; }
,[PSCustomObject]@{ OriginalValue = 'usa'; RegionId = 'usgovarizona'; RegionName = 'USGov Arizona'; }
,[PSCustomObject]@{ OriginalValue = 'uscentral'; RegionId = 'centralus'; RegionName = 'Central US'; }
,[PSCustomObject]@{ OriginalValue = 'usdodcentral'; RegionId = 'usdodcentral'; RegionName = 'USDoD Central'; }
,[PSCustomObject]@{ OriginalValue = 'usdodeast'; RegionId = 'usdodeast'; RegionName = 'USDoD East'; }
Expand Down Expand Up @@ -452,6 +466,7 @@ function Get-OpenDataRegion
,[PSCustomObject]@{ OriginalValue = 'ussw'; RegionId = 'ussecwest'; RegionName = 'USSec West'; }
,[PSCustomObject]@{ OriginalValue = 'usswc'; RegionId = 'ussecwestcentral'; RegionName = 'USSec West Central'; }
,[PSCustomObject]@{ OriginalValue = 'ustsc'; RegionId = 'southcentralusstg'; RegionName = 'South Central US STG'; }
,[PSCustomObject]@{ OriginalValue = 'usv'; RegionId = 'usgovvirginia'; RegionName = 'USGov Virginia'; }
,[PSCustomObject]@{ OriginalValue = 'usw3'; RegionId = 'westus3'; RegionName = 'West US 3'; }
,[PSCustomObject]@{ OriginalValue = 'uswest'; RegionId = 'westus'; RegionName = 'West US'; }
,[PSCustomObject]@{ OriginalValue = 'uswest2'; RegionId = 'westus2'; RegionName = 'West US 2'; }
Expand Down
Loading