Simplifying Cloud Management with Azure Automation Accounts

by

Managing cloud resources can feel like juggling too many balls at once—updates, monitoring, compliance, and resource optimization. That’s where Azure Automation Accounts come in—a powerful tool that automates time-consuming and repetitive tasks so IT pros can focus on what truly matters.

In this post, we’ll break down what Azure Automation Accounts are, how they work, their key features, real-world use cases, and how you can get started quickly.

What is an Azure Automation Account?

An Azure Automation Account is a centralized hub in Microsoft Azure where you can manage automation resources. It acts as a container for all the components you need to automate cloud tasks, such as:

  • Runbooks (scripts for tasks)
  • Schedules (timing your scripts)
  • Modules (PowerShell or Python libraries)
  • Hybrid workers (agents that run automation tasks on-prem or in other clouds)

In short, it’s your automation command center.

Key Features

Here are some standout features that make Azure Automation Accounts a must-have:

Runbooks

These are scripts that perform tasks like restarting VMs, rotating keys, or cleaning up unused resources. You can write them in PowerShell, Python, or use the Graphical Runbook Designer for drag-and-drop simplicity.

Scheduling

Automate tasks to run on a set schedule, like checking VM health every morning or scaling services during off-peak hours.

Hybrid Runbook Workers

Need to automate tasks on your on-prem servers? Hybrid Runbook Workers extend your automation capabilities beyond Azure.

Update Management

Keep your Windows and Linux VMs compliant with automated patching and update assessments.

Credential and Certificate Management

Securely store credentials, certificates, and other secrets directly in the automation account, keeping sensitive info safe.

Benefits for IT Professionals

  • Reduce Manual Effort: Save time by automating routine tasks.
  • Improve Consistency: Eliminate human error with repeatable scripts.
  • Boost Efficiency: Focus on strategic projects instead of repetitive admin.
  • Hybrid Flexibility: Automate tasks across on-prem, multi-cloud, and Azure environments.

PowerShell Script to create Azure Automation Account

I put together this PowerShell script to create Azure automation account.

Login to Azure using Connect-AzAccount

This script will,

  • Check and validate the name entered
  • Query Azure to ensure the location value entered is valid
  • Query logged in Azure subscription to validate the resource group name entered
  • Asks for the plan – Basic or Free
  • Asks if you want to generate and assign a new System Identity for this automation account
do {
    $AzAutomationAccountname = Read-Host -Prompt "Enter the name of the Automation Account"
    if ($AzAutomationAccountname -notmatch '^[a-zA-Z][-a-zA-Z0-9]{4,48}[a-zA-Z0-9]$') {
        Write-Host "Invalid name! Please follow these rules:" -ForegroundColor Red
        Write-Host "- Must be 6-50 characters long"
        Write-Host "- Must start with a letter"
        Write-Host "- Must end with a letter or number"
        Write-Host "- Can contain letters, numbers, and hyphens"
        Write-Host "Please try again." -ForegroundColor Yellow
    }
} while ($AzAutomationAccountname -notmatch '^[a-zA-Z][-a-zA-Z0-9]{4,48}[a-zA-Z0-9]$')


do {
    $AzAutomationAccountLocation = Read-Host -Prompt "Enter the location of the Automation Account"
    $validLocations = (Get-AzLocation).Location
    if ($AzAutomationAccountLocation -notin $validLocations) {
        Write-Host "Invalid location! Please enter one of these Azure locations:" -ForegroundColor Red
        $validLocations | Sort-Object | ForEach-Object { Write-Host "- $_" }
        Write-Host "Please try again." -ForegroundColor Yellow
    }
} while ($AzAutomationAccountLocation -notin $validLocations)


do {
    $AzAutomationAccountResourceGroupName = Read-Host -Prompt "Enter the name of the Resource Group for the Automation Account"
    $validResourceGroups = (Get-AzResourceGroup).ResourceGroupName
    if ($AzAutomationAccountResourceGroupName -notin $validResourceGroups) {
        Write-Host "Invalid Resource Group! Please enter one of these existing Resource Groups:" -ForegroundColor Red
        $validResourceGroups | Sort-Object | ForEach-Object { Write-Host "- $_" }
        Write-Host "Please try again." -ForegroundColor Yellow
    }
} while ($AzAutomationAccountResourceGroupName -notin $validResourceGroups)


do {
    Write-Host "Choose the Automation Account plan:"
    Write-Host "1. Basic"
    Write-Host "2. Free"
    $choice = Read-Host -Prompt "Enter your choice (1 or 2)"
    
    switch ($choice) {
        "1" { $AzAutomationAccountplan = "Basic" }
        "2" { $AzAutomationAccountplan = "Free" }
        default {
            Write-Host "Invalid choice! Please enter either 1 or 2" -ForegroundColor Red
            Write-Host "Please try again." -ForegroundColor Yellow
        }
    }
} while ($choice -notin "1","2")


$AzAutomationAccountAssignSystemIdentity = Read-Host -Prompt "Do you want to assign a system identity to the Automation Account? (Y/N)"
if ($AzAutomationAccountAssignSystemIdentity -eq "Y" -or $AzAutomationAccountAssignSystemIdentity -eq "y") {
    $AzAutomationAccountAssignSystemIdentity = $true
} elseif ($AzAutomationAccountAssignSystemIdentity -eq "N" -or $AzAutomationAccountAssignSystemIdentity -eq "n") {
    $AzAutomationAccountAssignSystemIdentity = $false
} else {
    Write-Host "Invalid input! Defaulting to No." -ForegroundColor Red
    $AzAutomationAccountAssignSystemIdentity = $false
}

Write-Host "`nReview your selections:" -ForegroundColor Cyan
Write-Host "Automation Account Name: $AzAutomationAccountname"
Write-Host "Location: $AzAutomationAccountLocation"
Write-Host "Resource Group: $AzAutomationAccountResourceGroupName"
Write-Host "Plan: $AzAutomationAccountplan"
Write-Host "System Identity: $(if ($AzAutomationAccountAssignSystemIdentity) { 'Yes' } else { 'No' })"

$confirm = Read-Host -Prompt "`nDo you want to proceed with these settings? (Y/N)"
if ($confirm -ne 'Y' -and $confirm -ne 'y') {
    Write-Host "Operation cancelled by user." -ForegroundColor Yellow
    exit
}


if ($AzAutomationAccountAssignSystemIdentity) {
    New-AzAutomationAccount -Name $AzAutomationAccountname -Location $AzAutomationAccountLocation -ResourceGroupName $AzAutomationAccountResourceGroupName -Plan $AzAutomationAccountplan -AssignSystemIdentity
} else {
    New-AzAutomationAccount -Name $AzAutomationAccountname -Location $AzAutomationAccountLocation -ResourceGroupName $AzAutomationAccountResourceGroupName -Plan $AzAutomationAccountplan
}

Azure Automation Accounts are an essential part of a smart cloud strategy. By leveraging them, IT professionals can reduce overhead, improve reliability, and maintain control over sprawling environments.

Whether you’re managing a handful of VMs or an enterprise-grade hybrid infrastructure, automation is your silent workhorse—and Azure makes it incredibly approachable.

Thank you for stopping by. ✌️

Azure Role-Based Access: Who’s Got the Keys to the Cloud Castle?

by

Alright, let’s talk about Azure Role-Based Access Control (RBAC)—the bouncer at the club, the gatekeeper of your cloud kingdom, the difference between “Oops, I deleted the production database” and “Phew, good thing I didn’t have permission for that.”

If you’re working with Microsoft Azure, RBAC is a must-know. It’s how you control who can do what in your cloud environment. Let’s break it down in a fun, easy-to-digest way.

What is Azure RBAC, and Why Should You Care?

Think of Azure RBAC like a high-tech office building with keycards. Not everyone should have access to every room, right? Your interns shouldn’t be able to access the CEO’s private office, and the janitor doesn’t need the nuclear launch codes.

RBAC works the same way in Azure:

  • You assign roles to users, groups, or applications instead of just giving them full access.
  • It’s based on the principle of least privilege, meaning people only get access to what they need—nothing more, nothing less.
  • It prevents chaos. Because let’s be real, one accidental click from an over-permissioned user can lead to disaster.

The Three Key Pieces of RBAC

Azure RBAC is built on three main pieces:

  1. Roles: These define what someone can do. Examples:
    • Owner – The boss. Can do anything and everything.
    • Contributor – Can create and manage resources but can’t assign roles.
    • Reader – Can look, but not touch.
    • Custom Roles – If the built-in roles aren’t enough, you can create your own.
  2. Scope: This defines where the role applies. It can be at:
    • Subscription level (the whole kingdom)
    • Resource group level (a city inside the kingdom)
    • Specific resources (a single castle or shop)
  3. Assignments: This is the who gets what role part. Assign a user, group, or service principal to a role at a given scope, and boom—permissions granted.

Real-World Example: The Coffee Shop Analogy ☕

Imagine you’re running a coffee shop:

  • The Owner (you) can do everything—order supplies, hire staff, make coffee, or even shut down the store.
  • The Baristas (contributors) can make coffee and manage the store but can’t hire or fire anyone.
  • The Customers (readers) can look at the menu, enjoy their coffee, but they’re not allowed behind the counter.

That’s Azure RBAC in action. Everyone gets access to what they need, but no one is accidentally pressing the “shutdown entire store” button.

Common RBAC Mistakes (And How to Avoid Them)

  1. Giving Everyone Owner or Contributor Roles – That’s like handing out master keys to your entire office. Keep permissions minimal!
  2. Not Using Groups – Assigning roles individually? Big mistake. Use Azure AD groups to manage permissions efficiently.
  3. Ignoring Scope – Always assign roles at the lowest necessary level to avoid over-permissioning.
  4. Forgetting to Review Roles Regularly – People leave jobs, projects change, and roles should be updated accordingly.

Final Thoughts: Lock It Down, But Keep It Practical

Azure RBAC is all about control, security, and making sure the right people have the right access. It’s not just an IT thing—it’s about keeping your cloud environment safe and sane.

So next time you’re setting up roles in Azure, ask yourself:

  • Does this person really need this level of access?
  • Could I use a lower scope?
  • Am I following best practices?

Get it right, and your cloud stays secure. Get it wrong, and… well, let’s just say you don’t want to be the person who accidentally gives the intern the power to delete the company’s entire infrastructure.

Thank you for stopping by.✌

How to Automatically Download and Use Azure Public IP Ranges with PowerShell

by

If you work with firewalls, proxies, or any system that restricts traffic based on IP addresses, you’re likely familiar with the challenges of maintaining access to dynamic cloud infrastructure. Microsoft understands this, which is why they publish the Azure IP Ranges and Service Tags – Public Cloud dataset — a JSON file containing up-to-date IP address ranges used by Azure services.

Why Microsoft Publishes Azure IP Ranges

Microsoft Azure’s cloud infrastructure spans global data centers, with thousands of services running behind constantly shifting sets of IP addresses. To help organizations:

  • Configure firewalls and security appliances
  • Whitelist Azure service IPs
  • Meet compliance or policy needs
  • Route traffic appropriately

…Microsoft provides this public JSON file that includes IP ranges tied to Service Tags like Storage, Sql, AppService, and many others, broken down by region.

If you are using Azure Firewall, then you can use these service tags directly in your firewall rules. More information can be found here. If you are using some other firewall, then you need to check if they support service tags directly e.g., Palo Alto Networks & External Dynamic Lists (EDL).

If you don’t have support for service tags in your firewall, then you need to use IP address prefixes directly. This is not ideal, since you need to update your firewall rules every time when new IP address prefixes are added or removed. This is why automating this process is important.

How Often Is the Data Updated?

Microsoft typically updates the Azure IP Ranges and Service Tags – Public Cloud file weekly, usually every Monday. Updates can reflect:

  • New IPs added for expanding infrastructure
  • Old ranges removed or reallocated
  • Changes to service or region mappings

Each release includes a "changeNumber" field and a "version" field to help you detect updates. Automation is key here — hence the script!

Changes, Changes…

Azure public IP addresses can change weekly! What does that mean for us?

Every week, there may be new IP addresses added to the list. Others may be removed if they’re no longer in use by Azure.

Why does that matter?

Let’s say an on-prem application needs access to an AzureSQL database, you previously added its IPs to your firewall allow-list. Then Azure updates its IP ranges, and the on-prem application keeps using the same IP or IP ranges that’s not in your allow-list. Boom — access denied. Not because you intended to block it, but because you didn’t know about the change.

It’s not just about adding the new IPs. You also need to handle removals to prevent your allow-lists from becoming bloated and insecure.

This isn’t a new problem in networking. But in Azure, the pace of change is faster — often weekly — and automation becomes essential.

How Is the JSON File Structured?

The JSON file is well-organized and structured to support automation. Here’s what it generally looks like:

jsonCopyEdit{
  "changeNumber": 20250401,
  "cloud": "Public",
  "values": [
    {
      "name": "AppService.WestUS",
      "id": "AppService.WestUS",
      "properties": {
        "region": "westus",
        "platform": "Azure",
        "systemService": "AppService",
        "addressPrefixes": [
          "13.64.0.0/18",
          "40.112.0.0/16"
        ]
      }
    },
    ...
  ]
}

Key fields:

  • values: Array of service tag entries
  • name / id: Service and region
  • properties.addressPrefixes: List of IP ranges (in CIDR format)

You can easily filter entries by region, service tag, or even specific prefixes depending on your needs.

Common Use Cases for Downloading Azure IPs

There are many real-world situations where access to this list is helpful:

  • Firewall Whitelisting: Allow only Azure Storage or Sql service traffic from a specific region.
  • Cloud Egress Policies: Identify what your workloads are connecting to by cross-referencing logs with Azure-owned IPs.
  • Network Audits & Compliance: Ensure your infrastructure is only communicating with approved external services.
  • CDN or WAF Configurations: Enable access to Azure Front Door, App Service, or other endpoints behind the scenes.
  • Automation Pipelines: Pull the list programmatically during CI/CD to dynamically configure network security settings.

PowerShell Script to Download the File

To help automate this process, I wrote a PowerShell script that downloads the latest Azure IP Ranges JSON file:

What this script does,

  • Fetches the HTML content of the page https://www.microsoft.com/en-us/download/details.aspx?id=56519 using Invoke-WebRequest.
  • Extracts the JSON file URL from the page content using a regex match.
  • Creates a folder named json_files if it does not already exist.
  • Extracts the JSON file name from the URL and constructs the file path within the json_files folder.
  • Checks if the JSON file already exists:
    • If it exists, skips the download.
    • If it does not exist, downloads the JSON file to the json_files folder.
  • Parses the downloaded JSON file into a PowerShell object using ConvertFrom-Json.
  • Creates a timestamped folder named AzureServicetags_<timestamp> in the directory called ‘AzureServicetags’ in the current directory.
  • Creates three subfolders within the timestamped folder:
    • All_IPs for all IP addresses.
    • IPv4_IPs for IPv4 addresses.
    • IPv6_IPs for IPv6 addresses.
  • Iterates through each service tag in the JSON content:
    • Extracts the service name and its associated IP address prefixes.
    • Separates the IP addresses into IPv4 and IPv6 categories using regex matching.
    • Creates text files for:
      • All IP addresses.
      • IPv4 addresses.
      • IPv6 addresses.
    • Writes the respective IP addresses into the corresponding text files.
    • Outputs a message indicating the creation of files for each service.
  • Skips services that do not have valid address prefixes and outputs a warning message.
  • Outputs a message if the JSON URL is not found.
# Define the URL of the download page
$pageUrl = "https://www.microsoft.com/en-us/download/details.aspx?id=56519"

# Fetch the page source
$response = Invoke-WebRequest -Uri $pageUrl -UseBasicParsing

# Extract the JSON URL for 'url'
$jsonUrl = ($response.Content -match '"url":"(https://download\.microsoft\.com/download/[^"]+)"') | Out-Null
$jsonUrl = $matches[1]

# Output the JSON URL
if ($jsonUrl) {
    Write-Output "JSON URL: $jsonUrl"

    # Create the folder 'json_files' if it doesn't exist
    $jsonFolder = "json_files"
    if (-not (Test-Path -Path $jsonFolder)) {
        New-Item -ItemType Directory -Path $jsonFolder | Out-Null
    }

    # Extract the file name from the JSON URL
    $jsonFileName = $jsonUrl.Split("/")[-1]
    $jsonFilePath = Join-Path -Path $jsonFolder -ChildPath $jsonFileName

    # Check if the JSON file already exists
    if (Test-Path -Path $jsonFilePath) {
        Write-Output "File '$jsonFileName' already exists. Skipping download."
    }
    else {
        # Download the JSON file into the 'json_files' folder
        Invoke-WebRequest -Uri $jsonUrl -OutFile $jsonFilePath
        Write-Output "Downloaded JSON file: $jsonFileName"
    }

    # Parse the JSON file
    $jsonContent = Get-Content -Path $jsonFilePath -Raw | ConvertFrom-Json

    # Create a folder with the current date and time stamp
    $timestamp = Get-Date -Format "yyyy-MM-dd_HH-mm"
    $folderName = ".\AzureServicetags\AzureServicetags_$timestamp"
    New-Item -ItemType Directory -Path $folderName | Out-Null

    # Create subfolders for All_IPs, IPv4_IPs, and IPv6_IPs
    $allIPsFolder = Join-Path -Path $folderName -ChildPath "All_IPs"
    $ipv4Folder = Join-Path -Path $folderName -ChildPath "IPv4_IPs"
    $ipv6Folder = Join-Path -Path $folderName -ChildPath "IPv6_IPs"

    foreach ($subFolder in @($allIPsFolder, $ipv4Folder, $ipv6Folder)) {
        if (-not (Test-Path -Path $subFolder)) {
            New-Item -ItemType Directory -Path $subFolder | Out-Null
        }
    }

    # Iterate through each service tag and create text files
    foreach ($service in $jsonContent.values) {
        $serviceName = $service.name

        if ($service -and $service.properties -and $service.properties.addressPrefixes) {
            $ipAddresses = $service.properties.addressPrefixes
            $ipv4Addresses = $ipAddresses | Where-Object { $_ -match '\.' -and $_ -notmatch '\:' }
            $ipv6Addresses = $ipAddresses | Where-Object { $_ -match '\:' }

            # Create files for all IPs, IPv4 IPs, and IPv6 IPs
            $allIPsFilePath = Join-Path -Path $allIPsFolder -ChildPath "$serviceName`_all_IPs.txt"
            $ipv4FilePath = Join-Path -Path $ipv4Folder -ChildPath "$serviceName`_v4_IPs.txt"
            $ipv6FilePath = Join-Path -Path $ipv6Folder -ChildPath "$serviceName`_v6_IPs.txt"

            $ipAddresses | Out-File -FilePath $allIPsFilePath -Encoding UTF8
            $ipv4Addresses | Out-File -FilePath $ipv4FilePath -Encoding UTF8
            $ipv6Addresses | Out-File -FilePath $ipv6FilePath -Encoding UTF8

            Write-Output "Created files for service: $serviceName"
        }
        else {
            Write-Warning "Service '$serviceName' does not have valid address prefixes. Skipping."
        }
    }
}
else {
    Write-Output "JSON URL not found."
}

Thanks for stopping by. ✌

Power BI – Restore Datasets to new on-premise Gateway when old Gateway has failed or Recovery Key is lost

by

Power BI on-premises data gateway is a service running on a Windows server working as a connecting platform between the Power BI cloud service and the on-premise data sources.

Setting up a data gateway on-premise is fairly a straightforward process. There can be instances where your on-premise gateway fails because of a hardware failure, issues due to updates or you may want to move the gateway instance to a new server then you realize you need the recovery key which is no where to be found.

Without a functioning gateway, the reports and dashboard in the Power BI cloud service with datasets that are connected to on-premise data sources will fail resulting in data to become stale. I’ll elaborate more on this issue in this post on how to restore datasets from an old or failed on-premise gateway to a new gateway.

I faced a similar scenario recently and it was a great learning experience. There are few methods using which you can resolve this issue. I’ll try to cover them all in as much detail as possible.

Manual Method

Well.. If you don’t have too many data sources on-premise or if you are just planning for a quick fix maybe because someone important in your organization needs this fixed and they notified you like an hour before their big meeting.

Here are the high-level steps,

Once you install and configure the data gateway, you can see and manage both the old and new instances from the Power BI portal.

To add a user as admin to the gateway in the portal, follow below steps.

This image has an empty alt attribute; its file name is image-15-1024x370.png

Search user using username or email address, select Admin and click Share.

This image has an empty alt attribute; its file name is image-16-1024x504.png

Now to add a new data source, from the page header in the Power BI service, select Settings gear icon > Manage gateways.

I have highlighted my failed gateway and the new gateway server in my case.

Your next step is to determine the data source of the affected dataset. To get this information, you’ll need access to the workspace. As you can see, I have a report named ‘AdventureWorksProducts‘ and the underlying dataset with the same name.

Under the Gateway connection section, you’ll find the necessary information to setup the data source in the new gateway.

Back in the Manage Gateways page and in the Data sources tab, Click on New

Choose the new gateway to create the connection on, then select the Data source type. In my scenario, I picked SQL Server.

Once you provide all the information, Select Create. You see Created New data source if the process succeeds and a new data source entry like in screenshot below.

If you’ve made it this far, you are almost at the end of this method. Now back to the dataset’s settings like we did earlier and on to the Gateway connection section. As a reminder, you’ll need access(Admin, Member or Contributor) to the workspace and to the dataset, also keep in mind that you also need admin permissions on the gateway.

You should see the new data source we created listed. Select it from the drop-down and click Apply

That should take care of the connection and to confirm, you can refresh your dataset to make sure the connection works ok.

Like I said earlier, this method should be good in a small environment or if you are in a hurry to get it fixed and worry about the bulk of things later. I’ll cover the semi-automated way in the coming sections. I use the word automated loosely here but it’s more like less clicks and not moving around in the BI portal as much.

Using a Service Account

In this method, I’m using a service account or in other words a regular user account without any roles assigned to it. This can be an AD synced account or a Azure AD cloud only account. This account will need a Power BI Pro license assigned to it.

Here are the high-level steps,

I’ve already covered the adding data source part to the gateway in the earlier section and the process is same for this method too. You can do it with PowerShell or REST APIs but I don’t believe there is a method to copy the data sources from one gateway to another.

Permissions

In this method, I’m using a service account which was granted Admin permissions for the gateways and set as Owner on the data source. You should be able to get away with just having the account set as user on the data source. This service account is also set as Admin on the workspace but Member or Contributor should do the trick.

You can grant the gateway admin permission in the portal which I’ve covered in the earlier method or use the below script to add the user as admin.

Connect-AzureAD
Connect-DataGatewayServiceAccount
Get-DataGatewayAccessToken

Get-DataGatewayCluster
$gw = Read-Host "Enter Gateway ID"
$user = Read-Host "Enter username to be added as gateway admin"
$userToAdd = (Get-AzureADUser -Filter "userPrincipalName eq '$user'").ObjectId
Get-DataGatewayRegion
$Region = Read-Host "Enter region value where IsDefaultPowerBIRegion is set to true"
Add-DataGatewayClusterUser -GatewayClusterId $gw -PrincipalObjectId $userToAdd -AllowedDataSourceTypes $null -Role Admin -RegionKey $Region

With all these permissions, the service account still needs to take ownership of the dataset to finish rebinding the data source to the new gateway. You won’t have to manually take ownership of the dataset, the script below will do it for you on the dataset you specify.

Rebind dataset

Before proceeding further make sure you have the Microsoft Power BI Cmdlets for PS installed and logged in to the Power BI service using PowerShell,

Connect-PowerBIServiceAccount
Get-PowerBIAccessToken

I don’t do Power BI administration on a daily basis and there was a learning curve for me to understand the inner workings. Here is the thought process that went into building this script.

  1. Get all the gateways the service account has access to
    • Using the output, determine and copy the new gateway ID and store it in a variable
  2. Using the variable from earlier step, return a list of data sources from the new gateway
    • Using the output, determine and copy the data source ID where the affected dataset should be mapped to and store it in a variable
  3. Returns a list of workspaces the user has access to
    • Using the output, determine and copy the workspace ID which has the affected dataset
  4. Using the variable from earlier step, return list of datasets from the specified workspace
    • Using the output, determine and copy the affected dataset’s ID
  5. Using the variable from step 3 and step 4, transfer ownership over the specified dataset to the service account
  6. Using variable from steps 1, 2, 3 and 4, bind the specified dataset from the specified workspace to the new gateway
$gateways = Invoke-PowerBIRestMethod -Url "gateways" -Method Get | ConvertFrom-Json
$gateways.value
Write-Host "Please copy the new Gateway ID from above output" -ForegroundColor Red
$newGWID = Read-Host "Please paste the new Gateway ID"

$GWdatasources = Invoke-PowerBIRestMethod -Url "gateways/$($newGWID)/datasources" -Method Get | ConvertFrom-Json
$GWdatasources.value
Write-Host "Please note down the Data Source ID used by the dataset that needs to be migrated from above output" -ForegroundColor Red
$datasourceObjectIds = Read-Host "Please paste the Data source ID"

$ws = Invoke-PowerBIRestMethod -Url 'groups' -Method Get | ConvertFrom-Json
$ws.value
Write-Host "Please note down the Workspace ID which has the dataset that needs to be migrated from above output" -ForegroundColor Red
$wsID = Read-Host "Please paste the Workspace ID"

$dataset = Invoke-PowerBIRestMethod -Url "groups/$($wsID)/datasets" -Method Get | ConvertFrom-Json
$dataset.value
Write-Host "Please note down the dataset ID that needs to be migrated from above output" -ForegroundColor Red
$dsID = Read-Host "Please paste the dataset ID"

#This below line is not needed if the service account already has ownership of the dataset and is safe to comment out
Invoke-PowerBIRestMethod -Url "https://api.powerbi.com/v1.0/myorg/groups/$($wsID)/datasets/$($dsID)/Default.TakeOver" -Method POST

try { $body = "{
  'gatewayObjectId': '$newGWID',
  'datasourceObjectIds': [
    '$datasourceObjectIds'
  ]
}"

Invoke-PowerBIRestMethod -Url "https://api.powerbi.com/v1.0/myorg/groups/$($wsID)/datasets/$($dsID)/Default.BindToGateway" -Body $body -Method POST
Write-Host "Dataset updated" }

catch {
  Write-Host "An error occurred"
}

You can adjust this script according to your needs as in some instances, your gateway ID, new data source ID and workspace ID will be the same, only the affected dataset ID will vary.

Using a Service Principal

In this method, I’m using a service principal to accomplish the same as above. One added advantage of using this method is, the Power BI Dataset can be setup to refresh without an actual user account. This would be great from an automation point of view and to avoid being tied to a specific user.

Here are the high-level steps,

Create SPN

The az ad app is part of Azure CLI and not a PS cmdlet. You’ll need to have Azure CLI installed and do az login as well before running this.

Connect-AzureAD 
Connect-AzAccount
az login

You can create an Azure AD application which will be the service principal from the portal and grant the and grant the ‘Dataset.ReadWrite.All’ API permission or use the below lines to create it. I’ve detailed how to determine the API ID and Permission ID in this blog post here.

A new Azure AD group is also needed and the Azure AD application has be made a member of this group. The below lines will accomplish that and if you have an existing group you have in mind, you can use that too. I’ll go over the reason for creating this group later in this section.

$appname = Read-Host "Enter a name Azure AD Application's Display Name"
$ObjID = New-AzureADApplication -DisplayName $appname | Select ObjectId
Add-AzADAppPermission -ObjectId $ObjID.ObjectId -ApiId 00000009-0000-0000-c000-000000000000 -PermissionId 322b68b2-0804-416e-86a5-d772c567b6e6 -Type Scope
Start-Sleep -Seconds 60
az ad app permission admin-consent --id $ObjID.ObjectId
Get-AzureADApplication -Filter "DisplayName eq '$appname'" | fl

$grpName = Read-Host "Enter a name for new Azure AD group"
$grpID = (New-AzureADGroup -DisplayName $grpName -MailEnabled $false -SecurityEnabled $true -MailNickName "NotSet").ObjectId
Get-AzureADGroup -ObjectId $grpID
Add-AzureADGroupMember -ObjectId $grpID -RefObjectId $spnToAdd
Get-AzureADGroupMember -ObjectId $grpID

The Get-AzureADApplication cmdlet will list the API permissions we applied. This can be verified in the ‘App registrations‘ blade from the Azure AD portal too.

Create a new Secret in this Azure AD application. You can also achieve this by using PowerShell. This secret value is needed for authentication while running the script later this section.

Remember to copy the secret value as it’ll be masked forever.

And we can also make sure of the group we created and it’s membership. I named the group, ‘PBI-API‘ in Azure AD.

For an Azure AD app to be able to access the Power BI content and APIs, the following settings need to be enabled in Azure AD portal. This is where the Azure AD group comes into play.

Go to Tenant settings in the Power BI Admin portal, and scroll down to Developer settings

  • Enable the Allow service principals to use Power BI APIs
  • Enable the Allow service principals to create and use profiles

Create SPN profile

I noticed that the SPN way of doing things worked in one instance without having a service principal profile created by the service principal. Profiles can be created using Profiles REST API. I’ve included the below lines which will create a profile for the SPN.

$prof = Read-Host "Enter a name for SPN's profile"

$body = "{
    'displayName' : '$prof'
}"

Invoke-PowerBIRestMethod -Url 'https://api.powerbi.com/v1.0/myorg/profiles' -Body $body -Method POST

A service principal can also call GET Profiles REST API to get a list of its profiles.

Invoke-PowerBIRestMethod -Url 'profiles' -Method Get

Permissions

Next, the service principal needs permissions on the dataset. We can achieve this by granting permissions to the service principal on the workspace.

Note: Adding the Azure AD group that has SPN as members doesn’t work

This next step is kind of where things get tricky.

What are we trying to achieve here?

  • Grant the service principal, admin permissions on the new gateway
  • Grant the service principal, user permissions on the gateway data source

Reason why it is tricky is, I first tried adding the Azure AD group the above permissions and it allowed me to add it but the script which comes later in this section didn’t work as expected. Based on further research, I realized that the SPN needs to be granted the above access directly instead of using the Azure AD group. Also, at the time of writing this post, adding SPN the above permissions using the portal is not supported. Hence, we’ll have to use PowerShell cmdlets,

Before proceeding further, please connect to the AzAccount and PowerBIService using the below cmdlets,

Connect-AzAccount
Connect-PowerBIServiceAccount
Get-PowerBIAccessToken

The below script will add the permissions I mentioned above and display the same at the end of executing the cmdlets. One good thing about the part where you add permissions to the gateway, data sources and workspaces is, it is a one-time deal.

Get-DataGatewayCluster
$gw = Read-Host "Enter Gateway ID"
$spn = Read-Host "Enter App name to be added as gateway admin"
$spnToAdd = (Get-AzADServicePrincipal -DisplayName $spn).Id
Get-DataGatewayRegion
$Region = Read-Host "Enter region value where IsDefaultPowerBIRegion is set to true"
Add-DataGatewayClusterUser -GatewayClusterId $gw -PrincipalObjectId $spnToAdd -AllowedDataSourceTypes $null -Role Admin -RegionKey $Region
Get-DataGatewayCluster -GatewayClusterId $gw | Select -ExpandProperty Permissions | ft
Get-DataGatewayClusterDatasource -GatewayClusterId $gw
$gwDSID = Read-Host "Enter Gateway Cluster DatasourceId"
Add-DataGatewayClusterDatasourceUser -GatewayClusterId $gw -GatewayClusterDatasourceId $gwDSID -DatasourceUserAccessRight Read -Identifier $spnToAdd
Get-DataGatewayClusterDatasourceUser -GatewayClusterId $gw -GatewayClusterDatasourceId $gwDSID

With all the permissions for the SPN now in place, we are ready to take ownership of the affected datasets in the workspaces and bind it with the new data source on the new gateway

Rebind dataset

In this SPN method, Instead of logging in with a username and password, you’ll have to login with the Application ID and secret

$Tenant = Read-Host "Enter Azure AD Tenant ID"
Connect-PowerBIServiceAccount -Tenant $Tenant -ServicePrincipal -Credential (Get-Credential) #user = Application (client) ID | Password is the secret value we created earlier in this section
Get-PowerBIAccessToken

The script is pretty much the same as in earlier section but only runs in the SPN context.

$gateways = Invoke-PowerBIRestMethod -Url "gateways" -Method Get | ConvertFrom-Json
$gateways.value
Write-Host "Please copy the new Gateway ID from above output" -ForegroundColor Red
$newGWID = Read-Host "Please paste the new Gateway ID"

$GWdatasources = Invoke-PowerBIRestMethod -Url "gateways/$($newGWID)/datasources" -Method Get | ConvertFrom-Json
$GWdatasources.value
Write-Host "Please note down the Data Source ID used by the dataset that needs to be migrated from above output" -ForegroundColor Red
$datasourceObjectIds = Read-Host "Please paste the Data source ID"

$ws = Invoke-PowerBIRestMethod -Url 'groups' -Method Get | ConvertFrom-Json
$ws.value
Write-Host "Please note down the Workspace ID which has the dataset that needs to be migrated from above output" -ForegroundColor Red
$wsID = Read-Host "Please paste the Workspace ID"

$dataset = Invoke-PowerBIRestMethod -Url "groups/$($wsID)/datasets" -Method Get | ConvertFrom-Json
$dataset.value
Write-Host "Please note down the dataset ID that needs to be migrated from above output" -ForegroundColor Red
$dsID = Read-Host "Please paste the dataset ID"

Invoke-PowerBIRestMethod -Url "https://api.powerbi.com/v1.0/myorg/groups/$($wsID)/datasets/$($dsID)/Default.TakeOver" -Method POST

try { $body = "{
  'gatewayObjectId': '$newGWID',
  'datasourceObjectIds': [
    '$datasourceObjectIds'
  ]
}"

Invoke-PowerBIRestMethod -Url "https://api.powerbi.com/v1.0/myorg/groups/$($wsID)/datasets/$($dsID)/Default.BindToGateway" -Body $body -Method POST
Write-Host "Dataset updated" }

catch {
  Write-Host "An error occurred"
}

Similar to the earlier section, you can adjust this script according to your needs as in some instances, your gateway ID, new data source ID and workspace ID will be the same, only the affected dataset ID will vary.

Needless to say, you can test if this was successful by doing a ‘Refresh now‘ on the dataset.

Issues you may encounter and How to fix it

Issue: You may encounter below status codes while running the Invoke-PowerBIRestMethod

Response status code : 404 (Not Found)
Response status code : 400 (Bad Request)

Fix or workaround: Well.. If you’ve already browsed though community.powerbi.com, then might have already realized that you are not alone dealing with these error codes. Usually this means you are requesting the Power BI REST API endpoints for data that doesn’t exist or you or the SPN that’s requesting the resource doesn’t have the necessary permissions to it. These best way to troubleshoot is to run these requests one at a time to determine where you it is failing or understand which resource you don’t have permissions to.

Issue: Applied permissions don’t reflect in the portal

Fix or workaround: I noticed that some of the changes takes time. Give it a few minutes before you go changing more things and you lose track of all the things you’ve changed in the process. If the permissions still didn’t show up for a while, use PowerShell cmdlets to verify if the permissions you’ve set was applied or not.

I’ll keep experimenting other scenarios and I’ll update the issues I come across later on.

This was one of those really lengthy posts but hey..as long as there is a solution at the end..Hopefully..am I right?..😁🤷‍♂️

Thank you for stopping by.✌

Azure AD – Improve Authenticator Notifications with Additional Context and Number Matching

by

As I have covered several times before, disabling basic authentication in one of the best things you can do in your O365 tenant for security.

MFA helps protect user’s account and prevents attacks. It is not perfect by any means but it is being improved. I’m a big fan of the Authenticator App. I try not to use the SMS or voice call options. Whenever I get a chance I always advocate the users I work with, to stick with the App. If your organization is yet to roll out MFA, it is time to take a hard look and make some drastic changes.

Microsoft in their November 18 Azure AD Identity blog revealed two new features for the Authenticator app. IMO, all O365 tenants should strongly consider enabling these two features below.

  • Number matching in Microsoft Authenticator
  • Additional context in Microsoft Authenticator

Number Matching

When a user responds to MFA challenge, they will see a number in the application or in the webpage which is challenging them and the user must enter this number in the Authenticator app to complete the process. This process is already part of the passwordless authentication method.

Additional Context

The Authenticator will also display the name of the app requesting MFA and also the user’s sign-in location. The sign-in location is based on the user’s public IP address. The location may not be accurate at times. This is because the IP location tagging and based on what I saw it is not the exact location of where the application’s traffic origin but usually close enough.

Application prompt on the webpage
Authenticator prompt

How to enable number matching with additional context in Azure AD

  • Open Azure AD admin center(https://aad.portal.azure.com/)
  • Click on the Security tab –> Authentication methods
  • Select Microsoft Authenticator
  • Toggle ENABLE to Yes
  • Toggle TARGET to All users
    • Depending on how you decide to roll out this feature, you can select a Azure AD group by selecting Select users, Select the group and follow along the next steps
  • Click on the three dots and Configure
  • Set
    • Authentication mode = Any
    • Require number matching = Enabled
    • Show additional context in notifications = Enabled
  • Click Done
  • Click Save
Microsoft Authenticator settings
Configure settings for All Users

In the drop down for ‘Require number matching’ and ‘Show additional context in notifications’, there is a ‘Microsoft Managed‘ option. It means this functionality will be enabled by default for all tenants after the feature is generally available. Currently it is in public preview.

Thank you for stopping by.✌