Latest

Page File functions added to AZSBTools PowerShell module to Get, Set, Remove page file(s)


3 new functions have been added to the AZSBTools PowerShell module to view, create and modify page file settings on a Windows computer running Windows 2008/Windows 7 and above versions.

Get-PageFile

This simple function takes no parameters and returns a PS custom object for each page file that has the following 3 properties:

  • DriveLetter: such as ‘c’, or ‘e’, …
  • InitialSizeMB: such as 1024 (0 value indicates a system-managed page file)
  • MaximumSizeMB: such as 4096 (0 value indicates a system-managed page file)

For example:

Set-PageFile

This function changes the page file setting on a given drive letter to the specified initial and maximum size in MB. It takes one parameter that’s similar to the PS custom object returned by the Get-PageFile function.

The -PageFile parameter of the Set-PageFile function accepts a PS Custom Object containing the following 3 properties:

  • DriveLetter such as ‘c’
  • InitialSizeMB such as 1024 (0 value indicate system managed page file)
  • MaximumSizeMB such as 4096 (0 value indicate system managed page file)

This object can be constructed manually as in:

$PageFile = [PSCustomObject]@{
    DriveLetter   = 'c'
    InitialSizeMB = 0 
    MaximumSizeMB = 0 
}

or obtained from the Get-PageFile function

For example, to configure all page files on all drives to system managed size:

Get-PageFile | foreach { $_.InitialSizeMB = 0; $_.MaximumSizeMB = 0; $_ } | Set-PageFile

Note that changes to page file require a reboot to take effect. Rebooting is not part of this function.

Remove-PageFile

Finally this simple function will remove page file from a given drive. It takes one parameter being the drive letter such as ‘e’

The 3 functions can be used in user scripts to move page file from one drive to another. For example:

Set-PageFile -PageFile ([PSCustomObject]@{
    DriveLetter   = 'e' 
    InitialSizeMB = 0
    MaximumSizeMB = 0
})
Remove-PageFile 'c' -EA 0 


To use the AZSBTools PowerShell module which is available in the PowerShell Gallery, you need PowerShell 5. To view your PowerShell version, in an elevated PowerShell ISE window type

$PSVersionTable

To download and install the latest version of AZSBTools from the PowerShell Gallery and its dependencies, type

Set-PSRepository -Name PSGallery -InstallationPolicy Trusted

To trust the Microsoft PowerShell Gallery repository, then

Install-Module AZSBTools,AzureRM -Force -AllowClobber

AZSBTools contains functions that depend on AzureRM modules, and they’re typically installed together.

To load the AZSBTools, and AzureRM modules type:

Import-Module AZSBTools,AzureRM -DisableNameChecking

To view a list of cmdlets/functions in AZSBTools, type

Get-Command -Module AZSBTools

To view the built-in help of one of the AZSBTools functions/cmdlets, type

help <function/cmdlet name> -show

such as

help New-SBAZServicePrincipal -show

Advertisements

Validate-NameResolution cmdlet added to AZSBTools PowerShell module


In the course of IAAS VM migrations from on-premises to Azure, the VM IP address changes. In Windows VM we typically invoke a command like

ipconfig /registerdns

or

Register-DNSClient

which is part of the DnsClient PowerShell module.

The Validate-NameResolution cmdlet part of the AZSBTools module, queries each DNS server in the current AD to make sure that all DCs in the current AD forest resolve a given computer name to the same IP. This helps to diagnose instances where DNS partition replication is not functioning properly, where some DC’s resolve a given computer name to the old on-premises IP while others resolve the same name to the new Azure IP.

This cmdlet takes one required parameter -ComputerName which accepts one or more computer names

Example: Validate-NameResolution -ComputerName ‘myTestPC’

The cmdlet outputs interim information to the console like:

In addition, it also returns PSCustom Objects, one for each resolved IP address with the following properties: ComputerName, ResolvesTo, and DNSServer similar to:


To use the AZSBTools PowerShell module which is available in the PowerShell Gallery, you need PowerShell 5. To view your PowerShell version, in an elevated PowerShell ISE window type

$PSVersionTable

To download and install the latest version of AZSBTools from the PowerShell Gallery and its dependencies, type

Set-PSRepository -Name PSGallery -InstallationPolicy Trusted 
Install-Module AZSBTools,AzureRM -Force

AZSBTools contains functions that depend on AzureRM module, and they’re typically installed together.

To load the AZSBTools and AzureRM modules type:

Import-Module AZSBTools,AzureRM -DisableNameChecking

To view a list of cmdlets/functions in SB-Tools, type

Get-Command -Module AZSBTools

To view the built-in help of one of the AZSBTools functions/cmdlets, type

help <function/cmdlet name> -show

such as

help New-SBAZServicePrincipal -show

Azure storage – features and pricing – June 2018


In the ever evolving Azure storage list of offerings, it may be hard to fully realize the available Azure storage offerings and their general cost structure at a given point in time. This post lays out a general summary of Azure storage offerings and costs, from the prospective of a consultant trying to make a recommendation to a large client as what storage type/options to use for which workload and why.

Storage account type

Classic storage

  • This is of type ‘Microsoft.ClassicStorage/storageAccounts’
  • This is considered legacy and should be migrated out of to GPv1 which provides backward compatibility to Azure classic (ASM) services

General purpose v1 (GPv1)

  • This is of type ‘Microsoft.Storage/storageAccounts’, kind ‘Storage’
  • Does not support Cool and Archive access tier attributes
  • Lower read/write transaction cost compared to GPv2
  • Can be used with Azure classic (ASM) services

General purpose v2 (GPv2)

  • This is of type ‘Microsoft.Storage/storageAccounts’, kind ‘StorageV2’
  • Supports Cool and Archive access tier attributes
  • Access Tier attribute (Hot/Cool) is exposed at the account level
  • Cannot be used with Azure classic (ASM) services
  • Compared to Blob Storage account: GPv2 charge for Cool early deletion but not Cool data writes (per GB)

Blob storage

  • This is of type ‘Microsoft.Storage/storageAccounts’, kind ‘BlobStorage’
  • This is a sub-type of GPv2 that supports only Block Blobs – not Page Blobs, Azure files, …
  • Minor pricing difference: charge for Cool data writes (per GB) but not Cool early deletion
  • Features similar to GPv2:
    • Supports Cool and Archive access tier attributes
    • Access Tier attribute (Hot/Cool) is exposed at the account level
    • Cannot be used with Azure classic (ASM) services

GPv1 and Blob Block accounts can be upgraded to a GPv2 account. This change cannot be reversed.

Set-AzureRmStorageAccount -ResourceGroupName <resource-group> -AccountName <storage-account> -UpgradeToStorageV2

Data Access Tiers

Data Access Tiers are Hot, Cool, and Archive. They represent 3 different physical storage platforms.

For the purpose of simplification, I will refer to LRS pricing in US East Azure region in US dollars for the next 450TB/month.

  • Data Access Tiers are supported in GPv2 accounts only (and the Block Blob storage sub-type) not GPv1
  • A blob cannot be read directly from the Archive tier. To read a blob in Archive, it must first be moved to Hot or Cool.
  • Data in Cool tier is subject to minimum stay period of 30 days. So, moving 1 TB of data to Cool tier for example, obligates the client to paying for that storage for 30 days at a minimum even if the data is moved or deleted before the end of the 30 days. This is implemented via the ‘Early deletion charge’ which is prorated.
  • Data in Archive tier is subject to minimum stay period of 180 days.

Hot

  • This is the regular standard tier where data can be routinely read and written
  • Storage cost is 2 cents per GB per month
  • Lowest read and write IO charges (5 cents per 10k write, 0.4 cents per 10k read)
  • No data retrieval charge (This is the cost to copy data to Hot tier from another tier before it can be read – already in Hot tier)

Cool

  • 25% cheaper storage cost down to 1.5 cents per GB per month
  • More costly read and write IO charges (10 cents per 10k write = 200% Hot, 1 cent per 10k read = 250% Hot)
  • 1 cent per GB data retrieval charge (this is the cost to copy the data to Hot tier, which is a required interim step to read data that resides in Cool tier)

Archive

  • 90% cheaper storage cost down to 0.2 cents per GB per month
  • Most costly read and write IO charges (10 cents per 10k write = 200% Hot, 5 dollars per 10k read = 125,000% Hot)
  • 2 cent per GB data retrieval charge (this is the cost to copy the data to Hot tier, which is a required interim step to read data that resides in Archive tier)
  • The archive tier can only be applied at the blob level, not at the storage account level
  • Blobs in the archive storage tier have several hours of latency (Is this tier using tape not disk!?)

Geo-replication

Geo-replication refers to automatically keeping a copy of a given Storage Account data in another Azure region that’s 400 miles or more away from the primary site. The primary Azure site is the Azure region where the Storage account resides and where the usual read/write transactions occur. The choice of a secondary site is a constant determined by Microsoft and is available if the client chooses the geo-replication feature of a Storage Account. The following is the list of the Azure region pairs as of 18 June 2018

Data in the secondary Azure region cannot be accessed. It is only used under the following conditions:

  • Microsoft declares a region-wide outage. For example East US. This is a Microsoft triggered – not client triggered event
  • Microsoft will first try to restore the service in that region. During that time data is not accessible for read or write. It’s unclear how long will Microsoft pursue this effort before moving to geo-failover.
  • Microsoft initiates geo-failover from the primary to secondary region. That’s all data of all tenants in the primary region.
    • This is essentially a DNS change to re-point the storage end points’ fully qualified domain names to the secondary region
    • The RPO (Recovery Point Objective) is 15 minutes. That’s to say up to 15 minutes worth of data may be lost.
    • The RTO (Recovery Time Objective) is unclear. That’s the time between primary site outage to data availability for read and write on the secondary site
    • At the end of the geo-failover, read/write access is restored (for geo-replicated storage accounts only of course)
    • At a later time, Microsoft will perform a geo-failback which is the same process in the reverse direction
  • This is a process that never happened before. No Azure data center ever sustained a complete loss.
  • It’s unclear when failback will be triggered, whether it will include down-time, or another 15 minute data loss.

A storage account can be configured in one of several choices with respect to geo-replication:

LRS (Locally redundant storage)

When a write request is sent to Azure Storage Account, the transaction is fully replicated on three different physical disks across three fault domains and upgrade domains inside the primary location, then success is returned back to the client.

GRS (Geo-redundant storage)

In addition to triple synchronous local writes, GRS adds triple asynchronous remote writes of each data block.  Data is asynchronously replicated to the secondary Azure site within 15 minutes.

ZRS (Zone-redundant storage)

ZRS is similar to LRS but it provides slightly more durability than LRS (12 9’s instead of 11 9’s for LRS over a given year).

It’s ony available to GPv2 accounts.

RA-GRS (Read access geo-redundant storage)

RA-GRS is similar to GRS but it provides read access to the data in the secondary Azure site.

 

 

 

 

 

 

Azure Cloud Shell


In a prior post I went over getting started with Azure automation using Azure Automation Account. Another way to run PowerShell scripts against an Azure subscription is to use Azure Cloud Shell. I think of Azure Automation Account as PaaS PowerShell whereas Azure Cloud Shell is more like IaaS PowerShell (more like PowerShell web access running on an Azure container).

Access

Azure Cloud Shell can be access from https://shell.azure.com/ or by clicking the Cloud Shell icon in the Azure Portal

Features

We have most of the common PowerShell features such as tab completion. Initially Get-Module shows:

Copy/paste works, although keyboard shortcuts like CTRL-C and CTRL-V do not.

Unlike Azure Automation Account, we can install and import PS modules directly from the shell:

We get drive y: as an Azure file share for persistent storage, since these PS sessions are not..

We have access to more than PowerShell in this shell, such as Azure CLI and Python, which I choose to completely ignore in this post, and focus on PowerShell only 🙂

The Azure: drive provides access to the available subscriptions and their objects:

So, we can list VMs under a given subscription by simply iterating the objects under azure:\subscription_name\VirtualMachines !!

You can tell that the Azure: drive provider is using the required AzureRM cmadlets to fetch the requested objects. In this example, it’s calling Get-AzureRmVM cmdlet

We can upload files to the y: drive Azure file share directly from the Azure portal:

 

Azure Automation – getting started


Azure Automation allows Azure administrators to run PowerShell and other scripts against an Azure subscription. They provide several benefits versus running the same scripts from the user desktop computer including:

  • Scripts run in Azure and are not dependent on the end-user desktop
  • Scripts are highly available by design.
  • Scheduling is a built-in feature
  • Authentication is streamlined for both classic ASM and current ARM resources

To get started with Azure Automation;

  1. Create an Azure Automation account
  2. Install needed PowerShell modules
  3. Create, run, test, schedule scripts

Create an Azure Automation account

In the current portal, Create Resource > Monitoring and Management > Automation > Create

In the ‘Add Automation Account’ blade enter/select a name for the Automation Account, Azure Subscription, Resource Group, and Azure Location

Azure will take a few minutes to create the automation account and associated objects.

We can now run scripts against the Azure subscription selected above. Here are some examples:

Create a test script

In the Automation Account blade, click Runbooks

Click ‘Add a runbook’ link on the top to create a new runbook of type PowerShell

Azure creates the runbook/script, and opens the ‘Edit PowerShell Runbook’ blade

Type in the desired command, click Save, then ‘Test pane’

In the ‘Test’ blade, click ‘Start’. Azure will queue and execute the script

Notes:

  • This is not like the PowerShell ISE. There’s no auto-completion for one thing.
  • If Azure comes across a bad command, it will try to execute THE ENTIRE SCRIPT repeatedly, and is likely to get stuck.
  • This shell does not support user interaction. So, any cmdlet that would typically require a user confirmation/interaction of any type will fail. For example, Install-Module cmdlet will fail since it requires user approval/interaction to install PowerShellGet.

Install needed modules

To see available modules click ‘Modules’ in the Automation Account blade

Click ‘Browse Gallery’ on top and search for the desired module

These modules come for the Microsoft PowerShell Gallery.

Click on the desired module, view its functions, and click Import to import it to this automation shell

Now that the module is imported, we can use it in scripting in this particular automation shell:

 

 

Azure Data Box


Azure Data Box is Microsoft’s parallel of AWS’  Snowball Edge or/and Google Transfer Appliance. It’s the evolution of Azure Import/Export service that allows a client to use client-provided disk to import/export data to/from Azure. As of April 2018;

Basic information

  • Azure Data Box is a 45 lb. NAS
  • 80 TB Usable Storage / 100 TB Physical Storage
  • Cost $80 + shipping both ways + Egress charges if exporting from Azure
  • 7-10 days processing time from device receipt date

Use Case

File share transfer/initial seeding

  • Azure Data Box connects to the client network as an IP NAS
  • Client uses WAImportExport free tool to copy BitLocker encrypted data to the Azure Data Box at local on-premises LAN speeds
  • At LAN speed of 1 Gbps and sustained local disk transfer rate of 128 MB/s, it takes about 7.5 days to copy 80 TB of data, provided the client does not require/impose copy throttling to maintain adequate data access performance.

Benefits/Limitations

  • Azure Data Box provides a free short term rental of a 100TB NAS. Alternatively a client can use their own disk(s) or NAS devices to transfer data to Azure using the same steps.
  • Azure Data Box is not intended for VM replication or migration.
  • It takes about 3 weeks between start of 80TB local data copy on-premises to data being available in Azure Storage Account (1+ week for local copy, 1+ week for Microsoft processing + shipping time)
  • Azure Data Box is useful in case of transferring mostly static data from on-premises to Azure. Daily data change rate must be below 5% (otherwise 100% of the data could have changed during the 20 day between start copy and data available in Azure)

 

Expand-Json cmdlet to expand custom PowerShell object in a more readable format added to AZSBTools PowerShell module


Microsoft Azure REST API version 2 (ARM – Azure Resource Manager) takes input request body and returns output in JSON format. Consequently, Azure PowerShell cmdelts and Azure CLI tend to use similar JSON objects for input, also known as ARM Templates.

For example, using this PowerShell cmdlet:

Get-AzureRmResource -ResourceId /subscriptions/xxxxx/resourceGroups

where xxxxx is your Azure subscription Id, may return output similar to:

Name : prod-mgt
ResourceId : /subscriptions/xxxxx/resourceGroups/prod-mgt
ResourceGroupName : prod-mgt
Location : eastus
SubscriptionId : xxxxx
Properties : @{provisioningState=Succeeded}

Name : TestAuto1
ResourceId : /subscriptions/xxxxx/resourceGroups/TestAuto1
ResourceGroupName : TestAuto1
Location : westeurope
SubscriptionId : xxxxx
Properties : @{provisioningState=Succeeded}

What the PowerShell cmdlet did is to send a GET request to the Azure Management API that looks partially like:

https://management.azure.com/subscriptions/xxxxx/resourceGroups?api-version=2014-04-01

Which returned JSON output similar to:

{
  "value": [
    {
      "id": "/subscriptions/xxxxx/resourceGroups/prod-mgt",
      "name": "prod-mgt",
      "location": "eastus",
      "properties": {
        "provisioningState": "Succeeded"
      }
    },
    {
      "id": "/subscriptions/xxxxx/resourceGroups/TestAuto1",
      "name": TestAuto1
      "location": "westeurope",
      "properties": {
        "provisioningState": "Succeeded"
      }
    }
  ]
}

In the course of working with Azure ARM templates, such as this template to create a Storage Account:

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "storageAccountType": {
      "type": "string",
      "defaultValue": "Standard_LRS",
      "allowedValues": [
        "Standard_LRS",
        "Standard_GRS",
        "Standard_ZRS",
        "Premium_LRS"
      ],
     "metadata": {
       "description": "Storage Account type"
     }
   }
  },
  "variables": {
    "storageAccountName": "[concat(uniquestring(resourceGroup().id), 'standardsa')]"
  },
  "resources": [
    {
      "type": "Microsoft.Storage/storageAccounts",
      "name": "[variables('storageAccountName')]",
      "apiVersion": "2016-01-01",
      "location": "[resourceGroup().location]",
      "sku": {
        "name": "[parameters('storageAccountType')]"
      },
      "kind": "Storage", 
      "properties": {
      }
    }
  ],
  "outputs": {
    "storageAccountName": {
      "type": "string",
      "value": "[variables('storageAccountName')]"
    }
  }
}

It may not be very clear what are the objects in the template and their hierarchy. Using the ConvertFrom-Json cmdlet of the Microsoft.PowerShell.Utility module produces a PS custom object with display similar to:

Get-Content E:\Scripts\ARMTemplates\Storage1.json | ConvertFrom-Json

$schema : https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#
contentVersion : 1.0.0.0
parameters : @{storageAccountType=}
variables : @{storageAccountName=[concat(uniquestring(resourceGroup().id), ‘standardsa’)]}
resources : {@{type=Microsoft.Storage/storageAccounts; name=[variables(‘storageAccountName’)]; apiVersion=2016-01-01; location=[resourceGroup().location]; sku=; kind=Storage; properties=}}
outputs : @{storageAccountName=}

This is better but it doesn’t show some of the information in the source JSON file/ARM template. The new Expand-Json cmdlet further expands the ConvertFrom-Json output:

Get-Content E:\Scripts\ARMTemplates\Storage1.json | ConvertFrom-Json | Expand-JSON


To use the AZSBTools PowerShell module which is available in the PowerShell Gallery, you need PowerShell 5. To view your PowerShell version, in an elevated PowerShell ISE window type

$PSVersionTable

To download and install the latest version of AZSBTools from the PowerShell Gallery and its dependencies, type

Install-Module POSH-SSH,SB-Tools,AZSBTools,AzureRM -Force

AZSBTools contains functions that depend on POSH-SSH, SB-Tools, and AzureRM modules, and they’re typically installed together.

To load the POSH-SSH, SB-Tools, AZSBTools, and AzureRM modules type:

Import-Module POSH-SSH,SB-Tools,AZSBTools,AzureRM -DisableNameChecking

To view a list of cmdlets/functions in SB-Tools, type

Get-Command -Module AZSBTools

To view the built-in help of one of the AZSBTools functions/cmdlets, type

help <function/cmdlet name> -show

such as

help New-SBAZServicePrincipal -show

New-SBAZServicePrincipal cmdlet to create new Azure AD Service Principal added to AZSBTools PowerShell module


For the use case of running PowerShell scripts that perform tasks on objects in an Azure subscription, we need to be able to run such scripts under a user context other than the script author which is what typically happens during script development. A Service Principal is an Azure AD user intended for this purpose. The New-SBAZServicePrincipal function automates and simplifies the process of creating an Azure Service principal.

Parameters

The New-SBAZServicePrincipal function takes the following parameters

ServicePrincipalName

This parameter accepts one or more Service Principal names

Environment

This parameter accepts a value that represents which Azure cloud to create the SPN in. This parameter default to Azure Commercial cloud. As of 15 March 2018 that list is:

  • AzureCloud
  • AzureUSGovernment
  • AzureChinaCloud
  • AzureGermanCloud

To see the current list, use: (Get-AzureRMEnvironment).Name

Role

This parameter is used to assign Role/Permissions for the Service Principal in the current subscription.
The default value is ‘Owner’ role.
As of 16 March 2018 the following default roles are defined:
API Management Service Contributor
Application Insights Component Contributor
Automation Operator
BizTalk Contributor
Classic Network Contributor
Classic Storage Account Contributor
Classic Storage Account Key Operator Service Role
Classic Virtual Machine Contributor
ClearDB MySQL DB Contributor
Contributor
Cosmos DB Account Reader Role
Data Factory Contributor
Data Lake Analytics Developer
DevTest Labs User
DNS Zone Contributor
DocumentDB Account Contributor
Intelligent Systems Account Contributor
Log Analytics Contributor
Log Analytics Reader
Network Contributor
New Relic APM Account Contributor
Owner
Reader
Redis Cache Contributor
Scheduler Job Collections Contributor
Search Service Contributor
Security Manager
SQL DB Contributor
SQL Security Manager
SQL Server Contributor
Storage Account Contributor
Storage Account Key Operator Service Role
Traffic Manager Contributor
User Access Administrator
Virtual Machine Contributor
Web Plan Contributor
Website Contributor
For more details on roles, type in:

Get-AzureRmRoleDefinition | select name,description,actions | Out-GridView

Output

The New-SBAZServicePrincipal function returns a PS Object for each input Service Principal Name containing the following properties:
ServicePrincipalName
TenantId
Environment
Role

Details

The New-SBAZServicePrincipal function performs the following tasks for each provided Service Principal name:

  1. Create/Validate Azure AD App. The Azure AD App is required to create a Service Principal. It carries the same name and has an initial URL matching the same name as well
  2. Create/Validate Azure AD Service Principal. The user is prompted to enter the desired password for the SPN. The password is encrypted and saved in the user’s temp folder for use with future automations
  3. Assign the provided Role to the SPN for the current subscription. By default this is the ‘Owner’ role. This allows the created SPN to perform all tasks against the current subscription.

Registered Apps can be also viewed in the Azure portal under Azure Active Directory/App Registrations blade:

Example

$SPList = New-SBAZServicePrincipal -ServicePrincipalName PowerShell01,samtest1

This example creates 2 Service Prinsipals; PowerShell01 and samtest1 in the default Azure Commercial cloud, and assigns them the default Owner Role in the current subscription.

The New-SBAZServicePrincipal function first pops the Azure login Window to identify which subscription to use:

This function has been tested with both Azure Commercial and Azure US GOV clouds.

Next enter the desired password for each of the 2 provided Service Principals:

The function saves the encrypted password to the user temp folder for future use/automation.

It also display console output similar to:

The Service Principals can be used now to run other PowerShell scripts

The newly registered/validated Apps can also be viewed from the Azure Portal


To use the AZSBTools PowerShell module which is available in the PowerShell Gallery, you need PowerShell 5. To view your PowerShell version, in an elevated PowerShell ISE window type

$PSVersionTable

To download and install the latest version of AZSBTools from the PowerShell Gallery and its dependencies, type

Install-Module POSH-SSH,SB-Tools,AZSBTools,AzureRM -Force

AZSBTools contains functions that depend on POSH-SSH, SB-Tools, and AzureRM modules, and they’re typically installed together.

To load the POSH-SSH, SB-Tools, AZSBTools, and AzureRM modules type:

Import-Module POSH-SSH,SB-Tools,AZSBTools,AzureRM -DisableNameChecking

To view a list of cmdlets/functions in SB-Tools, type

Get-Command -Module AZSBTools

To view the built-in help of one of the AZSBTools functions/cmdlets, type

help <function/cmdlet name> -show

such as

help New-SBAZServicePrincipal -show

Practical guide to PowerShell Arrays


Microsoft defines Arrays in PowerShell as ‘data structures designed to store collections of items’. This posting will go over practical aspects of arrays in PowerShell 5.1

An array is a data structure that holds an ordered list of items.

Array items are not unique (can have duplicates)

Array items are referred to as elements. Although array elements can be of different data types, in most practical use cases, they’re all of the same type.

Referencing array elements

Array elements are referred to by their order in the array where the first element in the array is order #0. They follow the array variable enclosed in square brackets [ ] as in:

In this example, $b[2] refers to the third element of the array which has the value of 4

The last element in an array is number -1, the element before last is -2, the 2nd element from the end is -3, …

We can reference more than one element of an array by using the comma (,), the range operator (..), or the plus sign (+)

In this array $c we have 10 random numbers each is between 0 and 9:

The comma ‘,’ is used to separate multiple array elements. For example to reference the 3rd and 6th elements of the array we use $c[2,5] and we expect to get 1 and 4 values in this example.

The range operator ‘..’ is used to get a sequential range of elements. To get elements 4 through 7 we can use $c[3..6]

The range operator circles the array boundary. For example, $c[-2..2] will list the element before last [-2], the last element [-1], the first element [0], the second element [1], and the third element [2] in this order

Although not very practical, you can mix individual elements and ranges when referencing array elements. Ranges are to be preceded by the plus sign (+). Individual elements are to be preceded by the comma (,) except after a range where they must be preceded by the plus sign (+) as well:

What we see often in practical use of arrays:

  • Referencing one or more array elements separated by commas as $c[1,4,6]
  • Referencing a sequential range of array elements as $c[3..5]
  • Referencing the last array element $c[-1]
  • We often don’t know how many elements an array has. We can find that out by using $c.Count or $c.Length or $c.GetUpperBound(0)+1 or $c.GetLength(0). In our example, array $c has 10 elements.

Declaring a variable as an array

There are several ways to declare an array. The standard/default way is to let PowerShell figure it out.

In this example, PowerShell figured out that $d should be an array based on the data that we’re trying to store in it.

We can specify that by using the array sub-expression operator @(). This is particularly useful when we want to make sure a variable is an array

Take this practical example:

Get-Service Win* | foreach {
    if ($PSItem.Status -eq 'Running') { $myList +=$PSItem }
}
$myList

We get errors on the last 3 cycles of this 4 cycle iteration:

The reason is that on cycle 1 of this 4 cycle iteration, PowerShell auto-declared the variable $myList as a ‘System.ComponentModel.Component’ which is the object returned from the Get-Service cmdlet. We actually want $myList to be an array of such objects that can hold 0, 1, or more items. There are 2 ways to do this:

  1. Declare $myList as an array upfront using the array sub-expression operator @()

    or
  2. Assign the resulting collection of the 4 cycle loop in one step instead of assigning each found value at the end of the if statement

The third way to declare an array is to type cast it. This only works in cases where all elements of the array are of the same data type. For example:

or

Here $myIPList is an array of objects, each is of data type ‘System.Net.IPAddress’

Removing array duplicates

Removing array duplicates is a simple matter of using ‘select -unique‘ as in:

Finding one or more values in an array

Finding a single value is a simple matter of using -match or -eq operators. For example:

In this example, the 8th and 10th elements have the value 3

Finding elements in array $a that also exist in array $b (AKA intersection of arrays $a and $b) can be done using ‘where’ statement and -in operator

This method produces the same result if the array elements are unique and sorted as in:

($d | select -Unique | where { $PSItem -in ($c | select -Unique) } | sort ) -join ', '
($c | select -Unique | where { $PSItem -in ($d | select -Unique) } | sort ) -join ', '

Adding/Removing array elements

Most arrays are of fixed length, which means the .add(), .remove(), .RemoveAt(), and the .Insert() array methods will fail.

To add element to an array, we recreate/reassign values to the array. For example, to add value 17 at the end of the $c array:

This makes 17 the 11th element of the $c array

To insert 17 as the 3rd element of the $c array we rebuild $c as in:

$c = $c[0,1] + 17 + $c[2..9]

This inserts 17 value as the 3rd element in the $c array

To remove a value from an array we can use the -notmatch operator. Note that this will remove all values that match. For example this removes the 17 value:

$c = $c -notmatch 17 

This example removes all 3 values:

$c = $c -notmatch 3

Whereas this example removes the first 3 value only:

$c = 0..($c.Length-1) | foreach { if ($PSItem -ne $c.IndexOf(3)) { $c[$PSItem] } }

This example uses the array object .IndexOf() method to determine the first occurrence of the element with the 3 value. It then loops through the array elements using foreach and compiles a new array skipping the 8th element identified by $c.IndexOf(3), and reassigns this new array to variable $c

To remove the 4th and 8th array elements for example regardless of their values, we use an expression similar to:

$c = 0..($c.Length-1) | foreach { if ($PSItem -notin 3,7) { $c[$PSItem] } }

Multi-dimensional arrays

Multi-dimensional arrays are rarely used and can be constructed using .NET as in:

$2D = [int[,]]::new(3,4)

This creates a 2 dimensional array. The first dimension 0 is 3 elements long, numbered 0..2 and the second dimension 1 is 4 elements long numbered 0..3

This sample code shows how to identify the count of array dimensions and the length of each dimension:

$2D = [int[,]]::new(3,4)
"2D is of type '$($2D.GetType().BaseType)'"
"It has '$($2D.Rank)' dimensions"
1..$2D.Rank | foreach { "Dimension '$($PSItem-1)' has '$($2D.GetUpperBound($PSItem-1))' elements" }

This example shows how to populate and reference multi-dimensional arrays:

"Example populating and referencing 2D:"
$i = 0
foreach ($Dim1 in 0..$2D.GetUpperBound(0)) { 
    foreach ($Dim2 in 0..$2D.GetUpperBound(1)) {
       $2D[$Dim1,$Dim2] = $i
       $i++
    }
}
foreach ($Dim1 in 0..$2D.GetUpperBound(0)) { 
    foreach ($Dim2 in 0..$2D.GetUpperBound(1)) {
       "2D[$Dim1,$Dim2] = $($2D[$Dim1,$Dim2])"
    }
}

 

Get-WindowsOpenPorts and Get-LinuxOpenPorts functions


Get-WindowsOpenPorts and Get-LinuxOpenPorts are 2 functions of the SB-Tools PowerShell module which is available in the PowerShell Gallery. These are not port scanners.

Get-WindowsOpenPorts

This function returns an array of PS object, one for each open port on the target Windows computer. Each object has the following properties:

  • ComputerName: String, such as myPC.mydomain.com
  • Layer3Protocol: String, such as IPv4 or IPv6
  • Layer4Protocol: String, such as TCP or UDP
  • LocalAddress: System.Net.IPAddress, such as 10.11.12.13
  • LocalPort: Int32, such as 80 or 139
  • State: String, such as LISTENING

for example:

$Session = New-PSSession -ComputerName abc3.xyz.klm.com -Credential (Get-SBCredential xyz\myuser) 
$WinPorts = Get-WindowsOpenPorts -Session $Session 
$WinPorts | Format-Table -AutoSize

This cmdlet/function takes a required parameter ‘Session’, which is of type ‘System.Management.Automation.Runspaces.PSSession’ that can be obtained via New-PSSession cmdlet of the ‘Microsoft.PowerShell.Core’ module.

It also takes 2 optional parameters that serve to filter its output:

-Layer3 parameter takes either ‘IPv4’ or ‘IPv6’ values or both, and will output only the records that match this criteria

-Layer4 parameter takes either ‘TCP’ or ‘UDP’ values or both, and will output only the records that match this criteria

By default, this cmdlet will filter on IPv4/TCP only.

This cmdlet uses the IPGlobalProperties.GetActiveTcpListeners() method of the System.Net.NetworkInformation.IPGlobalProperties class. It also parses netstat command output to obtain the Layer4Protocol and State properties.

Get-LinuxOpenPorts

Similarly, this function returns an array of PS object, one for each open port on the target Linux computer. Each object has the following properties:

  • ComputerName: String, such as myPC.mydomain.com
  • LocalAddress: String, such as 10.11.12.13, or ::1 (IPv6)
  • LocalPort: Int, such as 80 or 139
  • Process: String
  • Protocol: String, such as TCP, UDP, TCP6, UDP6, or RAW6
  • RemoteAddress: String, such as 11.12.13.14, or 0.0.0.0, or ::
  • RemotePort: String, such as 389 or *
  • State: String, such as LISTEN, ESTABLISHED, CLOSE_WAIT, TIME_WAIT, or LAST_ACK

for example:

$Session = New-SSHSession -ComputerName abc10.xyz.klm.com -Credential (Get-SBCredential myuser) -AcceptKey
$LinuxPorts = Get-LinuxOpenPorts -Session $Session -Verbose
$LinuxPorts | Format-Table -AutoSize

This cmdlet/function takes a required parameter ‘Session’, which is of type ‘SSH.SshSession’ that can be obtained via New-SSHSession cmdlet of the ‘POSH-SSH’ module.

It also takes 1 optional parameter that serves to filter its output:

-Protocol parameter takes one or more of the following values: TCP, UDP, TCP6, UDP6, RAW6, ALL, and will output only the records that match this criteria

By default, this cmdlet will filter on ‘ALL’

This command invokes ‘netstat -anp’ on the provided Linux host and parses its output to produce the resulting PS objects.


To use the SB-Tools PowerShell module which is available in the PowerShell Gallery, you need PowerShell 5. To view your PowerShell version, in an elevated PowerShell ISE window type

$PSVersionTable

To download and install the latest version of SB-Tools from the PowerShell Gallery, type

Install-Module SB-Tools,POSH-SSH -Force

SB-Tools contains functions that depend on POSH-SSH module, and they’re typically installed together.

To load the SB-Tools and POSH-SSH modules type:

Import-Module SB-Tools,POSH-SSH -DisableNameChecking

To view a list of cmdlets/functions in SB-Tools, type

Get-Command -Module SB-Tools

To view the built-in help of one of the SB-Tools functions/cmdlets, type

help <function/cmdlet name> -show

such as

help Convert-IpAddressToMaskLength -show