Posts tagged “Azure

StorSimple 8k software release 4.0


Around mid February 2017, Microsoft released StorSimple software version 4.0 (17820). This is a release that includes firmware and driver updates that require using Maintenance mode and the serial console.

Using this PowerShell script to save the Version 4.0 cmdlets and compare them to Version 3.0, I got:

storsimple40-a

Trying the new cmdlets, the Get-HCSControllerReplacementStatus cmdlet returns a message like:

storsimple40-b

The Get-HCSRehydrationJob returns no output (no restore jobs are running)

The Invoke-HCSDisgnostics seems pretty useful and returns output similar to:

storsimple40-c

The cmdlet takes a little while to run. In this case it took 14 minutes and 38 seconds:

storsimple40-d

It returns data from its several sections like;

System Information section:

storsimple40-e

This is output similar to what we get from the Get-HCSSystem cmdlet for both controllers.

Update Availability section:

storsimple40-f

This is output similar to Get-HCSUpdateAvailability cmdlet, although the MaintenanceModeUpdatesTitle property is empty !!??

storsimple40-g

Cluster Information section:

storsimple40-h

This is new exposed information. I’m guessing this is the output of some Get-HCSCluster cmdlet, but this is pure speculation on my part. I’m also guessing that this is a list of clustered roles in a traditional Server 2012 R2 failover cluster.

Service Information section:

storsimple40-i

This is also new exposed information. Get-Service is not an exposed cmdlet.

Failed Hardware Components section:

storsimple40-j

This is new exposed information. This device is in good working order, so this list may be false warnings.

Firmware Information section:

storsimple40-k

This output is similar to what we get from Get-HCSFirmwareVersion cmdlet

Network Diagnostics section:

storsimple40-l

Most of this information is not new, but it’s nicely bundled into one section.

Performance Diagnostics section:

storsimple40-m

Finally, this section provides new information about read and write latency to the configured Azure Storage accounts.

The full list of exposed cmdlets in Version 4.0 is:

Clear-DnsClientCache
Set-CloudPlatform
Select-Object
Restart-HcsController
Resolve-DnsName
Out-String
Out-Default
Set-HcsBackupApplianceMode
Measure-Object
Invoke-HcsmServiceDataEncryptionKeyChange
Invoke-HcsDiagnostics
Get-History
Get-Help
Get-HcsWuaVersion
Get-HcsWebProxy
Invoke-HcsSetupWizard
Set-HcsDnsClientServerAddress
Set-HcsNetInterface
Set-HcsNtpClientServerAddress
Test-HcsNtp
Test-HcsmConnection
Test-Connection
Sync-HcsTime
Stop-HcsController
Start-Sleep
Start-HcsUpdate
Start-HcsPeerController
Start-HcsHotfix
Start-HcsFirmwareCheck
Set-HcsWebProxy
Set-HcsSystem
Set-HcsRemoteManagementCert
Set-HcsRehydrationJob
Set-HcsPassword
Get-HcsUpdateStatus
Trace-HcsRoute
Get-HcsUpdateAvailability
Get-HcsSupportAccess
Enable-HcsRemoteManagement
Enable-HcsPing
Enable-HcsNetInterface
Disable-HcsWebProxy
Disable-HcsSupportAccess
Disable-HcsRemoteManagement
Enable-HcsSupportAccess
Disable-HcsPing
Test-NetConnection
Test-HcsStorageAccountCredential
TabExpansion2
Reset-HcsFactoryDefault
prompt
Get-NetAdapter
Disable-HcsNetInterface
Enable-HcsWebProxy
Enter-HcsMaintenanceMode
Enter-HcsSupportSession
Get-HcsRoutingTable
Get-HcsRemoteManagementCert
Get-HcsRehydrationJob
Get-HcsNtpClientServerAddress
Get-HcsNetInterface
Get-HcsFirmwareVersion
Get-HcsDnsClientServerAddress
Get-HCSControllerReplacementStatus
Get-HcsBackupApplianceMode
Get-Credential
Get-Command
Export-HcsSupportPackage
Export-HcsDataContainerConfig
Exit-PSSession
Exit-HcsMaintenanceMode
Get-HcsSystem
Update-Help

Advertisements

StorSimple 8k series as a backup target?


19 December 2016

After a conference call with Microsoft Azure StorSimple product team, they explained:

  •  “The maximum recommended full backup size when using an 8100 as a primary backup target is 10TiB. The maximum recommended full backup size when using an 8600 as a primary backup target is 20TiB”
  • “Backups will be written to array, such that they reside entirely within the local storage capacity”

Microsoft acknowledge the difficulty resulting from the maximum provisionable space being 200 TB on an 8100 device, which limits the ability to over-provision thin-provisioned tiered iSCSI volumes when expecting significant deduplication/compression savings with long term backup copy job Veeam files for example.

Conclusion

  • When used as a primary backup target, StorSimple 8k devices are intended for SMB clients with backup files under 10TB/20TB for the 8100/8600 models respectively
  •  Compared to using an Azure A4 VM with attached disks (page blobs), StorSimple provides 7-22% cost savings over 5 years

15 December 2016

On 13 December 2016, Microsoft announced the support of using StorSimple 8k devices as a backup target. Many customers have asked for StorSimple to support this workload. StorSimple hybrid cloud storage iSCSI SAN features automated tiering at the block level from its SSD to SAS to Azure tiers. This makes it a perfect fit for Primary Data Set for unstructured data such as file shares. It also features cloud snapshots which provide the additional functionality of data backup and disaster recovery. That’s primary storage, secondary storage (short term backups), long term storage (multiyear retention), off site storage, and multi-site storage, all in one solution.

However, the above features that lend themselves handy to the primary data set/unstructured data pose significant difficulties when trying to use this device as a backup target, such as:

  • Automated tiering: Many backup software packages (like Veeam) would do things like a forward incremental, synthetic full, backup copy job for long term retention. All of which would scan/access files that are typically dozens of TB each. This will cause the device to tier data to Azure and back to the local device in a way that slows things down to a crawl. DPM is even worse; specifically the way it allocates/controls volumes.
  • The arbitrary maximum allocatable space for a device (200TB for an 8100 device for example), makes it practically impossible to use the device as backup target for long term retention.
    • Example: 50 TB volume, need to retain 20 copies for long term backup. Even if change rate is very low and actual bits after deduplication and compression of 20 copies is 60 TB, we cannot provision 20x 50 TB volumes, or a 1 PB volume. Which makes the maximum workload size around 3TB if long term retention requires 20 recovery points. 3TB is way too small of a limit for enterprise clients who simply want to use Azure for long term backup where a single backup file is 10-200 TB.
  • The specific implementation of the backup catalog and who (the backup software versus StorSimple Manager service) has it.
  • Single unified tool for backup/recovery – now we have to use the backup software and StorSimple Manager, which do not communicate and are not aware of each other
  • Granular recoveries (single file/folder). Currently to recover a single file from snapshot, we must clone the entire volume.

In this article published 6 December 2016, Microsoft lays out their reference architecture for using StorSimple 8k device as a Primary Backup Target for Veeam

primarybackuptargetlogicaldiagram

There’s a number of best practices relating to how to configure Veeam and StorSimple in this use case, such as disabling deuplication, compression, and encryption on the Veeam side, dedicating the StorSimple device for the backup workload, …

The interesting part comes in when you look at scalability. Here’s Microsoft’s listed example of a 1 TB workload:

ss-backup-target03

This architecture suggests provisioning 5*5TB volumes for the daily backups and a 26TB volume for the weekly, monthly, and annual backups:

ss-backup-target04

This 1:26 ratio between the Primary Data Set and Vol6 used for the weekly, monthly, and annual backups suggests that the maximum supported Primary Data Set is 2.46 TB (maximum volume size is 64 TB) !!!???

ss-backup-target05

This reference architecture suggests that this solution may not work for a file share that is larger than 2.5TB or may need to be expanded beyond 2.5TB

Furthermore, this reference architecture suggests that the maximum Primary Data Set cannot exceed 2.66TB on an 8100 device, which has 200TB maximum allocatable capacity, reserving 64TB to be able to restore the 64TB Vol6

ss-backup-target06

It also suggests that the maximum Primary Data Set cannot exceed 8.55TB on an 8600 device, which has 500TB maximum allocatable capacity, reserving 64TB to be able to restore the 64TB Vol6

ss-backup-target07

Even if we consider cloud snapshots to be used only in case of total device loss – disaster recovery, and we allocate the maximum device capacity, the 8100 and 8600 devices can accommodate 3.93TB and 9.81TB respectively:

ss-backup-target08

Conclusion:

Although the allocation of 51TB of space to backup 1 TB of data resolves the tiering issue noted above, it significantly erodes the value proposition provided by StorSimple.



StorSimple 8k series software version reference


This post lists StorSimple software versions, their release dates, and major new features for reference. Microsoft does not publish release dates for StorSimple updates. The release dates below are from published documentation and/or first hand experience. They may be off by up to 15 days.

  • Version 4.0 (17820) – released 12 February 2017 – see release notes, and this post.
    • Major new features: Invoke-HCSDiagnostics new cmdlet, and heatmap based restores
  • Version 3.0 (17759) – released 6 September 2016 – see release notes, and this post.
    • Major new features: The use of a StorSimple as a backup target (9/9/2016 it’s unclear what that means)
  • Version 2.2 (17708) – see release notes
  • Version 2.1 (17705) – see release notes
  • Version 2.0 (17673) – released January 2016 – see release notes, this post, and this post
    • Major new features: Locally pinned volumes, new virtual device 8020 (64TB SSD), ‘proactive support’, OVA (preview)
  • Version 1.2 (17584) – released November 2015 – see release notesthis post, and this post
    • Major new features: (Azure-side) Migration from legacy 5k/7k devices to 8k devices, support for Azure US GOV, support for cloud storage from other public clouds as AWS/HP/OpenStack, update to latest API (this should allow us to manage the device in the new portal, yet this has not happened as of 9/9/2016)
  • Version 1.1 (17521) – released October 2015 – see release notes
  • Version 1.0 (17491) – released 15 September 2015 – see release notes and this post
  • Version 0.3 (remains 17361) – released February 2015 – see release notes
  • Version 0.2 (17361) – released January 2015 – see release notes and this post
  • Version 0.1 (17312) – released October 2014 – see release notes
  • Version GA (General Availability – 0.0 – Kernel 6.3.9600.17215) – released July 2014 – see release notes – This is the first Windows OS based StorSimple software after Microsoft’s acquisition of StorSimple company.
  • As Microsoft acquired StorSimple company, StorSimple 5k/7k series ran Linux OS based StorSimple software version 2.1.1.249 – August 2012

StorSimple Software update 3.0 (17759)


This post describes one experience of updating StorSimple 8100 series device from version 0.2 (17361) to current  (8 September 2016) version 3.0 (17759). It’s worth noting that:

  • StorSimple 8k series devices that shipped in mid 2015 came with software version 0.2
  • Typically, the device checks periodically for updates and when updates are found a note similar to this image is shown in the device/maintenance page: storsimple3-03
  • The device admin then picks the time when to deploy the updates, by clicking INSTALL UPDATES link. This kicks off an update job, which may take several hoursstorsimple3-01
  • This update method is known as updating StorSimple device using the classic Azure portal, as opposed to updating the StorSimple device using the serial interface by deploying the update as a hotfix.
  • Released updates may not show up, in spite of scanning for updates manually several times: storsimple3-04
    The image above was taken on 9 September 2016 (update 3.0 is the latest at this time). It shows that no updates are available even after scanning for updates several times. The reason is that Microsoft deploys updates in a ‘phased rollout’, so they’re not available in all regions at all times.
    storsimple3-05
  • Updates are cumulative. This means for a device running version 0.2 for example, we upgrade directly to 3.0 without the need to manually upgdate to any intermediary version first.
  • An update may include one or both of the following 2 types:
    • Software updates: This is an update of the core 2012 R2 server OS that’s running on the device. Microsoft identifies this type as a non intrusive update. It can be deployed while the device is in production, and should not affect mounted iSCSI volumes. Under the covers, the device controller0 and controller1 are 2 nodes in a traditional Microsoft failover cluster. The device uses the traditional Cluster Aware Update to update the 2 controllers. It updates and reboots the passive controller first, fails over the device (iSCSI target and other clustered roles) from one controller to the other, then updates and reboots the second controller. Again this should be a no-down-time process.
    • Maintenance mode updates:

      These are updates to shared components in the device that require down time. Typically we see LSI SAS controller updates and disk firmware updates in this category. Maintenance mode updates must be done from the serial interface console (not Azure web interface or PowerShell interface). The typical down time for a maintenance mode update is about 30 minutes, although I would schedule a 2 hour window to be safe. The maintenance mode update steps are:

      • On the file servers, offline all iSCSI volumes provisioned from this device.
      • Log in to the device serial interface with full access
      • Put the device in Maintenance mode: Enter-HcsMaintenanceMode, wait for the device to reboot
      • Identify available updates: Get-HcsUpdateAvailability, this should show available Maintenance mode updates (TRUE)
      • Start the update: Start-HcsUpdate
      • Monitor the update: Get-HcsUpdateStatus
      • When finished, exit maintenance mode: Exit-HcsMaintenanceMode, and wait for the device to reboot.

 


Migrating WordPress web site to Azure


From the IT prospective a WordPress web site requires:

  • A web server like Microsoft IIS or Apache
  • PHP
  • mySQL database

Migrating a WordPress website includes copying all its files/folder structure, and its mySQL database, and changing the wp-config.php file to point to the new mySQL database. These tasks could be complicated for a large site and may require specific skills related to web site configuration and mySQL database administration. This post goes over a very simple way to migrate a WordPress web site to Azure, using the Duplicator WordPress plugin.

Executive Summary

  • Add Duplicator WordPress Plugin
  • Create New Package
  • Create new Azure WebApp
  • Add mySQL database
  • Upload the Duplicator package to the new Azure WebApp
  • Run the Duplicator Package Installer

Add Duplicator WordPress Plugin

If you don’t have it already, add the WordPress Duplicator Plugin. On the Plugins page click Add New

WP01

Search for Duplicator, click Install Now

WP02

Create New Package

Click on Duplicator link on the left, then click Create New

WP03

Accept the defaults and click Next to scan your WordPress site

WP04

Duplicator scans your current WordPress site

WP05

and displays the result like:

WP06

In this example, I have a couple of warnings about large site size, and some large files. I check the box and click Build.

WP07

Duplicator builds the package:

WP08

The package consists of an Installer (installer.php file) and Archive (.zip file). I download both to my desktop. The zip file contains all the WordPress site files and folder structure + a scripted copy of the associated mySQL database

Create new Azure WebApp

In the Azure Portal, click New/Web+Mobile/Web App

WP09

In the Web App blade I type in the new Web App name ‘MyWebApp407’ which must be unique under .azurewebsites.net. I pick the Azure subscription from the Subscription drop down menu. I choose to create a new Resource Group. I give it a name; ‘MyWebApp-RG’. I click the arrow to create a new Service Plan

WP10

In the App Service Plan blade (middle) I click Create New,  type in MyWebApp-SP as the name, select East-US and accept the default Pricing teir of S1 Standard.

Finally, I click OK and Create

In a few minutes Azure complete MyWebApp deployment

WP11

Add mySQL database

I browse under Resource Groups/MyWebApp-RG, and click Add

WP12

I search for mySQL, and select MySQL Database by ClearDB

WP13

and click Create

WP14

I give it a name ‘MyWebAppDB’ (avoid using other than alphanumeric characters in DB name), pick East US for the location, click the arrow and OK to accept the terms, and finally click Create

WP15

Click Refresh and note the new blank mySQL database:

WP16

Upload the Duplicator package to the new Azure WebApp

If you browse to the new web site now you may see a temporary page like:

WP17

First zip the 2 files downloaded from the Duplicator Package above into 1 file:

WP18

Next browse to the KUDU page http://MyWebApp407.scm.azurewebsites.net

Click CMD under the Debug Console menu

WP19

Browse to d:\site\wwwroot

WP20

Drag the zip file from prior steps and drag it on the right side as shown below:

WP21

Azure will upload

WP22

and unzip the file

WP23

Run the Duplicator Package Installer

Browse to the installer.php file as in http://mywebapp407.azurewebsites.net/installer.php

You will see a page similar to

WP24

Back in the Azure Portal, click MyWebAppDB/Properties

WP25

Note the Database Name, Hostname, username, and password

WP26

Back in the installer.php screen, enter the required information as shown below:

WP27

Click ‘Connect and Remove All Data’, click ‘Test Connection’, check the box to acknowledge the notices, and click ‘Run Deployment’

Click OK to continue..

WP28

The installer extracts the Duplicator Package zip file restoring the file system and rebuilds the mySQL database from the script contained in the zip file

WP29

Accept the defaults and click Run Update

WP30

The Installer makes the selected changes to the WebApp config files

WP31

Follow the installer instructions to do final testing:

WP32

Step number 2 above is actually important. Clicking on the link next to ‘2.’ above will take you to the site admin login page:

WP33

Use the same credentials from the original site.

Adjust your permalinks setting as it is on the original site.:

WP34

As a last step, once the site users have tested that everything looks OK, add a custom domain to the site and switch the domain DNS records to point to your new Azure site.

 

 

 

 

 

 


Powershell script to Auto-Expand StorSimple volume based on amount of free space


StorSimple Hybrid Cloud Storage array is an on-premise iSCSI SAN that extends seamlessly to the cloud. iSCSI volumes provisioned from a StorSimple device can be expanded but cannot be shrunk. So, a typical recommendation here is to start a volume small and grow it as needed. Growing a volume is a process that does not require down time. This script grows a StorSimple volume automatically based on set conditions of volume free space and a not-to-exceed value.

The input region of this script is the one that should be edited by the script user:

Expand01

Similar to the script that monitors StorSimple Backups, the values for SubscriptionName, SubscriptionID, and StorSimpleManagerName variables can be found in the classic Azure Management Interface under your StorSimple Manager node Dashboard and Device pages:

Monitor-StorSimple05

and the RegistrationKey:

Monitor-StorSimple06

and the SSDeviceName (StorSimple Device Name)

Monitor-StorSimple07

The value for the SSVolumeName (StorSimple volume name) variable can be found under the device\volume container:

Expand02

Notify variable can be either $true or $false. This instructs the script whether or not to send email notification when an expansion is triggered,

Similarly, Expand variable can be either $true or $false. This instructs the script whether or not to expand the volume when an expansion is triggered, When set to $false (and Notify is set to $true) and an expansion is triggered, the script will send an email notification that an expansion is triggered but will not do the actual expansion.

ExpandThresholdGB and ExpandThresholdPercent variables are used by the script to identify the amount of free space on the volume below which a volume expansion is triggered. Only one of these variables is needed. If both are provided the script will use the larger value.

  • Example 1: If the volume size is 100 GB, and the ExpandThresholdGB is set to 10 (GB) and the ExpandThresholdPercent is set to 15 (%), the script will trigger a volume expansion if the amount of free space is at or below 15 GB
  • Example 2: If the volume size is 100 GB, and the ExpandThresholdGB is set to 10 (GB) and the ExpandThresholdPercent is set to 5 (%), the script will trigger a volume expansion if the amount of free space is at or below 10 GB

Similarly, the ExpandAmountGB and ExpandAmountPercent variables instruct the script on how much to expand the volume once expansion is triggered. Only one of these variables is needed. If both are provided the script will use the larger value.

  • Example 1: If the volume size is 100 GB, and the ExpandAmountGB is set to 10 (GB) and the ExpandAmountPercent is set to 15 (%), the script will expand the volume by 15 GB once expansion is triggered.
  • Example 2: If the volume size is 100 GB, and the ExpandAmountGB is set to 10 (GB) and the ExpandAmountPercent is set to 5 (%), the script will expand the volume by 10 GB once expansion is triggered.

The value assigned to the variable NotToExceedGB is used by the script as volume maximum size that the script must not exceed. For example, if the prior 4 variables instruct the script to expand a 900 GB volume by an additional 200 GB and the NotToExceedGB variable is set to 1024 (1 TB), the script will expand the volume by 124 GB only to reach the NotToExceedGB amount but to not to exceed it.

DiskNumber and DriveLetter are values that the script user should obtain from the server’s Disk Management screen of the file server using this iSCSI volume:

Expand05

As of the time of writing this post and script (1 April 2016), there’s no way to correlate a volume on a file server to a volume on a StorSimple device. For example, if you create 3 volumes of the same size on a StorSimple device and call them data1, data2, and data3, and present them to the same file server and format them with the same file system and block size, and use volume labels data1, data2, data3, there’s no way to tell if data1 on the StorSimple device is the volume labeled data1 on the file server. This is why it’s recommended to provision and format StorSimple volumes one at a time and use the same volume label when formatting the volume as the volume name on StorSimple. Long story short, it’s the user’s responsiblity to:

  1. Make sure the DrviveLetter and DiskNumber correspond to the SSVolumeName, and
  2. Update the DrviveLetter and DiskNumber values if they change on the file server due to adding or removing volumes.

One last point here; if this iSCSI volume is presented to a Windows Failover cluster, this script must be run on the owner node.

LogFile is the path to where the script will log its actions – each log line will be time stamped. This could be on a network share.

EmailSender is the name and email address you wish to have the email notification appear to come from. For example: StorSimple Volume Size Monitor <DoNotReply@YourDomain.com>

$EmailRecipients = @(
‘Sam Boutros <sboutros@vertitechit.com>’
‘Your Name <YourName@YourDomain.com>’
)

is an array that takes one or more email addresses in the format shown above.

SMTPServer is your SMTP relay server. You need to make necessary configuration/white-listing changes to allow your SMTP server to accept and relay SMTP email from the server running the script.

Sample script output:

Expand06

another example:

Expand04

and example of email notification:

Expand07

Possible future enhancements to this script include:

  1. Rewrite the script as a function so that it can handle several volumes
  2. Rewrite the script to use Powershell remoting, so that it does not have to run on the file server.
  3. Add functionality to detect if the target file server is a member of a failover cluster, and to automatically target the owner node.

 

 


Troubleshooting StorSimple high latency IO’s blocking low latency IO’s


By design StorSimple hybrid cloud storage tiers off automatically the oldest blocks from the local SSD tier down to the SAS tier as the SSD tier fills up (reaches ~80% capacity). In turn it also tiers down the oldest blocks from the SAS tier to the Azure tier as that fills up (reaches ~80% capacity).

This has the great benefits of:

  1. Automated tiering: This negates the need for data classification and the entirety of the efforts associated with that.
  2. Granular tiering: Tiering happens at the block level not at the file level. That’s 64KB for tiered volumes. So, a file can have some hot blocks in SSD, some older blocks in SAS, and some cold blocks that have been displaced all the way down to the Azure tier by warmer blocks (of the same or other files)

As of the time of writing this post (28 March 2016), tiering is fully automated and not configurable. The exception is ‘Locally Pinned Volume’ feature that comes with StorSimple software update 2.0 (17673) and above. A locally pinned volume loses the deduplication and compression features of a ‘Tiered Volume’, and always resides on the physical device. Currently no visibility is provided as to what tier a Locally Pinned Volume resides (SSD or SAS).

In the following scenario – take the example of an 8100 StorSimple device that has 15.8 TB local usable capacity (prior to dedplication and compression):

  1. Customer creates handful of volumes – about 30 TB provisioned out of 200 TB max allowed on the device, migrates some 25 TB of data:
    Capacity02
    The above ‘Primary’ capacity graph shows about 25 TB of data as it appears to the SMB file servers that consume the iSCSI volumes, while the below ‘Device’ capacity graph shows that about 10 TB of that 25 TB resides on the same device for the same time period.
    Capacity01
  2. Customer does an archive data dump, such as 2 TB of old backup or archive files. Any new data comes in as hot and in a ‘full’ device, it will displace older blocks to Azure. In this case, we have several TB of active production data that got inadvertently displaced to Azure. The following access pattern is observed:
    1. End user attempts to retrieve files. If the file blocks are in Azure, they will be retrieved, but to make room for them in the SSD tier, other blocks has be tiered down to the full SAS tier, which will have to tier off blocks back to Azure to make room for blocks coming down from SSD. So, a read operation has caused 2 tiering operations including a write operation to Azure. This is described as high latency IO operation.
    2. If this is taking several minutes, during the period where the device is handling high latency IO’s described above, if other users are requesting files that RESIDE ENTIRELY LOCALLY on the device (described as low latency IO operations), it has been observed that those read requests are slowed down as well to a crawl. That’s is high latency IO’s appear to block low latency IO’s.
    3. So in this scenario, a 2 TB archive data dump on an 8100 device with 10 TB on the device, result in the entire 10 TB being shuffled out to Azure and back in, few blocks at a time, until the 2 TB archive data ends up in Azure returning the device to its pre-incident status.

In my opinion, this is a situation to be avoided at all costs. Once it occurs, the device may exhibit very slow performance that may last for weeks until the archive data dump has made its way through the rest of the data on the device to Azure.

Best practices recommended to avoid this scenario:

  1. Adhere to the recommended device use cases, particularly unstructured data/file shares. StorSimple is not meant for multi-terabyte high performance SQL databases for example. Another example that is not recommended as a workload on StorSimple is large PST files. They’re essentially database file that are accessed frequently, and get scanned, indexed and accessed in their entirety.
  2. Do not run any workload or process that scans the active data set in its entirely. Anti-virus and anti-malware scans must be configured for incremental use or quick scans only, never for a full scan of all files on a volume. This applies to any process that may try to index, categorize, classify, or read all files on a volume. The exception is a process or application that reads files metadata and properties only – not open the files and reads inside of them. Reading metadata is OK because metadata always resides locally on the device.
  3. Carefully plan your data migration to StorSimple, putting emphasis on migrating the oldest data first. Robocopy can be a very helpful tool in the process.

I’m adding the following enhancements to my wishlist that I hope to see implemented by Microsoft in the next StorSimple software release:

  • Resolving the core issue of high latency IO’s seeming to block/impede low latency IO’s
  • More visibility into the device tiering metrics. Simply put, a storage admin needs to know when a StorSimple device is ‘full’ and is tiering off blocks from the primary data set to Azure. This knowledge is critical to avoid the situation described above. A metric of the amount of space available before the device is full, is even better to help provide predictability before reaching that point.
  • ‘Cloud Pinned Volume’ feature would be very helpful. This should allow the StorSimple storage admin to provision an iSCSI volume that resides always in Azure and does not affect the device heat map.

StorSimple 8k update to version 2.0 (17673)


StorSimple update 2.0 brings in a number of new exciting features such as Locally Pinned Volumes, OVA (On-premise Virtual Array), and enhanced SVA (StorSimple Virtual Array) model 8020 with 64TB capacity as opposed to 30 TB capacity of the prior model 1100 (now renamed 8010).

Update 2.0 is another intrusive update that requires down time. It includes LSI firmware update (KB 3121900), and SSD disk firmware update (KB 3121899).

Prior to the update, we can see the device running Software version 1.2 (17584)

StorSimple20-02

This can also be seen from the serial or Powershell interfaces by using the Get-HcsSystem cmdlet:

StorSimple20-03

Ensure that both controllers have routable IPs

As suggested by the update instructions, we ensure that both controllers 0 and 1 have routable IPs prior to start. To do so, I ping some external Internet IP address such as bing.com from each of the controllers’ fixed IPs:

From Controller 0 (the prompt must say ‘Controller0>’):

Test-HcsConnection -Source 10.1.2.86 -Destination bing.com

A positive response looks like:

StorSimple20-04

From Controller 1 (the prompt must say ‘Controller1>’):

Test-HcsConnection -Source 10.1.2.87 -Destination bing.com

StorSimple20-05

Phase I – Software update – start the update from the Azure Management Interface

In the classic portal, under the device Maintenance page, click Install Updates at the bottom:

StorSimple20-06

check the box and the check mark:

StorSimple20-07

Pre-upgrade checks are started:

StorSimple20-08

And a Software Update Job is created:

StorSimple20-09

successfully:

StorSimple20-10

Unlike prior updates, the 2.0 update starts on the passive controller:

StorSimple20-11

Under the StorSimple Manager/Jobs page, we can see an update job in progress:

StorSimple20-13

The controller being updated will reboot several times. During the update we’ll see unusual controller health and state information in the portal:

StorSimple20-12

This is normal while the update is in progress.

A few hours later, we can see that the passive controller has been patched to version 2.0

StorSimple20-16

and that a controller failover has occurred, where controller 1 is now active, and controller 0 (now passive) is being patched:

StorSimple20-14

About 4.5 hours the first phase of the update is finished:

StorSimple20-17

We can see the device in normal state and health under the Maintenance page:

StorSimple20-18

Phase II – Maintenance Mode LSI firmware update

Unfortunately this is an intrusive update that requires down time, similar to phase 2 of StorSimple version 1.2 update posted here.

To summarize the steps of maintenance mode updates:

  • Schedule a down-time window
  • Offline all StorSimple iSCSI volumes on the file servers
  • Run a manual cloud snapshot of all volumes
  • On the Device serial (not Powershell) interface, put the device in Maintenance mode:
    Enter-HcsMaintenanceMode
    Both controllers will reboot
  • Patch controller 0:
    Get-HcsUpdateAvailability
    Start-HcsUpdate
    Check update progress:
    Get-HcsUpdateStatus 
  • After controller o is patched repeat last step on controller 1 to patch it
  • Finally exit Maintenance mode:
    Exit-HcsMaintenanceMode
    Both controllers will reboot

The device is now back in normal operating condition, and we can online the volumes back on the file servers.


Setting up Azure AD Connect, 2-way directory synchronization, password write-back, online-password reset


For this demo, I will create a new Azure Active Directory (AAD) called Vertitech3AAD and a new on-premise Active Directory called Vertitech3OP.local (NetBIOS name Vertitech3OP) in a new 2012 R2 AD forest.

Create a new Azure Active Directory:

As of 24 February 2016, creating a directory is available only in the classic portal (https://manage.windowsazure.com). If you try to do it in the new portal (https://portal.azure.com):

AzureAD01

You’ll simply be redirected to the classic portal:
AzureAD02

I created Vertitech3AAD Azure Active directory in Azure, and created an on-premise AD domain called Vertitech3OP.local in new 2012 R2 forest:

AzureAD03

I can see the new AAD (Azure Active Directory) domain:

AzureAD04

Create new AAD Global Admin user:

We create a new AAD user for AD Connect because we need a Global Admin that has rights to a single AAD. In the new AAD I create a new user with Global Admin permissions:

AzureAD05

The new AAD user is created:

AzureAD06

Change the temporary user password:

Next, I must change the new user password. I browse to https://manage.windowsazure.com, log off and login again using the new user credentials and temp password:

AzureAD07

I’m then prompted to change my password:

AzureAD08

Download and install AD Connect on an on-premise machine:

AD Connect can be downloaded from the Azure AD page or this link.

AzureAD09

Install AD Connect

AzureAD10

AzureAD11

Using Express Settings:

AzureAD12

Enter the AAD Global Admin user name and password:

AzureAD13

And local (on-premise) AD credentials – this account needs to be member of the Enterprise Admins group:

AzureAD15

The message/recommendation about custom domain verification can be safely ignored.

AzureAD16

AD Connect uses SQL Express – but can be configured to use other on-premise full deployment of SQL:

AzureAD17

And we’re all done:

AzureAD18

I recorded the machine services in an XML file before installing AD Connect using the Powershell command:

Get-Service | Export-Clixml .\Services1.xml

After installing AD Connect, I ran this small script to identify new services added:

$Services1 = Import-Clixml .\Services1.xml
$NewServices = @()
(Import-Clixml .\Services2.xml) | % {
     if ($_.Name -notin $Services1.Name) {
         $NewServices += $_
     }
}
$NewServices | sort name | select name, displayname,status | FT -a

We can see 5 new services. Some are running under LocalSystem.

AzureAD20

In Computer Management under Local Users and Groups, we can see a number of new local groups that have been created during AD Connect installation:

AAD06

Only ADSyncAdmins local group has users. It has the local user account (service account for ADSync service) and the domain account that the AD Connect installation ran under,

And in Azure we can see a new Synchronization service account:

AzureAD22

Also, note that Directory integration is now Activated:

AzureAD34

To view synchronization activity, run Synchronization Service Manager (c:\program files\Microsoft Azure AD Sync\UIShell\miisclient.exe)

AAD07

User objects in the on-premise AD need to have inheritance enabled for AD Connect to work and synchronize these objects to Azure AD.

Enable Password Write-back:

We can also see Azure AD Connect icon on the desktop (shortcut to “C:\Program Files\Microsoft Azure Active Directory Connect\AzureADConnect.exe”)

AzureAD23

Which shows the following options:

AzureAD24

First option is to View Current Configuration:

AzureAD25

Note the default settings above. To change synchronization settings, click Customize Synchronization Options:

AzureAD26

Next we enter our Azure AD Global Admin user credentials:

AzureAD27

And our local (on-premise) AD admin credentials:

AzureAD28

We can select to synchronize all domains and OUs or specific domains and OUs:

AzureAD29

As well as optional features:

AzureAD30

I check the box to enable Password Write-back, and click Install to reconfigure the synchronization process:

AzureAD31

AzureAD32


To test synchronization, I create a local AD user:

AzureAD33

By default AD Connect synchronizes every 30 minutes. To force a manual synchronization I use this Powershell cmdlet on the AD Connect machine:

Start-ADSyncSyncCycle -PolicyType Delta

Now I can see the user in Azure:

AzureAD35

Configure Password Reset Policy:

In the Azure classic portal at https://manage.windowsazure.com, browse to your directory/configure page:

Pwdreset03

Click the Yes button for ‘Users Enabled for Password Reset’.

Don’t forget to click the Save icon on the bottom center to save and apply your new settings.

Accept the remaining default settings or customize them as needed under the ‘user password reset policy’ section.

I changed the default setting ‘Require Users to Register When Signing in’ from Yes to No. This feature will require users to enter Mobile Phone OR Alternate Email Address as configured in this section. You may want to warn users before hand to expect that requirement, or/and tackle any internal organization/privacy issues related to users’ alternate emails and mobile phone numbers.

One last note here; Password Reset Policy is a directory-wide setting. It will apply to all users. As of 7 March 2016, it cannot be configured to apply to a certain user/group/OU.

Finally, users can change their passwords online using the standard Azure password reset pages/links such as https://account.activedirectory.windowsazure.com/ChangePassword.aspx?BrandContextID=O365&ruO365=

Pwdreset01

which can be reached from the settings/password link under https://portal.office.com/account/ for example:

Pwdreset02