Posts tagged “Using Robocopy for file migration”

File share migration – Phase 4 – Dealing with files that failed to copy


3/17/2015 update:

I updated the Powershell script to report on files and folders copied and the duration it took, and export that information to CSV files.

Running the same command:

Get-FailedFiles (Get-ChildItem -Path .\logs | where { $_.Name.StartsWith('Robo') }).FullName

The script now produces output to the console like:

Migrate-Files-18

and saves two CSV files to the current folder. One contains the file/folder copy summary statistics displayed on the console, and the other contains the list of files that failed to copy and the error code

Following Phase 3 of file share migration, phase 4 will deal with the files that failed to copy. Most commonly this is because the files were open at the time Robocopy attempted to copy them and it could not get the necessary file lock. Step 1 of dealing with the files that failed to copy is to identify them. This Powershell script goes through one or more Robocopy log files and parses out the files that failed to copy. The script can be downloaded from the Microsoft script center repository.

Get-FailedFiles -RoboLog .\logs\Robo-Migrate-FileShares_NYFILSRV01P-2015-03-11_05-27-26AM.txt

The script returns a list of files that failed to copy and the corresponding error codes:

Migrate-Files-14This also can be run against a group of files as in:

$FailedFiles = Get-FailedFiles (Get-ChildItem -Path .\logs | where { $_.Name.StartsWith(‘Robo’) }).FullName

This command will parse all the files under .\logs subfolder that start with ‘Robo’ and return the files that failed to copy and their error codes. This can be presented:
$FailedFiles | FT -Auto # in tabular format

Migrate-Files-15$FailedFiles | Out-Gridview # in Powershell ISE Gridview

Migrate-Files-16

$FailedFiles | Export-Csv .\FailedFiles.CSV -NoType # or exported to CSV

Migrate-Files-17

 

March 12, 2015 | Categories: Powershell, Windows 2003 Migration | Tags: , , | 2 Comments

File share migration – Phase 2 – Robocopy


Following Phase 1 of file share discovery, phase 2 will do the initial seeding and copying of all data from the older Windows 2003 file server to the newer Windows 2012 R2 file server. The script can be downloaded from the Microsoft script center repository.

The same script can be used to run a refresh of the data. It will copy newer and changed files every time it runs. If interrupted, the script can simply be run again. Again only newer or changed files will be copied.

Requirements:

  • The same drives must exist on the new 2012 R2 server as the old 2003 server. So, if we have drive letters F, H, M on the older server we must have the same drive letters on the new server.
  • The drives on the new servers must have enough disk space to accommodate all copied data. If the drives on the new file server are formatted using a larger allocation unit size such as 64 KB, actual disk space used on the new server may exceed disk space used on the old server particularly if copying many files under 64 KB each.

Here’s an example output when running this script:

Migrate-Files-09Notes:

  • Robocopy copies NTFS permissions on files and folders assigned to domain users and groups. It will not copy permissions to files and folders assigned to local users or groups.
  • Robocopy is unaware of inherited NTFS permissions, and will not copy inherited permissions. This is why this script starts from the root of each drive letter to ensure that all NTFS permissions are captured.
  • This script is intended to run in the context of a domain user account with local administrative permissions on both the source and target servers.
  • Robocopy will copy files with paths longer than 256 characters
  • Depending on how much data is being copied, network speed, storage subsystem speed, and whether /IPG option is used with Robocopy to throttle down the copy process, this can take several hours, days, or weeks.

Robocopy options used:

  • /XJ :: eXclude Junction points. (normally included by default).
  • /R:0 :: number of Retries on failed copies. This is set to not retry failed files. Failed files will be addressed in next phases of the file migration.
  • /W:0 :: Wait time between retries: default is 30 seconds. Choosing no waiting here. Again will be addressed in next phases.
  • /IT :: Include Tweaked files (A Tweaked file is defined to be one that exists in both the source and destination, with identical size and time stamp, but different attribute settings).
  • /MIR :: MIRror a directory tree (equivalent to /E plus /PURGE).
  • /COPYALL :: COPY ALL file info (equivalent to /COPY:DATSOU).
  • /NP :: No Progress – don’t display percentage copied.
  • /TEE :: output to console window, as well as the log file.
  • /ZB :: use restartable mode; if access denied use Backup mode.
  • /LOG+:file :: output status to LOG file (append to existing log).

Since this will be run against production servers, you may want to consider the /IPG option:

/IPG:n :: Inter-Packet Gap (ms), to free bandwidth on slow lines.

For example, using /IPG:100 will slow down copying by waiting 100 ms between each packet to reduce the load on the source server. This will also make the migration process and file copy longer.

March 11, 2015 | Categories: Powershell, Windows 2003 Migration | Tags: , , | 5 Comments