1 0 Archive | PowerShell RSS feed for this section
post icon

Organizing your music with Powershell

So my wife just swapped her iPhone 4 for a new HTC Titan Windows Phone thanks to the recent Smoked by Windows Phone challenge.   The first thing she asked me about is how to get all of her music over to the new phone.  I’ve been using Zune Pass quite happily for a while however she has still continued to buy music from iTunes.   Her library, combined with the one that I had been amassing over the last several yeas before Zune Pass, meant that we have quite a bit of owned music.  Unfortunately like my pictures in the previous blog post thanks to multiple system backups, multiple computers, and the general fact that most of my files are in a sad state of unorganized I’m left with no consolidated music library.

Powershell to the rescue!!

I took the basic idea from the picture script I posted earlier and combined it with an open source DLL I found online which can query the metadata from a music file.  Grab the latest Windows package from http://download.banshee.fm/taglib-sharp/ and extract the taglib-sharp.dll file into the same folder as your powershell script.  

Here is the script –  it will remove duplicates and create a new folder with all of your music across multiple locations containing the unique files in the directory hierarchy ARTIST\ALBUM\TITLE.

#This is really divided into two scripts.   The first one searches for our audio files and creates and MD5 hash of each one. 
#Merging them all into the same folder allows us to eleminate duplicates.
#The second script reads the audio tags and puts them back out into folders based on artist, album, and title.


#Function to calculate the MD5 hash of a file
function Get-MD5([System.IO.FileInfo] $file = $(throw 'Usage: Get-MD5 [System.IO.FileInfo]'))
{
    # This Get-MD5 function sourced from:
    # http://blogs.msdn.com/powershell/archive/2006/04/25/583225.aspx
    $stream = $null;
    $cryptoServiceProvider = [System.Security.Cryptography.MD5CryptoServiceProvider];
    $hashAlgorithm = new-object $cryptoServiceProvider
    $stream = $file.OpenRead();
    $hashByteArray = $hashAlgorithm.ComputeHash($stream);
    $stream.Close();

    ## We have to be sure that we close the file stream if any exceptions are thrown.
    trap
    {
        if ($stream -ne $null) { $stream.Close(); }
        break;
    }

    $md5 = ""
    foreach ($byte in $hashByteArray)
      {
        $md5 = $md5 + $byte.ToString("X2");
      }
return $md5;
}

#Figure out where we are at and if there is a subfolder called output.  If not we will create one.   This is where we will put all of our images.
$currdir =  split-path -parent $MyInvocation.MyCommand.Definition
$outputdir = $currdir + "\output\"
if (!(Test-Path -path $outputdir))
{
    New-Item $outputdir -type directory
}

$tmpdir = $currdir + "\tmp\"
if (!(Test-Path -path $tmpdir))
{
    New-Item $tmpdir -type directory
}

#Using Get-ChildItem we search for all files matching our extension recurisvley from the location of the script down.
$files = Get-ChildItem -Exclude $outputdir -Recurse -Include *.m4a,*.mp3,*.wma

#We're going to keep track of how many files we process and put a unique number in the file for each one (eliminates all possibility of duplicate filename)
$i = 0;
foreach($f in $files) {
    #Increment our counter
    $i++;

    #Calcualte the MD5 of the original file so that we can look for duplicates later
    $md5 = Get-MD5($f);

    #The target filename will be the output directory with the MD5 hash as the filename and the original file extension
    $targetname = $tmpdir + $md5 + $f.extension;

    #Write the file to the output folder, if there are duplicate files Copy-Item's default behavior is to overwrite.   This eleminates the dupes.
    Copy-Item  $f.fullname $targetname

    #Could delete the old version, I'm leaving it as a backup so I've commented this out.
    #Remove-Item $f.fullname;    

    Write-Output $targetname

    #Write percent compelted of current operation
    $percent = [System.Math]::Round((($i / $files.Count)  * 100), 2)
    Write-Progress -Activity "Calculating hashes..."  -PercentComplete $percent -CurrentOperation "$percent% complete" -Status "Please wait."
}

#Now that we've zapped our duplicates we can move the files out to their target locations
#We loop through them again extracting the important bits from the audio tags
#Format will be /$artist/$album/$title.Extension

#Load the taglib library assuming the DLL is in the same folder (be sure to Unblock it).  
#You can get the DLL from http://download.banshee.fm/taglib-sharp/ -- I'm using 2.0.4.0 which was latest at time of writing this
[Reflection.Assembly]::LoadFile( (Resolve-Path ".\taglib-sharp.dll") )

#List of characters that we can't use
$invalid_characters = "[{0}]" -f ([Regex]::Escape([String][System.IO.Path]::GetInvalidPathChars()) + "/", "\", "*", "?", ":")  

$files = Get-ChildItem $tmpdir -Recurse
$i = 0;
foreach($f in $files) {
    $i++;

    #Load up the audio file into TagLib
    $audiofile = [TagLib.File]::Create($f.fullname);                

    if($audiofile.Tag.AlbumArtists) {
        $artist = [string] $audiofile.Tag.AlbumArtists
    } elseif ($audiofile.Tag.FirstArtist) {
        $artist = [string] $audiofile.Tag.FirstArtist
        $audiofile.Tag.AlbumArtists = $artist
        $audiofile.Save()
    } else {
        $artist = "Unknown"
    }

    if($audiofile.Tag.Album) {
        $album = [string] $audiofile.Tag.Album
    } else {
        $album = "Unknown"
    }

    if ($audiofile.Tag.Title) {
        $title = [string] $audiofile.Tag.Title
    } else {
        $title = "Unknown"
    }

    $artist = [string][Regex]::Replace($artist, $invalid_characters, '')
    $album = [string][Regex]::Replace($album, $invalid_characters, '')
    $title = [string][Regex]::Replace($title, $invalid_characters, '')

    #Where are we putting the new file?
    $targetname = $outputdir  + $artist + "\" + $album + "\" + $title  

    #Make sure that our folders exist (one for each month under the year) and if not create them
        if (!(Test-Path -path ($outputdir + $artist)))
        {
          $output =  New-Item ($outputdir + $artist) -type directory
        }

        if (!(Test-Path -path ($outputdir + $artist + "\" + $album)))
        {
          $output =  New-Item ($outputdir + $artist + "\" + $album) -type directory
        }

        $dupe = $true
        $x = 1
        While($dupe) {
            $x++
            if (!(Test-Path -Path ($targetname + $f.extension)))
            {
                $targetname = $targetname + $f.extension
                $dupe = $false
            } elseif (!(Test-Path -Path ($targetname + "-" + $x + $f.extension)))
            {
                $targetname = $targetname + "-" + $x + $f.extension
                $dupe = $false
            }
        }

        #Move the source file to it's new home. 
        Move-Item -Path $f.fullname -Destination $targetname
        Write-Output $targetname

        #Write percent compelted of current operation
        $percent = (($i / $files.Count)  * 100)
        Write-Progress -Activity "Moving files..."  -PercentComplete $percent -CurrentOperation "$percent% complete" -Status "Please wait."
    }
Leave a Comment
April 27, 2012
post icon

Organizing lots of pictures

If you’re anything like me you’ve been taking digital pictures for a long time now.   You’ve used various strategies and tools for organizing them over the years.   You’ve gone through several computers and moved the files around countless times.   Where does that leave you?  With an unorganized mess.

With library software like Picasa you can import all of those pictures and you can have some semblance of organization by way of the user interface but it doesn’t really solve the problem at the core.   The files are a mess.

I wanted to find a way to give a consistent filename to all of my pictures, organize them into folders based on the month and year they were taken, and remove duplicates.   At first it sounded like a tall order as I couldn’t find any off the shelf tools to do this.  Thankfully with just a little bit of time in Powershell I was able to put together a script that accomplished this for me.

 

The script does the following:

    • Identify all of the existing pictures
    • Query the EXIF data
    • Calculate an MD5 has of the file
    • Create a new copy based on the data in a “staging” folder
    • Parse the new file name to get the MD5 and look for duplicates
    • Delete the duplicate version
    • Move the files to the YYYY\MM folders

 

The first task of querying the EXIF data of files was actually the hardest.   I found a couple of blog articles that touched on this.   The synopsis is that you have to use the .NET System.Drawing DLL.   We can use Get-ChildItem to recurse a directory structure looking for files of a specific type.   For each file that we find we instantiate a new Bitmap object which contains the EXIF properties.   We can extract and update these.   I thought it might be useful to store the original path as a “Comment” in the EXIF in case it contained some relevant information that I later want to turn into a tag.

Below is the complete script.

#Load the .net System.Drawing assembly for examining the EXIF data of the pictures.
[reflection.assembly]::loadfile( "C:\Windows\Microsoft.NET\Framework\v2.0.50727\System.Drawing.dll")

#Function to calculate the MD5 hash of a file
function Get-MD5([System.IO.FileInfo] $file = $(throw 'Usage: Get-MD5 [System.IO.FileInfo]'))
{
    # This Get-MD5 function sourced from:
    # http://blogs.msdn.com/powershell/archive/2006/04/25/583225.aspx
    $stream = $null;
    $cryptoServiceProvider = [System.Security.Cryptography.MD5CryptoServiceProvider];
    $hashAlgorithm = new-object $cryptoServiceProvider
    $stream = $file.OpenRead();
    $hashByteArray = $hashAlgorithm.ComputeHash($stream);
    $stream.Close();

    ## We have to be sure that we close the file stream if any exceptions are thrown.
    trap
    {
        if ($stream -ne $null) { $stream.Close(); }
        break;
    }

    $md5 = ""
    foreach ($byte in $hashByteArray)
      {
        $md5 = $md5 + $byte.ToString("X2");
      }
return $md5;
}

#Figure out where we are at and if there is a subfolder called output.  If not we will create one.   This is where we will put all of our images.
$currdir =  split-path -parent $MyInvocation.MyCommand.Definition
$outputdir = $currdir + "\output\"

if (!(Test-Path -path $outputdir))
{
    New-Item $outputdir -type directory
}

#What files should we look for?   Typically this would be *.jpg.
$ext = "*.jpg"

#Using Get-ChildItem we search for all files matching our extension recurisvley from the location of the script down.
$files = Get-ChildItem -r -Include $ext

#We're going to keep track of how many files we process and put a unique number in the file for each one (eliminates all possibility of duplicate filename)
$i = 0;
foreach($f in $files) {
    #Increment our counter
    $i++;

    #Load up the .net system.drawing.bitmap object for the current file.  We will use this to access and update the exif data.
    $img=New-Object -TypeName system.drawing.bitmap -ArgumentList $f.fullname;

    #We grab up the camera date, height, and width.  We use try catch in case the property isn't availabe and set a default value.
    #For more details on properties available check:
    #http://blogs.technet.com/b/jamesone/archive/2007/07/13/exploring-photographic-exif-data-using-powershell-of-course.aspx

    Try
     {
        #The value is a byte array which we need to convert (assuming ASCII character set)
        $date = [System.Text.Encoding]::ASCII.GetString($img.GetPropertyItem(36867).Value);
     }
    Catch [system.exception]
     {
        #Default value in case we can't access the EXIF
        $date = "0000:00:00 00:00:00";
      }

    #Grab the height and width of our object
    $height = $img.Height;
    $width = $img.Width; 

    #The date is returned as a null terminated string with spaces and colons in it.   We replace all of these with dashes to make it filename friendly.
    $date = (($date.Replace("`0", "")).Replace(" ","-")).Replace(":","-");

    #Calcualte the MD5 of the original file so that we can look for duplicates later
    $md5 = Get-MD5($f);

    #The target filename will be the output directory with the variables concatnated in the below format.
    #Format will be YYYY-MM-DD-HH-MM-SS-WIDTHxHEIGHT-MD5-ID.Extension
    $filename = $outputdir + [string]::Format("{0}-{1}x{2}-{3}-{4}{5}", $date, $width, $height, $md5, $i, $f.extension);

    #We want to save the current path as a comment so we create a new property of type 40092 (comment)
    $property = $img.PropertyItems[0];
    $property.Id = 40092;
    $property.Type = 1;

    #It needs a string array so we pass in the current path ($f.fullname) and convert it to an array
    $property.Value = [system.text.encoding]::Unicode.GetBytes($f.fullname + ":" + $md5);
    $property.Len = $property.Value.Count;
    $img.SetPropertyItem($property);

    #We will save our image from memory in the path of our new file.
    $img.Save($filename);
    $img.Dispose();

    #Could delete the old version, I'm leaving it as a backup so I've commented this out.
    #Remove-Item $f.fullname;

    #Let the user know what the current status is and which files are being moved.    
    Write-Output "Copying $f to $filename";
}

#This is a one liner to split on filename, find the duplicates by MD5, ignore the first result and then delete the rest
#The 7th item in the filename format is the MD5 hence the hard coded index
$o = Get-ChildItem $outputdir `
| Select-Object @{Name="MD5";Expression={($_.Name).Split("-")[7]}}, @{Name="Filename";Expression={$_.Fullname}} `
| Group-Object md5 `
| ?{ $_.Count -gt 1 } `
| % {($null, $rest) = $_.Group; $rest;} `
| Select-Object Filename  `
| % { Write-Output "Removing duplicate $_"; Remove-Item $_.Filename }

Write-Output $o

#Now that we've zapped our duplicates we can move the files out to their target locations
#We loop through them again extracting the important bits from the filename
#Format will be YYYY-MM-DD-HH-MM-SS-WIDTHxHEIGHT-ID.Extension
$files = Get-ChildItem $outputdir
$i = 0;
foreach($f in $files) {
    $i++;

    #split string on dash and create an array of attributes
    $name = ($f.name).split("-");
    $year = $name[0];
    $month  = $name[1];
    $day = $name[2];
    $hour = $name[3];
    $min = $name[4];
    $sec = $name[5];
    $size = $name[6];
    $md5 = $name[7];

    #Where are we putting the new file?
    $targetname = $outputdir  + $year + "\" + $month + "\" + [string]::Format("{0}-{1}-{2}-{3}-{4}-{5}-{6}-{7}{8}", $year, $month, $day, $hour, $min, $sec, $size, $i, $f.extension);

    #Make sure that our folders exist (one for each month under the year) and if not create them
    if (!(Test-Path -path ($outputdir + $year)))
    {
        New-Item ($outputdir + $year) -type directory
    }

    if (!(Test-Path -path ($outputdir + $year + "\" + $month)))
    {
        New-Item ($outputdir + $year + "\" + $month) -type directory
    }

    #Move the source file to it's new home.  -Force tells it to overwrite if the file already exists.
    Write-Output "Moving $f to $targetname";
    Move-Item -Path $f.fullname -Destination $targetname -Force
}

 

 

Here are a couple of references that I found helpful in building this script:

 

http://blogs.technet.com/b/jamesone/archive/2007/07/13/exploring-photographic-exif-data-using-powershell-of-course.aspx

http://blog.codeassassin.com/2007/10/13/find-duplicate-files-with-powershell/

Leave a Comment
February 3, 2012
post icon

Disable email notifications in SharePoint 2010

Out of the box in SharePoint 2010 all users who have a profile are also defaulted to having email notifications set to “on”.   This might make sense in a small scale implementation but for my customer that was unacceptable.  I created a powershell script that iterates through the user profiles and turns the notifications off.  This script is similar to ones I’ve posted before for working with the user profile however it uses different fields.  The SharePoint field is called SPS-EmailOptin. This field disables both SharePoint and NewsGator emails.

Of course in future releases these field names might change so you should verify before using this script.  In my case this was SharePoint 2010 October CU and NewsGator Social Sites 1.2.2419.

#Load the SharePoint snap-in
Add-PsSnapin Microsoft.SharePoint.PowerShell;

#Load the SharePoint assemblies
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server");
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server.UserProfiles");

#Specify the MySite URL
$MySiteUrl = "http://sharepoint.vallery.net/";

#Get the server context for the profile manager
$site = Get-SPSite $MySiteUrl;
$ServerContext = Get-SPServiceContext $site;
$UPManager = new-object Microsoft.Office.Server.UserProfiles.UserProfileManager($ServerContext);

#Count variables
$ucount = 0;

$enumProfiles = $UPManager.GetEnumerator();
"Total User Profiles available:" + $UPManager.Count
$count=0;

#Loop through the profile entries and update the property
#Recieve Instant Notifications - NGAllowMetaEmail (bool)
#24 Hour Digest Email - NGReceiveDigestEmail (bool)
#RSS NewsFeed Email - NGAllowRssEmail (bool)
#SharePoint Notification emails - SPS-EmailOptin (int)
#This field has 3 values one for each email type

foreach ($oUser in $enumProfiles)
{
    $count = $count + 1;
    $u = $oUser.Item("Accountname");
    Write-Output "($count):  Setting values for $u";

    $oUser["NGAllowMetaEmail"].Value = $false;
    $oUser["NGReceiveDigestEmail"].Value = $false;
    $oUser["NGAllowRssEmail"].Value = $false;
    $oUser["SPS-EmailOptin"].Value = 111; 

    $oUser.Commit();
} 

#Dispose of site object
$site.Dispose();
Leave a Comment
January 13, 2011
post icon

Enumerating user profile property fields in SharePoint 2010 with PowerShell

I’ve been working quite a bit with SharePoint 2010 lately and have written a number of PowerShell scripts that I think will be useful to folks. This is the first of these.

This script connects to the User Profile managed service application and iterates through all of the properties that have been configured dumping the result to XML. The script additionally pulls in any mappings to active directory.

I’m currently working on a script that will import this XML and update the properties accordingly. I hope to post that soon as well.

#Define our configuration. This is the name you gave the import connection to AD
$url = "http://sharepoint.vallery.net/";
$connectionName = "Profile Sync";

#Setup our SharePoint objects
$site = Get-SPSite $url;
$serviceContext = Get-SPServiceContext($site);
$upManager = new-object Microsoft.Office.Server.UserProfiles.UserProfileConfigManager($serviceContext);
$syncConnection = $upManager.ConnectionManager[$connectionName];

#This is a collection of mappings to AD that we will use later 
$pmc = $syncConnection.PropertyMapping;

#This is a collection of all of the properties which we will iterate
$properties = $upManager.GetProperties();

# Create a new XML writer settings object 
$settings = New-Object system.Xml.XmlWriterSettings;
$settings.Indent = $true;
$settings.OmitXmlDeclaration = $false;
$settings.NewLineOnAttributes = $true;

# Create a new string writer to capture the output 
$sw = new-object System.IO.StringWriter;

# Create a new XmlWriter 
$writer = [system.xml.XmlWriter]::Create($sw, $settings); 

#Start the document and add the root node
$writer.WriteStartDocument();
$writer.WriteStartElement("properties");

#Iterate through the properties
foreach ($item in $properties);
{

    #Create the property element
    $writer.WriteStartElement("property");

    #Add in the fields as attributes
    $writer.WriteAttributeString("Name", $item.Name);
    $writer.WriteAttributeString("DisplayName",$item.DisplayName);
    $writer.WriteAttributeString("ManagedPropertyName",$item.ManagedPropertyName);
    $writer.WriteAttributeString("Type",$item.Type);
    $writer.WriteAttributeString("ChoiceList",$item.ChoiceList);
    $writer.WriteAttributeString("Description",$item.Description);
    $writer.WriteAttributeString("URI",$item.URI);
    $writer.WriteAttributeString("IsSystem",$item.IsSystem);
    $writer.WriteAttributeString("AllowPolicyOverride",$item.AllowPolicyOverride);
    $writer.WriteAttributeString("IsUserEditable",$item.IsUserEditable);
    $writer.WriteAttributeString("IsAdminEditable",$item.IsAdminEditable);
    $writer.WriteAttributeString("IsImported",$item.IsImported);
    $writer.WriteAttributeString("Length",$item.Length);
    $writer.WriteAttributeString("IsMultivalued",$item.IsMultivalued);
    $writer.WriteAttributeString("ChoiceType",$item.ChoiceType);
    $writer.WriteAttributeString("DefaultPrivacy",$item.DefaultPrivacy);
    $writer.WriteAttributeString("UserOverridePrivacy",$item.UserOverridePrivacy);
    $writer.WriteAttributeString("IsReplicable",$item.IsReplicable);
    $writer.WriteAttributeString("PrivacyPolicy",$item.PrivacyPolicy);
    $writer.WriteAttributeString("DisplayOrder",$item.DisplayOrder);
    $writer.WriteAttributeString("IsColleagueEventLog",$item.IsColleagueEventLog);
    $writer.WriteAttributeString("IsAlias",$item.IsAlias);
    $writer.WriteAttributeString("IsSearchable",$item.IsSearchable);
    $writer.WriteAttributeString("IsUpgrade",$item.IsUpgrade);
    $writer.WriteAttributeString("IsUpgradePrivate",$item.IsUpgradePrivate);
    $writer.WriteAttributeString("IsVisibleOnEditor",$item.IsVisibleOnEditor);
    $writer.WriteAttributeString("IsVisibleOnViewer",$item.IsVisibleOnViewer);
    $writer.WriteAttributeString("IsTaxonomic",$item.IsTaxonomic);
    $writer.WriteAttributeString("Separator",$item.Separator);
    $writer.WriteAttributeString("MaximumShown",$item.MaximumShown);
    $writer.WriteAttributeString("IsSection",$item.IsSection);
    $writer.WriteAttributeString("IsRequired",$item.IsRequired);
    $writer.WriteAttributeString("SubtypeName",$item.SubtypeName);

    #Look up any AD mappings in the PropertyManagerCollection and include them
    $writer.WriteAttributeString("IsImport",$pmc.Item($item.Name).IsImport);
    $writer.WriteAttributeString("IsExport",$pmc.Item($item.Name).IsExport);
    $writer.WriteAttributeString("DataSourcePropertyName",$pmc.Item($item.Name).DataSourcePropertyName);
    $writer.WriteAttributeString("OriginalDataSourcePropertyName",$pmc.Item($item.Name).OriginalDataSourcePropertyName);
    $writer.WriteAttributeString("AssociationName",$pmc.Item($item.Name).AssociationName);
    $writer.WriteAttributeString("Connection",$pmc.Item($item.Name).Connection.DisplayName);
    $writer.WriteEndElement(); 

}

#Finish up
$writer.WriteEndElement();
$writer.WriteEndDocument();
$writer.Flush();
$writer.Close(); 

#Capture the output into a string
$result = $sw.ToString();

# Write the XML out
Write-Output $result;

And here is an example of the XML output:

Leave a Comment
November 30, 2010