Faster filtering in PowerShell than using Where-Object

In a project I'm using a PowerShell script to read in a lot of .csv files and then do some lookups between these in order to get the wanted output. This all works well, but has been a bit slow lately due to a lot more data in the files.

I narrowed the speed issue down to a few lines where I iterate over some files in order to find a few rows that I need. Basically, the Where-Object cmdlet is the culprit.

A simplified example is below:

$mycsvfile = Import-Csv .\mydata.csv
$dataiwant = $mycsvfile | Where-Object {$_.idno -eq 5}

I had a few similar lines in the script, making the time to select the data add upp to 9 seconds, which was far to long in this case.

I found this post that show a few variants on filtering collections, and by simply using the PS v4 .Where() notation to do the same thing, I could bring this down to a single second.

$mycsvfile = Import-Csv .\mydata.csv
$dataiwant = $mycsvfile.Where({$_.idno -eq 5})

So lesson learned: PowerShell is evolving quickly and what I thought was a nice way to do something might very well be just fine, but there might also be a quicker way just around the corner. In this example, I'm sacrificing the streaming capabilities, but gain a lot of performance, just by changing a few characters.

"Clever" uninstall of msi packages/applications using PowerShell

When I created my own PowerShell script library for BizTalk deployment automation I ran across the need to uninstall applications, both BizTalk applications and non-BizTalk ones, by only knowing their name.

At first, I solved it using a WMI query like so:

$name = "application name"

$product = Get-WmiObject -Class Win32_Product -Filter "name='$name'" -ComputerName "localhost"

[void]$product.Uninstall()

This is not a recommended way of doing it though. It is both very slow and also causes a bit of spamming in the Windows Eventlog since the query in fact does a reconfigure of ALL applications installed. This reconfiguration can also cause a bit of other issues in some cases.

I then created another way of trying to uninstall applications in a more failsafe and secure way by using msiexec with the uninstall flag. The tricky part was to find a way to get the product key in order to be able to use msiexec since it requires this for uninstalling an application. The result can be found below.

The script function will take the application name as argument. It will then via the registry (note that this is configured for x64, so change the path if you are running x86) look up the application settings. If the application can be found via name (it should), we extract the product key. Then this is used as a parameter to msiexec.

If the product key cannot be found (it happens), we will instead try to read the uninstall string that is set when installing the application. Windows will run this string when you choose to uninstall an application, so why do not we use it? If found, we extract the product key and do our msiexec call.

If all fail, we throw an exception to be caught in the real part of the script.

This is the full script function:

Function Uninstall-Program([string]$name)
{
    $success = $false

    # Read installation information from the registry
    $registryLocation = Get-ChildItem "HKLM:\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\"
    foreach ($registryItem in $registryLocation)
    {
        # If we get a match on the application name
        if ((Get-itemproperty $registryItem.PSPath).DisplayName -eq $name)
        {
            # Get the product code if possible
            $productCode = (Get-itemproperty $registryItem.PSPath).ProductCode
           
            # If a product code is available, uninstall using it
            if ([string]::IsNullOrEmpty($productCode) -eq $false)
            {
                Write-Host "Uninstalling $name, ProductCode:$code"
           
                $args="/uninstall $code"

                [diagnostics.process]::start("msiexec", $args).WaitForExit()
               
                $success = $true
            }
            # If there is no product code, try to read the uninstall string
            else
            {
                $uninstallString = (Get-itemproperty $registryItem.PSPath).UninstallString
               
                if ([string]::IsNullOrEmpty($uninstallString) -eq $false)
                {
                    # Grab the product key and create an argument string
                    $match = [RegEx]::Match($uninstallString, "{.*?}")
                    $args = "/x $($match.Value) /qb"

                    [diagnostics.process]::start("msiexec", $args).WaitForExit()
                   
                    $success = $true
                }
                else { throw "Unable to uninstall $name" }
            }
        }
    }
   
    if ($success -eq $false)
    { throw "Unable to find application $name" }
}

Using PowerShell to remove all old files in a folder structure

A common task when working with BizTalk installations is to clean out old data from folder structures. May it be old log files, automatically saved messages from a send port, database backups or any other sort of data that is obsolete after a certain period of time.

When I started working with BizTalk, the norm was to use bat files or VB Script. These can still be seen in many places since they a) work just fine, and b) the developers/administrators hasn't learned a new way to script these kind of things, mostly because a) they work just fine.

I myself as mentioned earlier prefer to use PowerShell nowadays.

In order to recursivly delete old files and folders, one line of PowerShell script is all that is needed:

get-childitem -path 'c:\temp\test' -recurse | where -FilterScript {$_.LastWriteTime -le [System.DateTime]::Now.AddDays(-30)} | Remove-Item -recurse -force

This script simply traverses the specified folder (why not change it to take the folder and date as a parameter) and does so recursivly. All files older than 30 days in this case is then deleted. By using the -force parameter, locked and hidden files will not cause an error.

In order to have this run automatically on a server, it needs some adjustments. Add a Scheduled Task and set the Program/script to run to Powershell and then add the arguments as so:

-noninteractive -command "& {get-childitem -path 'c:\temp\test' -recurse | where -FilterScript {$_.LastWriteTime -le [System.DateTime]::Now.AddDays(-30)} | Remove-Item -recurse -force}"

Set the security options for the task to "Run whether user is logged on or not" and also check the "Run with highest privileges" checkbox.

Why I created my own PowerShell scripts for BizTalk deployment

Most of us working with BizTalk has at one point or another started scripting our deployments instead of doing them manually. I have during the years seen many variants of automating the deployment process. At one place the "old" way of using bat-files calling BTSTASK where used extensively. At another, a custom .Net program with a GUI was developed that would call the different API:s available. I myself resorted to using PowerShell to create a fully automatic deployment routine.

My scripts has two parts. One with different functions that I wrote and one part that encapsulates the functions and executes them in the correct order. As input I create an XML file that specifies which .msi packages that are to be deployed, which applications to remove/update/add, settings for the IIS virtual directories and so on. Since it is quite common during complicated deployments that something will err, I added functions to test run the scripts in advance. The test run simply checks for dependencies in the BizTalk configuration, making sure that applications can be modified as configured. It also checks that all files needed are provided in the setup package.

When creating the scripts, I briefly looked at using the BizTalk Deployment Framework and other extensions to PowerShell, but opted on developing it all in my own scripts. This due to several reasons, each quite strong in my case:

• I would learn more about PowerShell as well as the API:s and tools found in BizTalk
• I would know exactly where things could go wrong and what to do about it.
• It allowed me a greater flexibility since nothing was hidden in dll files or needed installation. As long as PowerShell was installed on the servers, the scripts would work. Nothing else is needed. This is also something that administrators liked. Some of them are a bit weary about installing "some stuff I found on the net". 

This post is the theoretical background to posts that are to follow that will showcase and explain my scripts in individual blocks. I have already started in advance with the post Recycle IIS application pools using PowerShell, and more snippets are to come.

Script updates of contacts in Outlook

I wanted to have all of my colleagues contact information in my iPhone so I thought of the simplest way to handle it. I quickly decided that the best way would be to simple add the appropriate domain accounts as contacts in Outlook which in a few seconds work gave me about 240 new contacts with the correct information entered as they appear in our Active Directory.

I then realised that they all had the internal speed dial phone number entered as their work number. This is a four digit number which is the same as the last four digits in the externally used number. Hence I needed a simple way to automatically add a prefix to the number for each contact, but only those that had four digits in the field since my address book is filled with external contacts as well.
After fiddling briefly with macros, csv-files and other means to get this to work, I had an epiphany and thought of PowerShell.

Three rows of PowerShell code written, tested and executed in a few minutes solved my problem

$outlook = new-object -com outlook.application
$contacts = $outlook.Session.GetDefaultFolder(10)
$contacts.Items | % { if($_.BusinessTelephoneNumber.Length -eq 4) { $_.BusinessTelephoneNumber = "123-45" + $_.BusinessTelephoneNumber; $_.save() } }

Using PowerShell to automatically update lots and lots of data will most likely be the first I think of the next time a similar task appears. This was almost ridiculously easy.

Find all possible parameters for an MSI package installation using msiexec

While working on a library of powershell scripts to do unattended installations of BizTalk applications (and all adjacent files and packages) I needed to find out how to specify the settings for an MSI package in order to do a complete unattended install of it using msiexec.exe.

The MSI I was working with was a setup package for a WCF service. Since this installs to the IIS, both website, virtual directory as well as application pool is needed to be specified during the installation. The question is, what are the correct parameter switches for setting these?

Simple enough, these can be found by doing an install of the MSI and logging a verbose output to file. First, run msiexec with logging enabled:

msiexec /I package.msi /L*V installationlog.txt

Then look in the logfile for the text PROPERTY CHANGE. In the following example, the virtual directory is set using the property TARGETVDIR which then also can be used as a parameter to the msiexec command to set the property from outside the GUI:

Action start 15:41:42: WEBCA_TARGETVDIR.
MSI (c) (F4:8C) [15:41:42:943]: Note: 1: 2235 2: 3: ExtendedType 4: SELECT `Action`,`Type`,`Source`,`Target`, NULL, `ExtendedType` FROM `CustomAction` WHERE `Action` = 'WEBCA_TARGETVDIR'
MSI (c) (F4:8C) [15:41:42:943]: PROPERTY CHANGE: Adding TARGETVDIR property. Its value is 'MyWcfServiceLibrary'.
Action ended 15:41:42: WEBCA_TARGETVDIR. Return value 1.
MSI (c) (F4:8C) [15:41:42:943]: Doing action: WEBCA_SetTARGETSITE
Action 15:41:42: WEBCA_SetTARGETSITE.
Action start 15:41:42: WEBCA_SetTARGETSITE.

Note that the custom parameters are not to be set as normal switches with a leading slash /. In my case, the command will look like this:

msiexec /I package.msi /qb TARGETSITE="/LM/W3SVC/1" TARGET VDIR="MyWCFLibrary" TARGETAPPPOOL="BtsAppPoolC"

This will do a complete unattended install of the WCF service to IIS with basic UI and set the needed properties to my preferred values instead of the defaults.