Using PowerShell to remove all old files in a folder structure

A common task when working with BizTalk installations is to clean out old data from folder structures. May it be old log files, automatically saved messages from a send port, database backups or any other sort of data that is obsolete after a certain period of time.

When I started working with BizTalk, the norm was to use bat files or VB Script. These can still be seen in many places since they a) work just fine, and b) the developers/administrators hasn't learned a new way to script these kind of things, mostly because a) they work just fine.

I myself as mentioned earlier prefer to use PowerShell nowadays.

In order to recursivly delete old files and folders, one line of PowerShell script is all that is needed:

get-childitem -path 'c:\temp\test' -recurse | where -FilterScript {$_.LastWriteTime -le [System.DateTime]::Now.AddDays(-30)} | Remove-Item -recurse -force

This script simply traverses the specified folder (why not change it to take the folder and date as a parameter) and does so recursivly. All files older than 30 days in this case is then deleted. By using the -force parameter, locked and hidden files will not cause an error.

In order to have this run automatically on a server, it needs some adjustments. Add a Scheduled Task and set the Program/script to run to Powershell and then add the arguments as so:

-noninteractive -command "& {get-childitem -path 'c:\temp\test' -recurse | where -FilterScript {$_.LastWriteTime -le [System.DateTime]::Now.AddDays(-30)} | Remove-Item -recurse -force}"

Set the security options for the task to "Run whether user is logged on or not" and also check the "Run with highest privileges" checkbox.

Why I created my own PowerShell scripts for BizTalk deployment

Most of us working with BizTalk has at one point or another started scripting our deployments instead of doing them manually. I have during the years seen many variants of automating the deployment process. At one place the "old" way of using bat-files calling BTSTASK where used extensively. At another, a custom .Net program with a GUI was developed that would call the different API:s available. I myself resorted to using PowerShell to create a fully automatic deployment routine.

My scripts has two parts. One with different functions that I wrote and one part that encapsulates the functions and executes them in the correct order. As input I create an XML file that specifies which .msi packages that are to be deployed, which applications to remove/update/add, settings for the IIS virtual directories and so on. Since it is quite common during complicated deployments that something will err, I added functions to test run the scripts in advance. The test run simply checks for dependencies in the BizTalk configuration, making sure that applications can be modified as configured. It also checks that all files needed are provided in the setup package.

When creating the scripts, I briefly looked at using the BizTalk Deployment Framework and other extensions to PowerShell, but opted on developing it all in my own scripts. This due to several reasons, each quite strong in my case:

• I would learn more about PowerShell as well as the API:s and tools found in BizTalk
• I would know exactly where things could go wrong and what to do about it.
• It allowed me a greater flexibility since nothing was hidden in dll files or needed installation. As long as PowerShell was installed on the servers, the scripts would work. Nothing else is needed. This is also something that administrators liked. Some of them are a bit weary about installing "some stuff I found on the net". 

This post is the theoretical background to posts that are to follow that will showcase and explain my scripts in individual blocks. I have already started in advance with the post Recycle IIS application pools using PowerShell, and more snippets are to come.

BizTalk Map Viewer: Graphical interface for the BizTalk Map Documenter

Quite a while ago during an assignment we needed a way to display the logic in BizTalk maps for developers that didn't have BizTalk installed on their workstations. We quickly turned to the BizTalk Map Documenter which simply is an XSLT file built to take a standard BizTalk map file (.btm) and output it in a human readable format as an HTML page.

In order to use the XSLT file you need to execute it, preferably with the utility msxsl.exe which can take the .btm file and XSLT file as arguments and then output the result to be viewed by the user. However, this caused a bit too much command line hacking in our case and I took an evening in the hotel bar to quickly whip together a GUI for it and thus, the BizTalk Map Viewer was born.

The application is very simple and in it's basic form simply allows you to open a BizTalk map file which then will be parsed by the BizTalk Map Documenter XSLT file and the result will be shown in the application window. I added functionality to save, print and copy the result. I also added functionality to register the application as a valid right-click option for .btm files. If enabled, you will get the option to "Open with BizTalk Map Viewer" when right-clicking on BizTalk map files which then will open the selected file in the application. Just be sure to have placed the application executable in a good folder before registering it since the current path will be used.

Feel free to download the code and executable.

Use it as you please.

A take on documenting integration platforms using Visio

Earlier this year I wrote a post about having Visio update a field on a shape automatically when editing any other field, making it possible to always know when the metadata was last changed. In the comments I was asked to share the Visio sheet which it now is time to do. This is the result of that request, in some way documenting my attempt to use Visio to document an integration platform. Hopefully you can gain some ideas to use in your documentation endeavours.


In almost all of my encounters of different integration platforms at various companies, there has been a high level view of the entire integration setup showing the various message flows available. This map has usually been constructed in Visio. There has also been two types of layouts, lets call them the structured layout and the unstructured layout.

The following is an example of the unstructured layout. It's common and does a fairly good job at describing the full environment. It is often seen when there has not been a true approach to working with integrations.

Most of the time they are filled with objects, with varying descriptions, showing the entire network.You can see that two systems interact, but you do not know in which way. It is also often hard to see how everything is connected if you as in the example has an integration hub in the middle, doing some message routing.

Hence we get the structured layout as is seen below. In this case, there is a very solid structure with the integration platform in the middle, and the integrated systems on both sides with arrows showing each and every message flowing through.

These maps are good at showing the message flows as every message is entered, but it doesn't give you the overall picture that the unstructured layout gives you since a lot of fluff is stripped away. Usually there has also been an improvement from the unstructured layout in that each message flow has been given an identity which then maps to more detailed documentation in a file archive or document portal.


So, earlier this year I did a test with a variant of the structured layout, seeing if I could document the current state of integrations at my current job. The map I was given from the beginning was according to the unstructured layout. A lot of information was out of date, systems had been taken out, others taken in, versions changed, message flows altered and so on. I had to create an up to date picture of the integrations, so that I could commence working with them.

I took a brand new Visio document and started documenting. I created a "system" shape that had the following shape data:

• System name
• System name 2
• System version
• Server name
• Database
• Database version

In my Swedish Visio it looks like this:

In order to make it easier updating and changing these properties/metadata for all shapes, I changed the Shape Sheet to make the form data window open when doubleclicking on the shape. You can do this by editing the Shape Sheet property "EventDblClick" under "Events" to read "=DOCMD(1312)".

I then did the same thing for the arrow shape adding metadata that I want to know for each message:

• ID
• Object name
• Status (i.e. Planned, Under development, In use, To be removed)
• Transport (i.e. Web service, File, FTP, WCF...)
• Type (i.e. Xml, Flat file...)
• Frequency
• Automatic (i.e. a drop down giving the option of Yes or No, indicating whether the message transfer is completely automatic)
• Sync/Async
• One way/Request-response
• From system
• To system
• Comment

Many of the fields are defined in the form data editor as dropdown menus as to make the data more consistent and easier to edit. I also added the "From system" and "To system" in order to get good documentation automatically created by exporting the form data to Excel. As with the System shapes, I added the doubleclick event to the shape.

I also altered the shape so that the choices made in the form data will be visible in the shape. Depending on the "Status", the arrow will be yellow (planned), green (under development), black (in use) or red (to be removed). This is done by editing the Data Graphic and setting that each value will match a specific color.

Also, setting the "Automatic" to "No" will show a little man besides the ID and Object name. This can be set in the Shape Sheet as well under "Text fields" and "Value". In my case, the formula is: =IF(STRSAME(Prop.Automatic,"No",TRUE)," 웃","")&" "&Prop.ID&". "&Prop.Objectname&" ". But is of course dependent on the name you give each property in the form data.

In the same manner, I have a setting that will change the arrows on the shape so that I get an indication if it is a one way transfer or a request-response one, by adding a little unfilled arrow in the other direction of the data flow.

I have also adjusted where the text appears on the connector according to this post from the Visio Guy, so that the text is always at the end of the line instead of in the middle.

So, is this a good way of documenting message flows?

Well, in this case, it has been very helpful. I have a single document that shows me quite a lot of the current setup and state. It eases on the job of manually editing shapes to indicate whether it is a planned message flow, if it is request-response, a manual job and so on. Instead, I just select the desired property and let Visio format the shape accordingly, giving me a neat view of the state.

After documenting all message flows in this Visio document, I will most likely strip it down. I will remove a lot of the metadata in the connection shapes, and have this solely in the integration specification document that I will create for each message. I do not like to have duplicate information in my documentation, and I feel that that sort of data will belong better in the written documentation than in the Visio sheet. I will still keep the color coding depending on Status, as well as some other visual cues that is helpful when looking at the overall view of the integration platform.

Book review: (MCTS): Microsoft BizTalk Server 2010 (70-595) Certification Guide

I got the opportunity to review the book (MCTS): Microsoft BizTalk Server 2010 (70-595) Certification Guide published by Packt Publishing, a company that is pretty well renowned for their good collection of IT books. The book itself is written by Johan Hedberg, Kent Weare and Morten la Cour.

I received the book just before the weekend and spent the spare time over the next few days reading it through and checking out the exercises that accompanies the book.

As the book both exclaims on the front as well as the writers themselves add in the first chapter, this book is a guide to help you study and prepare for the latest BizTalk exam called 70-595, TS: Developing Business Process and Integration Solutions by Using Microsoft BizTalk Server 2010 which I wrote about last year, but still have left to take due to too little time. Since the prerequisites for the exam is that you should have at least two years of experience of working with BizTalk solutions, the book aims to be a quick guide to many areas of BizTalk development making the focus the parts you need to be able to complete the test.

The book is structured accoring to the areas covered by the exam in order to give you a better chance of focusing on the areas you lack expertise in as well as knowing the structure on the test. Chapter 1 through 8 will cover the technical bits of BizTalk, and does a very good job at this. Each chapter will end with a few questions making sure that some vital parts of the chapter has stuck, as common with IT books. These chapters reflects the major topics that the exam is composed of, making it a breeze to know which areas you need to study more thoroughly.

When starting reading the book, I immediately got the feeling that the writers didn't quite stick to the view of the readers having more than two years of BizTalk development and hence not having to run through simple tasks that you learn during the first hours of a basic BizTalk course. Pretty much the entirety of chapter one is like this, and parts of the other chapters as well. But that is a minor fluke since most of the book contains just the right amount of information, or maybe a tad to little. Just as Mikael Sand mentions in his review, there are parts of the book that will tell you how to do things, but not why. I also have to add that there are parts that while describing a satisfactory way to solve the problem, it doesn't adhere to best practice. In other places, the writers will push you in the direction of complying with the established non-formal best practice in a very good way that is seldom seen in books of this type (SOA Patterns With BizTalk Server 2009 by Richard Seroter is one of few exceptions).

After the technical chapters, you get chapter 9 and 10 which gives you a more hands-on preparation of the actual test. Chapter 9 runs through tips on how to study for the test as well as how the actual test will take place. Chapter 10 is a collection of more practice questions that will mimic the real ones.

Now, bear in mind that I have yet to take the 70-595 test. I have however taken the 2006 version (70-235) and can say that based on my knowledge from that test as well as my knowledge from BizTalk development and architecture, this book covers the major parts in a very good way. I believe myself that a good way of studying for the test would be this book, reading it through, having a BizTalk setup to do labs and practical work in, and then of course a browser with MSDN/Technet available for all that in-depth knowledge hunting that I think is a must. It's simply just not good enough to know how you do something, you also should know why you should do it that way, and in which way that will affect your platform and solution. The book does a pretty good job at this by every now and then redirecting the reader to MSDN for more information.


All in all, this book actually surprised me as soon as I got past chapter 1, and I can recommend it as a good study guide to aid you (alongside the vast resources found online) to successfully complete the 70-595 exam.

How to move the LiteSpeedCentral database

We are running LiteSpeed at work in order to compress our backups in SQL Server. I was tasked with moving the LiteSpeedCentral database from one SQL server to another. A simple task one might think.

By a quick search, I found a support ticket that says that you simply do a backup/restore of the database and a reconfiguration of the LiteSpeed client on each server. Simple enough, I did that.

The reconfiguration could have been easier, i.e. swap "old server" for "new server" instead of having to reconfigure everything, but sure enough, it worked.

I then got a lot of warnings on my old SQL Server telling me that some service on the LiteSpeed application server is trying to connect to the database but is failing.

I did some digging trying to find out what was causing this. The warnings came with an interval of fifteen minutes which narrowed it down to the SQL Server Agent job LiteSpeed for SQL Server Update Native Backup statistics and more specifically the query exec [LiteSpeedLocal]..[LiteSpeed_ImportNativeHistory]. When I ran this manually, it executed without errors but I still got the login failures logged on the old SQL Server.

No matter what I did to reconfigure the LiteSpeed application I could get rid of this warning. Finally I managed to find a dicussion on the Quest support forum that pointed me to the registry where I could find entries that were left pointing to the old Central Database Server. They were at HKLM/SOFTWARE/Imceda/SQLLiteSpeed/Engine/ActivityServers/[Database]/ Deleting these entries solved my issue.

So seriously Quest. Is it so hard to have a properly documented procedure for moving the database? It should be a pretty common task for customers to have to do..

Script updates of contacts in Outlook

I wanted to have all of my colleagues contact information in my iPhone so I thought of the simplest way to handle it. I quickly decided that the best way would be to simple add the appropriate domain accounts as contacts in Outlook which in a few seconds work gave me about 240 new contacts with the correct information entered as they appear in our Active Directory.

I then realised that they all had the internal speed dial phone number entered as their work number. This is a four digit number which is the same as the last four digits in the externally used number. Hence I needed a simple way to automatically add a prefix to the number for each contact, but only those that had four digits in the field since my address book is filled with external contacts as well.
After fiddling briefly with macros, csv-files and other means to get this to work, I had an epiphany and thought of PowerShell.

Three rows of PowerShell code written, tested and executed in a few minutes solved my problem

$outlook = new-object -com outlook.application
$contacts = $outlook.Session.GetDefaultFolder(10)
$contacts.Items | % { if($_.BusinessTelephoneNumber.Length -eq 4) { $_.BusinessTelephoneNumber = "123-45" + $_.BusinessTelephoneNumber; $_.save() } }

Using PowerShell to automatically update lots and lots of data will most likely be the first I think of the next time a similar task appears. This was almost ridiculously easy.

Having Visio update shape data automatically

Part of my job is to have an up to date documentation of all integrations we have. I am used to using Visio for this, and have at my previous clients kept it quite simple with manual formatting for visualizing if a message is under construction or to be deleted etc.

At this moment, I am trying out having "a lot" of documentation gathered in the Visio document. Besides having shapes for "Systems" and connections between these that each is equivalent of a "Message", I have added custom data fields on these that allow me to document metadata on each message flow. Part of the metadata I have on a message is Id, Name, Trigger, Status, Transport type and so on. Some of these fields are then used to dynamically alter the format of the connection to give a visual cue to the underlying data. If the Status is "Planned" I give the connection a green color, if the connection is marked as "Request/Response" I visualize this by having a symbol indicating so.

This all works fine and gives me one single document for the overview of the integrations. I then thought of implementing a change list so that I could follow up on revisions of the diagram. Having a normal text based change list on a separate sheet would be possible, but would incur manual work to update it and also not be mandatory (meaning that it would be forgotten and hence useless). Instead, I opted for an extra custom data field on the shapes used indicating the last time the shape was updated (data wise, not if it was moved). This is handled automatically by code that triggers when the shape data is updated.

I have one class module called "UpdateShapeTimeStamp":

Dim WithEvents appObj As Visio.Application

Private Sub appObj_CellChanged(ByVal Cell As IVCell)
    If (Cell.Shape.CellExists("Prop.Updated", 0)) Then
        Cell.Shape.CellsU("Prop.Updated").FormulaU = Chr(34) & Now() & Chr(34)
    End If
End Sub

Private Sub Class_Initialize()
    Set appObj = Application
End Sub

Private Sub Class_Terminate()
    Set appObj = Nothing
End Sub

and the Document code to enable this code:

Dim UpdateShapeTimeStampClass As UpdateShapeTimeStamp

Private Sub Document_BeforeDocumentClose(ByVal Doc As IVDocument)
    Set UpdateShapeTimeStampClass = Nothing
End Sub

Private Sub Document_DocumentOpened(ByVal Doc As IVDocument)
    Set UpdateShapeTimeStampClass = New UpdateShapeTimeStamp
End Sub

Now when changing the shape data on any shape, the event will trigger and check if the custom field "Updated" is available (meaning that it is one of my custom shapes) and if so, set it to the current datetime.

This allows me to have an automatic tracking of all changes in the diagram for each shape. Neat!

Outlook 2010 macro to copy item links to the Windows clipboard

I have used OneNote for a few years to handle all my notes regarding different projects and all information I gather during work. It works very well, and I have during this time looked at how to use Outlook in a more advanced way. Of course, I want to tie the two products together.

In the basic form, it is possible to create OneNote note taking pages from an email or appointment in OneNote. This is nice, since I can simply bring up the context menu and choose "OneNote" in order to create a place for all my notes from this specific meeting (and also write down things to bring up before the meeting occurs so that I am prepared). I have however often run into the issue that a lot of scheduled meetings will be accompanied by separate emails containing information that is relevant. In some way I'd like to group these separate items together, preferably in the OneNote page that I have created for the meeting.

I looked at how OneNote creates a page for an email, which is in the same way as for an appointment. It will bring with it a few pieces of information and also create a link back to the email item in Outlook. This link is what I am interested in.

I looked into different ways of creating it. Most promising was a utility called Linker that can copy the internal ID of an Outlook item and put it in the clipboard. The flaw was that it prefixes the ID with "outlook:". This works natively for Outlook 2003. Outlook 2007 can handle it with a registry hack. Outlook 2010 will not handle it at all.

I at least got a bit closer to a solution.

I then found out that a proper prefix in Outlook 2010 is "onenote:outlook?folder=Contacts&entryid=". I looked into how to get the ID from the Outlook item via VBA code and it proved quite easy. I then hacked together a small Macro that would grab the ID from the currently selected item, create the correct HTML link with the prefix above, and then put it in the clipboard. This did of course not work.

The clipboard is an intricate piece of Windows that can handle a lot of different data. Putting raw text into it is simple, putting an HTML link into it a bit more difficult. I turned to Google and found a support article titled "How to add HTML code to the clipboard by using Visual Basic".

This helped me create the more intricate string used to identify HTML in Windows. Using this, I had it working.

After a while I noticed that special characters (å ä ö specifically since I'm in Sweden) did not work and got encoded in a wrong way. After trying different ways of encoding the string I finally resorted to try and put the entity name ä in the string and this worked! It is HTML I'm working with after all. A quick google gave me this nice piece of code to encode special characters in a string, and now I have it all working flawlessly.

My workflow now is as follows:
An appointment is made in Outlook, by me or someone else. I rightclick this and add a OneNote page with information regarding this meeting under a tab for the specific project. Those emails that are of interest for this meeting is then linked in the OneNote page using my Macro. I simply click the email in Outlook, hit my shortcut button to copy the link, and then paste it in OneNote. I have created the macro so that the link description is fetched from the subject in the email. Perfect!

I also noticed that the macro will create links to pretty much anything in Outlook. Emails, appointments, contacts and so on. This way I can link together all items in one OneNote page so I don't have to browse or search in Outlook for whatever I need.

Here is the complete code for the macro if you are interested:
Private Declare Function CloseClipboard Lib "user32" () As Long
Private Declare Function OpenClipboard Lib "user32" (ByVal hWnd As Long) _
   As Long
Private Declare Function GlobalAlloc Lib "kernel32" ( _
   ByVal wFlags As Long, ByVal dwBytes As Long) As Long
Private Declare Function SetClipboardData Lib "user32" ( _
   ByVal wFormat As Long, ByVal hMem As Long) As Long
Private Declare Function EmptyClipboard Lib "user32" () As Long
Private Declare Function RegisterClipboardFormat Lib "user32" Alias _
   "RegisterClipboardFormatA" (ByVal lpString As String) As Long
Private Declare Function GlobalLock Lib "kernel32" (ByVal hMem As Long) _
   As Long
Private Declare Function GlobalUnlock Lib "kernel32" ( _
   ByVal hMem As Long) As Long
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" ( _
   pDest As Any, pSource As Any, ByVal cbLength As Long)
Private Declare Function GetClipboardData Lib "user32" ( _
   ByVal wFormat As Long) As Long
Private Declare Function lstrlen Lib "kernel32" Alias "lstrlenA" ( _
   ByVal lpData As Long) As Long
Private Const m_sDescription = _
                  "Version:1.0" & vbCrLf & _
                  "StartHTML:aaaaaaaaaa" & vbCrLf & _
                  "EndHTML:bbbbbbbbbb" & vbCrLf & _
                  "StartFragment:cccccccccc" & vbCrLf & _
                  "EndFragment:dddddddddd" & vbCrLf
                 
Private m_cfHTMLClipFormat As Long


Function RegisterCF() As Long
   'Register the HTML clipboard format
   If (m_cfHTMLClipFormat = 0) Then
      m_cfHTMLClipFormat = RegisterClipboardFormat("HTML Format")
   End If
   RegisterCF = m_cfHTMLClipFormat
End Function


Public Sub PutHTMLClipboard(sHtmlFragment As String, _
   Optional sContextStart As String = "", _
   Optional sContextEnd As String = "")
  
   Dim sData As String
  
   If RegisterCF = 0 Then Exit Sub
  
   'Add the starting and ending tags for the HTML fragment
   sContextStart = sContextStart & ""
   sContextEnd = "" & sContextEnd
  
   'Build the HTML given the description, the fragment and the context.
   'And, replace the offset place holders in the description with values
   'for the offsets of StartHMTL, EndHTML, StartFragment and EndFragment.
   sData = m_sDescription & sContextStart & sHtmlFragment & sContextEnd
   sData = Replace(sData, "aaaaaaaaaa", _
                   Format(Len(m_sDescription), "0000000000"))
   sData = Replace(sData, "bbbbbbbbbb", Format(Len(sData), "0000000000"))
   sData = Replace(sData, "cccccccccc", Format(Len(m_sDescription & _
                   sContextStart), "0000000000"))
   sData = Replace(sData, "dddddddddd", Format(Len(m_sDescription & _
                   sContextStart & sHtmlFragment), "0000000000"))
   'Add the HTML code to the clipboard
   If CBool(OpenClipboard(0)) Then
  
      Dim hMemHandle As Long, lpData As Long
     
      hMemHandle = GlobalAlloc(0, Len(sData) + 10)
     
      If CBool(hMemHandle) Then
         lpData = GlobalLock(hMemHandle)
         If lpData <> 0 Then
           
            CopyMemory ByVal lpData, ByVal sData, Len(sData)
            GlobalUnlock hMemHandle
            EmptyClipboard
            SetClipboardData m_cfHTMLClipFormat, hMemHandle
         End If
      End If
  
      Call CloseClipboard
   End If
End Sub


' Add the current selected item as Clipboard link
Sub AddOutlookItemAsClipboardLink()
    Dim linkString As String
    linkString = "{1}"
    linkString = Replace(linkString, "{0}", ActiveExplorer.Selection.Item(1).EntryID)
    linkString = Replace(linkString, "{1}", EncodeString(ActiveExplorer.Selection.Item(1).Subject()))
       
    PutHTMLClipboard (linkString)
End Sub


' Encodes special characters to their entity equivalent
Function EncodeString(ByVal strOriginal) As String
    Dim currChar, i, sOut, CharList
    CharList = "óáéíúÁÉÍÓÚ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿×÷ÀÂÃÄÅÆÇÈÊËÌÎÏÐÑÒÔÕÖØÙÛÜÝÞßàâãäåæçèêëìîïðñòôõöøùûüýþÿ"
    sOut = strOriginal
    For i = 1 To Len(CharList)
        currChar = Mid(CharList, i, 1)
        sOut = Replace(sOut, currChar, "&#x" & Hex(AscW(currChar)) & ";")
    Next
    EncodeString = sOut
End Function

Convert C# to PowerShell code

Yesterday I saw a link on our company's intranet linking to a colleague's blog. It proved to be a very nice read. Among the .Net Reflector addins available on CodePlex, there's one for PowerShell allowing you to convert C# code to PowerShell. That will come in handy a lot of times since I definately write C# code faster than PowerShell code. Thanks for that tip!
The only downside is that Reflector for the last year requires a license fee and I have since switched over to ILSpy. With this and the other plugins available, it's either time to switch over to Reflector again, or rewrite some of the plugins to work with ILSpy.