Slow performance in PowerShell Get-ChildItem over UNC paths

I got a ticket submitted to me that proved to be quite interesting. The basics where that a specific maintenance script were running for ages on a server causing some issues.

The script itself was quite simple, it did a recursive gci on a path and performed an action on each item matching a where clause that identified empty folders. Nothing wrong there. However, even if the specified path contained a massive amount of folders and files, it still shouldn't have been running for days (literally) causing the ticket to be created.

When looking into the matter, I found this blog post on the Windows PowerShell blog that goes into detail why Get-ChildItem has slow performance at times. The main culprit is the .NET APIs that is too chatty and causes a lot of overhead traffic over the network when trying to query for files. This was fixed in PowerShell 3.0 that uses new API:s.

A quick test using a folder with 700 subfolders and a total of 40 000 files where I execute the script line

(gci d:\temp -Recurse | where-object {$_.PSIsContainer}) | Where-object {($_.GetFiles().Count + $_.GetDirectories().Count) -eq 0} | out-file d:\dir.txt

reveals the following execution time numbers:

Using PowerShell 2.0 and accessing the path as an UNC: 100s
Using PowerShell 3.0 and accessing the path as an UNC: 33s
Using PowerShell 2.0 and accessing the path locally: 6s
Using PowerShell 3.0 and accessing the path locally: 5s

In my case, the server was both running PowerShell 2.0 and accessing the path as an UNC causing a major performance hit. The solution is simply to both run the script locally on the server where the task has to be performed as well as upgrading to at least PowerShell version 3.0. As can be seen from my quick test, the best performance gain is clearly to run the script locally giving about 17 times better performance.

While no rocket science that accessing tons of files locally has to be faster then doing it over the network, it is still fairly common to see scripts executed on application servers and performing tasks on file shares.

This also proves that it is important in our business to stay on top of new versions of tools and frameworks in order to catch these types of improvements. I can be certain that there is quite a number of people out there bashing gci heavily for it's slow performance when in fact it has improved a lot between PowerShell 2.0 and 3.0.

BizTalk Performance - a short recap from BizTalk User Group Sweden 2010-05-19

Yesterday I attended BizTalk User Group Swedens gathering in Stockholm that this time focused on performance and optimization in BizTalk. A very nice seminar as always.

I didn't take any notes, but will write a short recap of the things that I remember the best from this evening. Some parts that I knew of, and some that I will explore further in some labs. So, here we go: BizTalk performance best practices:

1. Plan your SQL storage
   This is an entire bookshelf of information on its own and something that most of the BizTalk developers won't get in touch with. However, it is very crucial for the overall performance of the platform and time and money should be spent here. Arranging LUNs, RAID arrays and similar hardware configurations are the base. It is also quite possible to increase performance by splitting the datafiles to several files and filegroups as described in the following links: http://msdn.microsoft.com/en-us/library/ee377060%28BTS.10%29.aspx
   http://msdn.microsoft.com/en-us/library/ee377069%28BTS.10%29.aspx
2. Optimize SQL Server settings
   As the backbone of BizTalk, SQL Server optimizations is a way to gain performance boosts. The most prominent example from yesterdays session was enabling the T1118 flag. The only downside seems to be a possible increase in physical data allocation. http://support.microsoft.com/kb/328551
3. Turn off antivirus
   A risky move for some and might not play well with company policies. However, antivirus will eat a lot of the processing power and speed available.
4. Use XslCompiledTransform for maps
   The current implementation of the mapping engine in BizTalk uses the old and deprecated XslTransform class. By wrapping calls to XslCompiledTransform instead, a very noticable performance increase can be seen. At the session, about 2/3 of the execution time was removed from the first call to a map using a ~35MB XML file. The gain is made both for small files as well as larger ones. http://blogs.msdn.com/paolos/archive/2010/04/08/how-to-boost-message-transformations-using-the-xslcompiledtransform-class-extended.aspx
5. Optimize the WCF endpoint bindings
   This was covered in some part, but I'll take a different but short approach on the subject. Default is to fetch data in a buffered transfermode which enables the WCF message security and reliability. However, if performance is needed, examine using the streamed transfermode instead. On larger messages, this will have impact.
   Changing message encoding can also give a performance boost. Default is text encoding which with binary attachments/messages will result in a base-64 encoding and an increase in message size. MTOM or binary encoding will solve this.
   It is also interesting to look at the actual binding since NET.TCP can utilize binary encoding and also has the best performance. The penalty is severe though since it's both not interoperable and can be useless due to network restrictions.
6. Watch those persistance points
   If you have more complex orchestrations, take a look at optimizing the number of persistance points. I have orchestrations that can make up to thirty calls to external services. Each of these calls will result in a persistance point. I don't need to keep state and transactions in them, so by wrapping everything in an atomic scope, the number of writes to the database is vastly reduced. Sanket has a good explanation of persistance points on his blog: http://blogs.msdn.com/sanket/archive/2006/11/12/understanding-persistence-points-in-biztalk-orchestration.aspx

There is more to performance than this. Way more. But it's some of the simpler things you have to think of when working with BizTalk. The seminar was captured on tape, so both the slides and entire presentation should be available in a short future.