BizTalk with codeplex SFTP adapter, doing a file move on AfterGet

A client has a setup with the Codeplex SFTP adapter, reading files from a site and then deleting them when read using the Delete parameter in the AfterGet property within the receive location.

A need arised where the files would need to be moved to an archive folder in the SFTP site instead of being deleted.

There is no mention of this in the documentation of the adapter, however, by checking the underlying code one can see how we can do the move.

The OnBatchComplete method located in BatchHandler.cs, will be called when a message has been sent successfully to BizTalk and hence be removed/renamed on the SFTP server. The line

string renameFileName = CommonFunctions.CombinePath(Path.GetDirectoryName(fileName), batchMessage.AfterGetFilename);

Is the interesting part. We see that when the Rename option is chosen in the adapter properties, the  CombinePath method will be called to create the new full path for the file based on the current directory where it is located, and a new filename. This can be utilized by simply setting the new filename to include a relative path as well as the filename and by doing that create a move of the file and have the rename optional. The latter since the macro replacements will be done after the creation of the full path.

Suppose we read files located in /home/marcus/btstest/ and after reading these should be moved to /home/marcus/btstest/archive/. In this case, you would set the AfterGet File Name property to read archive/%SourceFileName%

If you instead want to move the files to a directory outside of your current path, you can do so by specifying the parent by the usual two dots (..) like for instance ../btstestarchive/%SourceFileName% for the directory located at /home/marcus/btstestarchive/.

One issue can occur though if you use this technique to move files to a file archive in order to have a log on the ftp server side of all files that has been processed. If the filename is already available in the archive folder and the adapter tries to move yet another file to this folder with the same name, it will fail. It will be logged in the event log as "Method: Blogical.Shared.Adapters.Sftp.SharpSsh.Sftp.Rename
Error: Unable to rename /home/marcus/btstest/testfile.xml to /home/marcus/btstest/archive/testfile.xml", but the original file will be left in the pickup folder on the SFTP site.

BizTalk will not try to process this file again, until you restart the host that is. Then it will be picked up once more, sent to the messagebox and the rename function will fail again. You could make it a bit safer by using the %DateTime% or %UniversalDateTime% macros that also are available besides %SourceFileName%. Yet another option is to extend the BatchHandler.cs with the %MessageID% macro and create a GUID to add to the filename in order to get it truly unique.

Leaving password blank in BizTalk 2013 SFTP adapter will cause host instance to run at 100% CPU without warning

A very interesting tidbit appeared on a server running BizTalk 2013 and using the brand new SFTP adapter that is bundled with the product.

As soon as a file was sent to an SFTP server, the host instance running the adapter would spin up to 100% CPU and stay there. The file would not be sent and no warnings or errors written to the event log.

After being tasked with trying to find out the issue behind this, I started with the normal tasks coupled with a host spinning up without any reason: checking the tracking settings, looking at the messagebox regarding rouge messages, zombies etc. Nothing could be found.

It wasn't until I tried to set up a copy of the flow on another 2013 server that I noticed that the password field in the Send Port properties was blank. Even with an entered password of "" the password field should display a bunch of stars to mask the password (or lack of one).

I set the password to the correct one, and started the host again. Now the file was sent and the host stayed at normal CPU load.

When looking into this, I noticed that when exporting the binding for the send port when the password wasn't set, the CustomProps in the TransportTypeData element never included the Password element. When setting either a blank password or a proper one, the binding will include a password placeholder of which can be translated to NULL, i.e. there is no password defined (it is never written to the binding file for security reasons). But if you never set a password, this field is never written to the binding file at all!

This is important to be aware of since I also noticed that if you try to set the password to "blank", by entering a character and then deleting it again and then saving, it is not entered since the GUI will interpret it as nothing has changed and therefore not set the password field to blank, leaving it completely undefined. Instead you have to enter a dummy character, save, enter the property dialog again and delete the character before doing the final save. That way the blank password will be properly saved.

Dangers when upgrading the Codeplex SFTP adapter

I encountered some dangers that one can run into when upgrading the BizTalk SFTP adapter that can be found on Codeplex.

On one installation where the adapter is used did the issue of the adapter stop working frequently arise. When the issue was fixed in a subsequent version of the adapter, it was upgraded, but the problem remained.

When I looked at the server, I noticed that the adapter never had got upgraded, even though they very well installed the new version. I then replicated what I believe to be the steps taken during the upgrade in order to fully understand what had happened.

I learned that when upgrading version 1.3.3 to 1.4.0, it is necessary to completely uninstall the previous version before installing the new. This was never done during the upgrade process. The old version remained installed and the setup program for the new version reported back that the installation was successful. The BizTalk administrator never noticed the dual entries in the Add/Remove programs window nor the old timestamps in the adapter installation folder (which was how I noticed it from the beginning).

Uninstalling the two adapter entries in the control panel and then reinstalling the new version solved the issue as can be seen in the installation directory.

I then also noticed that when properly upgrading the adapter, receive locations bound to the SFTP adapter will not start unless you for each and every one open the adapter properties windows and save it. This is due to a difference in the properties that will make the bindings fail otherwise, rendering the port to shut down.