Connecting BizTalk to Oracle RIB using JMS a.k.a netsh/netmon to the rescue

I was tasked with setting up a connection between BizTalk and an Oracle RIB installation using JMS. The JMS connection was handled using the JNBridge adapter.

Hooking up BizTalk to JMS doesn't pose too much of an issue thanks to good documentation from JNBridge. Most of the issues that occur is about getting proper access to the servers, having the correct parameters for the JMS topics, setting the Java parameters right and so on.

As soon as those things got properly aligned, we successfully received messages published in the RIB to BizTalk. The next part was to send messages to the RIB.

We could see messages leave BizTalk successfully, but nothing ever appeared in the JMS topic. No errors were logged anywhere on either side.

After some error searching, I turned to do a trace using netsh.exe and analyzing this in NetMon. This showed that we had some handshaking going on coupled with the adapter running checks against the JMS setup to make sure that we could put a message on the topic. This resulted in an ORA-24033: no recipients for message error.

Unfortunately, this error message is eaten up by the framework and not forwarded to the adapter so that it can be logged properly. The only way to see it is by running a trace on the traffic.

The error message we got is explained in the documentation as:

ORA-24033: no recipients for message

Cause: An enqueue was performed on a queue that has been set up for multiple dequeuers but there were neither explicit recipients specified in the call nor were any queue subscribers determined to be recipients for this message.

Action: Either pass a list of recipients in the enqueue call or add subscribers to the queue for receiving this message.

I looked around for a solution to this error since we couldn't find any issues with the topic itself. There were a subscriber properly set up and no issues regarding access were to be found.

In an older blog post I could find something to point me in the right direction. In order to send messages to Oracle RIB, you have to set a threadValue parameter in the JMS header (which can be read about in the RIB documenation if you know where to look). This is in turn used in the subscription rule on the other end. If this parameter isn't set, there is no subscriber available since it is a mandatory field in the selector.

After setting a threadValue element to "1" using a custom pipeline component, the messages was successfully sent to the RIB which both can be seen in the topic itself as in the trace where the update to the table is visible with the message as a parameter.

NetMon or a similar tool is invaluable when tracking down similar issues. It is not the first time that NetMon (or WireShark) has provided information that is needed in order to get things to work.

Leaving password blank in BizTalk 2013 SFTP adapter will cause host instance to run at 100% CPU without warning

A very interesting tidbit appeared on a server running BizTalk 2013 and using the brand new SFTP adapter that is bundled with the product.

As soon as a file was sent to an SFTP server, the host instance running the adapter would spin up to 100% CPU and stay there. The file would not be sent and no warnings or errors written to the event log.

After being tasked with trying to find out the issue behind this, I started with the normal tasks coupled with a host spinning up without any reason: checking the tracking settings, looking at the messagebox regarding rouge messages, zombies etc. Nothing could be found.

It wasn't until I tried to set up a copy of the flow on another 2013 server that I noticed that the password field in the Send Port properties was blank. Even with an entered password of "" the password field should display a bunch of stars to mask the password (or lack of one).

I set the password to the correct one, and started the host again. Now the file was sent and the host stayed at normal CPU load.

When looking into this, I noticed that when exporting the binding for the send port when the password wasn't set, the CustomProps in the TransportTypeData element never included the Password element. When setting either a blank password or a proper one, the binding will include a password placeholder of which can be translated to NULL, i.e. there is no password defined (it is never written to the binding file for security reasons). But if you never set a password, this field is never written to the binding file at all!

This is important to be aware of since I also noticed that if you try to set the password to "blank", by entering a character and then deleting it again and then saving, it is not entered since the GUI will interpret it as nothing has changed and therefore not set the password field to blank, leaving it completely undefined. Instead you have to enter a dummy character, save, enter the property dialog again and delete the character before doing the final save. That way the blank password will be properly saved.

Remember to set the child delimeter even for single element lines in a flat file schema if using wrap characters

I briefly ran into this when creating a larger flat file schema for a message. The input message had line tags that I could use to identify the different rows. Most of the rows had several distinct fields to read, but there were rows with only the tag and a string of text encapsulated in quotation marks.

As an example, we'll use an input file like so:

#FIRST 12345 "This is the first string"
#SECOND "This is the second string"

This can then be used to create a schema which identifies the two distincs rows based on their tag (#FIRST and #SECOND respectively) and then split the fields on a delimiter of 0x20 (the space character).

The schema can be like this:

 <?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns="http://TestProject.Schemas" xmlns:b="http://schemas.microsoft.com/BizTalk/2003" elementFormDefault="qualified" targetNamespace="http://TestProject.Schemas" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
      <b:schemaInfo standard="Flat File" codepage="65001" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" root_reference="Root" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="Root">
    <xs:annotation>
      <xs:appinfo>
        <b:recordInfo structure="delimited" child_delimiter_type="hex" child_delimiter="0xD 0xA" child_order="postfix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
      </xs:appinfo>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:annotation>
          <xs:appinfo>
            <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
          </xs:appinfo>
        </xs:annotation>
        <xs:element name="First">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo tag_name="#FIRST" structure="delimited" child_delimiter_type="hex" child_delimiter="0x20" child_order="prefix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Id" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Text" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" sequence_number="2" wrap_char_type="char" wrap_char="&quot;" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="Second">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo tag_name="#SECOND" structure="delimited" child_order="prefix" sequence_number="2" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Text" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" sequence_number="1" wrap_char_type="char" wrap_char="&quot;" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

This will however generate an output like so:

<Root xmlns="http://TestProject.Schemas">
    <First>
        <Id>12345</Id>
        <Text>This is the first string</Text>
    </First>
    <Second>
        <Text>"This is the second string"</Text>
    </Second>
</Root>

Notice the quotation marks that are still left in the string even if we have defined them as a wrap character in the schema for that field.

In order to make them disappear from the field properly in the same manner as in the first line of text, we have to set the Child Delimiter parameter on the "Second" child record:

 <xs:element name="Second">
    <xs:annotation>
        <xs:appinfo>
            <b:recordInfo tag_name="#SECOND" structure="delimited" child_order="prefix" sequence_number="2" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" child_delimiter_type="hex" child_delimiter="0x20" />
        </xs:appinfo>
    ...

Which in turn will make the quotation marks disappear from our element data.

MSDTC error after cloning a virtual machine

A common development task is to clone virtual servers for different development scenarios. I myself have a bunch of virtual servers configured that I clone in order to get new fresh installs to lab with. However, when cloning machines in a BizTalk setup, you have to be aware of the issues that can develop due to the need for unique identification keys in MSDTC. When cloning a server to two instances and using one for SQL Server and the other as the BizTalk application server, MSDTC will not work. When running DTCTester the following error will appear:

Error: SQLSTATE=24000,Native error=0,msg=[Microsoft][ODBC SQL Server Driver]Invalid cursor state

 The reason in this case is that the CID GUID is not unique on the servers.

Run regedit and browse to HKEY_CLASSES_ROOT\CID\ and find the GUID where Description is set to MSDTC (it is one of the four). The GUID should be exactly the same on the other server.

  

In order to fix this, you have to uninstall and then install MSDTC again (or you could go the wild path and just set another GUID). Uninstalling/installing MSDTC is done via the command prompt.

Open a command prompt as administrator and run the command

msdtc -uninstall

followed by

msdtc -install

Verify that the DTC coordinator has started up and check the CID values in the registry. When running MSDTCTester again it should show a successful result:

This error will be caught by the BizTalk MessageBox Viewer. It will show an error of

is duplicated on servers and - Check if < server1> and are not cluster nodes as CID must be unique on non-clustered servers!

Script BizTalk Application export to .msi with options using PowerShell

I've had the need to export BizTalk Applications using scripts, but with the requirement to not export web directories or global parties to the .msi file.

This can be done using the GUI:

However, it cannot be done (that I know of) using the btstask command with the exportapp parameter.

I opted for creating a PowerShell script that more or less extends the btstask exportapp command. The script takes a few parameters making it a bit more versatile:

$appName is the name of the BizTalk application to export
$exportPath is the path to write the .msi file to
$exportWebDir is a boolean indicating whether to also export web directories or not
$removeDefaultBinding is a boolean controlling whether the default bindings will be written to the .msi file or not. This is handy to remove in those cases you want to just keep bindings for dev/test/production environments.
$exportGlobalParties is a boolean controlling whether to include the Global Parties in the binding file or not. These will of course not be written if for instance you only have the default bindings and set the parameter $removeDefaultBinding to false and $exportGlobalParties to true, since the binding file never will be written at all.

The script basically dumps a ResourceSpecification file which then gets modified depending on the parameters entered. This ResourceSpecification is then used as input to the btstask exportapp command creating an .msi file according to our needs.

The full script looks like this:

param($appName, $exportPath, $exportWebDir, $removeDefaultBinding, $exportGlobalParties)

Write-Output "Exporting ResourceSpec..."
BTSTask ListApp /ApplicationName:$appName /ResourceSpec:$pwd\ResourceSpecTemp.xml

If (!($?))
{
    throw "Could not export resource specification. Verify application name."
}

Write-Output "Reading ResourceSpec..."
$xmlResource = [xml] (Get-Content $pwd\ResourceSpecTemp.xml)

If ($exportWebDir -eq $false)
{
    Write-Output "Removing web directories..."
    $delnodes = $xmlResource.SelectNodes("/*[local-name()='ResourceSpec' and namespace-uri()='http://schemas.microsoft.com/BizTalk/ApplicationDeployment/ResourceSpec/2004/12']/*[local-name()='Resources' and namespace-uri()='http://schemas.microsoft.com/BizTalk/ApplicationDeployment/ResourceSpec/2004/12']/*[local-name()='Resource' and namespace-uri()='http://schemas.microsoft.com/BizTalk/ApplicationDeployment/ResourceSpec/2004/12'][@Type='System.BizTalk:WebDirectory']")

    ForEach($delnode in $delnodes)
    {
        [void]$xmlResource.ResourceSpec.Resources.RemoveChild($delnode)
    }
}

If ($removeDefaultBinding -eq $true)
{
    Write-Output "Removing Default binding info..."
    $delnodes = $xmlResource.SelectNodes("/*[local-name()='ResourceSpec' and namespace-uri()='http://schemas.microsoft.com/BizTalk/ApplicationDeployment/ResourceSpec/2004/12']/*[local-name()='Resources' and namespace-uri()='http://schemas.microsoft.com/BizTalk/ApplicationDeployment/ResourceSpec/2004/12']/*[local-name()='Resource' and namespace-uri()='http://schemas.microsoft.com/BizTalk/ApplicationDeployment/ResourceSpec/2004/12'][@Type='System.BizTalk:BizTalkBinding'][@Luid='Application/$appName']")

    ForEach($delnode in $delnodes)
    {
        [void]$xmlResource.ResourceSpec.Resources.RemoveChild($delnode)
    }
}

Write-Output "Saving modified ResourceSpec..."
$xmlResource.Save("$pwd\ResourceSpecTemp.xml")

Write-Output "Exporting application $appName..."

If ($exportGlobalParties -eq $true)
{
    $globalPartiesParam = "/G"
}
else
{
    $globalPartiesParam = ""
}

BTSTask ExportApp /ApplicationName:$appName /Package:$exportPath\$appName.msi /ResourceSpec:$pwd\ResourceSpecTemp.xml $globalPartiesParam

If (!($?))
{
    throw "Could not export application. Verify application name and parameters."
}
       
Write-Output "Cleaning up..."
Remove-Item $pwd\ResourceSpecTemp.xml

Write-Output "DONE!"

Exit 0

And will be called like so:

ExportBtsMsi.ps1 BtsDemo1 "C:\" false false false

Which will result in output like this:

Run Microsoft pre-configured VHD evaluations without Hyper-V using VirtualBox

I needed to run the BizTalk 2010 VHD for demo purposes, but lacked access to a Windows 2008 Server with Hyper-V in order to run the virtual machine.

My first attempt was to simply mount the VHD file in a newly configured Virtual PC machine. This failed with the virtual machine rebooting shortly after start.

Next I tried to create a new virtual machine in VirtualBox, giving me pretty much the same result, but with a bluescreen flashing past just before reboot.

In the bluescreen that quickly flashes by, it is possible to see the error code 0x0000007B, also known as INACCESSIBLE_BOOT_DEVICE. This is an indication of a possible issue with the boot sector, device driver and similar things. It led me to take a peek at the storage settings for the virtual machine that showed me that the IDE controller had no disks attached (but a CD/DVD drive) and my added VHD file was attached to a SATA controller.

I removed the VHD file from the SATA controller and removed the controller completely. I then added a new harddisk to the IDE controller and assigned the VHD file to it, thinking that this most likely was how the VHD file was set up initially.

And indeed it was so. This made the virtual machine start up ok!

I did run into issues when logging on though since the admin password includes the character "@" which I was unable to type due to some issue with VirtualBox and the Alt Gr key on my keyboard that was needed to type the character in. This was however quickly remedied by using the ALT-sequence instead, typing ALT+064 for the "@" character.

After a day of testing, I cannot see any issues in the VM, but everything is running just fine! For those that are using VMWare instead, the same solution should work just as well.

"Clever" uninstall of msi packages/applications using PowerShell

When I created my own PowerShell script library for BizTalk deployment automation I ran across the need to uninstall applications, both BizTalk applications and non-BizTalk ones, by only knowing their name.

At first, I solved it using a WMI query like so:

$name = "application name"

$product = Get-WmiObject -Class Win32_Product -Filter "name='$name'" -ComputerName "localhost"

[void]$product.Uninstall()

This is not a recommended way of doing it though. It is both very slow and also causes a bit of spamming in the Windows Eventlog since the query in fact does a reconfigure of ALL applications installed. This reconfiguration can also cause a bit of other issues in some cases.

I then created another way of trying to uninstall applications in a more failsafe and secure way by using msiexec with the uninstall flag. The tricky part was to find a way to get the product key in order to be able to use msiexec since it requires this for uninstalling an application. The result can be found below.

The script function will take the application name as argument. It will then via the registry (note that this is configured for x64, so change the path if you are running x86) look up the application settings. If the application can be found via name (it should), we extract the product key. Then this is used as a parameter to msiexec.

If the product key cannot be found (it happens), we will instead try to read the uninstall string that is set when installing the application. Windows will run this string when you choose to uninstall an application, so why do not we use it? If found, we extract the product key and do our msiexec call.

If all fail, we throw an exception to be caught in the real part of the script.

This is the full script function:

Function Uninstall-Program([string]$name)
{
    $success = $false

    # Read installation information from the registry
    $registryLocation = Get-ChildItem "HKLM:\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\"
    foreach ($registryItem in $registryLocation)
    {
        # If we get a match on the application name
        if ((Get-itemproperty $registryItem.PSPath).DisplayName -eq $name)
        {
            # Get the product code if possible
            $productCode = (Get-itemproperty $registryItem.PSPath).ProductCode
           
            # If a product code is available, uninstall using it
            if ([string]::IsNullOrEmpty($productCode) -eq $false)
            {
                Write-Host "Uninstalling $name, ProductCode:$code"
           
                $args="/uninstall $code"

                [diagnostics.process]::start("msiexec", $args).WaitForExit()
               
                $success = $true
            }
            # If there is no product code, try to read the uninstall string
            else
            {
                $uninstallString = (Get-itemproperty $registryItem.PSPath).UninstallString
               
                if ([string]::IsNullOrEmpty($uninstallString) -eq $false)
                {
                    # Grab the product key and create an argument string
                    $match = [RegEx]::Match($uninstallString, "{.*?}")
                    $args = "/x $($match.Value) /qb"

                    [diagnostics.process]::start("msiexec", $args).WaitForExit()
                   
                    $success = $true
                }
                else { throw "Unable to uninstall $name" }
            }
        }
    }
   
    if ($success -eq $false)
    { throw "Unable to find application $name" }
}

Running an Integration Competence Center for small businesses

A while ago I saw a post from an old colleague on LinkedIn where he linked to an old article called Too small for an Integration Competency Center? Think again. I was intrigued to read the article since I am in the middle of establishing an ICC at the organization I'm at, and we are quite small in that regard.

The four building blocks involved are:

• Adopt common integration practices
• Standardize on a common integration platform
• Create a decentralized group that specializes in integration
• Form a centralized ICC

This has all been done at the organization I am at, with the experience I have from larger organization as a basis for the setup. In my case, with an IT department of a whopping four people, forming an ICC is no big challenge in that regard.

I have opted to include external parties in the ICC to form a team that is not bound to a strict organizational hierarchy, but rather be agile and flexible to the demands from the organization at stake.

The basis of the ICC is in my role as integration architect/supervisor in the IT department. Coupled to this I have included an external integration architect to have a form of sounding board. Most, if not all, integration development work is carried out by consultants which the external integration architect is responsible to coordinate. As needed, I also include internal resources in the ICC to help forming the ICC and drive the integration development forward. So basically, the ICC is in it's simple form just two persons, with several others coming and going as needed.

So far, this is working very well. The agility in the ICC is a strong factor to consider. A lot of times, people are included full time even if they are not needed full time. Since the external architect is contracted on an as-needed-basis, costs can be brought down. Of course, that puts a lot of burden on me.

However, I have strived to create a simple and effective process of the entire integration life cycle. Document templates are available for everything from requesting an integration (done by the organization) to detailed specification and testing of the completed development. The development guidelines and deliverance specifications are to be as standardized as possible to not bind anything to a single person or organization (except our own). I aim to be able to both replace the consultants involved as well as myself without having to explain anything about the setup and behavior of the ICC and involved parts.

As for the four building blocks above, I feel that all (almost) of them can be ticked off in our case. The platform is BizTalk, best practices are established through documentation and processes based on experience from me and consultants, a decentralized group is formed and the ICC is formed and handling most of the integration needs. What is left is the funding that is still divided and handled by separate organizational units rather than taken care of completely by a central organization. This is currently the biggest issue to handle since it hamper a true value-for-money path regarding integration.

But to summarize, in accordance to the article linked in the beginning of this post, you cannot have a too small of an organization for an ICC to be formed. By including resources on a need-to basis as well as including external consultants where needed, it can be handled in a very effective way. The important part is to have a clear process of the entire life cycle as well as clear boundaries between different parties involved. My role in the whole ICC is most importantly to coordinate the different tasks between the resources, and as long as that is done appropriately, everything is flowing smoothly. Having a centralized funding of the work is not necessary, but very (very!) preferable.

Simple dos and don'ts for versioning webservice interfaces (and similar integration interfaces)

Earlier this week I went to a user group gathering for the ERP system we utilize at work. The topic for the day was "integration". At the end of the day during an open Q and A session, I approached the ERP representatives with questions on how they worked with the web service API exposed from the ERP system. I asked them if using the current web services guaranteed that they would work without changes and need for deep regression testing when upgrading the system. I also asked them how new functionality to existing services would be implemented, from an integration point of view.

The answer was that current implementations would require normal regression testing after upgrading, but that there could be changes in the backend making the services work a bit different in the new versions, depending on the changes made in the business logic and GUI.

This made me a bit quezy and fidgety. In the best of worlds, I want a web service to be seen as a hard written contract between two parties, in this case the ERP system on one side and the integration platform on the other. By exposing a service, you should guarantee that the specification for the service is not to be changed and that the underlying logic is to be set in stone as well. That seemed to not be the case.

The discussion then evolved into the realm of versioning and planned changes of existing services. As of now, new functionality is added to the service without changing its name, version or namespace. Fields are simply added as optional.

This method is something I have several issues with, and which I also expressed at the session (maybe a bit too harsh, but it is an important subject to me). First off, by adding new functionality as optional fields, you will break one of the (many) benefits with web service contracts: hard typing of them! By doing it this way, the developers will make future maintenance of the services harder, for both themselves as well as those utilizing the services. Also, the clients will have to upgrade the integrations whether they want to or not as soon as the ERP system is to be upgraded and the services change.

I instead proposed that they should version their services and operations. When changing an operation, create a new version of it and as long as it is possible, make sure that the previous version(s) are still functional. If a service operation is to be removed from the system, mark it as obsolete and let the clients know which version of the ERP system it will be removed in, and preferably how to utilize other services and operations to enable the needed functionality. I myself think that this is a very obvious way of handling the dilemma. I have grown to understand that it is very rare that versioning is used though and I do my best preaching the above statements.

So please, make sure that you use some form of versioning strategy when building your interfaces. It will make my (and many others, I guarantee) life a lot easier.

BizTalk Server 2013 (formerly BizTalk Server 2010R2) is in beta

Late last night I saw some tweets about the new version of BizTalk getting to beta status. A quick look, and sure enough, BizTalk 2010R2 is renamed to BizTalk 2013 and a beta version is available for download.

I must say that this release is very intriguing. Among the more interesting new features for me is the REST capabilities, the new SharePoint adapter and the ESB toolkit integration (especially since I loathe the way the first version was to be setup and handled).

The new dependency function to check out how artifacts are dependent of eachother is also a nice addition.

Go check it out and hope for a quick release of the final product!