BizTalk Performance - a short recap from BizTalk User Group Sweden 2010-05-19

Yesterday I attended BizTalk User Group Swedens gathering in Stockholm that this time focused on performance and optimization in BizTalk. A very nice seminar as always.

I didn't take any notes, but will write a short recap of the things that I remember the best from this evening. Some parts that I knew of, and some that I will explore further in some labs. So, here we go: BizTalk performance best practices:

1. Plan your SQL storage
   This is an entire bookshelf of information on its own and something that most of the BizTalk developers won't get in touch with. However, it is very crucial for the overall performance of the platform and time and money should be spent here. Arranging LUNs, RAID arrays and similar hardware configurations are the base. It is also quite possible to increase performance by splitting the datafiles to several files and filegroups as described in the following links: http://msdn.microsoft.com/en-us/library/ee377060%28BTS.10%29.aspx
   http://msdn.microsoft.com/en-us/library/ee377069%28BTS.10%29.aspx
2. Optimize SQL Server settings
   As the backbone of BizTalk, SQL Server optimizations is a way to gain performance boosts. The most prominent example from yesterdays session was enabling the T1118 flag. The only downside seems to be a possible increase in physical data allocation. http://support.microsoft.com/kb/328551
3. Turn off antivirus
   A risky move for some and might not play well with company policies. However, antivirus will eat a lot of the processing power and speed available.
4. Use XslCompiledTransform for maps
   The current implementation of the mapping engine in BizTalk uses the old and deprecated XslTransform class. By wrapping calls to XslCompiledTransform instead, a very noticable performance increase can be seen. At the session, about 2/3 of the execution time was removed from the first call to a map using a ~35MB XML file. The gain is made both for small files as well as larger ones. http://blogs.msdn.com/paolos/archive/2010/04/08/how-to-boost-message-transformations-using-the-xslcompiledtransform-class-extended.aspx
5. Optimize the WCF endpoint bindings
   This was covered in some part, but I'll take a different but short approach on the subject. Default is to fetch data in a buffered transfermode which enables the WCF message security and reliability. However, if performance is needed, examine using the streamed transfermode instead. On larger messages, this will have impact.
   Changing message encoding can also give a performance boost. Default is text encoding which with binary attachments/messages will result in a base-64 encoding and an increase in message size. MTOM or binary encoding will solve this.
   It is also interesting to look at the actual binding since NET.TCP can utilize binary encoding and also has the best performance. The penalty is severe though since it's both not interoperable and can be useless due to network restrictions.
6. Watch those persistance points
   If you have more complex orchestrations, take a look at optimizing the number of persistance points. I have orchestrations that can make up to thirty calls to external services. Each of these calls will result in a persistance point. I don't need to keep state and transactions in them, so by wrapping everything in an atomic scope, the number of writes to the database is vastly reduced. Sanket has a good explanation of persistance points on his blog: http://blogs.msdn.com/sanket/archive/2006/11/12/understanding-persistence-points-in-biztalk-orchestration.aspx

There is more to performance than this. Way more. But it's some of the simpler things you have to think of when working with BizTalk. The seminar was captured on tape, so both the slides and entire presentation should be available in a short future.

BizTalk configuration error: Failed to connect to the SQL database 'SSODB'

Sometimes when configuring a fresh installation of BizTalk Server, the following error might appear during the configuration of the Enterprise SSO:

Failed to connect to the SQL database 'SSODB' on SQL Server ''. (SSO) Additional information: (0x80131700) (Win32)

Most interesting is that the error is due to a failure to connect when the configuration is meant to create the database in question.

I found a solution through this blogpost which while it is for Windows Vista and a bit different, still has the same solution as for the above problem. By some reason the Enterprise SSO is not correctly installed and by registering the SQL module and run the configuration tool again, it will work:

1. Start a Visual Studio command prompt.
2. run regasm "c:\Program Files\Common Files\Enterprise Single Sign-On\SSOSQL.dll"
3. Run the BizTalk Server configuration tool again

Error: 'Microsoft.XLANGs.BaseTypes.BPELExportable': inconsistent duplicate module attribute

While working on a solution we suddenly encountered the following error message while trying to build the code.

'Microsoft.XLANGs.BaseTypes.BPELExportable': inconsistent duplicate module attribute

It can be a nuisance to find the problem if you don't work with BizTalks BPEL capabilities that often. It is however quite easy to fix. The error message is due to one orchestration having a different Module Exportable setting than the other orchestrations in the project. Making sure that the setting match between all orchestrations will make the error message disappear.

Biztalk Server 2009r2 is to be known as BizTalk Server 2010

There was a mail discussion today between a number of consultants at work regarding the upcoming BizTalk Server version. It was previously known as BizTalk Server 2009r2 but was announced this week to be named BizTalk Server 2010.

There were a few concerned voices about what customers that just purchased BizTalk 2009 are going to think about Microsofts name changing move. I myself cannot see a reason to be too concerned though, but rather see it as the correct decision from both Microsoft as well as the customers that just got BizTalk Server 2009.

I believe (without having talked to anyone at Microsoft) that the reason for changing the name is mainly two-fold: Firstly, the name will synch with Visual Studio 2010. Secondly, and probably most important, the name change will reset the support cycle for the product. If a 2009r2 release were to be, its support cycle would be based on the release date of the 2009 version. Now they will reset it to the day in this year that the 2010 version will be out. Since BizTalk Server vNext will be a bigger change in the platform partly due to AppFabric, it's vital to have a longer support cycle for the versions being released before vNext.

While 2010 will bring a few neat things that I'm looking forward to (a proper SFTP-adapter and new mapping tool among others), the customers that have 2009 will still not miss out on any major things. It's especially not worth delaying a roll-out of the BizTalk platform (or upgrade) based on the improvements we get this year.

So all in all, it feels as the correct move by Microsoft, and everybody should be happy.

BAM deployment error: SQL Analysis Services 2008 Enterprise Edition is not configured. Can not create OLAP cubes for RTAs.

If the BAM OLAP databases are not set up in SQL Server when deploying BAM definitions by running bm.exe deploy-all, the following error message might appear.

Deploying View... ERROR: The BAM deployment failed.
SQL Analysis Services 2008 Enterprise Edition is not configured. Can not create OLAP cubes for RTAs.

Run BizTalk Server 2009 Configuration and select BAM Tools. Make sure that Enable Analysis Services for BAM aggregations is selected and the Data stores valid.

Click Apply configuration after making the necessary changes and then run bm.exe to deploy the configuration once again.

Adding an xml-stylesheet reference (or comment) to your XML file in BizTalk

A collegue of mine asked for help in adding a stylesheet reference declaration to an outgoing XML-file in BizTalk. The XML looked something like this:

<xml version="1.0" encoding="utf-16"?> 
<ns0:Report xmlns:ns0=http://www.customer.se/schema/System/ReportSchema_1.1>
  <ns0:Identification> …

But the desired output had this additional element declaring an xml-stylesheet.

<xml version="1.0" encoding="utf-16"?> 
<xml-stylesheet type="text/xsl" href="System_ReportSchema_1.1.xslt"?> 
<ns0:Report xmlns:ns0=http://www.customer.se/schema/System/ReportSchema_1.1>
  <ns0:Identification> …

The solution was to add the element to the XmlAsmProcessingInstructions in the XML pipeline properties in the send port. This property can also be used to add additional data that you want to add to your message, such as comments.

See http://msdn.microsoft.com/en-us/library/aa578687(BTS.20).aspx for more information.

Create child node only if input elements has appropriate data

I had a map where I needed to create the child element nodes only if some of the input nodes had a value. The rule was as follows:

For group messages, each person has to have a first name, surname, e-mailaddress and company. For sendlist messages, each person had to have first name AND/OR surname as well as a company.

The sending system would transmit messages that didn’t validate to these rules and I had to sort out the children in the messages that didn’t adhere to these rules. Easy enough to do in the maps.

By using the Logical String-functoid, I can validate if a node is a string or not, i.e. if a firstname or company is set for the person. The logical functoids (AND and OR) concatenate the results accordingly to my rules and I then use the output to decide whether to create the entire child or not in my output message.

UDDI 3 Configuration crashes when clicking on the Web Application page

When setting up UDDI 3 on a BizTalk multiserver configuration, the configuration tool might crash when clicking on the Web Application page. The following message will be logged in the event log:

Event Type:    Error
Event Source:    BizTalk DW Reporting
Event Category:    None
Event ID:    1000
Date:        2010-02-06
Time:        23:17:15
User:        N/A
Computer:    CONTBTS2K9
Description:

Faulting application configuration.exe, version 3.8.368.0, stamp 49b1da6c, faulting module configframeworkhelper.dll, version 3.8.368.0, stamp 49b1da6b, debug? 0, fault address 0x000823ba.

The reason for this error is that the account used to run the configuration tool is not local administrator on the SQL Server used for UDDI 3.

Installing UDDI 3 with a separate SQL Server

The installation documents doesn't explain how the setup for UDDI 3 is to be done when the SQL Server is installed on a separate machine than where the UDDI services are to be placed (in my case on the BizTalk 2009 server). Here is the steps I take.

1. Run setup.exe from the BizTalk media on the SQL Server and choose to install UDDI 3. Install only the database components.
   
   
   
   
   
   
   
   
2. When the installation finishes, choose to configure the UDDI services.
3. Select to do a custom configuration and specify the service account to use.
4. Go to the UDDI Services Database Component page of the configuration and choose to enable UDDI Services Database Component and deselect the SSL requirement.
   
   
   
   
   
   
   
   
   
5. Apply the configuration.
   A failure with the apply task will most likely be due to a lack of permissions to the SQL Server. Make sure that the account you are logged in as (or use to run the configuration tool) has the appropriate permissions to SQL Server.
   If successful, you should end up with a UDDI3 database on your SQL Server.
   
   
   
   
   
   
   
   
6. Start the UDDI3 setup from the BizTalk media on the BizTalk server.
7. Choose to install the components necessary, except for Database Components.
   
   
   
   
   
   
   
   
8. Finish the installation and select to run the configuration tool.
9. Enter the SQL Server name and the credentials for the service account and select to do a custom configuration.
10. Check the Enable UDDI Services Web Application box.
    
    
    
    
    
    
    
    Note! If you lack the proper credentials to the UDDI3 database on the SQL Server, you will see a red cross next to the Data Store name with an error message of "The database UDDI3 on BTSSQLSRV01 is not a valid UDDI database.". The database is in fact just fine, but the account used to run the configuration tool doesn't have all the required permissions to access the database.
    
    
11. Go to the next page and check the Enable UDDI Service Notification Service box. Enter the information for SMTP server and E-mail address.
    
    
    
    
    
    
    
12. Apply the configuration.
13. Go to the URL specified for the UDDI services and verify that everything is up and running.
    

Complete installation of ESB Toolkit 2.0

Here is my guide for installing the ESB Toolkit 2.0 in BizTalk 2009. It's for 32bit and everything is located on the same server.

1. Import ESB.Core into BizTalk
2. Import ESB.Exceptions into BizTalk (overwrite resources)
3. Import ESB.JSM into BizTalk (overwrite resources)
4. Install each msi onto the server as well (checkbox at the end of the import wizard)
5. Give SQL Service Account permission to BAM databases by running this query in SQL Management Studio
   USE BAMStarSchema
   GO
   CREATE USER SQLService FOR LOGIN [W1BTS3\SQLServiceAccount]
   GO
   EXEC sp_addrolemember db_datareader, SQLService
   GO
6. Deploy BAM activities
   "c:\program files\microsoft biztalk server 2009\tracking\bm.exe" deploy-all -definitionfile:"c:\program files\microsoft biztalk esb toolkit 2.0\bam\microsoft.biztalk.esb.bam.exceptions.xml"
   "c:\program files\microsoft biztalk server 2009\tracking\bm.exe" deploy-all -definitionfile:"c:\program files\microsoft biztalk esb toolkit 2.0\bam\microsoft.biztalk.esb.bam.itinerary.xml"
7. Run ESBConfigurationTool.exe and apply the configuration before hitting the next page.
8. Install UDDI from the BizTalk media. Make sure to uncheck the SSL requirement.
9. Run the UDDI Publisher tool \Bin\Microsoft.Practices.ESB.UDDIPublisher.exe).
10. Run "C:\Windows\Microsoft.NET\Framework\v3.0\Windows Communication Foundation\ServiceModelReg.exe" -r -y
11. Extract ESBSource.zip to C:\Projects\Microsoft.Practices.ESB
12. Remove the read-only attribute on all files and folders.
13. Open PowerShell and run Set-ExecutionPolicy unrestricted
14. Open ESB.AlertService sln, set setup project platform to x86
15. Build ESB.AlertService
16. Open ESB.Portal, verify web.config
17. Verify ExceptionService web.config
18. Build ESB.UDDI.PublisherService
19. Run portal install script
20. Enable TCP/IP connections in SQL Server
    

With this done, everything should be up and running properly.