In BizTalk application development we generally run into the situation where we need to create the custom pipeline with custom pipeline component to fulfil the requirement. A custom pipeline component is just a plain .NET class that implements several BizTalk interfaces. Here is the MSDN link for developing a custom pipeline component. Let’s consider a scenario where you need to change the namespace of a certain type of messages coming in from a business partner. You are receiving the messages through BizTalk and need to change the namespace before it can be published in the Message Box. The most general approach we developer take to implement the custom pipeline component would be something like the code displayed below. In the above pipeline components we are loading the entire document into memory using an XmlDocument object. This is not the best approach because the amount of space required by an instance of the XmlDocument class to load and create an in-memory representation of a XML document is up to 10 times the actual message size. In order to read a message, we should use an XmlTextReader object along with an instance of the following classes:

• VirtualStream (Microsoft.BizTalk.Streaming.dll) - The source code for this class is located in two locations under the Pipelines SDK as follows: SDK\Samples\Pipelines\ArbitraryXPathPropertyHandler and SDK\Samples\Pipelines\SchemaResolverComponent\SchemaResolverFlatFileDasm.
• ReadOnlySeekableStream (Microsoft.BizTalk.Streaming.dll).
• SeekAbleReadOnlyStream - The source code for this class is located in two locations under the Pipelines SDK as follows: SDK\Samples\Pipelines\ArbitraryXPathPropertyHandler and SDK\Samples\Pipelines\SchemaResolverComponent\SchemaResolverFlatFileDasm.

We should leverage the VirtualStream, ReadOnlySeekableStream, and SeekAbleReadOnlyStream class found in the Microsoft.BizTalk.Streaming.dll  assembly. The VirtualStream class for instance has the same functionality as a MemoryStream class in the System.IO namespace, however it uses disk space to store large streams instead of memory. Therefore, the benefit of using the VirtualStream class over XmlDocument is that you can gain performance when handling messages, since you do not need to load the entire document into memory using an object. Now lets say for the same scenario we considered above the size of the messages the partner send out are a substantial size i.e. 1 to 5 Mb and the frequency is quite higher on particular moments of the day. So now we should definitely try to change the namespace in a custom pipeline in the most optimal way. Therefore, we should build a custom pipeline component that leverages VirtualStream to replace the namespace based on runtime properties of a custom pipeline. This is very well explained in article Custom pipeline optimization using the Virtual Stream class by Steef-Jan Wiggers Sample code to leverage the Virtual Stream Class can be found at MSDN Code Gallery: BizTalk 2013 - Custom pipeline optimization using the Virtual Stream class Note: Custom pipeline with custom pipeline component should be tested separately under heavy load condition to observe their behavior when the system is working at full capacity and to find any possible bottlenecks.

Related Links

http://blogs.msdn.com/b/brajens/archive/2006/11/25/how-to-develop-biztalk-custom-pipeline-components-part1.aspx https://msdn.microsoft.com/en-us/library/ee377071(v=bts.70).aspx https://connectedpawns.wordpress.com/2015/01/09/optimising-pipeline-performance-xmlwriter-vs-xdocument/ http://blogical.se/blogs/johan/archive/2008/01/02/custom-pipeline-components-development-best-practices.aspx