Subscriptions in BizTalk

BizTalk Server is a message-based system; it is both data agnostic and transport agnostic, meaning that it can connect to a wide range of different transport protocols and systems and process a wide variety of different data formats.

Once a message has been processed by the adapter and associated pipeline, it needs to be routed to an interested party. A BizTalk orchestration or send port can register its interest in a specific message via subscriptions.

A subscription is made up of a number of conditions to ensure that a subscriber gets the message it requires. When a message is processed, metadata is derived from the message content, the transport adapter, and the BizTalk port that the message was received over and is written to the context of the message. These context properties are evaluated against the subscriptions in order to determine how to route the message.

Subscriptions are criteria on which message routing is determined. Orchestrations, send ports, and send port groups can each subscribe to messages. Each subscription allows the subscriber to initiate or continue the processing of a message. Subscriptions are managed by the MessageBox database.

When a message meets the specifications of a subscription, the message is passed from the MessageBox database to the subscriber. If multiple subscribers exist for a given message, each gets a copy of the message.

The above figure, outlines the steps in the publish/subscribe model, as follows:

1. Subscriptions are created for each subscriber (send port, send port group, or orchestration) by configuring a filter expression. For example, this expression could be for all PO messages with totals over $10,000.
2. A message is received, processed, and published to the BizTalk MessageBox database. Messages can be received through a variety of different adapters, including File, FTP, and HTTP.
3. Subscriptions, which are maintained by the MessageBox database, are evaluated to determine which subscriber(s) should receive a copy of the message.
4. Each subscriber will receive a copy of the message (with a unique message ID). The message stays in the MessageBox database until all subscribers have received and successfully processed the message.

Starting at the receive end, BizTalk Messaging Engine – also called Endpoint Manager or EPM – is responsible for hosting the receive port including the adapter and the pipeline.The adapter picks the message from the wire and passes it to the Receive Pipeline. Within the pipeline, the message passes through the various components. Finally an optional map can be used to transform the message before being sent to the MessageBox.The Message Agent sub-service, which runs as part of a BizTalk host instance – then passes the message to the Host queue for publishing into the Message Box.

In the case of BizTalk receiving a message, the Message Agent will receive a message once it has passed through an adapter and associated pipeline. The Message Agent must first write the promoted properties from the Message Context into the BizTalkMsgBoxDb database by using the bts_InsertProperty stored procedure, which ultimately places the properties into the MessageProps SQL table (all of which happens in a single roundtrip to the MessageBox database).

The promoted properties available will vary depending on the adapter and pipeline combination. The most common promoted property is MessageType (contains the namespace and root node of the associated schema), which provides the message type that is then used for orchestration subscriptions, for example.

The Message Agent then calls the bts_FindSubscriptions stored procedure held in the BizTalk MessageBox to establish which subscriptions, if any, match this message. This stored procedure queries the Subscription and Predicate SQL tables and joins them with the properties inserted for the batch of messages in the previous step to identify potential subscribers.

There is a series of Predicate SQL tables mentioned below which match all of the possible subscription expression combinations.

• EqualsPredicates,
• ExistsPredicates,
• GreaterThanPredicates,
• GreaterThanOrEqualPredicates,
• GreaterThanPredicates,
• LessThanOrEqualPredicates,
• LessThanPredicates, and
• NotEqualPredicates

Depending on the subscription, any number of the preceding SQL tables will be used to represent a subscription.

The Message Agent then calls the bts_InsertMessage stored procedure to insert the BizTalk message into the Spool SQL table, along with basic metadata about the message, such as the number of parts. As part of this stored procedure’s execution, it internally calls the int_InsertPart stored procedure, which places the raw body of the message into the Parts table.

As part of this stored procedure, the int_EvaluateSubscriptions stored procedure, which, among other validation steps, inserts a reference to the message into the relevant Host Queue SQL table, follows the naming convention <HostName>Q.

For example, the default BizTalkServerApplication host will be called BizTalkServerApplicationQ. This is useful when analyzing what work is currently waiting to be processed by which host in your system. See the Hosts in more details.

In the case of a multipart message, the BizTalk Message Agent then calls the bts_InsertMessage stored procedure again for each additional part of the message, which, in turn, is placed in the Parts table.

Subscription processing is complex, so in short you just need to remember that once a message arrives at the Message Agent, subscribers are evaluated, the message is inserted, and references are then added into the host instance queue, ready for processing by the subscriber hosted by the host instance.

Message De-queuing

Each host instance polls the database at a regular interval to look for new work in the queue. Multiple threads (called dequeue threads internally) are used by the host instance, and multiple machines may run a given host instance for resilience and scalability.

The default for the batch size is set to 20 and is configurable on a service-class-wide setting. You can also configure how aggressively the host instance calls the bts_DequeueMessages stored procedure, which in low-latency scenarios can improve the latency (although at the expense of your SQL server).

The architecture of BizTalk and its use of the MessageBox means that messages may be routed based on their content — typically referred to as content-based routing (CBR). CBR is an extremely powerful concept and one that is very easy to configure in BizTalk. You configure a receive location and one or more send ports, which subscribe to properties of the incoming messages. They are then sent to a remote system or endpoint using whatever transport is required.

CBR is a perfectly valid use of BizTalk and, when combined with custom pipeline components, can be all that is required to provide your solution. Most scenarios, however, use orchestrations, enabling BizTalk to be utilized to implement complex business processes.

Subscription Types

In the BizTalk Server, there are two main types of subscriptions: activation and instance.

An activation subscription is one specifying that a message that fulfills the subscription should activate, or create, a new instance of the subscriber when it is received. Examples of things that create activation subscriptions include send ports with filters or send ports that are bound to orchestrations, and orchestration receive shapes that have their Activate property set to true.

An instance subscription indicates that the messages that fulfill the subscription should be routed to an already-running instance of the subscriber. Examples of things that create instance subscriptions are orchestrations with correlated receives and request/response-style receive ports waiting for a response from the BizTalk Server.

Instance subscriptions are created for receive shapes that are following a correlation set, and they include the specific values that are included in the correlation set when it is initialized. The subscriber ID includes the orchestration instance ID.

Instance subscriptions come into play when a correlation set is initiated, as this is when subscriptions are created for all of those ports that follow this correlation set to receive messages. Since the correlation type defines the properties to be used for correlation, the orchestration engine can extract these properties from the message being sent or received by the initiating action. These values are then used to define subscriptions for all of the remaining actions, which follow this correlation set.

Resources:

https://msdn.microsoft.com/en-us/library/aa560414.aspx

http://geekswithblogs.net/cyoung/archive/2004/06/23/7007.aspx

http://mohamadhalabi.com/2013/12/30/biztalk-subscription-architecture-epm-and-message-agents/