BizTalk Settings Dashboard @ Host-Instance Level

Using the settings Dashboard you can modify the configuration information of a given host-instances, across a BizTalk group. This is further classified in two tabs:

  • Net CLR: Use this to update the number of Windows threads available in the .NET thread pool associated with an instance of a BizTalk host.
  • Orchestration Memory Throttling: Use this to control the Orchestration memory throttling.

Note that at the top of this dialog you can choose which host instance you are currently modifying. Settings you apply across this tab and the Orchestration Memory Throttling tab apply to a particular host instance.

.Net CLR

HostInstanceSetting.png

It is important to allocate enough threads to the .NET thread pool associated with an instance of a BizTalk host to prevent thread starvation. At the same time, care should be taken to prevent allocating more threads to the .NET thread pool associated with a host than is necessary.

The recommended value for CLR threading settings are shown below.

HostInstanceRecommededSettings.png

Orchestration Memory Throttling

MemoryThrottlings.png

Above figure shows throttling settings for orchestrations based on memory usage.

It makes sense for these to be per host instance (rather than host) because different instances may have different physical memory usage patterns. When memory use (either physical or virtual) reaches the optimal usage level, dehydration throttling begins in an effort to preserve available memory at the set level.

When memory reaches the maximal usage, dehydration throttling is at its most aggressive (highest severity).

OrchMemorySettingsDesc.png

A final topic regarding the Settings Dashboard is the ability to import and export all the settings discussed to a single Extensible Markup Language (XML) file.

Note the Import and Export buttons at the bottom of the dialog. You can use these for backup purposes, or (if the file is edited to account for differences in host/host-instance names) to duplicate settings for another BizTalk group.

Please refer the previous post for the setting @ host level.

Related Links

https://msdn.microsoft.com/en-us/library/ff629695.aspx

https://gautambiztalkblog.com/2015/05/04/biztalk-setting-dashboard-group-level/

BizTalk Setting Dashboard @ Host Level – Part 2

In this post I will discuss about the Rate-Based Throttling and Orchestration Throttling. Please refer the previous post for the General Setting and Resource-Based Throttling @ host level.

Rate Based Throttling

Rate-Based Throttling allows you to configure rate-based criteria when the host will go into a throttled mode.

RateBasedThrottling.png

The Publishing portion refers to the rate at which messages are published from this host (to downstream hosts).

The Delivery portion refers to the rate at which messages are delivered to the orchestration or messaging engine (within this host).

This set of controls lets you determine what will be the definition of too great a mismatch between inflow and outflow and what will be the resulting action.

Specifically, Minimum Number of Samples and Sampling Window Duration work together to define how much traffic will be observed to determine real operational rates.

Rate Overdrive Factor refers to the allowable mismatch in the producer/consumer relationship.

Maximum Throttling Delay is the maximum delay BizTalk will impose (at highest severity throttling).

Throttling Override allows you to manually introduce throttling, or disable it altogether. If you manually introduce it, Throttling Override Severity allows you to control the severity level for that case.

 

Publishing

PublishingThrottling.png

Delivery

DeliveryThrottling.png

 

Orchestration Throttling

OrchestartionThrottling.png

Orchestration Throttling allows for fine-grained control over dehydration/rehydration.

When Dehydration Behaviour is set to Custom, the Maximum Threshold is used to determine the maximum idle time (blocked and waiting for a message) before dehydration, and Minimum Threshold sets the minimum idle time. You can see by default the minimum is 1 second and the maximum is half an hour.

If you check the Subscriptions check box, you are overriding the behaviour of when the MessageBox will decide to pause/resume moving messages to the subscription instance based on the number waiting to be consumed.

OrchThrottlingDetails.png

Related Links

http://social.technet.microsoft.com/wiki/contents/articles/6983.biztalk-server-2010-host-throttling.aspx

https://msdn.microsoft.com/en-us/library/ff629757.aspx

https://msdn.microsoft.com/en-us/library/ff629703.aspx

BizTalk Setting Dashboard @ Host Level – Part 1

In last post I discussed about the Group Level Settings. This post is about Host Level Settings.

Using the settings Dashboard you can modify the configuration information of a given host, across a BizTalk group. This is further classified in four tabs:

  1. General Settings
  2. Resource-Based Throttling: monitors system resources such as threads, memory, and database size and can be applied to any service class.
  3. Rate-Based Throttling: Inbound (Published) & Outbound (Delivered)
  4. Orchestration Throttling: Controls the number of outstanding messages an Orchestration may have. Directly related to BtsNtSvc.exe.config

 

General Settings

HostSetting-General.png

Move Tracking Data to DTA DB

It refers to whether this host will use the tracking database (and tracking tables in the Message Box), and as a consequence, indicates whether this host (and all objects running in it) has read/write permissions to these locations.

If you do not select the check box, the host will have only write access to the tracking tables in the MessageBox database and will not have access to the Tracking database.

Trusted authentication

It refers to whether a particular host is allowed to collect authentication information (via digital signature, Windows authentication, and so on), and subsequently stamp a message with a party ID and a Windows Security ID.

If the host is not configured with Trusted Authentication, the Message Box just overwrites the party ID with the guest ID, and the SSID with that of the host instance’s service account.

32-Bit only

Indicate whether the host instance process should be created as 32-Bit on both 32-Bit and 64-Bit servers.

Basically 32-Bit Only forces 32-bit host instances on 64-bit servers, typically used when you have legacy native components involved (perhaps COM-based.). Also, certain adapters are 32- bit only.

Default app-domain for isolated adapter

Indicate whether the isolated adapter runs in the default app domain, or the domain of the caller.

Legacy whitespace behavior

It allows you to indicate that you want to preserve whitespace when creating maps.

Allow multiple responses

Indicate whether you want to enable multiple responses to be sent back to a 2-way receive location.

Response timeout

It basically indicates the default timeout for request-response messages (such as those originating from a Hypertext Transfer Protocol [HTTP] transport and configured as two way).

Maximum engine threads

Indicate the maximum number of messaging engine threads per CPU.

This option specifies the maximum number of threads that can be used by the End Point Manager (EPM). The EPM starts with the number of threads equivalent to 10% of this value and adds threads up to the specified value as load increases. The number of threads allocated is reduced as load is reduced or as necessary for throttling.

Show performance counters for

It allows you to set whether the Message Agent counters are showing activity for orchestrations or messaging activity.

When set to Messaging, Performance Monitor will display Message Agent counters for messaging. If the host contains orchestrations, no Message Agent output for the orchestration (XLANG) instances will display.

If the host only contains orchestrations, change the Show performance counters for setting to Orchestrations to display Message Agent counters for Orchestration instances. If the host only contains receive ports/send ports, keep the Messaging option to display Message Agent counters for Messaging instances.

Polling Intervals

Polling Intervals allows you to configure how often BizTalk will look for new messages and new orchestration activity. Tuning these parameters can help in low latency scenarios, depending on where overall processing time is spent.

  • Messaging – Set the BizTalk Server polling interval in milliseconds when BizTalk host instance is looking for new messages in the MessageBox.
  • Orchestrations – Set the BizTalk Server polling interval in milliseconds when BizTalk host instance is looking for new orchestrations in the database.

 

Resource Based Throttling

ResourceBaseThrottling.png

 

Per CPU Settings

RBT1.png

Memory Usage

RBT2.png

Severity

The Severity settings refer to what severity will be assigned for memory-triggered, database size-triggered, or in-flight message count-triggered throttling conditions. These values come into play when BizTalk is deciding what type of throttling instruction to issue because this is based on the condition with highest severity.

RBT3

Please refer the next post for the Rate-Based Throttling and Orchestration Throttling @ host level.

Related Links:

https://msdn.microsoft.com/en-us/library/ff629797.aspx

https://msdn.microsoft.com/en-us/library/ff629725.aspx

 

 

Configuring Hosts and Host Instances

BizTalk solutions typically consist of a collection of executable artifacts – receive locations, pipelines, orchestrations, send ports, etc.  BizTalk provides hosts as a way of logically grouping these artifacts and then allows you to instance each host on one or more BizTalk server.

A host instance represents a physical runtime process/windows service running on a BizTalk server.  This mechanism provides a very flexible solution in which to host BizTalk application components and there are several factors which should be taken into account when distributing your application across multiple hosts.

BizTalkArch.png

Although a single BizTalk Host can contain items that receive, send, and process messages, it is considered a best practice to create different hosts for each function to create security boundaries and for easier management and scalability.

In particular, it is recommend that you use different hosts for processing and for receive/send operations, and that you separate trusted and non-trusted items.

For example, if you receive one message, run an orchestration, and send ten messages, you want to separate the receive and send functionality into two separate hosts because the send items will have ten times more traffic than the receive items. If you receive one message, run an orchestration, and send one message, you can think of these items as a unit of work and group them into one single host. Alternatively, you can separate them into three different hosts to increase performance and flexibility.

Separating Host Instances by Functionality

A general best practice, is to use a separate host for receiving, processing, sending and tracking.

This provides flexibility when configuring the workload in your BizTalk group and is the primary means of distributing processing across a BizTalk group. This also allows you to stop one host without affecting other hosts. For example, you may want to stop sending messages to let them queue up in the MessageBox database, while still allowing the inbound receiving of messages to occur.

Separating host instances by functionality also provides the following benefits:

  • Each host instance has its own set of resources such as memory, handles, and threads in the .NET thread pool.
  • Multiple BizTalk Hosts will also reduce contention on the MessageBox database host queue tables since each host is assigned its own work queue tables in the MessageBox database.
  • Additional host separation should also be considered when dealing with artifacts with scale-out limitations; those which suit clustered BizTalk hosts (MSMQ receive, POP3 receive, in-order processing, etc).  In these cases, artefacts with scale-out limitations can be grouped into one host and clustered, while all others can remain in hosts which support multiple running host instances
  • Throttling is implemented in BizTalk Server at the host level. This allows you to set different throttling characteristics for each host.

Reliability

Separating application processing across hosts ultimately leads to multiple runtime processes hosting your application.  As BizTalk can host custom written components (custom adapters, custom pipeline components, custom functiods, etc), host separation can be used to provide process isolation and protection.

If you’re experiencing instability in any custom components, you should consider isolating them into their own BizTalk host.  This will help prevent these unstable components affecting the overall application processing.

Security

Host separation should also be a consideration when planning for security:

  • Each BizTalk host is assigned a Windows Group which is used to permission the host specific tables in the BizTalk databases.  This means that a BizTalk host provides a security boundary within the database and should be considered when dealing with highly sensitive data.
  • Certificates can be utilized in BizTalk for message signing and encryption.  Certificate thumbprints for these functions are assigned at the host level and therefore you may have a need to separate hosts in order to represent different identities.
  • All host instances are assigned a service account to run under.  Access to external systems is often a requirement for this account in order to process outbound messages through the adapters. In circumstances where access to sensitive external systems is restricted, host separation can be used to move the artifact which access these systems into their own dedicated hosts where specific restricted user accounts can be assigned.

The recommended starting point for host configuration/separation for any BizTalk solution is to separate receiving, orchestration processing, sending, and tracking functionality into separate hosts.  From this point, other aspects discussed above should be considered and tested to ensure the desired outcome of any changes is achieved.

Dedicated Tracking Host

A tracking host is very important for overall health of your BizTalk environment; it’s responsible for moving all the DTA and BAM tracking data from your message box database to BAM and DTA database. If you don’t have a tracking host, over a period your message box will get bloated and degrades you’re overall environment performance.

BizTalk Server is optimized for throughput, so the main orchestration and messaging engines do not actually move events or messages directly to the BizTalk Tracking (DTA) or Business Activity Monitoring (BAM) databases, since this would divert these engines from their primary job of executing business processes. Instead, BizTalk Server leaves the events and messages in the MessageBox database and marks them as requiring a move to the BizTalk Tracking or BAM databases. A background process (the tracking host) then moves the events to the BizTalk Tracking and BAM databases, while a SQL Server Agent job copies tracked messages to the BizTalk Tracking database.

Using a dedicated tracking host allows you to stop other BizTalk hosts without interfering with BizTalk Server tracking. The movement of tracking data out of the MessageBox database is critical for a healthy BizTalk Server system. If the BizTalk Host responsible for moving tracking data in the BizTalk group is stopped, the Tracking Data Decode service will not run. The impact of this is as follows:

  • HAT tracking data will not be moved from the MessageBox database to the BizTalk Tracking database.
  • BAM tracking data will not be moved from the MessageBox database to the BAM Primary Import database.
  • Because data is not moved, it cannot be deleted from the MessageBox database.
  • When the Tracking Data Decode service is stopped, tracking interceptors will still fire and write tracking data to the MessageBox database. If the data is not moved, this will cause the MessageBox database to become bloated, which will impact performance over time. Even if custom properties are not tracked or BAM profiles are not set up, by default some data is tracked (such as pipeline receive / send events and orchestration events)

High Availability for BizTalk Hosts

Following table summarizes the high-availability guidelines for the most common adapters in BizTalk Server.HAguidelines.png

Related Links:

http://social.technet.microsoft.com/wiki/contents/articles/7027.biztalk-server-host-architecture.aspx

https://msdn.microsoft.com/en-us/library/ee308915(v=bts.10).aspx

https://msdn.microsoft.com/en-us/library/aa577430.aspx

https://sandroaspbiztalkblog.wordpress.com/tag/host-instances/

BizTalk Host and Host Instance

A BizTalk host is a logical container representing one or more BizTalk Server run-time instances. Hosts are virtual process boundaries that provide an administrative and a security context for running instances of a BizTalk application. A BizTalk Host provides a container for items such as adapter handlers, receive locations (including pipelines), and orchestrations.

Once you have defined a BizTalk host, you can then create instances of it. When creating an instance, you specify the name of a physical server for the host instance to execute on along with a service account for the host to run under.

The following figure shows the relationship between servers, hosts, and host instances.

BizTalkHost.png

A BizTalk group is an organizational unit. Multiple computers running BizTalk Server can be in a single BizTalk group, but all servers in the group share a BizTalk Configuration database (the BizTalkMgmtDb database).

BizTalk-Group.png

When you create a new Host in BizTalk all you are doing is creating an entry in a table adm_Host in the BizTalk Management database and nothing more.  You can attach an Orchestration to this Host, but it won’t run until you create a Host Instance.  When you create a Host Instance you actually create a Windows Service and this is what does the work.  So you can start and stop Host Instances but Hosts have no such concept.

You cannot add a BizTalk server to the same host more than once. A single host instance can be added to multiple hosts.

Items—such as adapter handlers, receive locations (including pipelines), and orchestrations—contained in BizTalk hosts can perform receiving, sending and processing of messages.

For every host created, a corresponding set of queue table is made in BizTalk Message Box database

  • BizTalkServerApplicationQ (main queue) – host polls this table to collect message pending for processing.
  • BizTalkServerApplicationQ_Scheduled (scheduled queue) – this is created but never actually used by BizTalk.
  • InstanceStateMessageReferences_BizTalkServerApplication (state queue) – holds the list of messages that have been processed but are required later
  • BizTalkServerApplicationQ_Suspended (suspended queue) – suspended message are stored here.

Hosts can be either in-process or isolated.

In-process Hosts

An in-process host instance executes as a Windows service. The BizTalk tools automatically create this service when a new host is created.

In-process hosts represent service instances that an administrator creates, deletes, and fully controls with Windows Management Instrumentation (WMI) and the BizTalk Administration Console.

In-process hosts have the following characteristics:

  • You can enlist any orchestration into an in-process host.
  • An in-process host can host any send handler.
  • An in-process host can host any of the receive handlers except for SOAP and HTTP:
      • FILE
      • FTP
      • MQSeries
      • MSMQ
      • POP3
      • SQL
      • Windows SharePoint Services
  • The first in-process host you create in a BizTalk Server deployment is the default host and you cannot delete it. The BizTalk Message Queuing adapter uses the default host for static handler configuration. Adding an adapter automatically creates receive and send ports for the default host.

Isolated Hosts

The isolated host is used in conjunction with the HTTP and some WCF receive handlers. They primarily host adapters that are designed to run outside of the normal BizTalk Server runtime process; for example, external processes such as Internet Information Services (IIS) application pools.

Isolated hosts have the following characteristics:

  • You cannot enlist orchestrations into an isolated host.
  • An isolated host cannot host send handlers.
  • An isolated host can host only the receive handlers associated with HTTP/S and SOAP adapters (the isolated-type adapters).
  • An isolated host cannot host tracking.
  • An isolated host cannot be the default host.
  • The status of an isolated host is always Status Unavailable. BizTalk Server does not access the status information for external processes.

The BizTalkServerApplication host is the default host for any adapters or orchestrations that you create and for outbound SOAP messages generated by BizTalk, whereas the BizTalkServerIsolatedHost is used for messages received via the SOAP adapter.

There are several good reasons for this split. In-bound SOAP messages are handled by Internet Information Server (IIS), not the BizTalk Server engine, unlike other adapters. Also, you can think of messages being received from IIS as being far less trusted than those from other adapters; in many cases they will be received over an untrusted network, such as the Internet. Many adapters, such as the SOAP adapter, need to be hosted by processes other than the BizTalk NT Service. The isolated host model enables this and effectively means that an adapter may be hosted outside of BizTalk.

For these scenarios, the adapter actually loads the BizTalk Messaging Engine and, subsequently, the Message Agent into its process address space. This ensures that the same out-of-process communication is required regardless of whether the adapter is hosted by BizTalk or as an isolated adapter.

This split allows you to run an isolated host under a different service account with bare minimum permissions to the BizTalk MessageBox, delivering a great security solution. Of course, you can configure each BizTalk host instance with a different service account, enabling the same approach to be used throughout your deployment.

32 bit vs. 64 Bit Hosts

A 32-bit host instance can only address to a maximum of 2 GB. That limits the amount of memory that a process can take. Therefore, as a guideline, it is recommended to configure hosts as 64-bit hosts by not selecting 32-bit only as a 64-bit hosts can address a much higher memory space (up to 7 TB). However, the FTP, POP3 and SQL adapters are not supported to run on 64-bit host instances, and should be run in 32-bit host instances.

Trusted and Untrusted Hosts

BizTalk Server enables hosts identified as authentication trusted to indicate that the sender of a message that the trusted host is queuing to the MessageBox database is an entity other than the trusted host itself. The primary purposes of authentication trust are to enable pipelines to resolve to a Product ID (PID) and pass that PID along to consuming services for use in authorization and outbound party resolution, and to enable the transmission of the sender Windows Security ID (SSID) along to consuming services for use in orchestration action authorization.

For more on authentication and authorization, refer Security Features in BizTalk Server

Related links:

https://msdn.microsoft.com/en-us/library/ee276854(v=bts.10).aspx