(or how to do asynchronous processing of a synchronous request/response message)
[Note: this post follows on from my previous post How to validate Xml Documents against schemas in BizTalk. For example, if you validate a message in a pipeline and validation fails, how do you send a response back to a waiting web service client.]

For a while now, people have been struggling with how to asynchronously process request/response messages - basically, how to get away from having a request/response port in an orchestration bound to a request/response  receive location.

For example, Yossi Dohan blogged on this a while back, and came up with a solution involving multiple orchestrations:
http://www.sabratech.co.uk/blogs/yossidahan/2006/06/sync-to-async-conversion.html

The problem revolves around the fact that BizTalk will always demote the EpmRRCorrelationToken context property when your message leaves the orchestration if you try and manually set it yourself.

There is a way around this, but it does involve you writing some code.
In fact, it involves writing a custom adapter and/or pipeline component. This is how we solved it at one of my clients.
The trick is to promote the property outside of the orchestration – and that can only be done in a send pipeline or send adapter.
If you're not up to writing your own adapter, you can use the loopback adapter and a custom pipeline component.
What you have to do is promote the EpmRRCorrelationToken context property in your pipeline or your adapter.

Which means that with a custom pipeline/adapter, you can asynchronously process request/response messages without even needing orchestrations (e.g. in high-perf situations where you can do everything in pipelines).
The code to promote the property is very simple.

First of all (for those of you who haven't come across this), let's explain the issue (skip this and go to after the primer if you wish):
<Begin Primer>

---

Take a typical example of BizTalk work: exposing an orchestration as a web service.
You have an orchestration which takes in a message, does some processing, and then returns a response.
If you've used the Web Services Publishing Wizard, your orchestration will have s request/response port which is used to receive messages from the web service, and send a response.

When your request message is submitted to the BizTalk MessageBox (for pickup by the Receive port on your orchestration) there needs to be some way to return the response back to the same service instance which submitted the message in the first place. This will make sure the response goes back out as a response to the HTTP POST which submitted the request in the first place.

BizTalk achieves this by:
1.    Creating a new subscription using the ID of the service instance waiting for a response
2.    Setting correlation properties on the Request message which match this subscription
The two properties used are:

EpmRRCorrelationToken
RouteDirectToTP

When you send a message back through the Send port of your orchestration, once of the things that happens internally is that these properties (from the request message) are promoted onto the response message you're sending.
The orchestration places the message in the MessageBox, and the waiting ServiceInstance picks up the message as it has a matching subscription:

http://schemas.microsoft.com/BizTalk/2003/system-properties.EpmRRCorrelationToken == Svr:DANIEL-PC;PID:836;RRUid:{C5120F0D-06F7-4BDF-8A45-79B391AF56A0}  And
http://schemas.microsoft.com/BizTalk/2003/system-properties.RouteDirectToTP == True
Figure 1. The subscription for the ServiceInstance Response channel



Figure 2. The Context Properties on the Request message

So, if you want to create your own response message, all you have to do is promote the same properties onto a message (note that you can't just set them – they have to be marked as Promoted as well)
In fact, as the subscription is not MessageType specific, you can return any message you like back (as long as it deserialises into something expected by the web service code).

Here's an example of where you might want to do this: Failed Message Routing.
If the pipeline processing your request message throws an exception, and you have Enable routing for failed messages switched on in your receive port, then your message will be submitted to the message box, and a new Error Report message will be created with all the original message properties demoted, and some new error properties promoted:


Figure 3. The Context Properties for an Error Report message

As you can see although the RouteDirectToTP property has disappeared, the EpmRRCorrelationToken property (the one we need) is still there.

A new feature in BizTalk 2006 is that if a message fails in the pipeline, it is still written to the MessageBox and an error report generated i.e. you still have access to the original request message even if it's not XML.
In BizTalk 2004, if a pipeline exception occurred you'd lose the message (without writing a custom pipeline component).
If you subscribe to the Error Report, what you will receive is the original message (as opposed to the error report message which has no actual data).

So if we wanted to send this original message back to the waiting service instance, all we'd have to do is:
1.    Add the RouteDirectToTP property
2.    Promote both of these properties

And herein lies the problem that many people have discovered: orchestration send ports contain specific code which will not promote any user-set instances of EpmRRCorrelationToken.
The only time this property is automatically promoted is when it's part of a request/response port bound to the actual ReceiveLocation.

So what this means is that you can't promote this property in an orchestration.

---

<End Primer>

So how do you promote the EpmRRCorrelationToken property before the message gets into the MessageBox? Easy - you have to execute the following code from a custom adapter or pipeline component:

// Check if EpmRRCorrelationToken is present in message context
object value = inmsg.Context.Read("EpmRRCorrelationToken", "http://schemas.microsoft.com/BizTalk/2003/system-properties");
if (value != null)
{
      string empToken = (string)value;
      bool directToTP = true;
      inmsg.Context.Promote("EpmRRCorrelationToken", "http://schemas.microsoft.com/BizTalk/2003/system-properties", empToken);
      inmsg.Context.Promote("RouteDirectToTP", "http://schemas.microsoft.com/BizTalk/2003/system-properties", directToTP);
}
(ideally, you'd use the Types from Microsoft.BizTalk.GlobalPropertySchemas to obtain the context property names and namespaces, as this means you'll always have the correct namespace versions)

So all you have to do is create a SendPort which uses your custom adapter (or the Loopback adapter and a custom pipeline component with the above code), connect the SendPort to your orchestration (or create a filter on the SendPort if you're not using orchestrations) and the message will be submitted to the message box, to be picked up by the waiting service instance.

Bear in mind that your service instance will attempt to deserialise the message received into an appropriate response message - which means that if you're using typed messages (as opposed to XmlDocuments) you may need to run a map or create a new response message, otherwise the service instance will barf.

I have a full sample showing how to do all this without needing any orchestrations which I'll post up if anyone asks.
Additionally, I should get round to posting up the code for the custom adapter we use - it saves you having to write your own pipeline component.

Update (20/03/2008):
I had a reader email me and ask for the source code for the adapter I mentioned.
So I dug it out, cleaned the dust off it, and here it is.
There are two downloads:
1) EpmCorrelationAdapter.zip
This is the actual adapter project (all source included plus a Setup project which builds an MSI for the adapter).
It's a loopback adapter which promotes the EpmRRCorrelationToken property and (optionally) creates an ErrorReport response message.
So you can use this Adapter to do web service request/response processing without neededing an orchestration.

2) AdapterTestProject.zip
This is a BizTalk project and Web Service for testing the adapter and showing how to catch XmlValidation errors and return an error message without using an orchestration.


Get the files here:
EpmCorrelationAdapter.zip (63kb)
AdapterTestProject.zip (25kb)