How to validate Xml Documents against schemas in BizTalk

Daniel Probert — Sat, 01 Sep 2007 10:16:55 GMT

I got asked a question the other day: How would you validate an incoming message against a schema if the message was the request part of a request-response pair and you wanted to return a response if the request wasn't valid?

In the example given, an orchestration had been exposed as a web service, and the requirement was to validate the incoming message. If the message did not validate they wanted to return a response message with an error message in it.

I gave two of the ways I would do it, but that wasn't what they were expecting: they were expecting the simplest (and computationally slowest) way of doing it. And I realised that many people use this mechanism as they don't know there's any other way.

Why do I say this? I'll explain as I give my solutions.
First of all: The solution that was expected was to use an orchestration to do the validation - as the person explained to me, that was the only way to get the response message back to the same "connection" i.e. have it go back out as a response to the matching request.
As you'll see this is not true.
In this post I'll cover the ways to do validation.
In the next post, I'll cover how you correlate the response back to the client who is waiting for a response.

Let me say one thing: BizTalk is not magic. There is no magic (thanks Nakor). There's simply some COM+ applications, some .NET assemblies, instances of a Windows Service, some database tables... and a lot of unmanaged code.
What gets confusing[...]

Validating Schemas in WebMethods using Attributes

Daniel Probert — Sun, 15 Apr 2007 16:44:54 GMT

One of the most annoying things in BizTalk is when an error is generated in a Receive Pipeline indicating that a message has been suspended because of the content of the message doesn't validate against the given schema (for the moment, we'll gloss over the fact that in an ideal situation this should only ever happen during testing).

Using BizTalk 2006's new Failed Message Routing feature, it's fairly simple to write something which can deal with this situation (using either an Orchestration or SendPort subscribing to the Error Report).
Returning a message back to a Web Service caller indicating that there is something wrong with their message, is non-trivial – it's possible, but requires some in-depth knowledge of how instance subscriptions work.

However, there is an easier way: validating the message at the endpoint (i.e. inside the Web Service itself).
Aaron Skonnard and Dan Sullivan wrote an article for MSDN magazine in July 2003 on how to implement attribute based schema validation for Web Method parameters.

It's a great idea: decorate a Web Method with an attribute, and when the web method is called then any messages received will be validated against their schemas – if they don't validate, then a SOAP Fault is returned back to the caller[...]

Web Services and the 15 Second Delay

Daniel Probert — Sat, 10 Mar 2007 12:01:13 GMT

Background: The large(ish) BizTalk system I've been working on for the last few months consumes a few external Web Services. Whilst doing some early performance tuning, I noticed that I was getting pretty serious request-response latency from the primary BizTalk Web Service... Like over 45 secs, instead of the expected 0.5 – 2 secs...

Fortunately, I routinely put tracing/audit calls in my BizTalk code which includes timing info - and I always log the times around orchestration entry/exit... So I was quite surprised to see that a request to one of the external Web Services was taking 15secs to respond... especially as the web service was hosted (temporarily) on the same server...

Although I didn't have the source code, with the help of Lutz Roeder's trusty .NET Reflector I was poking around the innards within minutes… and couldn't find anything that would cause a delay of 15 secs.

What's more, after further testing I found that the problem only occurred the first time after IIS was restarted. After the first time, it would return within 2 secs (still not very good, but much better). One thing I noticed about the Web Service was that it made use of some code in an unmanaged assembly (old school C++ DLL)...

On a hunch, I wrote a simple test Web Service which simply referenced some functions in the DLL (using the DllImport attribute)... and bang, a 15 sec delay before the Web Service constructor was called.

I also tried the same trick in a console app... and there was no delay.

By this time I was fairly stumped. This issue only happened on our Load Test server: it didn't happen on the Dev or Test servers. And it seemed to only happen to code hosted under IIS.

I figured it was time to dig a bit deeper and see what IIS was doing: starting up Process Monitor (to monitor file and registry access) and TCP View (to monitor network activity), I called my test Web Service.

Filtering the Process Monitor logs to the w3wp.exe process (the IIS Worker Process for Application Pools) I noticed that w3wp was reading a lot of security and cryptography keys from the registry:

Hmm. Interesting.

Then I looked at the TCP View log. And I noticed something very interesting coming from the w3wp process: a connection was being attempted to http://crl.versign.com and was being blocked.

How do I know it was being blocked? Well, the TCP View log showed a SYN_SENT, which appeared and was visible for 15 secs before disappearing:

Now if you're not familiar with how TCP works, then you might miss the significance of this: When a TCP client attempts to contact a TCP server, the first thing performed is a handshake. This involves the client sending a SYN, the server responding with a SYN-ACK, and the client sending a SYN-ACK-ACK.

In the TCP View tool, the rightmost column indicates the state of the local TCP connection end-point. You'll see that in this case, the state is SYN_SENT: this state indicates that the local TCP client is waiting for a SYN-ACK response from the SYN it sent.

Why would this be? Well, many reasons, but mostly because the address can't be reached e.g. a proxy server is required, or a firewall rule is blocking the call.

And what's the relevance of crl.verisign.com? A CRL (Certificate Revocation List) is used to indicate which Certificates (issued by a Certificate Authority) are no longer valid.

And this is only done when you are using an assembly which has been digitally signed.

(Note: Microsoft became really really serious about CRL's back in 2001 when VeriSign accidentally released two Microsoft certificates to a third party – which meant that the third party could sign code that appeared to have come from Microsoft).

So what does this have to do with my Web Service and a 15 second delay?

Looking through the assemblies in the Web Service, I noticed that the unmanaged assembly was digitally signed. Aha! So when the unmanaged assembly was referenced in a Web Service, IIS would hand off control to the Windows Cryptography API (CAPI) and ask it to verify the certificate. And this involves retrieving a CRL for the certificate.

After doing some more poking around, I found this blog post (from one of the developers at Xceed Software, a company specializing in .NET components).

The thing that drew my attention was this comment:

Some people behind a very strict firewall, ignoring outgoing connections instead of rejecting them, had a timeout while loading our assemblies. That timeout (at least 15 seconds) is hardcoded in WinVerifyTrust!

Eureka ! That's exactly my problem!

And it explained why it was only happening on our Load Test server: Our Dev and Test servers have access to the internet, whilst the Load Test server does not: it has no proxy server configured.

To test this theory, I configured the Load Test server to use a proxy server. If you haven't done this before, you can't do it via Internet Explorer settings – that only configures a proxy for application which go via wininet.dll i.e. the Internet Explorer API, which uses the proxy server settings in the Internet Explorer/Internet Options/Connections/LAN Settings panel.

CAPI, however, uses winhttp.dll (Microsoft Windows HTTP Services). And to set the proxy server for that, you need to use the proxycfg command line tool which sets proxy server settings for winhttp.

If you run proxycfg –u, this will import your Internet Explorer proxy server settings:

So I did this, restarted IIS and presto! The problem went away.

However, giving the Load Test server access to the Internet is a fairly large security hole (at least it is in our network, where the Load test server is behind both the firewall and DMZ).

So, we contacted the vendor who supplied the unmanaged DLL, and managed to get them to release a new version which wasn't digitally signed.

Which solved the problem of the latency in the Web Service (which accounted for 15 secs of the 45 secs latency).

Solving the remaining 30 secs latency is the subject of next week's post.

BizBert - Web Services

How to validate Xml Documents against schemas in BizTalk

Validating Schemas in WebMethods using Attributes

Web Services and the 15 Second Delay