Interoperability Gotcha: OAEP Parameterization is not supported

@YaronNaveh

WSE2 is already obsolete but I still see people trying to reach interoperability from WSE2 to WCF. Surprisingly there are a number of interesting ws-security scenarios where this is possible. Today I will discuss one scenario where it may be harder.

WSE configuration allows to configure the algorithms we use for encryption:

<microsoft.web.services2>
...
     <keyAlgorithm name="RSA15" />
...
</microsoft.web.services2>

Some scenarios work great with RSA15 but when we change it to RSAOAEP we may see this exception from the WSE party:

OAEP Parameterization is not supported

Let's see what in the message that WCF sent to WSE may cause this:

<e:EncryptedKey xmlns:e="http://www.w3.org/2001/04/xmlenc#">
  <e:EncryptionMethod   Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p">
      <DigestMethod       Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"       xmlns="http://www.w3.org/2000/09/xmldsig#" />
  </e:EncryptionMethod>
</EncryptedKey>

We can see that the encryptedKey is indeed encrypted with the RSAOAEP algorithm. Also the RSAOAEP RFC states that:

The RSAES-OAEP-ENCRYPT algorithm... takes three parameters. The two user specified parameters are [MANDATORY]... The message digest function is indicated by the Algorithm attribute of a child ds:DigestMethod element...

Hence the DigestMethod element.

So why WSE throws an exception when a mandatory parameter is specified? Let's look at a WSE generated message to see if it produces anything different:

<e:EncryptedKey xmlns:e="http://www.w3.org/2001/04/xmlenc#">
  <e:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p">
    <ds:DigestMethod     xmlns:ds="http://www.w3.org/2000/09/xmldsig#"     Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
  </e:EncryptionMethod>
</EncryptedKey>

Looks the same to me. So let's drill down into the WSE stack to understand the root cause. The error seems to be thrown from within this method:

public override void set_Parameters(string value)
{
   if (((value != null) && (value.Length != 0)) && (value != this._parameters))
   {
     throw new NotSupportedException("OAEP Parameterization is not supported");
   }
}

Ok so we are comparing the current message parameters (value) to some default value (this._parameters). But what is this default? Let's look in the ctor:

public RSAOAEPKeyExchangeFormatter()
{
   this._parameters = "";
}

Oh boy. WSE2 compares xml fragments as strings! So it sees this:

<DigestMethod xmlns="http://www.w3.org/2000/09/xmldsig#" />

as different from this:

<ds:DigestMethod xmlns:ds="http://www.w3.org/2000/09/xmldsig#" />

While the above fragments have the same xml semantics.

Just in order to convince my self this is the issue, I have manually changed the soap so that it will have the WSE-style digest. This worked like a charm. This kind of gives us the general solution here which is to build a WSE filter that changes the soap format to the one WSe expects. Note that you can only change non-signed soap parts, which is usually the case with the encrypted key. The easier option of course is to revert to the RSA15 algorithm.

And let's not forget the main point which is to never compare xml as a string.

@YaronNaveh

What's next? get this blog rss updates or register for mail updates!