What is the best way to post signed content on the internet?

Question

I am currently working on an architecture, where users can post content any server. To ensure the content has actually been posted by a certain user (and has not been altered after being posted), a signature is created using the private key of the author of the content, whose public key is accessible for everyone on a centralized repository.

Problem is, I have no control over how the content is actually stored on these servers. So I might transmit the content e.g. as a JSON object with all data being base64-encoded and the signature is created using a hash of this the base64-encoded content concatenated in a certain order:

{
    "a": "b",
    "c": "d",
    "signature": "xyz"
}

with

signature := sign(PrivKey, hash(b + d);

Now the server will probably store the content of this in another way, e.g. a database. So maybe the encoding changes. Maybe a mysql_real_escape_string() is done in PHP so stuff gets lost. Now if one wants to check the signature there might be problems.

So usually when creating signatures you have a fixed encoding and a byte sequence (or string) with some kind of unambiguous delimiter - which is not the case here.

Hence the question: How to deal with signatures in this kinda scenario?

Answer 1

It is still required to have a specific message representation in bits or bytes to be able to sign it. There are two ways to do this:

1. just store the byte representation of the message and don't alter it afterwards (if the message is a string, first encode it with a well defined character encoding);
2. define a canonical representation of the message, you can either store the canonical representation the message directly or convert it in memory when you are updating the hash within your signature.

A canonical representation of a message is a special, unique representation of the data that somehow distinguishes it from all other possible messages; this may for instance also include sorting the entries of a table (as long as the order doesn't change the meaning of the table), removing whitespace etc.

XML encryption for instance contains canonicalization methods for XML encoding. Obviously it is not possible to define canonicalization for data that has no intrinsic structure. Another (even) more complicated canonical representation is DER for ASN.1 messages (e.g. X509 certificates themselves as well as within RSA signatures).

Answer 2

I think you're really asking two different questions:

How should data be signed?

I suggest using standard digital signature data format when possible, and "detached signatures" at other times. What this means in practice: PDF, Word, Excel and other file formats that provide for digital signatures should remain in those formats.

File formats that don't provide for digital signatures should be signed using a detached signature. The recommended standard for detached signatures is the .p7b file type–A PKCS#7 digital signature structure without the data. Here is an example of signing data with a detached signature from my company.

This means that the "Relying Party" -- the person downloading/receiving the information -- would download two files. The first is the original data file, unchanged. The second file will be the detached signature for the first.

Benefits The signed file formats that directly support digital signatures can have their signatures verified using the file's usual software app. Ie, the free Adobe PDF Reader app knows how to verify digitally signed PDFs. In the same way, MS Word know how to verify signed Word files.

And for the other file types, the associated detached signature file will guarantee to the recipient that the file was not modified since it was signed and who the signer was (depending on the trust issue, see below).

Re database storage -- you don't care how the data is stored on the different servers (database, file system, etc.) In any or all cases, the data should remain unchanged.

How to establish trust between the signer and the recipient I suggest that the organization create its own root certificate. You can then put the certificate as a file on your SSL web site. (Your web site's SSL certificate should be from a CA, eg Comodo, VeriSign, etc.) The result is that people who trust your web site's SSL certificate can then trust your organizational certificate. And your signers' certificates should be chained to your organization's certificate, thus establishing trust for the recipients.

This method of creating a self-signed organizational certificate is low cost and provides a high level of trust. But relying parties will need to download and install your organization's certificate.

If that is not good, you can get certificates for your signers from a public Certificate Authority (CA), but that will drive up the cost by at least an order of magnitude due to the charges from the CA. My company, CoSign, supports all of these configurations.