DTLS AES 256 CBC with SHA, how do I calculate the final encrypted handshake message

Question

I'm trying to understand how the final Encrypted Handshake message works

To do this, I'm trying to ensure that my algorithm gives the same result as this example that works:

https://www.cloudshark.org/captures/56acf0481a79

I'm quiet sure that DTLS 1.0 is following RFC 4347 due to the 0xFEFF header.

The known are

Preshared Key: 123456789012345678901234567890aa
Server Random: d87eeeb79b8c5bb29a6e01236ca75a00d515ac18a060e7b4dd4aa85d66130b41
Client Random: df14cead6b8a82a7f0fa710ed4437fa747f5f20e160b6865a3486ca3abc1c427
Cipher Suite: TLS_PSK_WITH_AES_256_CBC_SHA

I'm doing the following step:

Pre Master Secret

The premaster secret is formed as follows: if the PSK is N octets long, concatenate a uint16 with the value N, N zero octets, a second uint16 with the value N, and the PSK itself. (RFC 4279 Section 2)

  001000000000000000000000000000000000
  0010123456789012345678901234567890aa

The master_secret = PRF(pre_master_secret, "master secret", ClientHello.random + ServerHello.random) [0..47];

The PRF is defined as combining two different hashing functions. Section 5 of RFC 2246:

197c358a9de99d7c50120aea40af2095
c7c340719385f23f5355004c07d9f896
681942c494eb0d77992c3acf1bc92e4f 

The key_block = PRF(SecurityParameters.master_secret, "key expansion", ServerHello.random + ClientHello.Random) according to RFC4346 Section 6.3

• 20 bytes for a client MAC key (SHA1)
• 20 bytes for a server MAC key (SHA1)
• 32 bytes for a client encryption key (AES256)
• 32 bytes for a server encryption key (AES256)
• 16 bytes for a client IV (AES uses 128-bit blocks)
• 16 bytes for a server IV (AES uses 128-bit blocks)

In this case we need to generate 136 byte

4921654a071c95e2ddb8e3a8162258fa
acffdd8def0a0b7ce49f492a6f088af9
e539aae851232337c90564d6d4b01fb1
0b34466fe379e34b10b5738203453253
3fe0823297ca5c111b3d23dfb6145447
a638a84376f21a845de503b324f2beab
e145274f680519cc2ecc088e0bf6fb37
69b31c82df3ce706f6ac2cb45226234a
dbd564a2b43c79ee

If the above is correct then

Client Write Key: c90564d6d4b01fb10b34466fe379e34b
                  10b57382034532533fe0823297ca5c11
Client Write IV: 2ecc088e0bf6fb3769b31c82df3ce706

Therefore, if I decrypt the client handshake encrypted record with the key and IV above I get

7fd6314cf559a60c14a44a2fd4ac5494
1400000c000200000000000ce93fd3d8
557a3eb9574d25943e01f797b982a5ed
35ce268520ef7475144441ea03030303

How do I get the value above?

I understand that this needs to be hashed (the concatenation of Client Hello + Sever Hello + Sever Hello Done + Client Key Exchange)

010000390000000000000039feffdf14
cead6b8a82a7f0fa710ed4437fa747f5
f20e160b6865a3486ca3abc1c4270000
0006008d008c00ff0100000900230000
000f0001010200003600000000000000
36feffd87eeeb79b8c5bb29a6e01236c
a75a00d515ac18a060e7b4dd4aa85d66
130b4100008d00000eff010001000023
0000000f0001010e0000000001000000
00000010000011000100000000001100
0f436c69656e745f6964656e74697479

Can anyone help?

Answer 1

The first row of the encrypted finished data is IV. Therefore, in your decrypted output you need to remove the first 16 bytes, which gives us:

14 00 00 0C 00 02 00 00 00 00 00 0C E9 3F D3 D8
55 7A 3E B9 57 4D 25 94 3E 01 F7 97 B9 82 A5 ED
35 CE 26 85 20 EF 74 75 14 44 41 EA 03 03 03 03

Starting from the end we have the following sections:

PADDING (4 bytes)

The last four bytes (03 03 03 03) are padding so we may skip them.

MAC (20 bytes)

The next 20 bytes (3e 01 f7 97 b9 82 a5 ed 35 ce 26 85 20 ef 74 75 14 44 41 ea) are HMAC-SHA1 calculated in a peculiar way: you need to first append epoch + message sequence number (in our case 00 01 00 00 00 00 00 00), then add TLS record header (without the epoch and sequence number) with the content size set to the actual content size (0x18 in our case):

00 01 00 00 00 00 00 00 16 FE FF 00 18 14 00 00
0C 00 02 00 00 00 00 00 0C E9 3F D3 D8 55 7A 3E
B9 57 4D 25 94

Let's save it to a file named handshake-client-finished-without-mac.bin and call openssl:

$ openssl dgst -sha1 -hmac $(cat client-mac-key.bin) handshake-client-finished-without-mac.bin
HMAC-SHA1(handshake-client-finished-without-mac.bin)= 3e01f797b982a5ed35ce268520ef7475144441ea

The client MAC key (from the key block that you already generated) is:

49 21 65 4A 07 1C 95 E2 DD B8 E3 A8 16 22 58 FA
AC FF DD 8D

We just verified that the MAC is correct. Let's move forward with the message bytes.

VERIFY DATA (12 bytes)

Next 12 bytes (E9 3F D3 D8 55 7A 3E B9 57 4D 25 94) is the Verify Data. You correctly extracted the bytes from all the previous messages. We need now to calculate MD5 and SHA1 hash of those bytes:

$ openssl dgst -md5 to_hash.bin
MD5(to_hash.bin)= fcc3d19566dc07777834ebddf9dd5dc4

$ openssl dgst -sha1 to_hash.bin
SHA1(to_hash.bin)= 2b069b971ceb63e2f0e6d2687479a10b0aee6abd

Then combine them (md5+sha1) and save them to client-data-for-prf.bin. Finally, we calculate the 12 bytes using the PRF function (I used my PRF implementation from github):

.\PRF.exe -l "client finished" -s .\master-secret.bin -d .\client-data-for-prf.bin -o
 client-prf-result.bin -n 12

The client-prf-result.bin will contain bytes equal to the verify data from the encrypted message.

HANDSHAKE HEADER (12 bytes)

Offset Field
00     Handshake Type: Finished (20)
01     Length: 12
04     Message Sequence: 2
06    Fragment Offset: 0
09    Fragment Length: 12

Side note

If you are ever stuck again when playing with the TLS you may check Wireshark debug logs, which reveal a lot about what's happening on the wire. Right click on any of the DTLS frames and open the protocol preferences:

Then set the pre-shared key that you already have and pick a file where you would like to save the logs:

Click OK and Wireshark should decrypt your TLS data and give you a lot of detailed information in the log file.