Unable to decode COAP packets in wireshark after succesful DTLS decryption

Question

I am trying to debug an LWM2M protocol issue. I need to know what messages are transferred between the COAP server and the COAP client. As the traffic is encrypted using DTLS, I need to give the pre-shared key in wireshark to see the payload. Wireshark is successfully decrypting the payload, but I am not able to see the underlying COAP protocol messages. I am seeing just the raw data("Application data"):

But when I try sniffing unencrypted COAP traffic on a dummy setup, I could see the COAP messages properly:

How can I view the COAP protocol traffic using wireshark when it is encrypted?

Answer 1

The problem is that you're using a non-standard port number for COAPS instead of the IANA-registered port of 5684, and the Wireshark COAP dissector's proto_reg_handoff_coap() function only registers to the DTLS dissector with that fixed, non-configurable port (DEFAULT_COAPS_PORT). There are some possible ways to fix this and/or work-around it.

1. If possible, change your application to use the standard port, 5684.

2. Modify the Wireshark COAP dissector to also register to the DTLS dissector with your desired port, or better yet to allow the port to be configurable, perhaps even with a port range preference to allow multiple ports to be registered. You can reference the Wireshark Developer's Guide for information about compiling and developing for Wireshark on your particular platform.

3. Submit a Wireshark Issue, asking for an enhancement to the COAP dissector to allow the DTLS-registered port to be configurable. Again, a range preference would probably be even better, as it would allow more than just 1 port to be registered.

4. Assuming the first option isn't possible and the other options would take longer and not solve the immediate problem of dissecting the payload as COAP for your existing capture file, you could use a program such as TraceWrangler (or other such Capture file editors and/or anonymizers tools) to rewrite the UDP port value of 57845 to 5684 for all packets. That should allow the payload to be passed to the COAP dissector without requiring any other changes. If you do use TraceWrangler, then the basic steps would be:

   • Add Files: Choose your file, i.e., file.pcapng.
   • Taskname: Anonymize Files.
   • Payload: Deselect "Remove all unknown layers..." as you want to keep everything.
   • PCAPng: Action=Passthrough, as there's no need to replace original comments here.
   • Layer 4, UDP: Action=Replace. Select "Replace UDP ports by list", then choose Add. Enter 57845 for the Original port number and 5684 for the Replacement port number, then click Add.
   • Select Okay.
   • Select Run.
   • When the Status indicates, "Task complete", you should be able to open the newly created packet capture file, named file_anon.pcapng, in Wireshark and Wireshark should now recognize the payload as COAP.

DISCLAIMER: I have not tested this myself, but it should work.