nDPI :: Does Supplying Protocol List Delete "Default" Protocol List?

Question

I have been learning the nDPI library through a lot of trial-and-error and from help on this forum. I recently wrote a new version of my C program (written on Ubuntu, compiler is GCC) and believe I’m properly using the nDPI library. Here’s a broad overview:

#include "ndpi_config.h"

static struct ndpi_detection_module_struct *ndpi_info_mod = NULL;

int main(int argc, char ** argv){

        // Set up nDPI
        // This is adapted from the "nDPI Quick Start Guide"
        NDPI_PROTOCOL_BITMASK all;
        ndpi_info_mod = ndpi_init_detection_module(ndpi_no_prefs);
        if(ndpi_info_mod == NULL) return -1;
        NDPI_BITMASK_SET_ALL(all);
        ndpi_set_protocol_detection_bitmask2(ndpi_info_mod, &all);
        ndpi_load_protocols_file(ndpi_info_mod, 
            "/home/me/myProtocols.protos");
        ndpi_finalize_initalization( ndpi_info_mod ); // Init the Detect Mod

        u_char* buffer = malloc( sizeof(char) * 1000 );

        while(1){

                buffer = capturePacket(); //this works, packet stored as char*

                // ...build all necessary nDPI structs here...

                ndpi_protocol ret = ndpi_detection_process_packet(
                        ndpi_info_mod, flowStruct, buffer, bufferSize, 0, 
                        srcStruct, dstStruct );

                printf("This packet was::  %s  (%d)\n", 
                        ndpi_get_proto_name(ndpi_info_mod, ret.app_protocol),
                        ret.app_protocol );
        }
}

For testing purposes, I deliberately wrote a very short “myProtocols.protos” file:

# Comment field :: myProtocols.protos
tcp:5201@iPerf3
tcp:80@Some_Useful_Protocol

The above compiled and ran very well. I ran some test traffic, including iPerf3, HTTP, FTP, and SNMP. Here’s the output, edited for brevity:

This packet was::  iPerf3  (243)
This packet was::  Some_Useful_Protocol  (244)
This packet was::  Unknown  (0)
This packet was::  Unknown  (0)

…where the first packet was iPerf3, the second HTTP, the third FTP, and the fourth SNMP. Verifying with Wireshark, I’m confident that I’m sending genuine traffic. I’ve played around with nDPI enough to know that if you install it and use it without supplying a protocol file, it will still be able to recognize common applications like HTTP, FTP, SNMP, etc. I was expecting that if you supplied a protocol file, that file would supplement, not overwrite, the “baked in” protocol list. This paper suggests as much in the “Extending nDPI” section. Further, when I commented out this line…

ndpi_load_protocols_file(ndpi_info_mod, "/home/me/myProtocols.protos");

…the program’s output became this…

This packet was::  Unknown  (0)
This packet was::  Unknown  (0)
This packet was::  Unknown  (0)
This packet was::  Unknown  (0)

…which obviously isn’t right.

So it looks like my nDPI can only recognize the protocols listed in my protocol file, and no others. I’m guessing there are two possibilities here:

1. I’ve done something wrong in initiating nDPI and the “prebaked” list of protocols has not been tied in? Or erased? ...or something…?

2. My code that captures the packet is screwing something up and making it unreadable to nDPI.

To be honest, I suspect (2)… but on the off-chance that the problem could be (1), I wanted to ask this forum. Does anyone see something wrong with my approach? Thank you.

EDIT: I added the call to "ndpi_finalize_initalization()" which was missing in the original version of this post. Unfortunately, this does not change the program's behavior...

Answer 1

Loading a protocol file supplements the built-in protocols; it does not overwrite them.

Ensure the decoding works as expected before adding ndpi_load_protocols_file.

It would be better to post your question to nDPI's github issues: https://github.com/ntop/nDPI/issues