Which language and lib is better suited for high-performant network devices polling server (SNMP)?

Question

I'm working on network devices monitoring solution at work. Prototype is written on Python using PySNMP which is awfully slow and stresses CPU a lot with small amount of devices being monitored. So I thought of making some small high-performance SNMP polling server to get rid of PySnmp and split the program in two (data provider and data analyser/decision maker).

Net-SNMP is not a good choice for SNMPv3 (not thread-safe, not fully async due to synchronous engineID discovery, which is slow if device is not responding) and I don't like SNMP++, so I decided to try writing this server on other languages. Java is not an option (no JVM), so in the end it's Erlang or Haskell (I don't know both of them..). Erlang does have built-in SNMP support, but I don't need process/machine distribution.

So could you please recommend which lang and snmp protocol (manager) implementation is better suited for the task? Requirements:

• high performance (small CPU usage),
• stability,
• high concurrency (a lot of simultaneous polls),
• non-blocking requests,
• full Snmp V1, V2C and V3 support.

Answer 1

Erlang support for SNMP in OTP distribution as application snmp. It means it is stable, matured and battle tested in a corporate environment. It supports:

The SNMP development toolkit contains the following parts:

• An Extensible multi-lingual SNMP agent, which understands SNMPv1 (RFC1157), SNMPv2c (RFC1901, 1905, 1906 and 1907), SNMPv3 (RFC2271, 2272, 2273, 2274 and 2275), or any combination of these protocols.
• A multi-lingual SNMP manager.
• A MIB compiler, which understands SMIv1 (RFC1155, 1212, and 1215) and SMIv2 (RFC1902, 1903, and 1904).

From the point of language, Erlang is strict but dynamically typed functional language with strict evaluation and embedded concurrency (actor based) and strong reliability support. (You can think about is as DSL for writing reliable clustered services.) Erlang compile to bytecode or native (but not as efficient as Haskell) and runs in own VM BEAM. The Haskell is strict and static typed functional with lazy evaluation and no embedded support for concurrency and reliability, but there are solutions in modules. In my personal opinion, Erlang is much smaller language and simpler to learn for somebody going from an imperative language.

Erlang is often not recognized for its speed, but there are real-world applications where Erlang outperforms Java, C++ or C# solutions especially under concurrent load. There is even a study comparing real C++ application with Erlang counterpart. The result is really surprising.

I personally would choose Erlang ower Haskell. There is one major catch, for writing corporate grade solution the OTP is must. Where Erlang as language is small and easy, the OTP is much harder but concurrency and reliability are hard itself and OTP makes it actually easier.

As last remark, there is always possible rewrite time critical parts from Erlang to C or C++ as native implemented functions (NIF) and I found it surprisingly easy in comparison with my previous experience with writing similar functions for Perl or Python.

Answer 2

As a scripting language, I'd expect Python to be fairly slow.

C or C++ ought to be about as performant as you can get. C# or Java probably aren't bad. Erlang should be reasonable. GHC compiles Haskell to native machine code, so it ought to be fairly fast. In the end, this is going to come down to which library do you like the look of the most.

I have no idea what SNMP is or what it does. But I found a Haskell library for SNMP. According to the documentation, a trivial SNMP client might look something like this:

import Network.Protocol.Snmp
import Control.Applicative
import Network.Socket.ByteString (recv, sendAll)
import Network.Socket hiding (recv, sendAll)

-- create new empty packet
v2 :: Packet
v2 = initial Version2

community = Community "hello"

oi = Coupla [1,3,6,1,2,1,1,4,0] Zero

-- set community, oid
packet :: Community -> Coupla -> Packet
packet community oi = 
  setCommunityP community . setSuite (Suite [oi]) $ v2

-- here must be code for create udp socket
makeSocket :: Hostname -> Port -> IO Socket
makeSocket = undefined

main :: IO ()
main = do
   socket <- makeSocket "localhost" "161"
   sendAll socket $ encode $ setRequest (GetRequest 1 0 0) packet
   result <- decode <$> recv socket 1500 :: IO Packet
   print $ getSuite result 

If that looks OK to you, go knock yourself out...