How to parse rabbitmq status output?

Question

I installed RabbitMQ on Linux, it's a great piece of software.

When I run this command:

sudo rabbitmqctl status

I get a mess of output:

[{pid,18665},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.1.5"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.1.5"},
      {webmachine,"webmachine","1.10.3-rmq3.1.5-gite9359c7"},
      {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.1.5-git680dba8"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.1.5"},
      {rabbit,"RabbitMQ","3.1.5"},
      {os_mon,"CPO  CXC 138 46","2.2.7"},
      {inets,"INETS  CXC 138 49","5.7.1"},
      {xmerl,"XML parser","1.2.10"},
      {mnesia,"MNESIA  CXC 138 12","4.5"},
      {amqp_client,"RabbitMQ AMQP Client","3.1.5"},
      {sasl,"SASL  CXC 138 11","2.1.10"},
      {stdlib,"ERTS  CXC 138 10","1.17.5"},
      {kernel,"ERTS  CXC 138 10","2.14.5"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [rq:1] [async-threads:30] [kernel-poll:true]\n"},
 {memory,
     [{total,179426464},
      {connection_procs,300224},
      {queue_procs,14434024},
      {plugins,474968},
      {other_proc,9607952},
      {mnesia,89264},
      {mgmt_db,1539936},
      {msg_index,85175152},
      {other_ets,29060560},
      {binary,18243208},
      {code,17504466},
      {atom,1602617},
      {other_system,1394093}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,1522479923},
 {disk_free_limit,1000000000},
 {disk_free,58396659712},
 {file_descriptors,
     [{total_limit,924},{total_used,17},{sockets_limit,829},{sockets_used,4}]},
 {processes,[{limit,1048576},{used,233}]},
 {run_queue,0},
 {uptime,5169640}]

It looks like JSON, but it's not.

What data format is this? And how did you find out?

Closest thing I can find is this: http://erlang.org/doc/man/yecc.html

Answer 1

rabbitmqctl has a --formatter flag to request JSON formatted output optionally available. For instance:

sudo rabbitmqctl status --formatter json | jq .disk_free_limit
50000000

Answer 2

Here is a method I made in python for the purpose, which only requires PyYAML (available in pypi). First revision, might be sub-optimal or buggy, but works for me:

import re
import subprocess
import yaml


def fix_dicts(json_str_list, pos):
    '''this recursive function puts all comma-separted values into square
    brackets to make data look like normal 'key: value' dicts'''
    quoted_string = False
    value = True
    value_pos = 0
    commas = False
    is_list = False
    in_list = 0
    while pos < len(json_str_list):
        if not quoted_string:
            if json_str_list[pos] == '{':
                json_str_list, pos = fix_dicts(json_str_list, pos+1)
            elif json_str_list[pos] == '"':
                quoted_string = True
            elif json_str_list[pos] == ':':
                value = True
                value_pos = pos + 1
            elif json_str_list[pos] == '[':
                if value and not commas:
                    is_list = True
                    in_list += 1
            elif json_str_list[pos] == ']':
                in_list -= 1
            elif json_str_list[pos] == ',':
                commas = True
                if not in_list:
                    is_list = False
            elif json_str_list[pos] == '}':
                if not is_list and commas:
                    json_str_list = (json_str_list[:value_pos] + ['[']    +
                                     json_str_list[value_pos:pos] +    [']'] +
                                     json_str_list[pos:])
                pos += 2
                return json_str_list, pos
        elif json_str_list[pos] == '"':
            quoted_string = False
        pos += 1
    return json_str_list, pos


def squash_dicts(input_data):
    # recursively converts [{a:1},{b:2},{c:3}...] into {a:1, b:2, c:3}'''
    if type(input_data) is list:
        for i in range(len(input_data)):
            input_data[i] = squash_dicts(input_data[i]) 
        if all([type(e) is dict for e in input_data]):
            input_data = dict([(k,v) for e in input_data for k,v in e.items()])
    elif type(input_data) is dict:
        for k, v in input_data.items():
            input_data[k] = squash_dicts(v)
    return input_data


text = subprocess.check_output(['rabbitmqctl','status'])
text = text.splitlines()
text = text[1:]  # skip the first line "Status of node..."
text = ''.join(text)  # back into string for regex processing
# quote strings
bad_yaml = re.sub(r'([,{])([a-z_A-Z]+)([,}])', r'\1"\2"\3', text)
# change first element into a key - replacing ',' with ':'
bad_yaml = re.sub(r'({[^,]+),',r'\1:', bad_yaml)
bad_yaml_list = list(bad_yaml)  # into a list for another fix
good_yaml, _ = fix_dicts(bad_yaml_list, 0)
status_list = yaml.load(''.join(good_yaml))
status_dict = squash_dicts(status_list)
# now we can use "status_dict" - it's an ordinary dict
print(yaml.safe_dump(status_dict, default_flow_style=False))

Answer 3

rather than querying the rabbitctrl process I suggest querying the REST api which will return JSON.

GET: http://localhost:15672/api/overview

Here is the documentation:

http://hg.rabbitmq.com/rabbitmq-management/raw-file/3646dee55e02/priv/www-api/help.html

Answer 4

If anyone is interested in using the command line util instead of the UI plugin, you can use this open-source simple parser I wrote: https://github.com/yuvaltir/RabbitMQNet Example of use:

// assume file RmqStatus.txt includes the outputof the rabbitmqctl status command and that the node name is 'serverx'
var statusLines = File.ReadAllText("RmqStatus.txt");
RabbitStatusParser parser = new RabbitStatusParser();
var res = parser.ParseText(statusLines.Replace(Environment.NewLine,string.Empty)); 
var keyWord1 = "Status of node 'serverx@serverx'.memory.mgmt_db"
var mgmtDbMem = res[keyWord1];
var keyWord2 = "Status of node 'serverx@serverx'.file_descriptors.sockets_used"
var socketsUsed = res[keyWord2];

Answer 5

The output format of rabbitmqctl is an Erlang term or also Erlang ETF (External Term Format).

You can use the python library erl_terms to transform the output into something usable by Python:

from erl_terms import decode
from os import getuid
from re import sub
from subprocess import check_output


check_command = ['/usr/sbin/rabbitmqctl', '-q', 'status']

if getuid() != 0:
    check_command.insert(0, '/usr/bin/sudo')

status = check_output(check_command)

## Join into a single line string then add a period at the end to make it a valid erlang term
status = ''.join(status.splitlines()) + '.'
# Remove any literal \n's since the erlang_version item has one in it
status = sub('(?:\\\\n)+', '',  status)

# Decode this into a python object
status = decode(status)

# And now let's find just mem_stat for mgmt_db
for item in status[0]:
    if 'memory' in item:
        for mem_stat in item[1]:
            if 'mgmt_db' in mem_stat:
                print mem_stat[1]