MongoDB Sharded, Replica'd Cluster, Query commands for all configuration / running statuses?

Question

I find that I have gotten confused on all the different commands I can use as an admin to discover what might be wrong with my MongoDB cluster. For example, running the split command, I got an error that "pre-condition failed". I found the part of the code related to its error message, 13105, https://github.com/mongodb/mongo/blob/master/src/mongo/client/syncclusterconnection.cpp#L219 , but I'm still confused on what I am seeing.

So I wanted to systematically check every part of my cluster. These are the commands I have run and remember right off the bat, but I'm pretty sure I'm forgetting some! I was never a DBA before looking at Mongo, so it would really help me to have a debugging checklist of info to get.

So my question is, have I got all the commands I need to , to get the status and configuration of each aspect of my cluster from a Mongo Console? For example, I feel sure I was able to see the sizes of each shard chunk used, but I didn't get that yet. thank you!

Update 2014-11-22, modified the list below to include my summary findings and @wdberkeley's answer. //TIPS For MongoDB Debugging //1.Get General info on configs passed and the IP's (in Bash) ps aux | grep -in mongo //1A.Alternatively, to see details of configs on a single server use admin db.serverCmdLineOpts() //2.Shard chunk/ranges status (from the mongoS balancer node) sh.status() //2A.Shard Status, verbose output sh.status(true) //3.Replica Set Status (from a replica set node, NOT from a mongoS) rs.status() //4.Mongo Server info (from anywhere; VERY LONG output) db.serverStatus() //5.Log summary from configured file or copy/paste of system.output into a .log file (from Bash) mloginfo myCapturedLogs.log --distinct //6.Diagnostic tools (from Bash) mongostat mongotop //7.Check Q&A / Reference sites.
Refs; [6] The amazing mtools includes log parsing and log timeline visualization. https://github.com/rueckstiess/mtools [7] Q&A / Reference sites' URLS; 7A.mongo Shell Quick Reference http://docs.mongodb.org/manual/reference/mongo-shell/ ; 7B.StackOverflow; 7C.The stackexchange solely for dba questions, eg https://dba.stackexchange.com/questions/48232/mongodb-config-servers-not-in-sync ; 8.Mongo Diagnostics FAQ http://docs.mongodb.org/manual/faq/diagnostics/#faq-memory

Answer 1

sh.status, rs.status, and db.serverStatus are the main ones. Verbose output (sh.status(true) should list all the chunk sizes for you. There are other potentially useful functions, e.g. to see the parsed configuration options, you'd use db.serverCmdLineOpts. There's a reference for mongo shell functions that you can look at to see if there's any more functions you're interested in. There's also some command-line tools like mongostat and mongotop that will give you useful information on the activity of your cluster.

If you post the error and how you caused it, I can try to give more specific advice on what's worth looking at for that error as well.