I am trying to add Tomcat-JMX Integration to the Datadog Agent, in order to achieve Remote Monitoring .
I am following the docs https://docs.datadoghq.com/containers/guide/autodiscovery-with-jmx/?tab=helm#autodiscovery-annotations
This is the Environment I am using in my case. I am trying to integrate JMX in one my Deployment, in AKS environment and I am using Helm to upgrade my environment. as suggested in the documentation., I am adding the annotations in my deployment file in helm structure. My agent is JMX enabled, and i can see in deployment logs. I have configured a service for this deployment and its reflecting.
Issue I am facing : I am trying to add my annotations, in the deployment file, after deployment, it should be able make the remote connection , but its showing these logs in the agent. while i check the Java process., even its not reflecting this JMX process , it seems JMX process is not at all created, in my case.,
JMXFetch
Information
runtime_version : 11.0.18
version : 0.47.8
Initialized checks
tomcat
instance_name : tomcat-10.75.12.110-1099
message : Unable to instantiate or initialize instance 10.75.12.110:1099. Is the target JMX Server or JVM running? Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.75.12.110; nested exception is:
java.net.ConnectException: Connection refused (Connection refused)]
metric_count : 0
service_check_count : 0
status : ERROR
instance_name : tomcat-10.75.12.110-1099
message : Unable to instantiate or initialize instance 10.75.12.110:1099. Is the target JMX Server or JVM running? Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.75.12.110; nested exception is:
java.net.ConnectException: Connection refused (Connection refused)]
metric_count : 0
service_check_count : 0
status : ERROR
Failed checks
no checks
https://docs.datadoghq.com/containers/guide/autodiscovery-with-jmx/?tab=helm#available-jmx-integrations i have followed the above page for my annotations.
Following are the annotations i am using in my case
Annotations:
ad.datadoghq.com/{{ .Values.configmap.app.name }}-container.checks: >-
{
"tomcat": {
"init_config": {
"is_jmx": true,
"collect_default_metrics": true
},
"instances": [{
"host": "%%host%%",
"port": "1099"
}]
}
}
-----------
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: JAVA_OPTS
value: >-
-Xms256m -Xmx6144m
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.port=1099
-Dcom.sun.management.jmxremote.rmi.port=1099
-Djava.rmi.server.hostname=tomcat.<namespace>.svc.cluster.local
even i tried with adding -Djava.rmi.server.hostname=tomcat..svc.cluster.local , but no luck!
Even i tried by altering some of the unwanted annotations, I am not getting the expected result., ie., remote JMX connection.,
I would requet your help in this.,