Tomcat Manager is not accessible in multi-domain configuration

Question

Tomcat Manager is not accessible in my multi-domain configuration. I get a 404 even though the manager files are present in both domains.

Funny enough, I'm able to make this configuration work locally. However, when I transfer my configuration to my servers, it falls apart.

Here is my config file:

<?xml version='1.0' encoding='utf-8'?>

<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="8005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <!-- Security listener. Documentation at /docs/config/listeners.html
  <Listener className="org.apache.catalina.security.SecurityListener" />
  -->
  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->


    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
    -->
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443
         This connector uses the NIO implementation that requires the JSSE
         style configuration. When using the APR/native implementation, the
         OpenSSL style configuration is required as described in the APR/native
         documentation -->
    <!--
    <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
               maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />


    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->
      <!--
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
      -->

      <!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="my.intranet.com"  appBase="webapps/intranet"
            unpackWARs="true" autoDeploy="true">

            <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                    prefix="intra_localhost_access_log" suffix=".txt"
                    pattern="%h %l %u %t &quot;%r&quot; %s %b" />
      </Host>

      <Host name="my.internet.com"  appBase="webapps/internet"
            unpackWARs="true" autoDeploy="true">

            <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                    prefix="inter_localhost_access_log" suffix=".txt"
            pattern="%h %l %u %t &quot;%r&quot; %s %b" />
      </Host>
    </Engine>
  </Service>
</Server>

Directory listing for both intranet and internet in webapps, showing the manager having been copied in both folders:

[me@server internet]$ ls -l
drwxr-xr-x 2 tomcat appsadmins 4096 Jun  9 08:34 hello
drwxr-xr-x 5 tomcat appsadmins 4096 Jun  9 08:25 manager

[me@server intranet]$ ls -l
drwxr-xr-x 2 tomcat appsadmins     4096 Jun  9 08:34 hello
drwxr-xr-x 5 tomcat appsadmins     4096 Jun  9 08:25 manager

The /hello context works in both domains, showing individualized content for each domains.

I can also manually move WAR files into each intranet and internet folders to deploy. However, I'm asked to use the Tomcat Manager (which is the standard way for undeploying/deploying?)

Edit #1

This is currently on Tomcat 8/Java 8. However, I have obtained the same results on our other Tomcat7/Java7 servers. No problems in local VMs though.

Edit #2

Log files that seem to show that the manager has been deployed successfully.

22-Jun-2015 09:33:24.813 INFO [my.internet.com-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /path/to/tomcat/webapps/internet/manager has finished in 562 ms
22-Jun-2015 09:33:51.990 INFO [my.intranet.com-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /path/to/tomcat/webapps/intranet/manager has finished in 316 ms

Edit #3

I added webapps/host-manager/manager.xml to conf/Catalina/my.internet.com/ and to conf/Catalina/my.intranet.com

I still get a 404.

Edit #4

Reverted to a minimal configuration. 404 on manager/html still.

• cleared tomcat8/conf/Catalina
• cleared tomcat8/work
• restarted

The only configured host:

<Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

The access logs:

xxx.xxx.xxx.xxx - - [02/Jul/2015:10:17:41 -0400] "GET /manager/html HTTP/1.1" 404 1018

/docs and /example work though.

And again, in a CentOS 6.3 VM, I was able to mimic two domains without too much fuss.

Edit #5

Need to get back at this... on my destination server again.

Recap, configuration of internet domain:

<Host name="my.internet.com" appBase="inter"
    unpackWARs="true" autoDeploy="true">

    <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
            prefix="inter_localhost_access_log" suffix=".txt"
    pattern="%h %l %u %t &quot;%r&quot; %s %b" />
</Host>

Doing a tail -f logs/inter_localhost_access_log.2015-08-04.txt, I can see requests being logged when connecting to my.internet.com.

Accessing the manager/html, gives me 404

my_ip - - [04/Aug/2015:08:59:28 -0400] "GET /manager/html HTTP/1.1" 404 1028
my_ip - - [04/Aug/2015:08:59:28 -0400] "GET /favicon.ico HTTP/1.1" 200 21630
my_ip - - [04/Aug/2015:08:59:28 -0400] "GET /favicon.ico HTTP/1.1" 200 21630

Accessing /docs

my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs HTTP/1.1" 302 -
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/ HTTP/1.1" 200 16805
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/images/docs-stylesheet.css HTTP/1.1" 200 6082
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/images/tomcat.png HTTP/1.1" 200 8410
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/images/fonts/fonts.css HTTP/1.1" 200 1996
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/images/asf-feather.png HTTP/1.1" 200 40042
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/images/fonts/OpenSans600.woff HTTP/1.1" 200 22604
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/images/fonts/OpenSans400.woff HTTP/1.1" 200 21956
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/images/fonts/OpenSans700.woff HTTP/1.1" 200 22748
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /docs/images/fonts/OpenSans400italic.woff HTTP/1.1" 200 21092
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /favicon.ico HTTP/1.1" 200 21630
my_ip - - [04/Aug/2015:08:59:57 -0400] "GET /favicon.ico HTTP/1.1" 200 21630

File listing of inter

[me@server inter]$ ls -la
total 28
drwxrwsr-x  7 tomcat appsadmins 4096 Jun 30 08:15 .
drwxrwsr-x 11 tomcat appsadmins 4096 Jul 10 14:50 ..
drwxrwsr-x 14 tomcat appsadmins 4096 Jun 30 08:14 docs
drwxrwsr-x  6 tomcat appsadmins 4096 Jun 30 08:14 examples
drwxrwsr-x  5 tomcat appsadmins 4096 Jun 30 08:14 host-manager
drwxrwsr-x  5 tomcat appsadmins 4096 Jun 30 08:14 manager
drwxrwsr-x  3 tomcat appsadmins 4096 Jun 30 08:15 ROOT

Process information

[me@server inter]$ ps aux | grep tomcat
tomcat   31863  3.7  4.1 2914836 162184 pts/0  Sl   08:56   0:18 /usr/bin/java -Djava.util.logging.config.file=/path/to/tomcat8/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/path/to/tomcat8/endorsed -classpath /path/to/tomcat8/bin/bootstrap.jar:/path/to/tomcat8/bin/tomcat-juli.jar -Dcatalina.base=/path/to/tomcat8 -Dcatalina.home=/path/to/tomcat8 -Djava.io.tmpdir=/path/to/tomcat8/temp org.apache.catalina.startup.Bootstrap start

HTTP Headers in Firefox:

http://my.internet.com:8080/manager/index.jsp

GET /manager/index.jsp HTTP/1.1
Host: my.internet.com:8080
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: JSESSIONID=92691406245E9FA2F35105951A8363F8; __utma=200567291.1579910409.1418663486.1431011588.1437739620.4; __utmz=200567291.1418663486.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _ga=GA1.3.1579910409.1418663486; _dfo_lang=en
Connection: keep-alive

HTTP/1.1 302 Found
Server: Apache-Coyote/1.1
Location: http://my.internet.com:8080/manager/html
Content-Type: text/html;charset=ISO-8859-1
Content-Length: 0
Date: Tue, 04 Aug 2015 13:13:36 GMT
----------------------------------------------------------
http://my.internet.com:8080/manager/html

GET /manager/html HTTP/1.1
Host: my.internet.com:8080
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: JSESSIONID=92691406245E9FA2F35105951A8363F8; __utma=200567291.1579910409.1418663486.1431011588.1437739620.4; __utmz=200567291.1418663486.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _ga=GA1.3.1579910409.1418663486; 
Connection: keep-alive

HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 1018
Date: Tue, 04 Aug 2015 13:13:36 GMT

Edit #6

Permissions on /docs and /manager

[me@server docs]$ ls -l
total 1296
-rw-rwSr-- 1 tomcat appsadmins  21715 Jun 30 08:14 aio.html
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 api
drwxrwsr-x 3 tomcat appsadmins   4096 Jun 30 08:14 appdev
-rw-rwSr-- 1 tomcat appsadmins  13843 Jun 30 08:14 apr.html
drwxrwsr-x 4 tomcat appsadmins   4096 Jun 30 08:14 architecture
-rw-rwSr-- 1 tomcat appsadmins   7701 Jun 30 08:14 balancer-howto.html
-rw-rwSr-- 1 tomcat appsadmins  16305 Jun 30 08:14 building.html
-rw-rwSr-- 1 tomcat appsadmins  16480 Jun 30 08:14 BUILDING.txt
-rw-rwSr-- 1 tomcat appsadmins  10726 Jun 30 08:14 cgi-howto.html
-rw-rwSr-- 1 tomcat appsadmins 209429 Jun 30 08:14 changelog.html
-rw-rwSr-- 1 tomcat appsadmins  18330 Jun 30 08:14 class-loader-howto.html
-rw-rwSr-- 1 tomcat appsadmins  43860 Jun 30 08:14 cluster-howto.html
-rw-rwSr-- 1 tomcat appsadmins  10301 Jun 30 08:14 comments.html
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 config
-rw-rwSr-- 1 tomcat appsadmins   8806 Jun 30 08:14 connectors.html
-rw-rwSr-- 1 tomcat appsadmins  17936 Jun 30 08:14 default-servlet.html
-rw-rwSr-- 1 tomcat appsadmins  23038 Jun 30 08:14 deployer-howto.html
-rw-rwSr-- 1 tomcat appsadmins   8733 Jun 30 08:14 developers.html
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 elapi
-rw-rwSr-- 1 tomcat appsadmins  10577 Jun 30 08:14 extras.html
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 funcspecs
-rw-rwSr-- 1 tomcat appsadmins  30498 Jun 30 08:14 html-manager-howto.html
drwxrwsr-x 3 tomcat appsadmins   4096 Jun 30 08:14 images
-rw-rwSr-- 1 tomcat appsadmins  16805 Jun 30 08:14 index.html
-rw-rwSr-- 1 tomcat appsadmins  11995 Jun 30 08:14 introduction.html
-rw-rwSr-- 1 tomcat appsadmins  23447 Jun 30 08:14 jasper-howto.html
-rw-rwSr-- 1 tomcat appsadmins  66799 Jun 30 08:14 jdbc-pool.html
-rw-rwSr-- 1 tomcat appsadmins  35841 Jun 30 08:14 jndi-datasource-examples-howto.html
-rw-rwSr-- 1 tomcat appsadmins  54636 Jun 30 08:14 jndi-resources-howto.html
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 jspapi
-rw-rwSr-- 1 tomcat appsadmins  34918 Jun 30 08:14 logging.html
-rw-rwSr-- 1 tomcat appsadmins  75344 Jun 30 08:14 manager-howto.html
-rw-rwSr-- 1 tomcat appsadmins   8218 Jun 30 08:14 maven-jars.html
-rw-rwSr-- 1 tomcat appsadmins   8323 Jun 30 08:14 mbeans-descriptor-howto.html
-rw-rwSr-- 1 tomcat appsadmins  40235 Jun 30 08:14 monitoring.html
-rw-rwSr-- 1 tomcat appsadmins  13371 Jun 30 08:14 proxy-howto.html
-rw-rwSr-- 1 tomcat appsadmins  64018 Jun 30 08:14 realm-howto.html
-rw-rwSr-- 1 tomcat appsadmins   6954 Jun 30 08:14 RELEASE-NOTES.txt
-rw-rwSr-- 1 tomcat appsadmins  35358 Jun 30 08:14 rewrite.html
-rw-rwSr-- 1 tomcat appsadmins  16682 Jun 30 08:14 RUNNING.txt
-rw-rwSr-- 1 tomcat appsadmins  34336 Jun 30 08:14 security-howto.html
-rw-rwSr-- 1 tomcat appsadmins  30478 Jun 30 08:14 security-manager-howto.html
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 servletapi
-rw-rwSr-- 1 tomcat appsadmins  13047 Jun 30 08:14 setup.html
-rw-rwSr-- 1 tomcat appsadmins  17817 Jun 30 08:14 ssi-howto.html
-rw-rwSr-- 1 tomcat appsadmins  37032 Jun 30 08:14 ssl-howto.html
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 tribes
-rw-rwSr-- 1 tomcat appsadmins  11610 Jun 30 08:14 virtual-hosting-howto.html
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 WEB-INF
drwxrwsr-x 2 tomcat appsadmins   4096 Jun 30 08:14 websocketapi
-rw-rwSr-- 1 tomcat appsadmins  13888 Jun 30 08:14 web-socket-howto.html
-rw-rwSr-- 1 tomcat appsadmins  22776 Jun 30 08:14 windows-auth-howto.html
-rw-rwSr-- 1 tomcat appsadmins  22974 Jun 30 08:14 windows-service-howto.html

[me@server manager]$ ls -l
total 32
drwxrwsr-x 2 tomcat appsadmins 4096 Jun 30 08:14 images
-rw-rwSr-- 1 tomcat appsadmins  906 Jun 30 08:14 index.jsp
drwxrwsr-x 2 tomcat appsadmins 4096 Jun 30 08:14 META-INF
-rw-rwSr-- 1 tomcat appsadmins 4458 Jun 30 08:14 status.xsd
drwxrwsr-x 3 tomcat appsadmins 4096 Jun 30 08:14 WEB-INF
-rw-rwSr-- 1 tomcat appsadmins 4723 Jun 30 08:14 xform.xsl

Answer 1

I've come to the conclusion that the issues I was having were related to the way it was initially installed and configured.

I say this because if I get the latest version, extract it and launch it; the configs posted above work.

I'm unsure how the existing version was installed, I assume it was from a repo (I'm on RHEL 6.7).

Update

Using Apache to proxy to Tomcat, we have added ProxyPreserveHost On to our Apache to ensure the right manager is being accessed.

Answer 2

If you want the manager to work with multiple virtual hosts, you will need to configure the manager for each host.

Check if the following helps your

Virtual Hosting and Tomcat

Answer 3

Here are the steps that you could follow -

1. Shutdown tomcat, clear the CATALINA_HOME/works/* and CATALINA_HOME/logs/* and then start the tomcat. This way there would be less clutter while troubleshooting.

2. Try and figure out where the request is routed. You have mentioned 404 error which means tomcat tries to find manager and fails to do so. In such case, the request should either be logged in at inter_localhost_access_log.txt or intra_localhost_access_log.txt. This will help you to understand where the tomcat is trying to search for manager application.

3. Put the manager application where the tomcat is trying to find. Step 2. will give you a clue about where tomcat is trying to search for manager

Another point to be noted - Try keeping the appBase directories separate.

Eg - put the application into

mkdir $CATALINA_HOME/internet_webapp
mkdir $CATALINA_HOME/intranet_webapp

Overlapping directories are not allowed.