Hive Metastore Standalone and Apache Ranger Integration

Question

I'm trying to integrate HMS Standalone version and Apache Ranger Hive Plug-in. I've compiled the ranger-2.4.0-hive-plugin from source code of Ranger 2.4.0.

Situation:

1. Ranger Admin runs as Docker.
2. Hive Metastore Standalone is also runs in Docker with Hive Plugin enabled.

Ranger Plugin for hive has been enabled. Please restart hive to ensure that changes are effective.

I can verify the installation with this message.

3. There is no connection problem between two container. I can verify it the command;

   curl -u user:pass -X GET http://ranger_host:port/service/public/v2/api/service/ |jq

4. Both systems are up and running (as containerized). I've set log level to TRACE via log4j.But I can not see anything the HMS logs related to Apache Ranger Hive Plugin.

But /etc/ranger/hmstest/policycache directory is empty. Also Ranger logs does not container any exception related to HMS.

I can get the policy via Rest API Ranger for hmstest service. hmstest is defined as Hadoop SQL Service in Ranger Admin.

But it seems there is no agent to get policies on Hive Metastore Standalone Service.

I am using open jdk 11. Also I verified the metastore-site.xml and it containers Ranger Related configurations.

Any idea, what can cause the problem?

Used HMS: https://downloads.apache.org/hive/hive-standalone-metastore-3.0.0/hive-standalone-metastore-3.0.0-bin.tar.gz

Answer 1

You are trying to enforce standalone Hive Metastore (HMS) directly, but Ranger enforces security for Hive through Hive Server2 (HS2). If your goal is to achieve direct enforcement for the metastore, you have two options:

1. Configure HMS with an external database for which Ranger has a plugin. Enforce security policies on this external database using Ranger. In this scenario, you would be using a plugin specific to the external database, not the ranger-2.4.0-hive-plugin.
2. (This might be complicated) Hive-4 is not released yet, but the following Jira enables direct HMS enforcement in Hive https://issues.apache.org/jira/browse/HIVE-21753. In this case, you need to use Hive-4-alpha instead of Hive-3, uncomment this block in Ranger and recompiled the plugin (this might be just a trivial issue, not 100% sure), and as a last step configure Hive to use HMS as pre event listener:

<property>
<name>hive.security.authorization.manager</name>
<value>org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory</value>
</property>
<property>
<name>hive.metastore.pre.event.listeners</name> 
<value>...,org.apache.hadoop.hive.ql.security.authorization.plugin.metastore.HiveMetaStoreAuthorizer,...</value>
</property>

Apache Ranger community has not take any specific actions regarding HMS integration, and the second option might be challenging. However, if configured correctly, you should see HMS as a service in the Ranger Admin UI as below: