Integrating with Cloudera Navigator

Introduction

Gluent Offload Engine can integrate with Cloudera Navigator via the REST API to provide extra data governance information. This is configured in the Gluent Data Platform environment file.

Mandatory Parameters

Parameters that require modification are:

Parameter

Reference

DATA_GOVERNANCE_API_URL

For example http://nav01.example.com:7187/api

DATA_GOVERNANCE_API_USER

Username for connecting to the Cloudera Navigator REST API. This user must have any of the Managed and Custom Business Metadata Editor, Policy Editor or Full Administrator roles

DATA_GOVERNANCE_API_PASS

Password for DATA_GOVERNANCE_API_USER. Password encryption is supported using the Password Tool utility

CLOUDERA_NAVIGATOR_HIVE_SOURCE_ID

Cloudera Navigator entity id for the Hive source that will register metadata

The value for CLOUDERA_NAVIGATOR_HIVE_SOURCE_ID can be determined by taking the identity value from the output of a curl command like the example below:

$ curl "${DATA_GOVERNANCE_API_URL}/<API-VERSION>/entities?query=((type:SOURCE)AND(sourceType:Hive)AND(clusterName:<CLUSTER-NAME>))" -u username:password -X GET

Where:

  • <API-VERSION> is the correct API version, e.g. “v13”. This can be identified using: $ curl "${DATA_GOVERNANCE_API_URL}/version" -X GET

  • <CLUSTER-NAME> is the name of the cluster as shown in Cloudera Navigator UI

Note

If multiple entities are listed by the command above then choose the correct source for the Hive service you intend to use.

Optional Parameters

There are also configuration parameters that can be used to fine tune data governance metadata. These parameters are:

Parameter

Reference

DATA_GOVERNANCE_AUTO_TAGS

Tags to be attached to Hive objects created or modified by Gluent Offload Engine. This will be left as its default in most cases

DATA_GOVERNANCE_CUSTOM_TAGS

Tags to be attached to Hive objects created or modified by Gluent Offload Engine. Also available using the --data-governance-custom-tags command-line option

DATA_GOVERNANCE_AUTO_PROPERTIES

Dynamic properties to be attached to Hive objects created or modified by Gluent Offload Engine. These are referenced using specific identifiers (see DATA_GOVERNANCE_AUTO_PROPERTIES for details) and will usually be left at the default value

DATA_GOVERNANCE_CUSTOM_PROPERTIES

Key/value pairs to be attached to Hive objects created or modified by Gluent Offload Engine. Also available using the --data-governance-custom-properties command-line option

Environment Verification (Connect) will verify the Cloudera Navigator API configuration.

Important

Any changes made to the Gluent Data Platform environment file (offload.env) must be propagated across all installations.

Documentation Feedback

Send feedback on this documentation to: feedback@gluent.com