Glossary

Words and terms that have unique, special meaning within Gluent and for Gluent’s products.

(Adaptive) Join Filter Pulldown

JFPD

An optimization that enables filter predicates on joined tables (typically dimensions) to be pulled down and applied to the Offloaded Table in the Backend system (typically facts). Join Filter Pulldown will be applied either by Backend subqueries (where both join tables are in the Backend) or by retrieving the filter predicates from the Oracle Database instance and applying them to the Backend query in an adaptive way based on the nature of the data.

100-0

100-0 Hybrid View

Hybrid View (100-0)

For data that is fully-Offloaded or stored only in the Backend system, the Hybrid View is created as a simple query of the corresponding Hybrid External Table. The 100-0 label denotes that all (100%) of the data is accessed from the Backend system and none (0%) from the RDBMS.

100-10

100-10 Hybrid View

Hybrid View (100-10)

For partitioned data that is fully-Offloaded, it is possible to define a boundary for a 100-10 Hybrid View that accesses the RDBMS for the “hotter” partitions and the Backend system for the “colder” partitions (known as a 100-10 Hybrid View). The 100-10 label states that while all (100%) of the data is available in the Backend, the Hybrid View continues to access some (e.g. 10%) from the RDBMS: the actual percentages depend on the selection boundary specified for each Offloaded partitioned table.

90-10

90-10 Hybrid View

Hybrid View (90-10)

For tables that are partially-offloaded over time (i.e. (Sub)Partition-Based Offload, Predicate-Based Offload), a Hybrid View is created that appends the RDBMS-resident partitions in the Application Schema to the Hybrid External Table that represents Offloaded data. The 90-10 label denotes that most (e.g. 90%) of the data is accessed from the Backend and some (e.g. 10%) from the RDBMS, although this is just a notional label: the actual percentage split depends on how much data/how many partitions have been Offloaded.

Advanced Aggregation Pushdown

AAPD

Aggregate queries on Hybrid Views are pushed down to the Backend system using an alternative aggregated Hybrid View (created by a Present with aggregation) and a general Rewrite Rule. An advanced optimization also enables aggregations with filters to be pushed down without requiring filtered columns to be included in the grouping set. This enables low-granularity aggregations to be pushed down even in the presence of high-cardinality filter columns.

Application Schema

The customer’s application schema, containing the tables to be Offloaded or tables to be queried in tandem with Offloaded Data.

Backend

Backend data platform (such as Cloudera Data Hub, Cloudera Data Platform, Google BigQuery, Snowflake) which Gluent Data Platform is using as the target for Offload or source for Hybrid Queries and Present.

Base Table

When Incremental Update is enabled for an Offloaded Table, the table is renamed to an internal Gluent Data Platform format and is referred to as a Base Table. The original table name is adopted for the Delta View (User), which becomes the query object.

Changelog Sequence

A sequence created in the Hybrid Schema for use with Incremental Update. Each Changelog Sequence is coupled to a Changelog Table and is used to apply data modifications in the correct order.

Changelog Table

A table created in the Hybrid Schema for staging data changes captured by Incremental Update prior to Extraction. A Changelog Table is created for each application table that has Incremental Update enabled (unless the ORA_ROWSCN extraction method has been used).

Changelog Trigger

A trigger created on an application table to capture data changes for Incremental Update and store them in a Changelog Table, ready for copying to the Backend system. This is created to support the CHANGELOG, UPDATABLE_CHANGELOG and CHANGELOG_INSERT extraction methods.

Column Dropout

A Gluent Query Engine optimization whereby Smart Connector returns incomplete records when trailing fields are not in the projection (and therefore not required).

Compaction

An Incremental Update process that merges data changes in the Delta Table with the Base Table to create a new compacted Base Table. There are three compaction methods (In-Place, Full, Full Partition-Wise), with different characteristics and offline/online capabilities.

Configuration File

offload.env

Configuration file central to the operation of Gluent Data Platform.

Data Daemon

datad

Component of Gluent Query Engine that communicates with Backend systems on behalf of Smart Connector and Metadata Daemon via a Java daemon process.

Data Model Virtualization

The decoupling of the physical and logical structure of the data model or decoupling of the RDBMS logical data model from the Backend logical data model. Data Model Virtualization is enabled by Gluent Offload Engine’s Offload and Present components and supports Gluent Query Engine optimization features such as Join Pushdown.

Delta Table

A table stored in the Load Database to store data changes captured by Incremental Update.

Delta View

A view in the Backend system created to support Incremental Update. A Delta View merges data in the Delta Table with data in the Base Table to provide the latest state. When Incremental Update is enabled for an Offloaded Table, the table is renamed to a Base Table and the Delta View (User) takes on the original name.

Delta View (Internal)

An internal form of the Delta View, with an internal name.

Delta View (User)

The user-facing Delta View to be used for queries.

Delta Views

The Delta View (User) and Delta View (Internal).

Dependent View (General)

In general, a Dependent View is one with a direct or extended dependency to a table that is being Offloaded. Once a table with Dependent View(s) has been Offloaded, equivalent Hybrid Dependent View(s) are needed to replicate the Dependent Views’ functionality, but with Hybrid data. See Dependent View (Hybrid).

Dependent View (Hybrid)

Gluent Offload Engine-generated view created in the Hybrid Schema with a direct or extended dependency to a Hybrid View. Hybrid Dependent Views are created automatically during the Offload process as Hybrid equivalents of Dependent Views in the Application Schema. See Dependent View (General) for a definition of Application Schema Dependent Views.

Extraction

An Incremental Update process that extracts data changes from the RDBMS and moves them to a Delta Table. The extraction mechanism depends on the Extraction Method in use for an Offloaded Table.

Extraction Method

Describes the type of data capture mechanism to be used when enabling the Incremental Update feature for an Offloaded Table. There are six methods (ORA_ROWSCN, CHANGELOG, CHANGELOG_INSERT, UPDATABLE, UPDATABLE_CHANGELOG, UPDATABLE_INSERT), each creating a different set of Oracle Database and Backend supporting objects.

Full Offload

Offload of all the data in a table to a Backend system. All Oracle Database table types are candidates for Full Offload. Data that exists only in the Backend and is Presented to Oracle Database as Hybrid Objects is also considered fully-Offloaded. Tables that are Offloaded or Presented with Full Offload support are most commonly queried through a 100-0 Hybrid View (although a 100-10 Hybrid View can be defined for fully-Offloaded range-partitioned and list-partitioned tables if required).

Gluent Admin Schema

Owner schema for Gluent Data Platform and super-user for Offload. Default name is GLUENT_ADM.

Gluent Application Schema

Application schema for Gluent Data Platform and runtime user for Smart Connector and Offload Transport operations. Default name is GLUENT_APP.

Gluent Metadata Repository

A repository for metadata to support Gluent Data Platform operations such as the Offload and Present of tables. Uses the Gluent Repository Schema.

Gluent Node

A server, either physical or virtual, used to host Gluent Offload Engine and Data Daemon and any tools required for Offload Transport. This could be an edge node in a Hadoop cluster or a server with network connectivity to a Cloud Warehouse.

Gluent Offload Engine

Gluent Data Platform’s collection of software components for managing all Orchestration operations, comprising Offload, Present, Schema Sync, Diagnose, Offload Status Report and a number of other utilities.

Gluent Query Engine

Gluent Data Platform’s collection of software components for managing all Transparent Query operations, comprising Smart Connector, Metadata Daemon and Data Daemon.

Gluent Repository Schema

Database schema for Gluent Metadata Repository. Default name is GLUENT_REPO.

Hybrid External Table

Gluent Offload Engine-generated external table, created in the Hybrid Schema. Provides the query interface to Offloaded and/or Presented data. Not queried directly, it is made available via a corresponding Hybrid View.

Hybrid Objects

Gluent Offload Engine-generated Hybrid Views, Hybrid External Tables and Hybrid Rewrite Rules created in the Hybrid Schema to enable Offloaded and/or Presented data to be queried from the Oracle Database instance.

Hybrid Query

An RDBMS query that queries Offloaded and/or Presented Data through a Hybrid View, either fully or in part. Hybrid queries can query RDBMS-resident data in addition to Backend-resident data.

Hybrid Schema

Gluent Data Platform’s partner schema for the customer’s Application Schema, containing the Hybrid Objects necessary to query the customer’s Offloaded and Presented Data. Also contains synonyms for Application Schema tables and views that are not Offload or Present candidates. Default name is <APP_SCHEMA>_H.

Hybrid View

Gluent Offload Engine-generated view created in the Hybrid Schema to query Offloaded and/or Presented Data. For an Offloaded Table, the Hybrid View will usually have the same name as the original Application Schema table. For a Presented Table, the Hybrid View name can optionally be user-specified (mandatory for Presented aggregations). There are three variations on the Hybrid View format, depending on where the data is to be accessed from: 90-10 Hybrid View, 100-0 Hybrid View and 100-10 Hybrid View.

Incremental Data Append

A feature that enables different subsets of data to be Offloaded from a table or partition periodically. Tables that are Offloaded with Incremental Data Append support are queried through a 90-10 Hybrid View.

Incremental Offload

A set of Gluent Data Platform features that supports the modification of data in the Backend system following its initial Offload. Enables tables that are subject to data modification to be Offloaded and maintained periodically. Partition-Based Offload and Incremental Update are examples of Incremental Offload features.

Incremental Partition Append

Partitioned data that is partially-Offloaded and topped-up periodically with additional Offloaded partitions is supported for Incremental Partition Append. Tables that are Offloaded with Incremental Partition Append are queried through a 90-10 Hybrid View.

Incremental Update

Gluent Offload Engine’s data synchronization feature that enables data modifications (DML) in Oracle Database to be tracked, identified and copied to a Backend.

Internal Format

Gluent Query Engine (internally or via UDFs) converts values from a Backend system to Oracle Database internal binary formats for greater efficiency in external table ingestion.

Join Pushdown

Joins between Hybrid Views are pushed down to the Backend system using an alternative Hybrid View (created by either Offload Join or Present Join). Enables Offloaded Data to be stored more optimally in the Backend system (e.g. denormalized, pre-joined datasets through Data Model Virtualization) and queried from the Hybrid Schema using the join Hybrid View.

Load Database

Load Schema

A Gluent Offload Engine operational database/schema created in the Backend system to hold transient and operational Gluent Data Platform data in relation to Offloads and Hybrid Queries. Examples include transient Load Tables during Offload and operational tables to support Incremental Offload features and parallel Hybrid Queries.

Load Table

A temporary table in the Load Database/Schema to stage temporary data during an Offload Process. Data is copied to the load table before being loaded into the final table in the corresponding Backend application database.

Metadata Daemon

metad

Component of Gluent Query Engine that manages the building of Hybrid Queries and the transfer of metadata between Smart Connector and Data Daemon via a C++ daemon process.

Multitenant

Support for multiple source databases to Offload data to a single Backend system without risk of name collisions.

Offload

Offload Software

Gluent Offload Engine component that orchestrates the transfer and translation of data from a source RDBMS to the Backend. Includes the creation of Hybrid Objects in the Hybrid Schema to enable Offloaded Data to be queried and enables a range of optimization and manageability options such as Data Model Virtualization and Incremental Partition Append.

Offload Join

A Data Model Virtualization extension to Offload that materializes a user-specified join of Offloaded Tables in the Backend. Enables joins in Hybrid Queries to be pushed down through (Transparent) Join Pushdown.

Offload Process

The process by which data is copied to the Backend from a source RDBMS (e.g. perform an offload using Gluent Offload Engine).

Offload Transport

The process of extracting data from the source RDBMS, validating the data and loading it into the Backend system.

Offload Type

Indicates how a table was Offloaded, either as Incremental Data Append, either with Partition-Based Offload, Subpartition-Based Offload or Predicate-Based Offload (Offload Type INCREMENTAL) or as Full Offload (Offload Type FULL).

Offloaded

Offloaded Data

Offloaded Table

Data that has been copied from the source RDBMS to the Backend and is accessed via Hybrid View(s). An Offload can be for a table, one or more (sub)partitions or a set of data for a table.

Orchestration

Conceptual name given to all functions executed by Gluent Offload Engine in relation to Offloaded or Presented data.

Partition Functions

User-Defined Functions created by users to enable Gluent Offload Engine to create custom synthetic partitioning schemes.

Partition-Based Offload

Incremental Partition Append By Range Partition

Incremental Partition Append By List Partition

An Incremental Partition Append feature for partial-Offload, available for both range-partitioned and list-partitioned tables. Partition-Based Offload is often used to Offload partitions beyond an age threshold over time, retaining just the most recent period in the source RDBMS, but for list-partitioned tables also supports Offload of partitions in any sequence. Tables that are Offloaded with Partition-Based Offload are queried through a 90-10 Hybrid View.

Password Key File

A file containing a key used to generate encrypted password strings by the Password Tool utility.

Predicate Pushdown

Gluent Query Engine pushes predicates from Hybrid Queries down to the Backend SQL engine to optimize data retrieval.

Predicate-Based Offload

Incremental Data Append By Predicate

An Incremental Data Append feature for partial data Offload. Dataset(s) to Offload are identified by predicate(s) rather than partition boundary.

Present

Present Software

Gluent Offload Engine component that orchestrates the presentation of Backend-resident data to Oracle Database for query, including the creation of Hybrid Objects in the Hybrid Schema. Supports the presentation of previously-offloaded tables, Backend-only tables, Data Model Virtualization (Present Join) and optimizations such as aggregations.

Present Join

A Data Model Virtualization extension to Present that presents a user-specified join of Presented (and optionally, Offloaded Tables) in the Backend. Enables joins in Hybrid Queries to be pushed down through (Transparent) Join Pushdown.

Present Process

The process by which Backend-resident data is made available to Oracle Database for Hybrid Queries (e.g. present an Impala, BigQuery or Snowflake table or view to Oracle Database using Gluent Offload Engine).

Presented

Presented Data

Presented Table

Backend-resident data that can be queried from Oracle Database through a Hybrid View. Data can be presented as-is, aggregated or pre-joined and with optional type conversions. Present operates on tables or views.

Projection Optimization

Gluent Query Engine only projects and returns data for columns in the plan projection (for example, selected columns, columns used in join conditions, or columns involved in predicates), thereby reducing the volume of data that needs to be retrieved from the Backend system.

Rewrite Rule

Rewrite Rules

Gluent Offload Engine-generated Advanced Query Rewrite Equivalence. Created in the Hybrid Schema to enable Advanced Aggregation Pushdown (AAPD) and Transparent Join Pushdown, using Oracle Database’s query rewrite mechanism.

Run-Length Encoding

Method used by Smart Connector to minimize data returned to external table interface.

Smart Connector

Connector

Smart Connector is the Gluent Query Engine component that manages the execution of Hybrid Queries. It transfers query information to Metadata Daemon and manages the consumption of data from a Backend system into the RDBMS. Smart Connector utilizes a number of optimizations such as Predicate Pushdown, Advanced Join Filter Pulldown, Advanced Aggregation Pushdown, Projection Pushdown, and Column Dropout.

Subpartition-Based Offload

Incremental Partition Append By Range Subpartition

An Incremental Partition Append feature, similar to Partition-Based Offload, but using range subpartitions to determine the Offload boundary instead of top-level range partitions.

Transparent Join Pushdown

Similar to Join Pushdown, except join queries are transparently (automatically) re-written to use an alternative Hybrid View (created by either an Offload Join or Present Join operation) by Oracle Database’s Advanced Query Rewrite mechanism (using Rewrite Rules). A Data Model Virtualization feature that enables Offloaded Data to be stored more optimally in the Backend system (e.g. denormalized, pre-joined datasets), but queried and joined in its original relational format from the Hybrid Schema.

Transparent Query

Conceptual name given to all functions executed by Gluent Query Engine in support of a Hybrid Query.

UDFs

User-Defined Functions installed when using a Hadoop-based Backend to convert values to Internal Formats.

Updatable Trigger

A trigger on an Updatable View to capture changes and store them in a Changelog Table. This is created to support the UPDATABLE, UPDATABLE_CHANGELOG and UPDATABLE_INSERT Incremental Update extraction methods.

Updatable View

A view created in the Hybrid Schema to enable data modifications to be applied to data that has been Offloaded and subsequently removed from the RDBMS. This is created to support the UPDATABLE, UPDATABLE_CHANGELOG and UPDATABLE_INSERT Incremental Update extraction methods.

Documentation Feedback

Send feedback on this documentation to: feedback@gluent.com