For how many years have you been working
with physical servers that are starving your database of the memory
necessary to deploy important new performance features such as the Result
Cache, Memoptimize Pool, In-Memory Aggregation, In-Memory Column Store, and
Full Database Caching? Too long? Contact me to learn how to improve all
queries ... not just some queries.
Purpose
DBMS_HADOOP_INTERNAL is a private package that
provides a number of helper functions to DBMS_HADOOP
Given a Hive table, this function returns all the partitions that appear in the Hive table, but missing in the corresponding Oracle external table. We use the function ADDED_PARTNS() to obtain the hashed partitioned names from DBA_HIVE_TAB_PARTITIONS and then find the corresponding original partition specs.
dbms_hadoop_internal.added_hive_partns(
clus_id IN VARCHAR2,
db_name IN VARCHAR2,
tab_name IN VARCHAR2,
partnList IN CLOB,
mdata_compatible IN VARCHAR2)
RETURN CLOB;
Given a Hive table, this function returns all the partitions that appear in the Hive table, but missing in the corresponding Oracle external table
dbms_hadoop_internal.added_partns(
clus_id IN VARCHAR2,
db_name IN VARCHAR2,
tab_name IN VARCHAR2,
et_name IN VARCHAR2,
et_owner IN VARCHAR2,
mdata_compatible IN VARCHAR2)
RETURN CLOB;
Given a Hive table, this function returns all the partitions that appear in the corresponding Oracle external table, but in the given Hive table
dbms_hadoop_internal.dropped_partns(
clus_id IN VARCHAR2,
db_name IN VARCHAR2,
tab_name IN VARCHAR2,
et_name IN VARCHAR2,
et_owner IN VARCHAR2,
mdata_compatible IN VARCHAR2)
RETURN CLOB;
getHiveTable() is a pipelined table function that returns the rows back from C external procedures via ODCI to PL/SQL.
The rows sent from C external procedures actually originate from various Hive metastores and fetched via JNI calls made from hotspot JVM.
dbms_hadoop_internal.getHiveTable(
configDir IN VARCHAR2,
debugDir IN VARCHAR2,
clusterName IN VARCHAR2,
dbName IN VARCHAR2,
tblName IN VARCHAR2,
createPartitions IN VARCHAR2,
callType IN NUMBER)
RETURN hiveTypeSet PIPELINED USING HiveMetadata;
dbms_hadoop_internal.get_ddl(
secureConfigDir IN VARCHAR2,
secureDebugDir IN VARCHAR2,
secureClusterId IN VARCHAR2,
secureDbName IN VARCHAR2,
secureHiveTableName IN VARCHAR2,
createPartitions IN VARCHAR2)
RETURN CLOB;
Given a Hive table and its corresponding Oracle external table, this function returns the first incompatibility that is encountered
dbms_hadoop_internal.get_incompatibility(
clus_id IN VARCHAR2,
db_name IN VARCHAR2,
hive_tbl_name IN VARCHAR2,
et_tbl_name IN VARCHAR2,
et_tbl_owner IN VARCHAR2)
RETURN CLOB;
Given a Hive table, this function returns the object number of the Oracle external table, if one is present
dbms_hadoop_internal.get_objno_from_et(
cluster_id IN VARCHAR2,
table_name IN VARCHAR2,
xt_tab_name IN VARCHAR2,
xt_tab_owner IN NUMBER)
RETURN NUMBER;
Given a Hive table and its corresponding Oracle external table, this function checks whether the external table is metadata-compatible with the Hive table.
Metadata compatibility means (a) every column in the external table must be present in the Hive table (b) the datatype of each external table column must be the same or compatible with the datatype of the hive table column.
(c) The partition keys in the external table must be in the same order as the partition keys in the hive table.
dbms_hadoop_internal.is_metadata_compatible(
clus_id IN VARCHAR2,
db_name IN VARCHAR2,
hive_tbl_name IN VARCHAR2,
et_tbl_name IN VARCHAR2,
et_tbl_owner IN VARCHAR2)
RETURN VARCHAR2;
Given a Hive table and its corresponding Oracle external table, this function tells whether the external table is partition compatible with the hive table.
If the XT is exactly identical to the Hive table this function will return FALSE - the reason is that the user does not need to call the SYNC API.
dbms_hadoop_internal.is_partition_compatible(
mdata_compatible IN VARCHAR2,
partns_added IN CLOB,
partns_dropped IN CLOB)
RETURN VARCHAR2;