tencent cloud

Stream Compute Service

文档Stream Compute ServiceSQL Developer GuideReferencing Multiple External Catalogs Simultaneously

Referencing Multiple External Catalogs Simultaneously

下载
聚焦模式
字号
最后更新时间: 2026-08-11 14:58:16
本文档由 AI 翻译
Note:
Newly created clusters now support a single job using multiple external catalogs concurrently. If your current cluster edition does not yet support this feature, please submit a ticket to apply for a cluster upgrade.

Use Cases

This is designed for scenarios where a single job needs to reference multiple external catalogs concurrently. However, because multiple Hive/HDFS configuration files can cause conflicts, using multiple catalogs that require this configuration file support (for example, Hive Catalog or Paimon(hive) Catalog) simultaneously is not supported.

Usage

For the method to create an external Catalog, refer to Table Management.
For using multiple catalogs in an SQL job, refer to the following code:

-- 1. Define the Paimon destination table
CREATE TABLE IF NOT EXISTS `116-paimon-sink1`.`default`.`HiveSink` (
id INT PRIMARY KEY NOT ENFORCED, -- Paimon requires a primary key, and `id` serves as the primary key.
name STRING
) WITH (
);
-- 2. Define the Hive source table
CREATE TABLE IF NOT EXISTS `hive116-1`.`default`.`source` (
id INT,
name STRING,
dt STRING,
hr STRING
) PARTITIONED BY (dt, hr)
WITH
'connector' = 'hive',
'hive-version' = '3.1.1',
'hive-database' = 'default',
'partition.time-extractor.timestamp-pattern'='$dt $hr:00:00',
'sink.partition-commit.trigger'='partition-time',
'sink.partition-commit.delay'='10 min',
'sink.partition-commit.policy.kind'='metastore,success-file'
);


INSERT INTO `116-paimon-sink1`.`default`.`HiveSink`
SELECT
id,
`name`
FROM `hive116-1`.`default`.`source`;


帮助和支持

本页内容是否解决了您的问题?

填写满意度调查问卷,共创更好文档体验。

文档反馈