Releases Notes and Announcements
- Release Notes
Product Introduction
Purchase Guide
Getting Started
Operation Guide
- Overview of Upstream and Downstream Data
- Managing Jobs
- Developing Jobs
- Monitoring Jobs
- Job Logs
- Events and Diagnosis
- Crash-Scene Data Collection
- Managing Metadata
- Managing Checkpoints
- Tuning Jobs
- Managing Dependencies
- Managing Clusters
- Managing Permissions
SQL Developer Guide
- Overview
- Glossary and Data Types
- DDL Statements
- DML Statements
- Merging MySQL CDC Sources
- Database Synchronization (SQL) Capability
- Connectors
- SET Statement
- Operators and Built-in Functions
- Identifiers and Reserved Words
Python Developer Guide
Stream Lake Engine Setats
- Introduction to Setats
- Setats Cluster Management
- Job Development Guide
- Metrics
FAQ
Contact Us

Overview of Upstream and Downstream Data

Download

Mode fokus

Ukuran font

Terakhir diperbarui: 2026-08-06 15:17:00

Diterjemahkan oleh AI

Introduction to Upstream and Downstream Data for SQL Jobs
A data source (Source) refers to the upstream data input to a stream computing system. In current SCS SQL-mode jobs, data sources can include message queues such as Kafka and databases such as MySQL.
A data destination (Sink) refers to the destination where a stream computing system outputs processed results. In current SCS SQL-mode jobs, data destinations can include message queues such as Kafka, databases such as MySQL, and data analysis engines such as ES.
Users can also upload custom Connector packages to support more data sources and destinations.
For the concepts mentioned in this document, such as the differences between Tuple and Upsert data destinations, see the Glossary.
Upstream/Downstream
As a Streaming Data Source
As a Batch Data Source
As a Dimension Table
As an Append Data Destination
As an Upsert Data Destination
Message Queue Kafka
Supported
Supported (JAR Jobs)
-
Supported
-
Message Queue Upsert Kafka
Supported
Supported (JAR Jobs)
-
Supported
Supported
Message Queue CMQ
Supported
-
-
Supported
-
Message Queue TDMQ RabbitMQ
-
-
-
Supported
-
Database MySQL CDC
Supported
-
-
-
-
Database TDSQL-MySQL
Supported
-
-
-
-
Database Redis
-
-
Supported (Flink-1.11)
Supported
Supported
Database MongoDB CDC
Supported
-
-
-
-
Database MongoDB
-
-
-
Supported
-
Database PostgreSQL CDC
Supported (version 9.6 and above)
-
-
-
-
Cloud Data Warehouse PostgreSQL
-
Supported
Supported
Supported
Supported
Data Warehouse Hbase
-
Supported
Supported
Supported
Supported
Data Warehouse Doris
-
-
-
Supported
Supported
Data Warehouse Hive
Supported
-
Supported
Supported
-
Data Warehouse ClickHouse
-
Supported
Supported
Supported
Supported
Data Warehouse Kudu
-
Supported
-
Supported
Supported
Data Analytics Engine ES
-
Supported (Flink-1.13)
-
Supported
Supported
Log Consumption CLS
Supported
-
-
-
-
Database Connection JDBC
-
Supported
Supported
Supported
Supported
Filesystem(HDFS/COS)
Supported
Supported
-
Supported
-
Data Lake Iceberg
Supported
Supported
-
Supported
Supported
Data Lake Hudi
Supported
Supported
-
Supported
Supported
Message Queue Pulsar
Supported
Supported
-
Supported
Supported
DLC
-
-
-
Supported
Supported
Database SQL Server CDC
Supported
-
-
-
-
Database StarRocks
-
Supported
Supported
Supported
Supported
Note:
Elasticsearch supports versions 6.x and 7.x, but does not support version 5.x.
The mysql-cdc connector can be used with TDSQL-C.
Note
 For information on developing and using Sources and Sinks for SQL jobs, refer to the Upstream and Downstream Development Guide.
Introduction to Upstream and Downstream Data for JAR Jobs
After the VPC of a dedicated cluster establishes a peering connection with a user-specified VPC, JAR-mode jobs can access all network-accessible resources within that user-specific VPC. This not only supports the upstream and downstream components compatible with SQL jobs mentioned above, but also enables the use of various Tencent Cloud services within that VPC, such as message queues, databases, API services, and CVMs.
Additionally, you can purchase a NAT Gateway within this specific VPC and configure the routing table to access external internet addresses (such as APIs on the public network or externally self-built services), further enhancing the processing capability of stream computing jobs.
For information on developing and using Sources and Sinks for JAR jobs, refer to the DataStream Connectors section in the Flink official documentation.

Bantuan dan Dukungan

Apakah halaman ini membantu?

Anda juga dapat Menghubungi Penjualan atau Mengirimkan Tiket untuk meminta bantuan.

masukan

tencent cloud

Stream Compute Service

Overview of Upstream and Downstream Data

Introduction to Upstream and Downstream Data for SQL Jobs

Introduction to Upstream and Downstream Data for JAR Jobs

Bantuan dan Dukungan

Upstream/Downstream	As a Streaming Data Source	As a Batch Data Source	As a Dimension Table	As an Append Data Destination	As an Upsert Data Destination
Message Queue Kafka	Supported	Supported (JAR Jobs)	-	Supported	-
Message Queue Upsert Kafka	Supported	Supported (JAR Jobs)	-	Supported	Supported
Message Queue CMQ	Supported	-	-	Supported	-
Message Queue TDMQ RabbitMQ	-	-	-	Supported	-
Database MySQL CDC	Supported	-	-	-	-
Database TDSQL-MySQL	Supported	-	-	-	-
Database Redis	-	-	Supported (Flink-1.11)	Supported	Supported
Database MongoDB CDC	Supported	-	-	-	-
Database MongoDB	-	-	-	Supported	-
Database PostgreSQL CDC	Supported (version 9.6 and above)	-	-	-	-
Cloud Data Warehouse PostgreSQL	-	Supported	Supported	Supported	Supported
Data Warehouse Hbase	-	Supported	Supported	Supported	Supported
Data Warehouse Doris	-	-	-	Supported	Supported
Data Warehouse Hive	Supported	-	Supported	Supported	-
Data Warehouse ClickHouse	-	Supported	Supported	Supported	Supported
Data Warehouse Kudu	-	Supported	-	Supported	Supported
Data Analytics Engine ES	-	Supported (Flink-1.13)	-	Supported	Supported
Log Consumption CLS	Supported	-	-	-	-
Database Connection JDBC	-	Supported	Supported	Supported	Supported
Filesystem(HDFS/COS)	Supported	Supported	-	Supported	-
Data Lake Iceberg	Supported	Supported	-	Supported	Supported
Data Lake Hudi	Supported	Supported	-	Supported	Supported
Message Queue Pulsar	Supported	Supported	-	Supported	Supported
DLC	-	-	-	Supported	Supported
Database SQL Server CDC	Supported	-	-	-	-
Database StarRocks	-	Supported	Supported	Supported	Supported