tencent cloud

Data Transfer Service

DocumentaçãoData Transfer ServiceData Subscription 2.0MySQL Series Data SubscriptionCreating a MariaDB or Percona Data Subscription

Creating a MariaDB or Percona Data Subscription

Baixar
Modo Foco
Tamanho da Fonte
Última atualização: 2026-11-05 10:31:38
Traduzido e Verificado por IA
This scenario describes how to use DTS to create a data subscription task for MariaDB or Percona. Because the data subscription operations for MariaDB and Percona are the same, MariaDB is used as an example in the following introduction.

Prerequisites

Prepare the source database for subscription and ensure that its version meets the requirements. For more information, see databases supported by data subscription.
Binlog has been enabled on the source instance.
A subscription account has been created on the source instance. The account requires the following permissions: REPLICATION CLIENT, REPLICATION SLAVE, PROCESS, and SELECT on all objects. The specific grant statements are as follows:
create user 'account' IDENTIFIED BY 'password';
grant SELECT, REPLICATION CLIENT,REPLICATION SLAVE,PROCESS on *.* to 'account'@'%';
flush privileges;

Restrictions

Subscribed messages are stored in the built-in Kafka (single Topic) of DTS. Currently, the default retention period is 1 day, and the maximum storage per Topic is 500 GB. When the data retention period exceeds 1 day or the data volume exceeds 500 GB, the built-in Kafka starts to purge the earliest written data. Therefore, consume the data promptly to avoid data being purged before it is fully consumed.
The region for data consumption must be the same as the region of the subscription task.
Geometry-related data types are currently not supported.
During a subscription task, operations such as modifying subscription objects cause a task restart, which may result in duplicate data when users consume data from the Kafka client.
DTS transfers data by the smallest data unit. In incremental data transfer, each marked checkpoint is a data unit. If a data unit has just been transferred when a restart occurs, no data duplication will be caused. If a data unit is still being transferred when a restart occurs, the data unit needs to be pulled again after the restart to ensure data integrity, which will cause data duplication.
If you are concerned about duplicate data, configure deduplication logic when consuming data.

Supported SQL Operations for Subscription

Operation Type
Supported SQL Operations
DML
INSERT,UPDATE,DELETE
DDL
CREATE DATABASE,DROP DATABASE,CREATE TABLE,ALTER TABLE,DROP TABLE,RENAME TABLE

Operation Steps

1. Log in to the DTS console, select Data Subscription in the left sidebar, and click Create Subscription.
2. On the Create Subscription page, select the corresponding configuration, and click Buy Now.
Parameter
Description
Billing Mode
The yearly/monthly subscription and pay-as-you-go modes are supported.
Database
Select based on the specific database type.
Region
If the source database is a TencentDB instance, the region here must be consistent with the region of the source database instance. If the source database is not a TencentDB instance, the region here must be consistent with the region where data is consumed. If there is no special requirement for the data consumption region, select the region closest to the source database.
Edition
Select 2.0.
Specification
Select a specification based on your business requirements.
Tag
Set tag keys and values for subscription tasks to efficiently manage multiple tasks through tags.
Subscribed Instance Name
Select Name After Creation: The default task name will be the same as the task ID. You can rename the task after the subscription task is created.
Set it to Name Now. Enter the task name in the input box below.
Terms of Service
Make sure to select "I have read and agree to DTS Terms of Service".
3. After the purchase is successful, return to the subscription list, click Configure Subscription in the Operation column to configure the newly purchased subscription. You can use it only after the configuration is completed.
4. On the Configure data subscription page, select the data subscription task. After completing the configuration, click Test Connectivity. After the test is successful and the network is configured, click Save and Go Next.
Data Subscription Task Setting
Configuration Item
Description
Instance Type
Database type selected during creation. It cannot be modified.
We recommend selecting the primary database for subscription, as the subscription service imposes minimal load on the source database.
Access Type
Select based on your scenario. For preparation steps for different access types, see Preparation Overview.
Public Network: The source database is accessible through a public IP address.
Self-Built on CVM: The source database is deployed on Tencent CVM.
Direct Connect: The source database can connect to Tencent Cloud VPC through Direct Connect.
VPN Access: The source database can connect to Tencent Cloud VPC through VPN Connections.
Database: The source database is a TencentDB instance.
CCN: The source database can connect to Tencent Cloud VPC through CCN.
Public Network
Host address: IP address or domain name of the source database.
Port: port used by the source database.
Self-Built on CVM
CVM instance: The instance ID of the CVM.
Port: port used by the source database.
Direct Connect
VPC-based Direct Connect gateway: Only the VPC-based Direct Connect gateway is supported for the Direct Connect access type. Confirm the network type associated with the gateway.
VPC: Select the VPC and subnet associated with the Direct Connect gateway and VPN gateway.
Host address: IP address or domain name of the source database.
Port: port used by the source database.
VPN Access
VPN gateway: Select the VPN gateway instance for access.
VPC: Select the VPC and subnet associated with the Direct Connect gateway and VPN gateway.
Host address: IP address or domain name of the source database.
Port: port used by the source database.
Database
Cloud database instance: instance ID of the source database.
We recommend subscribing to the primary database because the subscription service places minimal load on the source database.
When the source database is TencentDB for MariaDB, you can select disaster recovery/read-only instances.
CCN
Host address: host IP address or domain name of the source database.
Port: port used by the source database.
VPC CCN: Specify the CCN instance name.
Access VPC: refers to the VPC in CCN that connects to the subscription link. Select a VPC other than the one to which the source database belongs from all VPCs associated with CCN.
For example, if a database in the Guangzhou region serves as the source database, select a VPC in another region, such as a VPC in Chengdu or Shanghai.
Subnet: name of the subnet in the selected VPC.
Access VPC region: The region of the source database you selected when purchasing the task must be consistent with the region of the access VPC. If they are inconsistent, the source database region will be automatically changed to the VPC access region.
Account/Password
Account and password of the source database.
Network Settings
Configuration Item
Description
Network
Specify in advance the VPC environment for accessing the Kafka of the subscription task (including the VPC and subnet). You can then consume the data subscription service through this VPC.
Note:
Access within the same VPC: All CVMs in the same VPC can access subscription data normally without additional configuration.
Cross-VPC access: If the consumer CVM is in another VPC, you need to configure CCN to enable cross-VPC access.
5. On the Subscription Type and Object Selection page, select the subscription type, and click Save Configuration.
Configuration Item
Description
Subscription Type
Data update: Subscribes to data updates for all objects in the source database, including INSERT, UPDATE, and DELETE operations.
Structure update: Subscribes to the creation, modification, and deletion of structures for all objects in the source database.
All instances: Subscribes to data updates and structure updates for all objects in the source database.
Kafka Version
Displays the version of the built-in Kafka in DTS. It cannot be modified.
Number of Topic Partitions
Set the number of partitions for topics in the built-in Kafka for data delivery. Increasing the number of partitions can improve the speed of data write operations and consumption. A single partition can guarantee message order, while multiple partitions cannot. If you have strict requirements on the order of consumed messages, select 1 as the number of partitions.
Topic Partition Policy
Partition by table name: Write data with the same table name into the same partition.
Partition by table name + primary key: Data with the same primary key is written to the same partition. This method is suitable for tables with hot data, as it distributes hot table data across different partitions to improve concurrent consumption efficiency.
6. On the pre-check page, the pre-check task is expected to run for 2 to 3 minutes. After the pre-check is passed, click Start to complete the data subscription task configuration.
Note:
If the check fails, see Check Failure Handling to fix the issue and perform the check again.
7. The subscription task will be initialized. This process is expected to take 3 - 4 minutes. After successful initialization, the task enters the Running state.
8. Create Consumer Group. Data Subscription for Kafka allows users to create multiple consumer groups (up to 10 per subscription task) for multi-point consumption. Data Subscription for Kafka relies on Kafka consumer groups, so you need to create a consumer group before consuming data.
9. After the subscription instance enters the Running state, you can start consuming data. Kafka consumption requires password authentication. For specific examples, see Data Consumption Demo, where we provide Demo code in multiple languages and explain the main consumption process and key data structures.
Anterior: Creating TDSQL for MySQL Data SubscriptionPróximo: Guide to Consuming MySQL Series Subscription Data

Ajuda e Suporte

Esta página foi útil?

comentários

Search
    Enter keywords to search documentation