Overview
During full data loading, there is certain resource overhead on read-write instances of TDSQL-C for MySQL, especially when one-time data loading in clusters with a large amount of data is performed. For instances with tight resources, this operation may affect the performance stability of the read-write instances to some extent. You can control the impact on read-write instances of TDSQL-C for MySQL during data loading through several methods, thereby reducing the resource consumption on the source. However, such operations may reduce the efficiency of loading data from row-based storage to columnar storage, so adjustments should be made based on the actual business needs.
Prerequisites
The cluster status is Running.
Controlling Data Loading Speed
Adjust the number of concurrent data reads for a single table in the scenario of full data loading.
Modify the replication_dumper_select_parallel parameter through the feature of adjusting parameters. This parameter indicates the concurrency for simultaneous data reads from the same object. A larger value allows for a broader range of data reads from the table. In scenarios with sufficient resources, this can improve the loading efficiency. If resources are insufficient, it is recommended to set this value smaller. See Setting Instance Parameters for operation details. Adjust the number of rows read from a single table in the scenario of full data loading.
Modify the replication_dumper_chunk_size parameter through the feature of adjusting parameters. This parameter indicates the number of data rows read in a single operation for the same object. A larger value allows for more data reads each time. In scenarios with sufficient resources, this can improve the loading efficiency. If resources are insufficient, it is recommended to set this value smaller. See Setting Instance Parameters for operation details. Adjust the total number of concurrent data reads in the scenario of full data loading.
Modify the replication_dumper_table_parallel parameter through the feature of adjusting parameters. This parameter indicates the number of tables from which data can be read simultaneously. A larger value allows more tables to be read simultaneously. In scenarios with sufficient resources, this can improve the loading efficiency. See Setting Instance Parameters for operation details.