consumerDemo-avro-flink\\src\\main\\resources\\avro-tools-1.8.2.jar: A tool for generating code related to the Avro protocol.consumerDemo-avro-flink\\src\\main\\java\\com\\tencent\\subscribe\\avro: The directory where the Avro tool generates code.consumerDemo-avro-flink\\src\\main\\resources\\Record.avsc: The protocol definition file.{"namespace": "com.tencent.subscribe.avro", //The last schema in Record.avsc, with "name" displayed as "Record""type": "record","name": "Record", //"name" is displayed as "Record", indicating the format of data consumed from kafka"fields": [{"name": "id", //id indicates a globally incremental ID. For more information about record values, see the following table."type": "long","doc": "unique id of this record in the whole stream"},{"name": "version", //version indicates the protocol version"type": "int","doc": "protocol version"},{"name": "messageType", //Message type"aliases": ["operation"],"type": {"namespace": "com.tencent.subscribe.avro","name": "MessageType","type": "enum","symbols": ["INSERT","UPDATE","DELETE","DDL","BEGIN","COMMIT","HEARTBEAT","CHECKPOINT","ROLLBACK"]}},{...},}
Field Name in Record | Description |
id | Globally incrementing ID. |
version | Protocol version. The current version is 1. |
messageType | Message type. Enum values: "INSERT", "UPDATE", "DELETE", "DDL", "BEGIN", "COMMIT", "HEARTBEAT", "CHECKPOINT". |
fileName | Name of the binlog file where the current record resides. |
position | End offset of the current record in the binlog, in the format of End_log_pos@binlog file number. For example, if the current record is in the file mysql-bin.000004 and its end offset is 2196, the value is "2196@4". |
safePosition | Start offset of the current transaction in the binlog, in the same format as above. |
timestamp | Time when written to the binlog, in unix timestamp format with second-level precision. |
gtid | Current gtid, for example, c7c98333-6006-11ed-bfc9-b8cef6e1a231:9. |
transactionId | Transaction ID. A transaction ID is generated only for commit events. |
serverId | Source database serverId. To view the source database server_id, refer to SHOW VARIABLES LIKE 'server_id'. |
threadId | Session ID that commits the current transaction. For reference, see SHOW processlist;. |
sourceType | Database type of the source database. The current version supports only MySQL. |
sourceVersion | Source database version. To view the source database version, refer to select version();. |
schemaName | Database name. |
tableName | Table name. |
objectName | Format: database_name.table_name. |
columns | Definitions of columns in the table. |
oldColumns | Data of the row before DML execution. For insert messages, this array is null. The array contains 12 types of elements: Integer, Character, Decimal, Float, Timestamp, DateTime, TimestampWithTimeZone, BinaryGeometry, TextGeometry, BinaryObject, TextObject, and EmptyObject. For details, see the definition in the demo. |
newColumns | Data of the row after DML execution. For delete messages, this array is null. The array contains 12 types of elements: Integer, Character, Decimal, Float, Timestamp, DateTime, TimestampWithTimeZone, BinaryGeometry, TextGeometry, BinaryObject, TextObject, and EmptyObject. For details, see the definition in the demo. |
sql | SQL statement of the DDL. |
executionTime | DDL execution duration, in seconds. |
heartbeatTimestamp | Timestamp of the heartbeat message, in seconds. Only heartbeat messages have this field. |
syncedGtid | GTID set parsed by DTS, in the format of c7c98333-6006-11ed-bfc9-b8cef6e1a231:1-13. |
fakeGtid | Whether it is a constructed fake GTID. If gtid_mode is not enabled, DTS constructs a GTID. |
pkNames | If the table in the source database has a primary key, the DML message carries this parameter; otherwise, it does not. |
readerTimestamp | Time when DTS processes this data, in unix timestamp format with millisecond-level precision. |
tags | |
total | If a message is fragmented, records the total number of fragments. This field is meaningless in the current version (version=1) and is reserved for future extension. |
index | If a message is fragmented, records the index of the current fragment. This field is meaningless in the current version (version=1) and is reserved for future extension. |
MySQL Type | Corresponding Avro Type |
MYSQL_TYPE_NULL | EmptyObject |
MYSQL_TYPE_INT8 | Integer |
MYSQL_TYPE_INT16 | Integer |
MYSQL_TYPE_INT24 | Integer |
MYSQL_TYPE_INT32 | Integer |
MYSQL_TYPE_INT64 | Integer |
MYSQL_TYPE_BIT | Integer |
MYSQL_TYPE_YEAR | DateTime |
MYSQL_TYPE_FLOAT | Float |
MYSQL_TYPE_DOUBLE | Float |
MYSQL_TYPE_VARCHAR | Character |
MYSQL_TYPE_STRING | Character. If the original type is binary, this type will correspond to BinaryObject. |
MYSQL_TYPE_VAR_STRING | Character. If the original type is varbinary, this type will correspond to BinaryObject. |
MYSQL_TYPE_TIMESTAMP | Timestamp |
MYSQL_TYPE_DATE | DateTime |
MYSQL_TYPE_TIME | DateTime |
MYSQL_TYPE_DATETIME | DateTime |
MYSQL_TYPE_TIMESTAMP_NEW | Timestamp |
MYSQL_TYPE_DATE_NEW | DateTime |
MYSQL_TYPE_TIME_NEW | DateTime |
MYSQL_TYPE_DATETIME_NEW | DateTime |
MYSQL_TYPE_ENUM | TextObject |
MYSQL_TYPE_SET | TextObject |
MYSQL_TYPE_DECIMAL | Decimal |
MYSQL_TYPE_DECIMAL_NEW | Decimal |
MYSQL_TYPE_JSON | TextObject |
MYSQL_TYPE_BLOB | BinaryObject |
MYSQL_TYPE_TINY_BLOB | BinaryObject |
MYSQL_TYPE_MEDIUM_BLOB | BinaryObject |
MYSQL_TYPE_LONG_BLOB | BinaryObject |
MYSQL_TYPE_GEOMETRY | BinaryObject |
Esta página foi útil?
Você também pode entrar em contato com a Equipe de vendas ou Enviar um tíquete em caso de ajuda.
comentários