When pt-archiver archives MySQL json fields, garbled characters appear

Polyglot Projects Percona Toolkit
1

The scenario is as follows:
This is the command executed:

pt-archiver --limit 5000 --txn-size 5000 --progress 5000 --statistics --bulk-insert --charset=utf8mb4 --no-check-charset --no-check-columns

Table character set:

CREATE TABLE `o_deposit_order` (
 `id` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL ,
 `merchant_name` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT '',
 `payer_information` json DEFAULT (json_object()) 
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci

The exception is that the data in the standard varchar fields in the table does not display garbled characters after archiving, but the data in the json fields does display garbled characters.

The source table data is as follows:

The sql is:

SELECT merchant_name, payer_information FROM o_deposit_order limit 3;

The data is:

`test商户  {"ip": "1.1.1.1", "os": "macOS 10.15.7)", "time": 3, "brand": "其他", "equipment": "电脑"}

The archive table is:
SQL:

SELECT merchant_name ,payer_information FROM o_deposit_order_history

Data:

test商户 {"ip": "1.1.1.1", "os": "未知系统 未知版本", "time": 0, "brand": "其他", "equipment": "未知设备", "homePlace": "ä¸å›½-广东省-广州市-电信"}

pt tool version: 3.7.0
mysql version: gcp sql 8.0.42

2

一模一样的问题

same problem. my json data contains chinese characters as above

pt tool version is 3.7.0 and mysql version is 8.0.31.

I found that this question has been raised for 5 years and has not beed solved yet.

see that: Jira

4

@Veitor I gave up and wrote a similar archiving tool based on its principle. It has solved the existing bugs I found, such as garbled JSON fields and data anomalies caused by line breaks when batch inserting data. It is currently working well and the performance is sufficient for my table data of tens of millions.