Hi ,
We are getting below error during pt-archival of 1 table’s
Unicode non-character U+FFFF is illegal for open interchange at /bin/pt-archiver line 6923.
Kindly check and help for this
Hello @SumitKumar
Are you using the latest version of pt-archival ?
What is the MySQL/PS version and charset of a table?
Are you using the latest version of pt-archival ?
VERSION is pt-archiver 3.5.7
What is the MySQL/PS version and charset of a table?
version: 8.0.22-commercial MySQL Enterprise Server - Commercial
Table charset – ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
Thanks
Please update
Hi
Requesting…Can any one help for this ?
Hi @SumitKumar,
Let’s do a thing. Provide us your table definition and command; I’ll try to reproduce.
We already know pt-archiver 3.5.7 and mysql 8.0.22
@kedarpercona Please check below as requested
Table Definition 
Create Table: CREATE TABLE od_audit (
AUDIT_ID varchar(32) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
EVENT_ID int NOT NULL,
APP_SERVER varchar(20) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
CLIENT_IP_ADDR varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL,
WEB_SERVER varchar(20) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
AUDIT_DATE datetime NOT NULL,
REFERENCE varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
SESSION_ID varchar(150) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
OD_LOGIN_ID varchar(120) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
USER_NO varchar(120) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
OD_GCIF varchar(120) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
OD_PARENT_GCIF varchar(120) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
SIMULATION_MODE varchar(1) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
CHANNEL varchar(20) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
BROWSER_NAME varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
AUDIT_DATA longtext COLLATE utf8_unicode_ci,
DSA_DATA text CHARACTER SET utf8 COLLATE utf8_unicode_ci,
DEVICE_BAND_ID varchar(10) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
USER_AGENT varchar(500) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
WORKSPACE_ID varchar(50) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
LAYOUT_ID varchar(50) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
WIDGET_ID varchar(50) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
GEO_LOCATION varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
REQUEST_ID varchar(45) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
REQUEST_URI varchar(256) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
REFERENCE_KEY varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF1 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF2 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF3 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF4 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF5 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF6 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF7 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF8 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF9 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
UDF10 varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (AUDIT_ID),
UNIQUE KEY SYS_C008745 (AUDIT_ID),
KEY OD_AUDIT_IDX4 (USER_NO,SESSION_ID,OD_GCIF),
KEY OD_AUDIT_IDX2 (OD_GCIF,EVENT_ID,AUDIT_DATE),
KEY OD_AUDIT_IDX3 (OD_GCIF,OD_LOGIN_ID,EVENT_ID,AUDIT_DATE),
KEY OD_AUDIT_TRAIL_IDX1 (REFERENCE)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
1 row in set (0.00 sec)
Command is:-
TABLE=od_audit
pt-archiver --source h=localhost,u=$USERNAME,p=$PASSWORD,D=$DATABASE,t=$TABLE --where ‘AUDIT_DATE < now()’ --no-check-charset --limit=5000 --commit-each --file ${ARCHIVAL_PATH}/%Y-%m-%d-%D.%t 2>> $LOGPATH/arch_cp.err
RETURN_STATUS1=$?
status_check
Had a quick test and you rcommand is ok. The place where it appears to have stuck or “erroring” appears to be complaining about the data it is processing.
Unicode non-character U+FFFF is illegal for open interchange - Your data appears to have these non-character data included causing the archiver to fail. Is it possible to clean your data from such special characters?
Hi @kedarpercona ,
How we find these special character and clean ?
Hi Team,
Is it possible to clean your data from such special characters?
We can can change manually such type character from the tables , any other solution please help ?
Ignore above comment
Is it possible to clean your data from such special characters?
We can not change manually such type character from the tables , any other solution please help ?
So did this work for you: PT- Archiver Error - #7 by SumitKumar