Error_code: 1677 on Rolling upgrade

[giomanda]

Hi

I have a percona cluster of three servers and i am trying to perform rolling schema upgrade with RSU + node dropping method.
Basicaly what you do is:

1.remove the a server from the cluster,
2.change OSU method to RSU,
3.perform the schema change
4. Add server back to cluster

I did the above method on a small table and it all worked well how ever when i tried on a much bigger table (took like 5 hours to complete) when added the server back to cluster i got the following error:

150331 9:05:09 [Note] WSREP: Receiving IST: 278908 writesets, seqnos 42240-321148
150331 9:05:10 [ERROR] Slave SQL: Column 3 of table ‘main_db.allRecords’ cannot be converted from type ‘varchar(120)’ to type ‘varchar(100)’, Error_code: 1677
150331 9:05:10 [Warning] WSREP: RBR event 2 Write_rows apply warning: 3, 42266
150331 9:05:10 [Warning] WSREP: Failed to apply app buffer: seqno: 42266, status:1
at galera/src/replicator_smm.cpp:apply_wscoll():55

Weird thing is that the change i made was indeed on relevant table on Column 3 from VARCAHR(40) to VARCHAR(100)…
Any ideas?

Thanks

[wagnerbianchi]

Do you have any other columns with VARCHAR data type on the same table? Can you check if you have the same column order of the same table among the nodes? How about the SQL_MODE on the new node? It seems that some values are being truncated and due to that, just the node where the transaction has started has the information while other nodes don’t have it. Check data truncation or SQL_MODE…and provide more details about that.