# Rocksdb: Alter table fails silently and drops all data: Global seqno is required, but disabled, **URL:** \ **Category:** Percona Server for MySQL\ **Created:** [December 9, 2023, 1:33pm UTC](https://forums.percona.com/t/rocksdb-alter-table-fails-silently-and-drops-all-data-global-seqno-is-required-but-disabled/27038 "2023-12-09T13:33:11Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![peterdk](https://avatars.discourse-cdn.com/v4/letter/p/898d66/32.png) [@peterdk](https://forums.percona.com/u/peterdk)\ **Post date:** [December 9, 2023, 1:33pm UTC](https://forums.percona.com/t/rocksdb-alter-table-fails-silently-and-drops-all-data-global-seqno-is-required-but-disabled/27038/1 "2023-12-09T13:33:11Z")
I did alter a fairly small rocksdb table yesterday, and it reported that everything went ok. So I was happy. Today I did another table which was a bit larger, and added a field there as well. It reported that all went fine, but I noticed inphpmyadmin afterwards that all data was gone… I checked the logs, and found the following message: > Plugin rocksdb reported: ‘Failed to bulk load. status code = 4, status = Invalid argument: Global seqno is required, but disabled, IngestExternalFileOptions={move\_files=1,failed\_move\_fall\_back\_to\_copy=1,snapshot\_consistency=0,allow\_global\_seqno=0,allow\_blocking\_flush=0,ingest\_behind=0,write\_global\_seqno=1,verify\_checksums\_before\_ingest=0,verify\_checksums\_readahead\_size=0,verify\_file\_checksum=1,fail\_if\_not\_bottommost\_level=0}’ I double checked the table change of yesterday, and surely, that one also had no data in it anymore besides new data from yesterday onwards… The logs had a identical message I found out. Seems like a big issue! What I did was: ```auto set session rocksdb_bulk_load=1; ALTER TABLE `foo` ADD `bar` TINYINT NULL DEFAULT NULL AFTER `foobar`; ``` Can I get my data back besides restoring a backup? Is this a bug? Should I set a certain variable for this to work? My config: > rocksdb-override-cf-options=‘default={compression=kZSTD;bottommost\_compression=kZSTD;compression\_opts=-14:12:0;level\_compaction\_dynamic\_level\_bytes=true;write\_buffer\_size=128m;target\_file\_size\_base=32m;max\_bytes\_for\_level\_base=512m;block\_based\_table\_factory={format\_version=5;index\_block\_restart\_interval=16;cache\_index\_and\_filter\_blocks=1;filter\_policy=bloomfilter:12:false;whole\_key\_filtering=1}}’ > rocksdb\_block\_size=16384 > rocksdb\_max\_open\_files=-1 > rocksdb\_max\_background\_jobs=8 > rocksdb\_flush\_log\_at\_trx\_commit=2 > rocksdb\_large\_prefix=1 > open\_files\_limit = 102400 > rocksdb\_block\_cache\_size = 12G > rocksdb\_max\_total\_wal\_size=1G > transaction-isolation=READ-COMMITTED > rocksdb\_max\_row\_locks=1073741824 > Ver 8.0.34-26 for Linux on x86\_64 (Percona Server (GPL), Release ‘26’, Revision ‘0fe62c85’) > Ubuntu 22.04.3 LTS ---
**Author:** ![peterdk](https://avatars.discourse-cdn.com/v4/letter/p/898d66/32.png) [@peterdk](https://forums.percona.com/u/peterdk)\ **Post date:** [December 9, 2023, 7:13pm UTC](https://forums.percona.com/t/rocksdb-alter-table-fails-silently-and-drops-all-data-global-seqno-is-required-but-disabled/27038/2 "2023-12-09T19:13:09Z")
I performed a restore of the staging DB, but the restoring fails as well, with same error… ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [December 11, 2023, 5:40pm UTC](https://forums.percona.com/t/rocksdb-alter-table-fails-silently-and-drops-all-data-global-seqno-is-required-but-disabled/27038/3 "2023-12-11T17:40:47Z")
Hi @peterdk, I’m not seeing anything in the manual that says you should set bulk loading ON when doing normal table alters. The manual says you should do this only when converting from other table engines, or when doing LOAD DATA. The manual also states “The data may not be visible until bulk load mode is ended (i.e. the rocksdb\_bulk\_load is set to zero again)” Additionally, “None of the data being bulk loaded can overlap with existing data in the table.” which is pretty much what you are doing with an ALTER. And finally, “Inserting one or more rows out of order will result in an error and may result in some of the data being inserted in the table and some not.” It is recommended that you do not use bulk loading for simple ALTER table to add columns. You should opt to use pt-online-schema-change instead. ---
**Author:** ![peterdk](https://avatars.discourse-cdn.com/v4/letter/p/898d66/32.png) [@peterdk](https://forums.percona.com/u/peterdk)\ **Post date:** [December 11, 2023, 6:18pm UTC](https://forums.percona.com/t/rocksdb-alter-table-fails-silently-and-drops-all-data-global-seqno-is-required-but-disabled/27038/4 "2023-12-11T18:18:46Z")
Hi @matthewb Thanks for the reply. It’s interesting. I thought I did read that bulk loading setting would speed up alter tables. But I might have mistaken indeed. Strangely I did encounter the error as well when importing logical backup of a table. I spun up a instance of MariaDB and there all imports went without issue. So I am not sure if this was my error or Percona Server. I’ll try to reproduce the import error again in Percona, and report back. This might take a few days. ---
**Author:** ![peterdk](https://avatars.discourse-cdn.com/v4/letter/p/898d66/32.png) [@peterdk](https://forums.percona.com/u/peterdk)\ **Post date:** [December 20, 2023, 2:54pm UTC](https://forums.percona.com/t/rocksdb-alter-table-fails-silently-and-drops-all-data-global-seqno-is-required-but-disabled/27038/5 "2023-12-20T14:54:21Z")
Hi, I did experiment a bit more, and read more about it. And you are right, bulk load should not be used for ALTER TABLE commands. I guess I thought so, because it was needed to do for TokuDB tables. I do still get weird errors when importing data into a fresh table when using bulk load. But it might be caused by the /tmp folder not having enough space. I’ll research a bit more. ---
**Author:** ![peterdk](https://avatars.discourse-cdn.com/v4/letter/p/898d66/32.png) [@peterdk](https://forums.percona.com/u/peterdk)\ **Post date:** [December 26, 2023, 3:17pm UTC](https://forums.percona.com/t/rocksdb-alter-table-fails-silently-and-drops-all-data-global-seqno-is-required-but-disabled/27038/6 "2023-12-26T15:17:56Z")
So, when inserting all data without using bulk\_load\_unsorted, everything works as expected. Table changes work as well, without setting bulk\_load, as indicated by Matthew. My DB server is humming along again. ---
**Author:** ![jinyou.ma](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/jinyou.ma/32/19714_2.png) [@jinyou.ma](https://forums.percona.com/u/jinyou.ma)\ **Post date:** [November 12, 2025, 3:48am UTC](https://forums.percona.com/t/rocksdb-alter-table-fails-silently-and-drops-all-data-global-seqno-is-required-but-disabled/27038/7 "2025-11-12T03:48:37Z")
Hi @peterdk Please check the bug report [Jira](https://perconadev.atlassian.net/browse/PS-9097)