Skip to content

Leave generated columns out of Export to SQL - #4198

Open
anandghegde wants to merge 1 commit into
sqlitebrowser:masterfrom
anandghegde:fix-sql-export-generated-columns
Open

anandghegde wants to merge 1 commit into
sqlitebrowser:masterfrom
anandghegde:fix-sql-export-generated-columns

Conversation

@anandghegde

Copy link
Copy Markdown

Fixes #2978.

Export → Database to SQL file writes every column of a table into its INSERT statements, generated columns included. SQLite doesn't allow inserting into a generated column, so the exported file can't be imported again:

CREATE TABLE orders(id INTEGER PRIMARY KEY, qty INTEGER, price REAL,
                    total REAL GENERATED ALWAYS AS (qty*price) STORED,
                    label TEXT GENERATED ALWAYS AS ('order ' || id) VIRTUAL,
                    note TEXT);
  • Default export: INSERT INTO "orders" VALUES (1,2,9.5,19.0,'order 1','O''Brien'); fails with table orders has 4 columns but 6 values were supplied.
  • With "Keep column names in INSERT INTO": fails with cannot INSERT into generated column "total".

On #2978 it was suggested that the export should keep all columns. The sqlite3 shell's .dump leaves generated columns out (INSERT INTO t VALUES(1);), because SQLite recomputes them on import. This change makes DB4S behave the same way, so its SQL export can be re-imported.

Change

In DBBrowserDB::dump(), tables that have at least one generated column select, and name in the INSERT statements, only their other columns. Tables without generated columns keep the existing SELECT *, so their export is byte-for-byte unchanged. The same file already skips generated columns this way in emptyInsertStmt() and when copying data during alterTable().

Test plan

Built on macOS (Apple Silicon, Qt 5.15.16 from the project tap) with -DENABLE_TESTING=ON. ctest passes (4/4). None of those tests cover the export.

Exported the orders table above, which has a stored and a virtual generated column, a quoted value and a NULL, together with a plain table. I used each export option and re-imported every file into a fresh database with sqlite3:

Export option INSERT output for orders Re-import
Default INSERT INTO "orders" VALUES (1,2,9.5,'O''Brien'); ✅ data identical to the original, generated values included
Keep column names INSERT INTO "orders" ("id","qty","price","note") VALUES (...) ✅ identical
Multiple rows per INSERT INSERT INTO "orders" VALUES (1,2,9.5,'O''Brien'),\n (2,3,1.25,NULL); ✅ identical

The plain table exports as before in all three.

CSV import into an existing table with generated columns has a similar problem (ImportCsvDialog.cpp). I've kept this PR to the SQL export and can send that as a separate PR if it's wanted.

Export > Database to SQL file wrote every column into the INSERT
statements, including generated columns. SQLite refuses to insert into a
generated column, so importing the file failed with "table X has N columns
but M values were supplied", or with "cannot INSERT into generated column"
when column names were kept.

Only for tables that have generated columns, select and insert the other
columns instead, the same as the sqlite3 shell's .dump. The generated values
are recomputed on import. Other tables still export with SELECT *.

Fixes sqlitebrowser#2978
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Export Database in SQL file : ERROR, Insert into generated column

1 participant