Repository navigation
Better default options and configuration for the CSV import #1395
Description
@karim came up with a good default behaviour for the CSV import in #1349:
I just want to say something. 😃
Regarding the expected behavior of DB4S when importing empty values from a CSV.
- If the column has a default value then it gets priority over everything else.
- This is the default action for SQLite, it fills columns with default values when a value is not given.
- The column being
NOT NULLor not doesn't matter, since a value will be set for it anyway.- ✔️ No need for a checkbox, it should be the right thing to do. 👍
- If the column does not have a default value.
- Check if the column does not have
NOT NULL, if true:
- Then just add
NULLto it, it should be the default.- Be consistent. CSV value is
NULL, SQLite table doesn't forbidNULL, so just addNULL.- ✔️ Also, no need for a checkbox, and this is what everyone here wants. 🎉
- Else if it does have
NOT NULL, then we have two options, either:
- Fail the import (make sense since you are importing
NULLto aNOT NULLcolumn), or- Set a default value for it (e.g.
0forINTEGER,""forTEXT)
- Probably by adding a checkbox to "set default values for empty fields in NOT NULL columns" (need a better wording 😄)
We should implement some rules in the import code to reflect this and maybe add some configuration options to make any deviation from this default an explicit decision by the user.
Activity
justinclift commentedon May 22, 2018
Oh, we should probably also have an option to turn off the data type autodetection. Or at least have the values given in quotes
"foo"always be strings.I personally find the autodetection useful, but it was pointed out in #1382 that for some use cases it's a bad thing and leads to data loss.
Personally, I'm really bothered by our lack of warning to users when we mis-import stuff too, but I have no solid idea how to fix it just yet.
justinclift commentedon May 22, 2018
... and there's #1347 as well. That one looks reasonably simple though. 😄
Implement better default behaviour for CSV import
MKleusberg commentedon May 25, 2018
Just pushed a first attempt at implementing @karim's set of rules above. We're not failing the import by default as suggested in 2.II.a, but there is a new option to make it fail in that case.
A few more notes and questions:
MKleusberg commentedon Jun 7, 2018
If nobody has reservations about my decisions regarding question 2 and 3 in my last comment, I would close this issue because the other issues mentioned by @justinclift already have their own tickets 😄
justinclift commentedon Jun 7, 2018
No reservations from me.
I reckon we should tweak the names and tooltip descriptions a bit for the CSV options, but that's easily a follow up thing for later. 😄
Implement better default behaviour for CSV import
MKleusberg commentedon Jun 7, 2018
Ok cool, I'll close this issue then 😄