You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As far as I can tell, there is not a way to import NULL values from a CSV. I suggest an additional checkbox in the "Import CSV file" window of "Treat missing values as NULL". Or I'm curious if you all have another idea to handle this.
Thanks!
Yep, that would be useful. The generally referenced CSV spec (an RFC) doesn't take into consideration database things. eg default values, NULL values, and similar
On that note, we should probably also have the choice of treating missing (eg empty) values as "use the default value for the field, if defined". 😉
A crappy hacky workaround which I've used in the past is to use a value thats either 'nonsense' or known. So if a date field was to have NULL values, use a date thats otherwise invalid (eg, 99/99/9999) or is outside the usual range (eg, 01/01/2000) then simply do a 'update tablename set fieldname = null where fieldname='99/99/9999' etc.
Yup, told you it was hacky. 🔢
We're trying to migrate from the no-longer maintained Firefox extension (which treated blank columns as NULL) to DB Browser and this difference bit us unexpectedly. I'd vote for an option to treat blanks as NULL on the import screen.
Oops. We really should get a review done of #1349.
As a thought for now, while we wait for the code review to get done... would it help if I make a one-off build of DB4S (our abbreviation for DB Browser) including the code in that Pull Request?
Looks great! Thanks team.
This is my first look at the nightly build versions. I like the other changes I noticed: all caps for the suggested SQL commands, in the "Execute SQL" tab moving "Save as CSV/View" to the tool bar. Do you have a new official release planned?
We're hoping for some time in the next few weeks. As a rough guide 😉, we're generally wanting to get all of the Pull Requests completed, and fix any other bug type things that seem important enough to get included as well.
Thanks for testing, @mkweskin! I'll close this issue then 😄
By the way, if you're interested in helping with the next release, maybe @justinclift can build a once-off build for #1394 for you to play around with. The changes in this pull request are large enough that some real-world test results could be helpful 😉
Yep, that's pretty easy to do, especially for OSX builds. It's 99.5% scripted, so I just need to point the script at the right bits and launch it.
@mkweskin I'm happy to create an OSX build with the multi-threading code for #1394 included, if you'd be ok to try things out with it and report back weirdness/bugs/etc? 😄
Sure thing, happy to do more testing. Let me know where to download this one-off build when it's ready. It sounds like you're saying it's different than the nightly builds.
Yeah. The nightly builds (done automatically) use whatever code is in our master branch.
I made a copy of the nightly build script a while ago, and adjusted it so it can be told to build from a different source. It also uploads to a different place on our nightlies server.
I'll get the build we're discussing here launched in a few minutes. It takes a while to complete though (~1/2 from memory). 😄
Activity
justinclift commentedon Mar 8, 2018
Yep, that would be useful. The generally referenced CSV spec (an RFC) doesn't take into consideration database things. eg default values, NULL values, and similar
On that note, we should probably also have the choice of treating missing (eg empty) values as "use the default value for the field, if defined". 😉
mkweskin commentedon Mar 8, 2018
Agree about the use default option. I tried defining default as NULL and doing an import and saw that the default wasn't used.
chrisjlocke commentedon Mar 8, 2018
A crappy hacky workaround which I've used in the past is to use a value thats either 'nonsense' or known. So if a date field was to have NULL values, use a date thats otherwise invalid (eg, 99/99/9999) or is outside the usual range (eg, 01/01/2000) then simply do a 'update tablename set fieldname = null where fieldname='99/99/9999' etc.
Yup, told you it was hacky. 🔢
justinclift commentedon Mar 8, 2018
Still, that's a useful workaround for now. 😄
remram44 commentedon Mar 15, 2018
Interestingly the behavior matches sqlite3's
.importcommand. But I agree that it's a terrible default.remram44 commentedon Mar 16, 2018
The current code looks at the column type to decide what to do, but only has special behavior for
INTEGERcolumns:sqlitebrowser/src/ImportCsvDialog.cpp
Lines 560 to 565 in a242a58
aelfric commentedon Apr 25, 2018
We're trying to migrate from the no-longer maintained Firefox extension (which treated blank columns as NULL) to DB Browser and this difference bit us unexpectedly. I'd vote for an option to treat blanks as NULL on the import screen.
justinclift commentedon Apr 25, 2018
Oops. We really should get a review done of #1349.
As a thought for now, while we wait for the code review to get done... would it help if I make a one-off build of DB4S (our abbreviation for DB Browser) including the code in that Pull Request?
MKleusberg commentedon May 22, 2018
Just merged that pull request 😄 Can you check tomorrow's nightly build @aelfric and @mkweskin to see if it's now working for you as expected? 😄
mkweskin commentedon May 23, 2018
Looks great! Thanks team.
This is my first look at the nightly build versions. I like the other changes I noticed: all caps for the suggested SQL commands, in the "Execute SQL" tab moving "Save as CSV/View" to the tool bar. Do you have a new official release planned?
justinclift commentedon May 23, 2018
We're hoping for some time in the next few weeks. As a rough guide 😉, we're generally wanting to get all of the Pull Requests completed, and fix any other bug type things that seem important enough to get included as well.
MKleusberg commentedon May 24, 2018
Thanks for testing, @mkweskin! I'll close this issue then 😄
By the way, if you're interested in helping with the next release, maybe @justinclift can build a once-off build for #1394 for you to play around with. The changes in this pull request are large enough that some real-world test results could be helpful 😉
justinclift commentedon May 24, 2018
Yep, that's pretty easy to do, especially for OSX builds. It's 99.5% scripted, so I just need to point the script at the right bits and launch it.
@mkweskin I'm happy to create an OSX build with the multi-threading code for #1394 included, if you'd be ok to try things out with it and report back weirdness/bugs/etc? 😄
mkweskin commentedon May 24, 2018
Sure thing, happy to do more testing. Let me know where to download this one-off build when it's ready. It sounds like you're saying it's different than the nightly builds.
justinclift commentedon May 24, 2018
Yeah. The nightly builds (done automatically) use whatever code is in our
masterbranch.I made a copy of the nightly build script a while ago, and adjusted it so it can be told to build from a different source. It also uploads to a different place on our nightlies server.
I'll get the build we're discussing here launched in a few minutes. It takes a while to complete though (~1/2 from memory). 😄
justinclift commentedon May 24, 2018
k, just started the script. In theory - if I didn't forgot something 😉 - then the builds should appear here in about 30-40 mins:
https://nightlies.sqlitebrowser.org/osx-onceoffs/
There will be 2 versions. One with standard SQLite support, and one with SQLCipher support. Use whichever appeals. 😄
justinclift commentedon May 24, 2018
Playing around with the one including SQLCipher support. Seems to "work ok" in very light initial testing now.
I'd better get the DBHub.io migration stuff completed today, so we have some large example data sets we can easily try with too. 😄