You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Table export to CSV crashes for large tables with out-of-memory exception. #107
It appears that the table export to CSV builds the export file content first in RAM before writing it to the specified file. If the table is very large the process is hitting the 2GB RAM limit of Windows 32bit applications. Then it crashes. No file gets written at all.
You are right, for streaming our sqltablemodel is not really useful as it will not free any unused data.
But doing real streaming for import and export shouldn't be to hard to implement and was on my list anyway.
Thanks for testing Jens. 😄 I'll close this issue here, because the original reported problem is fixed.
With the quote characters, they're supposed to be optional. Testing here just now, in Export → "Table as CSV file" dialog there is a "Quote character" option. Change that from the default of ", to the third line down (no character). That seemed to work for me, although I don't have a lot of data in the example database I just tried it on. Does that work for you?
Activity
rp- commentedon Sep 19, 2014
You are right, for streaming our sqltablemodel is not really useful as it will not free any unused data.
But doing real streaming for import and export shouldn't be to hard to implement and was on my list anyway.
rp- commentedon Oct 3, 2014
csv data is now streamed out and shouldn't fill up memory.
50731cd
justinclift commentedon Oct 5, 2014
@beyonav Would you have time to test this out in the Windows nightly builds, and confirm back to us if it works?
justinclift commentedon Oct 6, 2014
Oops, I should have mentioned. 😄
You can download the nightly windows builds from here:
http://rp.oldsch00l.com/sqlitebrowser/sqlitebrowser.exe.xz
.xz is a compressed file file, you can use 7-zip to uncompress it before installing.
justinclift commentedon Oct 6, 2014
Thanks for testing Jens. 😄 I'll close this issue here, because the original reported problem is fixed.
With the quote characters, they're supposed to be optional. Testing here just now, in Export → "Table as CSV file" dialog there is a "Quote character" option. Change that from the default of ", to the third line down (no character). That seemed to work for me, although I don't have a lot of data in the example database I just tried it on. Does that work for you?
justinclift commentedon Oct 6, 2014
Is this the right spec? https://tools.ietf.org/html/rfc4180
We had some discussion about CSV handing in #84. I guess we need to tweak things further still. 😄
justinclift commentedon Oct 6, 2014
Hmmm, we should probably turn this into a new Issue, for better tracking.
Are you ok to do that? If not, just say so and I'll create one. 😄