Repository navigation
[Bug]: No error check for export errors (insufficient space) #3243
Description
What did you do?
Browse/filter some data.
Click the Save Table As... - Export to CSV menu.
Waiting until it says completed.
What did you expect to see?
Success, or errors.
If possible, like number of lines and file size. And the window should be fast responding.
What did you see instead?
The program said the "Export completed.".
The available space of the target driver is not enough. It may become close to 0 bytes, or tens of MB available (because other system occupies, such as the target directory compressed by NTFS).
You can create a virtual disk to reproduce it, such as a fixed 200 MB VHD disk.
The exported CSV file is incomplete, it is cut off as space exhausted.
If I did not notice the disk space, I might be mistakenly assuming that CSV is a complete copy and potentially causes data loss.
I suspect that if the disk is released during the full disk, the result file will be broken that may be a correct ending but no middle contents.
https://github.com/sqlitebrowser/sqlitebrowser/blob/master/src/ExportDataDialog.cpp#L375
DB4S Version
3.12.2
What OS are you seeing the problem on?
Windows
OS version
Windows 10
Relevant log output
No response
Prevention against duplicate issues
- I have searched for similar issues
Activity
Export: fix no error check for export errors
mgrojo commentedon Dec 22, 2022
@yfdyh000, thanks for the report. It should be fixed in tomorrow's nightly build. Please, close this issue when you can confirm it.
yfdyh000 commentedon Dec 23, 2022
This problem still occurs in the latest nightly build.
mgrojo commentedon Dec 23, 2022
You don't get this error dialog?
yfdyh000 commentedon Dec 23, 2022
Sorry, I have a stupid mistake when switch version.
When Export CSV, the warning work. Although, it still says "Export complete." too.
When Export SQL, the same issue still exists, no warning.
When Export JSON, I will get an incomplete JSON file that is slightly smaller than available space, no warning.
justinclift commentedon Dec 25, 2022
@yfdyh000 Just to check, how much disk space is actually free on the Windows drive you're writing to?
Also, is it running in WSL(2) or similar? Not sure if that makes a difference, am asking that just in case it turns out to be. 😄
yfdyh000 commentedon Dec 25, 2022
For testing, total 100MB or 200MB fixed VHD disk in Windows 10. sqlitebrowser run on Windows 10.
Export: more error checks
mgrojo commentedon Dec 26, 2022
I've added error checks in the SQL export and made some tuning in the CSV and JSON versions. Moreover, the final message after exporting all the selected tables shows a different message warning about the export process having finished with errors. The message "Export completed" is only displayed when there have been no error.
In my tests, I still see some cases where even trying to write in a full partition, and after creating an empty file, Qt or Linux does not report any failure. I think it is in cases when all the content written is buffered in memory and the error is not detected even after closing the file.
justinclift commentedon Dec 26, 2022
That's a bit worrying. Doing some looking online shows Qt has a "Flush" function:
https://doc.qt.io/qt-5/qfiledevice.html#flush
However, it's unclear what that actually does. eg:
https://bugreports.qt.io/browse/QTBUG-8544
This could be one of those "can of worms" things we might not want to open. Not sure. 😉
mgrojo commentedon Dec 26, 2022
I knew
close()calledflush()but I just noticed that it ignores errors from flush (why?!). If it actually flushes the file, a call toflush()before checking for errors might give some error. I will try that.Export: use QSaveFile to get better error check on full partition
mgrojo commentedon Dec 27, 2022
I've made a change that might improve the full disk condition, even for smaller files: changing use of
QFilebyQSaveFile, which, according to Qt documentation, should handle that case. Nonetheless, there is a bug that causes that implementation to still fail under Linux, see https://bugreports.qt.io/browse/QTBUG-75077In any case, I hope the situation is better in other OS. @yfdyh000, could you try the next nightly on Windows and see what happens?
yfdyh000 commentedon Dec 29, 2022
Looks good for SQL and CSV export. JSON export still has problem, I see "completed" and an incomplete file.
mgrojo commentedon Dec 29, 2022
The file handing for JSON is doing the same thing, the single difference is in the writing phase. For the JSON case, a JSON object is generated in memory, and then it's dumped to the file in a single write() call. I don't know how that could affect the result. Maybe Windows is affected by the Qt bug, although in a less general case.
Export: fix no error check for export errors
Export: more error checks
Export: use QSaveFile to get better error check on full partition