Skip to content

[Cell Editor] Data detection / Qt edit gadget strip [CR's] from BLOB data. #1793

Open

Description

@sky5walk

Details for the issue

Bug taken from Closed Issue

What did you do?

Create a BLOB column.
Populate with any multiline text.
View entry in Cell Editor and observe with the available options:
Text
Binary
Image
XML
JSON

What did you expect to see?

When selecting Binary, I expected to see the EXACT hex information I provided.
And the exact Byte count I provided.

What did you see instead?

The hex information shows my [CR] characters are stripped from the [CR+LF] line endings.
Only Hex 0A remains.
The DB4S Character(s) count is NOT accurate since the [CR]'s are missing.

Useful extra information

The Qt editor/DB4S code does reapply the [CR+LF] characters on export or cut and paste clipboard?
All very confusing.
The prior Issue mentioned dropping the Qt editor gadget and just use the Scintilla gadget for all modes.

What operating system are you using?

  • Windows: ( version:10 Pro_ )
  • Linux: ( distro: ___ )
  • Mac OS: ( version: ___ )
  • Other: ___

What is your DB4S version?

  • 3.11.1
  • 3.11.0
  • 3.10.1
  • Other: ___3.11.99 v190304

Did you also

Activity

  1. added
    bugConfirmed bugs or reports that are very likely to be bugs.
    on Mar 8, 2019
  2. mgrojo commented on Mar 8, 2019

    @mgrojo
    Member

    Thanks @sky5walk. Have you confirmed that the Scintilla widget can load a Windows text file (a JSON or XML file with CR+LF endings) and behaves correctly in Windows, both in terms of displaying the lines and CR not being stripped?

  3. sky5walk commented on Mar 9, 2019

    @sky5walk
    Author

    Yes, but with a simple test:

    1. Navigate to a null BLOB cell.
    2. SET cell editor mode to JSON
    3. Paste/Import json file contents with [CRLF] line endings.
    4. Observe Character(s) calculated by DB4S.
    5. Repeat 1-4 with other cell editor modes.

    NOTE: Once the [CR] characters are stripped, the character count is invalidated.
    Pasting with Binary cell mode corrupts the data?
    I don't understand Binary?

    INPUT -> JSON w/[CRLF] from Clipboard

    CellMode Character(s)
    TEXT FAIL <-- [CR]'s stripped
    JSON PASS
    XML PASS <-- Auto-detection reassigns mode to JSON.
    BINARY FAIL <-- Why is input data altered?

    INPUT -> JSON w/[CRLF] from imported file.

    CellMode Character(s)
    TEXT FAIL <-- [CR]'s stripped
    JSON PASS
    XML PASS <-- Auto-detection reassigns mode to JSON.
    BINARY PASS <-- Auto-detection reassigns mode to JSON.

    Conclusion:

    Auto-detection should be disabled when user selects BINARY mode.
    Scintilla gadget should be used for all non-binary views.

  4. sky5walk commented on Mar 10, 2019

    @sky5walk
    Author

    Yay 😄, the cell editor Scintilla gadget does not strip [CR]'s and I now have identical Character counts.
    BUG(s):
    If I hit the auto-detect wheel, the font is changed in the cell editor Scintilla gadget.
    If I hit XML or any non-TEXT mode and return to TEXT, the font is changed.
    It returns to my setting after closing and reopening DB4S.

    Do we really need the auto-detect wheel❓

  5. Johntaa commented on Mar 10, 2019

    @Johntaa

    Database cell editor doesn't accept LF & CR in text , I used to insert html entity
    to substitute the LF. Hope it will be solved in the next release.

  6. sky5walk commented on Mar 10, 2019

    @sky5walk
    Author

    Ooops, [LF]'s are converted to [CRLF]'s now. 😮
    Still so much auto-detection??
    Even if I paste into BLOB with Binary mode.

  7. mgrojo commented on Mar 10, 2019

    @mgrojo
    Member

    Ooops, [LF]'s are converted to [CRLF]'s now. 😮
    Still so much auto-detection??

    If the only problem is when pasting, I think that using this Scintilla option will fix it:
    https://www.scintilla.org/ScintillaDoc.html#SCI_GETPASTECONVERTENDINGS

    The drawback is that pasting data from sources of different modes will convert the text in a mixture. This will probably be unusual when working in only one OS with the same DB.

    If I hit the auto-detect wheel, the font is changed in the cell editor Scintilla gadget.
    If I hit XML or any non-TEXT mode and return to TEXT, the font is changed.
    It returns to my setting after closing and reopening DB4S.

    I can't reproduce this. The text mode is supposed to use the same font as the Data Browser while the other modes use the same font as the SQL editor (a monotype font, more adequate for a data definition or programming language). This was already so, but now it is trickier because the widget is the same and the font has to switch. I had problems with this, but now is working good for me, so once again a platform issue?

  8. sky5walk commented on Mar 10, 2019

    @sky5walk
    Author

    Yes, it seems to respect the content with File Import.

    Only TEXT mode changes the font.
    The other modes keep my original font.

  9. mgrojo commented on Mar 10, 2019

    @mgrojo
    Member

    You're right, the font used in the text mode is not the one configured for the data browser, as before, I think it's the font used by default by QScintilla. It is ignoring the setting. I'll have to make some testing or ask in the QScintilla mailing list.

  10. mgrojo commented on Mar 10, 2019

    @mgrojo
    Member

    It should be fixed in tomorrow's nightly build.

  11. sky5walk commented on Mar 11, 2019

    @sky5walk
    Author

    Nightly v190311 fixes the font change event. 👍
    If @Johntaa is ok with the LF issue, (I do not have his use model), I would close this issue.

  12. justinclift commented on Mar 23, 2019

    @justinclift
    Member

    @Johntaa Would you have time to check the nightly builds, and see if the problem is fixed from your point of view? 😄

    If you do, the nightly builds are here:

        https://nightlies.sqlitebrowser.org/latest/

  13. Johntaa commented on Mar 24, 2019

    @Johntaa

    @justinclift , Sorry, I tried many times but the nightly builds cannot show the text field any more in "Edit Data Cell" window, only if the language of the text is English, but not if text is in Arabic. This is a new issue that was not exist in the earlier releases. you can try the attached file, check the text field.
    sample.zip

  14. 7 remaining items

  15. scottfurry commented on Jun 23, 2019

    @scottfurry
    Contributor
  16. mgrojo commented on Jun 23, 2019

    @mgrojo
    Member

    You're right. I was thinking in using either the QChar::script() function (I think only Arabic, Syriac and Hebrew in that list are RTL scripts) or the QChar::direction() function (R, AL, RLE, RLO, RLI are RTL characters, according to Unicode).

  17. mgrojo commented on Jun 23, 2019

    @mgrojo
    Member

    Google search: https://doc.qt.io/qt-5/qguiapplication.html#layoutDirection-prop

    It is another option. It would be simple, but then a user with a RTL layout could be using English or another language in the DB or viceversa, an English interface in the application and a DB in Arabic or Hebrew.

  18. scottfurry commented on Jun 23, 2019

    @scottfurry
    Contributor

    A little more digging found that functionality exists in QApplication. This would be detectable at startup in the main.cpp and could be set accordingly. The layout direction would then propagate downward to all created child widgets.

  19. justinclift commented on Jun 24, 2019

    @justinclift
    Member

    Sounds like it might be worth exploring. 😄

  20. Johntaa commented on Jun 25, 2019

    @Johntaa

    Sounds like it might be worth exploring. smile

    Until you decide which option is the right one , isn't it possible to develop 3.12 as a new branch to follow up with the new updates other than Qt widget. and keep the Qt widget .
    Just a Question :)

  21. justinclift commented on Jun 25, 2019

    @justinclift
    Member

    Oh yeah, there's a bunch of different possibilities for doing that. It just depends on who wants to take it up and how they want to do it. 😄

  22. mgrojo commented on Apr 26, 2020

    @mgrojo
    Member

    @sky5walk Are all the problems address in 3.12 alpha?

  23. sky5walk commented on Apr 26, 2020

    @sky5walk
    Author

    Ok, almost done.
    For existing database content, the binary view now shows the [CR][LF] characters.
    But not when I paste new data into a blob field?
    The [CR]'s are not deleted, but they are not shown in the binary view.

  24. justinclift commented on Apr 27, 2020

    @justinclift
    Member

    As a general thought, I've been wondering if we should try building with the newest Qt release (5.14.2) too.

    Apparently that has changes to better enable accessibility, which would really help with Community members needing to use screen readers and such.

    Suspecting the newer Qt version might muck with some of our widgets (including the cell editor) again if we try though. 🤷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugConfirmed bugs or reports that are very likely to be bugs.response requested

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions