This release of csv2db fixes an Oracle Database driver deprecation issue and moves the minimum oracledb driver version to 2.0.0.

This release of csv2db brings a lot of new and exciting functionality and enhancements.

Highlights 🎉 🍾 🎇

• New command line options to control error scenarios during the data load and more. 🛠️
• csv2db now also releases as a Python package on PyPI.org, making it directly installable via pip install csv2db. 🐍 📦

Added 🆕

• New option --log to log erroneous lines in a *.bad file of the same name (ER #46)
• New option --ignore to ignore errors in file and continue loading (ER #46)
• New option --truncate to empty (truncate) the target table before the load (ER #45)
• New option --encoding to specify the file encoding, if not compatible with UTF-8 (bug #50)
• New option --case-insensitive-identifiers to uppercase all identifiers (ER #60)
• New option --quote-identifiers to quote all table and column identifiers (ER #61)
• Release csv2db as a Python package (ER #53)

Changed 🔄

• Read all input files with UTF-8 encoding by default
• Upgraded to oracledb driver for Oracle Database
• Upgraded to psycopg (3) driver for Postgres
• Remove usage of psycopg2.extras.execute_batch() as psycopg 3 now supports executemany() natively
• Change testing strategy to separate DB loading tests
• Implemented GitHub Action testing

Fixed ✅

• Raise an error when empty column names in the header are encountered (bug #56)

This release of csv2db brings some bug fixes and further enhancements. Most noticeable in this release are the following changes:

• csv2db now uses a different, much, much faster method to load data for Postgres which brings the performance for Postgres on-par with the other databases
• csv2db now uses UTF-8 for client-side connection encoding for Oracle Database (was defaulted before to ASCII by the Oracle driver)

Changes:

• Using psycopg2.extras.execute_batch() to speed up data load with Postgres (ER #39)
• Use UTF-8 connection encoding for Oracle (bug #36)
• Added PROTOCOL=TCPIP and AUTHENTICATION=SERVER to the Db2 connect string.
• Convert row to tuple on array append (bug #35)
• Enhanced error reporting (ER #37)

This release introduces interactive password entry.

If the password hasn't been provided via the command line, csv2db will prompt for it. This method is generally more secure because the password is never exposed in clear text.

This release reflects the description change of csv2db from The CSV command line loader to The CSV to database command line loader.

This release does not include any additional functionality nor bug fixes.

This release includes support for Windows!
From now on you can also easily run csv2db via the csv2db.bat file.

This release brings a lot of new functionality to csv2db!

First and foremost, this release introduces support for SQL Server!

We also got a brand new icon for csv2db which we are particularly proud of.

                  ___       ____  
  ___________   _|__ \ ____/ / /_ 
 / ___/ ___/ | / /_/ // __  / __ \
/ /__(__  )| |/ / __// /_/ / /_/ /
\___/____/ |___/____/\____/_____/ 
                                  

Mandatory command line arguments are now checked and csv2db exits immediately if no files were found, as to before where it unnecessarily connected to and disconnected from the database.
csv2db also now has different exit codes for different termination conditions, see Exit codes in the README for more information.
We have added more precise status output, more debug statements during the file reading operations and csv2db now also prints the failed data record, if any, in the debug output to help you find out what data may be corrupted.

csv2db now provides color output, yellow for debug statements, red for errors. This can be turned off by setting the $NO_COLOR environment variable to any value, supporting the NO_COLOR standard.

We have changed the default data type for the generate functionality to VARCHAR(1000) to be out of the box compatible with all supported databases.

csv2db has also seen some bug fixes in this release:

The column order is now just as in the files from the header, as to before random.
The internal record list is now cleared on error, to start with clean data from the next file, if any.
Autocommit is explicitly turned off as some drivers have it on by default.

We hope that you enjoy this brand new version of csv2db!

This release includes a bug fix for Oracle Database directpath loading and other minor bug fixes

This release introduces support for (custom) quote characters. Values inside quote characters won't be split even if the delimiting character(s) is(are) contained within the value.

Let's assume we have a line 1234,"This is but a test, to demonstrate the quote characters". Without quote character support the values would have been split into:
1234
,"This is but a test
to demonstrate the quote characters"

With this release the line is split into the correct values of:
1234
This is but a test, to demonstrate the quote characters

This release also introduces support for direct-path loading with Oracle Database for faster loading.

Support for user-defined column separators

This release added support for user-defined column separators, making csv2db a universal character delimited file loading tool for databases. The default separation character remains , providing backwards compatibility. Other separators can be passed on via the -s or --separator parameter.

Bug fixes

• Minor bug fix for avoiding to flush an empty data set (bug #6)