Skip to content

Cursor.executemany is unnecessarily slow #332

@klothe

Description

@klothe

The Cursor.executemany method is implemented by repeatedly calling Cursor.execute, rather than combining all the parameter lists into one statement (relevant code is here: https://github.com/pymssql/pymssql/blob/master/pymssql.pyx#L480).

For example, instead of

INSERT INTO table (x, y) VALUES (1, 2), (3, 4), (5, 6);

it produces:

INSERT INTO table (x, y) VALUES (1, 2);
INSERT INTO table (x, y) VALUES (3, 4);
INSERT INTO table (x, y) VALUES (5, 6);

where each INSERT statement requires a separate round trip between the client and the database server.

This is allowed by the DBAPI specification, but it defeats the point of executemany, because its performance no better than having the consumer call execute repeatedly.

This makes pymssql unusable when trying to insert many rows, especially over a slow network connection. Some other DBAPI implementations, such psycopg2, implement executemany by generating a single insert statement, which is fast; pymssql should do the same thing.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions