Skip to content

Reduce temporary string creation during types load#5986

Merged
NinoFloris merged 6 commits intonpgsql:mainfrom
bbowyersmyth:users/bruceb/typeStrings
Oct 29, 2025
Merged

Reduce temporary string creation during types load#5986
NinoFloris merged 6 commits intonpgsql:mainfrom
bbowyersmyth:users/bruceb/typeStrings

Conversation

@bbowyersmyth
Copy link
Contributor

Reduce string creation by using the span overloads of string.Concat.

NpgsqlDatabaseInfo.cs - InternalName returns a new string every time. Store it in a variable and only do the Dictionary key lookup once in the common case.

ToDefaultMultirangeName does not look to be correct in both existing cases where the final string will be too long. In the first case (range -> multirange) the original name + "multi" is checked against NAMEDATALEN then the final name is trimmed to 58 characters.
It looks like it should be

            if (str.Length > NAMEDATALEN)
                str = str.Slice(0, NAMEDATALEN);

In the second case the original name + "multi" is checked against NAMEDATALEN but "_multirange" is being appended later.
It looks like it should be

            if (str.Length + "_multirange".Length > NAMEDATALEN)

I can change that as part of this PR if they are incorrect.

@NinoFloris
Copy link
Member

It looks like it should be

Quickly glancing at the implementation I believe the idea is that because we know 'range' exists we should only add "multi" to know the actual length after replacement. I agree the other side - where 'range' is not present - has a bug, it should not sum the count with "multi" but "_multirange" as you say. That seems like a careless copy paste on my part. Could you add that fix to the PR?

@bbowyersmyth
Copy link
Contributor Author

str = str.Slice(0, NAMEDATALEN - "multi".Length);
is
str = str.Slice(0, 58);
and that is the final name. PG just trims it to 63 after doing the replace.

@NinoFloris
Copy link
Member

Right, that slice bound should be adjusted, as you concluded correctly 'multirange' is already present in the final string.

@NinoFloris
Copy link
Member

NinoFloris commented Feb 6, 2025

Something like this?

// Static transform as defined by https://www.postgresql.org/docs/current/sql-createtype.html#SQL-CREATETYPE-RANGE
// Manual testing on PG confirmed it's only the first occurence of 'range' that gets replaced.
public DataTypeName ToDefaultMultirangeName()
{
    var nameSpan = UnqualifiedNameSpan;
    if (nameSpan.IndexOf("multirange".AsSpan(), StringComparison.Ordinal) != -1)
        return this;

    var rangeIndex = nameSpan.IndexOf("range", StringComparison.Ordinal);
    if (rangeIndex != -1)
    {
        nameSpan = string.Concat(nameSpan.Slice(0, rangeIndex), "multirange", nameSpan.Slice(rangeIndex + "range".Length));
        if (nameSpan.Length > NAMEDATALEN)
            nameSpan = nameSpan.Slice(0, NAMEDATALEN);

        return new(string.Concat(Schema, ".", nameSpan));
    }

    if (nameSpan.Length > NAMEDATALEN - "_multirange".Length)
        nameSpan = nameSpan.Slice(0, NAMEDATALEN - "_multirange".Length);

    return new(string.Concat(Schema, ".", nameSpan, "_multirange"));
}

@bbowyersmyth
Copy link
Contributor Author

The "range" -> "multirange" replace result will still need to be limited to NAMEDATALEN characters

@NinoFloris
Copy link
Member

Edited, it's ok to merge it into this PR.

@bbowyersmyth
Copy link
Contributor Author

We will only be able to use the AsSpan if the length is equal or greater than NAMEDATALEN. How does the last commit look?

@NinoFloris
Copy link
Member

Ahaha well that serves me, trying to do this off the cuff from github.

Copy link
Member

@NinoFloris NinoFloris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind adding a few tests to DataTypeNameTests?

Also while you're in this area, it's not something we have to address right away but I think we might be able to use the new GetAlternateLookup mechanism to remove the string allocations in NpgsqlDatabaseInfo.TryGetPostgresTypeByName

@bbowyersmyth
Copy link
Contributor Author

@NinoFloris Added a few tests. That turned up another bug in FromDisplayName where !displayNameSpan.Slice(schemaEndIndex).StartsWith("_".AsSpan(), StringComparison.Ordinal) was never false. Changed the code to match its intention but I can't say if its correct or not to ignore qualified arrays.

@NinoFloris
Copy link
Member

Right, the comment is essentially referring to the following strange case in PG:

SELECT 'pg_catalog._int4'::regtype, 'pg_catalog.int4[]'::regtype;

both resolve successfully and return the display name integer[]

@bbowyersmyth bbowyersmyth force-pushed the users/bruceb/typeStrings branch from 1093a75 to 4cf98aa Compare February 15, 2025 06:03
@NinoFloris
Copy link
Member

I will take another look, apologies for the delay, I even had a pending review from earlier. Don't hesitate to ping me. I sometimes just need to be reminded again.

@NinoFloris NinoFloris force-pushed the users/bruceb/typeStrings branch from 4cf98aa to b624b98 Compare October 29, 2025 18:54
Copy link
Member

@NinoFloris NinoFloris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the work you put into this, especially the tests are much appreciated.

I've pushed a few other changes to avoid allocating intermediate strings, and addressed any feedback I had.

LGTM!

@NinoFloris NinoFloris force-pushed the users/bruceb/typeStrings branch from b624b98 to e58a2ee Compare October 29, 2025 19:50
@NinoFloris NinoFloris merged commit 4f7cf71 into npgsql:main Oct 29, 2025
16 checks passed
@bbowyersmyth bbowyersmyth deleted the users/bruceb/typeStrings branch October 30, 2025 02:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants