Skip to content

Ignore relations with more than 32k members#716

Merged
pnorman merged 1 commit into
osm2pgsql-dev:masterfrom
lonvia:restrict-number-of-relation-members
Mar 13, 2017
Merged

Ignore relations with more than 32k members#716
pnorman merged 1 commit into
osm2pgsql-dev:masterfrom
lonvia:restrict-number-of-relation-members

Conversation

@lonvia

@lonvia lonvia commented Mar 13, 2017

Copy link
Copy Markdown
Collaborator

There is a hard restriction in the pgsql middle tables, where relation members are indexed using a smallint. This change just restricts to the technical limit to fix the issue with the diff file.

Fixes #713.

There is a hard restriction in the pgsql middle tables, where
relation members are indexed using a smallint. To remain
consistent, drop all relations which exceed this size.

Fixes osm2pgsql-dev#713.
@pnorman pnorman merged commit 1888048 into osm2pgsql-dev:master Mar 13, 2017
@mboeringa

Copy link
Copy Markdown

@lonvia ,

Just a minor remark: It may be difficult or impossible to implement at this particular stage of processing or code, but wouldn't it be better to reject the entire changeset containing the offending relation wholesale, instead of just dropping the relation?

If I understand the code right, this will just drop the relation, but potentially leave an inconsistently imported changeset in the database.

@pnorman

pnorman commented Mar 14, 2017

Copy link
Copy Markdown
Collaborator

Just a minor remark: It may be difficult or impossible to implement at this particular stage of processing or code, but wouldn't it be better to reject the entire changeset containing the offending relation wholesale, instead of just dropping the relation?

osm2pgsql has nothing to do with changesets and really knows nothing about them. It doesn't import changesets, which are intended to be useful to humans. Unless -x is used it doesn't store this information in the DB.

Even if it were possible, I'd still do it this way.

@pnorman

pnorman commented Mar 14, 2017

Copy link
Copy Markdown
Collaborator

It's also worth noting that this PR doesn't introduce a restriction, the restriction was always there.

@lonvia lonvia deleted the restrict-number-of-relation-members branch August 25, 2017 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pgsql-mid fails on relations with more than 32767 members

3 participants