Skip to content

[FLINK-40489][table] Add UUID_V4() and UUID_V7() functions - #29154

Open
manner wants to merge 3 commits into
apache:masterfrom
manner:FLINK-40489
Open

[FLINK-40489][table] Add UUID_V4() and UUID_V7() functions#29154
manner wants to merge 3 commits into
apache:masterfrom
manner:FLINK-40489

Conversation

@manner

@manner manner commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

This pull request adds two new built-in SQL scalar functions, UUID_V4() and UUID_V7(), generating RFC 9562 UUID values — a random version 4 UUID and a time-ordered version 7 UUID.

Brief change log

  • Added BuiltInFunctionDefinitions.UUID_V4 / UUID_V7
  • Added UuidGenerationUtils with generateV4()/generateV7(), producing the 16-byte internal representation of a UUID value directly (no round-trip through java.util.UUID)
  • Added UuidV4Function / UuidV7Function runtime classes
  • Added uuidV4() / uuidV7() to the Java and Scala Table API expression DSL
  • Documented both functions in sql_functions.yml (English and Chinese)

Verifying this change

This change added tests and can be verified as follows:

  • Added UuidGenerationUtilsTest, directly asserting on the generated bytes: correct length, version nibble, variant bits, and (for v7) that the embedded timestamp matches System.currentTimeMillis() at call time
  • Added UuidFunctionsITCase, covering the SQL and Table API return type (UUID) and the version nibble in the canonical string form, executed end-to-end against a MiniCluster

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yesExpressions.java (@PublicEvolving) gains two new additive methods, uuidV4() and uuidV7()
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): yes — only exercised when a query explicitly calls UUID_V4()/UUID_V7()
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? docs (sql_functions.yml) and JavaDocs

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Sonnet 5)

@flinkbot

flinkbot commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Comment on lines +89 to +95
/**
* Holder for the {@link SecureRandom} instance, so that seeding it is deferred until a {@code
* UUID} is actually generated for the first time.
*/
private static final class Holder {
static final SecureRandom SECURE_RANDOM = new SecureRandom();
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why a class and not just a field?

Suggested change
/**
* Holder for the {@link SecureRandom} instance, so that seeding it is deferred until a {@code
* UUID} is actually generated for the first time.
*/
private static final class Holder {
static final SecureRandom SECURE_RANDOM = new SecureRandom();
}
/**
* Holder for the {@link SecureRandom} instance, so that seeding it is deferred until a {@code
* UUID} is actually generated for the first time.
*/
private static final SecureRandom SECURE_RANDOM = new SecureRandom();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to keep the SecureRandom lazily instantiated, so it's only created when actually used (similar to java.util.UUID). But since both/all methods in the class use the RNG, the class is never loaded without needing it, so the holder doesn't help at all.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that generateV4 uses UUIDs RNG, it might make sense to add this back.

@github-actions github-actions Bot added the community-reviewed PR has been reviewed by the community. label Sep 10, 2026

@dylanhz dylanhz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @manner. I left some doc comments.
And please also update pyflink.table.tests.test_expression.PyFlinkBatchExpressionTests.test_expression.

Comment thread docs/data/sql_functions_zh.yml Outdated
Comment thread docs/data/sql_functions_zh.yml Outdated
table: uuid()
description: Returns an UUID (Universally Unique Identifier) string (e.g., "3d3c68f7-f608-473f-b60c-b0c44ad4cc4e") according to RFC 4122 type 4 (pseudo randomly generated) UUID. The UUID is generated using a cryptographically strong pseudo random number generator.
description: Returns an UUID (Universally Unique Identifier) string (e.g., "3d3c68f7-f608-473f-b60c-b0c44ad4cc4e") according to RFC 9562 version 4 (pseudo randomly generated) UUID. The UUID is generated using a cryptographically strong pseudo random number generator.
- sql: UUID_V4()

@davidradl davidradl Sep 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if UUID() already generates v4, so we need a second function doing the same thing? Are they alias's - the implementation does not use the same code for each.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing UUID function returns a string, while the added functions return the newly added UUID type (FLINK-40486, #29041).

Comment thread docs/data/sql_functions.yml
* Runtime helpers for generating {@code UUID} values, stored as their 16-byte big-endian encoding.
*/
@Internal
public final class UuidGenerationUtils {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you reconsider the purpose of this utility class now that v4 delegates to the JDK? Calling v4 also unnecessarily initializes the SecureRandom used only by v7.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Let's move the methods to the corresponding functions and drop the UuidGenerationUtils class.

@twalthr twalthr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @manner. I left some comments.

description: Returns an UUID (Universally Unique Identifier) string (e.g., "3d3c68f7-f608-473f-b60c-b0c44ad4cc4e") according to RFC 9562 version 4 (pseudo randomly generated) UUID. The UUID is generated using a cryptographically strong pseudo random number generator.
- sql: UUID_V4()
table: uuidV4()
description: Returns a random RFC 9562 version 4 UUID value. The UUID is generated using a cryptographically strong pseudo random number generator.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add "Compared to UUID, this function returns a value of UUID data type."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add this to all JavaDocs etc.

table: uuid()
description: |
根据 RFC 4122 类型 4(伪随机生成)UUID,返回 UUID(通用唯一标识符)字符串。
根据 RFC 9562 类型 4(伪随机生成)UUID,返回 UUID(通用唯一标识符)字符串。

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only add english text here. English speakers cannot review/approve otherwise.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was proposed by @dylanhz in #29154 (comment).
In the existing description I only replaced the outdated RFC number.

* Runtime helpers for generating {@code UUID} values, stored as their 16-byte big-endian encoding.
*/
@Internal
public final class UuidGenerationUtils {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Let's move the methods to the corresponding functions and drop the UuidGenerationUtils class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants