Skip to content

Conversation

@vsai12
Copy link
Contributor

@vsai12 vsai12 commented Oct 29, 2025

Summary

Implements core audit log sequencer (PR # 2) with guaranteed gap-free sequence numbers (1, 2, 3...) for reliable log aggregation and integrity monitoring.

Key Features

  • Gap-free sequencing: Single-writer pattern ensures no gaps under concurrent load (verified with 10k events from 100 concurrent clients)
  • Backpressure handling: Non-blocking queue submission with error propagation when overloaded
  • Service layer: Clean separation of domain logic, data layer, and orchestration
  • UTF-8 safe validation: Payload size limits with character boundary preservation
  • Graceful shutdown: Context-based timeout coordination matching server patterns

Implementation Details

  • Config management with production defaults and comprehensive validation
  • Store method EnsureUniqueBytebaseID with collision retry (up to 5 attempts)
  • Database migration: UNIQUE constraint on (bytebase_id, sequence_number)
  • Error handling: common.Error with proper codes (Invalid, Internal, SizeExceeded)
  • Test coverage: 12 tests including race detector and concurrent load verification

Files Changed

  • backend/common/audit/: config.go, logger.go, service.go, validation.go, instance.go (refactored)
  • backend/store/audit_log.go: Added EnsureUniqueBytebaseID method
  • backend/migrator/migration/3.9/0011##audit_log_unique_bytebase_id.sql: UNIQUE index
  • Tests: 575 LOC comprehensive test suite

Dependencies

Requires PR #17897 (proto fields + Bytebase ID generation)

@cla-bot cla-bot bot added the cla-signed label Oct 29, 2025
@github-actions
Copy link

github-actions bot commented Oct 29, 2025

The latest Buf updates on your PR. Results from workflow Proto linter / lint-protos (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed⏩ skippedOct 30, 2025, 2:24 AM

@d-bytebase d-bytebase requested a review from h3n4l October 29, 2025 10:19
@vsai12 vsai12 marked this pull request as draft October 30, 2025 02:28
Comment on lines +30 to +32
stdoutQueue chan *storepb.AuditLog // Sequencer → Stdout writer (PR #3)
dbQueue chan *storepb.AuditLog // Sequencer → DB batcher (PR #3)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(PR #3)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants