[EPIC] Deployment of the Search Update Pipeline on Flink / k8s
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Gehel
	Jun 27 2023, 1:25 PM

Related Objects
Search...

Status	Assigned	Task
Resolved	Gehel	T317045 [Epic] Re-architect the Search Update Pipeline
Resolved	Gehel	T340548 [EPIC] Deployment of the Search Update Pipeline on Flink / k8s
Resolved	Ottomata	T328561 [Event Platform] Flink Operations
Resolved	gmodena	T328563 [Flink Operations] How to handle restarting a Flink application
Open	None	T328565 [Flink Operations] Automate Replay of Failed Events
Resolved	gmodena	T328569 [Flink Operation] How to handle app upgrades
Resolved	bking	T344614 Add Zookeeper config to 'rdf-streaming-updater' test service on DSE cluster
Resolved	bking	T345957 Restore dse-k8s' rdf-streaming-updater from savepoint/improve bootstrapping process
Resolved	bking	T346048 Troubleshoot rdf-streaming-updater/dse-k8s cluster
Resolved	Gehel	T341625 Requesting permission to use kafka-main cluster to transport CirrusSearch updates
Resolved	MatthewVernon	T342620 Storage request: swift s3 bucket for flink search-update-pipeline checkpointing
Resolved	bking	T347075 Deploy test instance of cirrus updater in k8s
Resolved	bking	T348350 Set requests (not limits) for cirrus-streaming-updater in k8s
Resolved	EBernhardson	T350186 Cirrus-streaming-updater test: validate relforge indices are correctly updated
Resolved	EBernhardson	T350299 EventBus change events involving redirect changes are sometimes incorrect
Resolved	dcausse	T326914 Migrate the WDQS streaming updater from FlinkKafkaConsumer/Producer to KafkaSource/Sink
Resolved	bking	T349772 Create dashboards/alerts for new Cirrus Streaming Updater
Resolved	bking	T359213 Adapt Flink-related rdf-streaming-updater alerts for Cirrus Streaming Updater
Invalid	None	T349848 Determine and control cirrus streaming updater's usage of MWAPI resources
Invalid	None	T350185 Estimate cirrus streaming updater's usage of MWAPI
Resolved	dcausse	T350826 Test backfilling for cirrus-streaming-updater
Resolved	EBernhardson	T358518 Deploy streaming updater for 100% of writes to cloudelastic

Event Timeline

Gehel created this task.Jun 27 2023, 1:25 PM

Gehel added subtasks: T328561: [Event Platform] Flink Operations, T326409: Migrate the wdqs streaming updater flink jobs to flink-k8s-operator deployment model.

bking moved this task from needs triage to Current work on the Discovery-Search board.Jul 10 2023, 3:14 PM

bking edited projects, added Discovery-Search (Current work); removed Discovery-Search.

TJones renamed this task from Deployment of the Search Update Pipeline on Flink / k8s to [EPIC] Deployment of the Search Update Pipeline on Flink / k8s.Jul 10 2023, 3:41 PM

TJones added a project: Epic.

TJones moved this task from Incoming to Epics on the Discovery-Search (Current work) board.

Gehel triaged this task as High priority.Jul 19 2023, 8:40 AM

Gehel moved this task from Incoming to Epics on the Data-Platform-SRE board.Jul 19 2023, 3:38 PM

Gehel added a subtask: T341625: Requesting permission to use kafka-main cluster to transport CirrusSearch updates.Jul 25 2023, 1:38 PM

Gehel added a subtask: T342149: Test common operations in the flink operator/k8s/Flink ZK environment.

Gehel added a subtask: T342620: Storage request: swift s3 bucket for flink search-update-pipeline checkpointing.Aug 7 2023, 6:44 PM

MatthewVernon closed subtask T342620: Storage request: swift s3 bucket for flink search-update-pipeline checkpointing as Resolved.Aug 18 2023, 9:23 AM

bking added a subtask: T344614: Add Zookeeper config to 'rdf-streaming-updater' test service on DSE cluster.Aug 21 2023, 2:33 PM

Gehel closed subtask T341625: Requesting permission to use kafka-main cluster to transport CirrusSearch updates as Resolved.Sep 15 2023, 9:26 AM

Gehel removed a subtask: T342149: Test common operations in the flink operator/k8s/Flink ZK environment.Sep 20 2023, 1:41 PM

Gehel closed subtask T344614: Add Zookeeper config to 'rdf-streaming-updater' test service on DSE cluster as Resolved.