Add Ingestion Job management API for Feast Core#548

Merged

feast-ci-bot merged 66 commits into

feast-dev:masterfrom

mrzzy:job-management-api

Apr 7, 2020

Commits on Apr 6, 2020

Added protobuf definitions for a Job Management API
Zhu Zhanyan
committed
Added query methods to JobRepository to query jobs by store and featureset
Zhu Zhanyan
committed
Added hashCode() & equals() to Job model compare and hash jobs
Zhu Zhanyan
committed
Added toIngestionProto() to Job object to convert Job model to ingestion job proto
Zhu Zhanyan
committed
Added query methods to FeatureSetRepository to query by exact (name, project) or (name,version)
Zhu Zhanyan
committed
Added listJobs() to JobService to handle request list ingestion jobs requests
Zhu Zhanyan
committed
Added missing filter field in ListIngestionJobsRequest protobuf
Zhu Zhanyan
committed
Added code to setup mockups for testing JobService
Zhu Zhanyan
committed
Revert "Added hashCode() & equals() to Job model compare and hash jobs"
Zhu Zhanyan
committed
Added JobServiceTest unit tests for JobService's listJobs()
Zhu Zhanyan
committed
Added stopJobs() to JobService to handle requests to stop jobs
Zhu Zhanyan
committed
Added getTransitionalStates() to JobStatus to return collection of transitional states
Zhu Zhanyan
committed
Changed stopJobs() to throw unsupported error on transitional job statuses
Zhu Zhanyan
committed
Moved conversion of JobStatus to IngestionJobStatus proto to JobStatus.
Zhu Zhanyan
committed
Make findFeatureSet() match only one featureset.
Zhu Zhanyan
committed
Added matchFeatureSets() to SpecService to match Feature Sets from References
Zhu Zhanyan
committed
Refactor JobService listJobs() to use SpecService to provide featureset matching
Zhu Zhanyan
committed
Revert findBy methods added to FeatureSetRepository as no longer used.
Zhu Zhanyan
committed
Added restartJob() to JobManagers to restart ingestion/import jobs
Zhu Zhanyan
committed
Added restartJob() to JobService to restart ingestion jobs
Zhu Zhanyan
committed
Update job model (due to new extId) when restartJob() in JobService
Zhu Zhanyan
committed
Use assertThat() & equalTo() instead of assertEquals() in JobService
Zhu Zhanyan
committed
Revert "Use assertThat() & equalTo() instead of assertEquals() in JobService" due to failed tests
Zhu Zhanyan
committed
Use hamcrest’s assertThat() and equalTo() instead of Junit's assertEquals
Zhu Zhanyan
committed
Hook up JobService list, stop, restart job methods to CoreServiceImpl.
Zhu Zhanyan
committed
Throw InvalidArgumentException instead when calling listJobs() with invalid FeatureSetReference
Zhu Zhanyan
committed
Fixed typos in javadocs: InvalidArgumentException should be IllegalArgumentException
Zhu Zhanyan
committed
Fixed in findByFeatureSetsIn() query in JobRepository
Zhu Zhanyan
committed
Renamed toIngestionProto() methods to toProto() to follow code convention
Zhu Zhanyan
committed
Make JobService's listJobs() a transaction to prevent DB data race conditions
Zhu Zhanyan
committed
Moved matchFeatureSets() in SpecService to private method in JobService
Zhu Zhanyan
committed
Make the map that maps between JobStatus and IngestionJobStatus static
Zhu Zhanyan
committed
Make JobService's listJobs() to return all ingestion jobs on empty filter
Zhu Zhanyan
committed
Fixed issue where the jobManager map that JobService built used wrong keys
Zhu Zhanyan
committed
Fix issue where actual Job Status is not synced with database.
Zhu Zhanyan
committed
Log stopJob() & restartJob() operations to make debugging easier
Zhu Zhanyan
committed
Use Runner.name() instead of runner.toString() to build JobManager map
Zhu Zhanyan
committed
Move documentation on JobService operations to CoreService protobuf definition
Zhu Zhanyan
committed
Added IngestJob to python sdk as native representation of IngestionJob proto
Zhu Zhanyan
committed
Make empty filter on JobService's listJobs() select all ingestion jobs
Zhu Zhanyan
committed
Added bindings for Job management API to python sdk client.
Zhu Zhanyan
committed
Fixed __connect_core() to connect to Feast CoreService when calling on Job API calls
Zhu Zhanyan
committed
Auto reload IngestJob.status and IngestJob.external_id on get property.
Zhu Zhanyan
committed
Added IngestJob.wait() to wait for job status to transtion
Zhu Zhanyan
committed
Added basic job api e2e test to exercise job api
Zhu Zhanyan
committed
Reorder the operations e2etest to make sure that jobs are running after test
Zhu Zhanyan
committed
Added e2e test for all types exercising job api
Zhu Zhanyan
committed
Fixed typo in function arguments
Zhu Zhanyan
committed
Added unit tests for Ingestion Job API additions in python sdk
Zhu Zhanyan
committed
Rename "ingestion" to "ingest" for more consistent naming
Zhu Zhanyan
committed
Disable support for restarting Job in a terminal state due to possible race conditions
Zhu Zhanyan
committed
Added __str__ and __repr__ to IngestJob to render ingestjob in human readable string
Zhu Zhanyan
committed
Added FeatureSetRef to represent references to featursets
Zhu Zhanyan
committed
Admend client's list_ingest_jobs() to accept feature references directly
Zhu Zhanyan
committed
Fixed typo in IngestJob.store property
Zhu Zhanyan
committed
Fixed issue with FeatureSetRef.from_str not converting version to int
Zhu Zhanyan
committed
Make the grpc error message more apparent on stop_ingest_job() and restart_ingest_job()
Zhu Zhanyan
committed
Added feast ingest-job list, describe, stop, restart to CLI
Zhu Zhanyan
committed
Rename Job to RetrievalJob to prevent confusion with IngestJob
Zhu Zhanyan
committed
Updated e2e tests to use FeatureSetRef in list_ingest_jobs()
Zhu Zhanyan
committed
Fixed due e2e tests to cater to new limitations on stop_ingest_job()
Zhu Zhanyan
committed
Increase timeout on test_all_types_ingest_jobs() e2e test.
Zhu Zhanyan
committed

Commits on Apr 7, 2020

Configure IngestJob.wait() to backoff with a exponentially larger wait duration
Zhu Zhanyan
committed
Added print statements to debug e2e test failure.
Zhu Zhanyan
committed
Revert "Added print statements to debug e2e test failure."
Zhu Zhanyan
committed
Fixed issue of test waiting for aborted job to become running causing timeout.
Zhu Zhanyan
committed