Skip to content

Add job management APIs #302

@zhilingc

Description

@zhilingc

We should add a job management to the core API for

  • tracking status of jobs
  • cleaning up jobs

For retrieval, we could possibly retrieve jobs based on the existing GetFeatureSets and GetStores filters:

rpc GetJobs(GetJobsRequest) returns (GetJobsResponse);

// Retrieves matching jobs given feature set and/or store filters. If none are provided,
// all active jobs will be returned.
message GetJobsRequest {
    // Filter by feature set name and version
    GetFeatureSetsRequest.Filter feature_set = 1;

    // Filter by store name
    GetStoresRequest.Filter store = 2;
}

message GetJobsResponse {

    enum JobStatus {
        // Job state unknown
        UNKNOWN = 0;

        // New feature set or feature set version created
        PENDING = 1;

        // Error occurred while trying to apply changes
        RUNNING = 2;

        COMPLETED = 3;

        /** When user sent abort command, but it's still running */
        ABORTING = 4;

        /** User initiated abort job */
        ABORTED = 5;

        /**
        * Runner’s reported that the import job failed to run or there is a failure during job
        * submission.
        */
        ERROR = 6;

        /** job has been suspended and waiting for cleanup */
        SUSPENDING = 7;

        /** job has been suspended */
        SUSPENDED = 8;
    }

    message Job {
        string name = 1;
        JobStatus status = 2;
        repeated feast.core.FeatureSet feature_sets = 3;
        feast.core.Store store = 4;
        feast.core.Source source = 5;
    }
    repeated Job jobs = 1;
}

Jobs should be aborted by name.

rpc AbortJob(AbortJobRequest) returns (AbortJobResponse);

message AbortJobRequest {
    string job_name = 1;
}

message AbortJobResponse {
    enum Status {
        INVALID = 0;
        SUCCESS = 1;
        ERROR = 2;
    }

    // Feature set response has been enriched with version and source information
    Status status = 1;
}

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions