Skip to content

Throw more informative exception when write_triggering_frequency_seconds is missing #917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 2 commits into from
Aug 1, 2020
Merged

Throw more informative exception when write_triggering_frequency_seconds is missing #917

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8c576e1
add checkArgument
pyalex Aug 1, 2020
2d218ac
add default write_triggering_frequency_seconds
pyalex Aug 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions infra/charts/feast/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ feast-batch-serving:
staging_location: gs://<bucket_name>/feast-staging-location
initial_retry_delay_seconds: 3
total_timeout_seconds: 21600
write_triggering_frequency_seconds: 600
subscriptions:
- name: "*"
project: "*"
Expand Down Expand Up @@ -280,6 +281,7 @@ feast-batch-serving:
staging_location: gs://<bucket_name>/feast-staging-location
initial_retry_delay_seconds: 3
total_timeout_seconds: 21600
write_triggering_frequency_seconds: 600
subscriptions:
- name: "*"
project: "*"
Expand Down
1 change: 1 addition & 0 deletions infra/charts/feast/README.md.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ feast-batch-serving:
staging_location: gs://<bucket_name>/feast-staging-location
initial_retry_delay_seconds: 3
total_timeout_seconds: 21600
write_triggering_frequency_seconds: 600
subscriptions:
- name: "*"
project: "*"
Expand Down
1 change: 1 addition & 0 deletions infra/charts/feast/values-batch-serving.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ feast-batch-serving:
staging_location: gs://<bucket_name>/feast-staging-location
initial_retry_delay_seconds: 3
total_timeout_seconds: 21600
write_triggering_frequency_seconds: 600
subscriptions:
- name: "*"
project: "*"
Expand Down
1 change: 1 addition & 0 deletions infra/docker-compose/serving/batch-serving.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ feast:
staging_location: gs://gcs_bucket/prefix
initial_retry_delay_seconds: 1
total_timeout_seconds: 21600
write_triggering_frequency_seconds: 600
subscriptions:
- name: "*"
project: "*"
Expand Down
5 changes: 3 additions & 2 deletions protos/feast/core/Store.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ message Store {
int32 initial_backoff_ms = 3;
// Optional. Maximum total number of retries for connecting to Redis. Default to zero retries.
int32 max_retries = 4;
// Optional. how often flush data to redis
// Optional. How often flush data to redis
int32 flush_frequency_seconds = 5;
}

Expand All @@ -118,6 +118,7 @@ message Store {
string staging_location = 3;
int32 initial_retry_delay_seconds = 4;
int32 total_timeout_seconds = 5;
// Required. Frequency of running BQ load job and flushing all collected rows to BQ table
int32 write_triggering_frequency_seconds = 6;
}

Expand All @@ -131,7 +132,7 @@ message Store {
string connection_string = 1;
int32 initial_backoff_ms = 2;
int32 max_retries = 3;
// Optional. how often flush data to redis
// Optional. How often flush data to redis
int32 flush_frequency_seconds = 4;
}

Expand Down
8 changes: 8 additions & 0 deletions .../bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryFeatureSink.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
*/
package feast.storage.connectors.bigquery.writer;

import static com.google.common.base.Preconditions.checkArgument;

import com.google.api.services.bigquery.model.TableSchema;
import com.google.auto.value.AutoValue;
import com.google.cloud.bigquery.*;
Expand Down Expand Up @@ -62,6 +64,12 @@ public abstract class BigQueryFeatureSink implements FeatureSink {
* @return {@link BigQueryFeatureSink.Builder}
*/
public static FeatureSink fromConfig(BigQueryConfig config) {
checkArgument(
config.getWriteTriggeringFrequencySeconds() > 0,
"Invalid configuration: "
+ "write_triggering_frequency_seconds in BigQueryConfig must be positive integer. "
+ "Please fix that in your serving configuration.");

return BigQueryFeatureSink.builder()
.setDatasetId(config.getDatasetId())
.setProjectId(config.getProjectId())
Expand Down