Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs.feldera.com/docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ import TabItem from '@theme/TabItem';
## 0.226.0

The Delta Lake connector's `skip_unused_columns` property has been deprecated. Use
table-level [`skip_unused_colums`](https://docs.feldera.com/sql/grammar#skip-unused-columns)
table-level [`skip_unused_colums`](https://docs.feldera.com/sql/grammar#ignoring-unused-columns)
instead.

## 0.201.0
Expand Down
2 changes: 1 addition & 1 deletion docs.feldera.com/docs/connectors/sources/delta.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ exactly once fault tolerance.
| `cdc_delete_filer` | string | | <p>A predicate that determines whether the record represents a deletion.</p><p>This setting is only valid in the `cdc` mode. It specifies a predicate applied to each row in the Delta table to determine whether the row represents a deletion event. Its value must be a valid Boolean SQL expression that can be used in a query of the form `SELECT * from <table> WHERE <cdc_delete_filter>`.</p>|
| `cdc_order_by` | string | | <p>An expression that determines the ordering of updates in the Delta table.</p><p>This setting is only valid in the `cdc` mode. It specifies a predicate applied to each row in the Delta table to determine the order in which updates in the table should be applied. Its value must be a valid SQL expression that can be used in a query of the form `SELECT * from <table> ORDER BY <cdc_order_by>`.</p>|
| `num_parsers` | string | | The number of parallel parsing tasks the connector uses to process data read from the table. Increasing this value can enhance performance by allowing more concurrent processing. Recommended range: 1–10. The default is 4.|
| `skip_unused_columns` (<b>DEPRECATED</b>) | bool | false | <p>This property is deprecated. Use the [table-level `skip_unused_columns` property](/sql/grammar#skip-unused-columns) instead.</p><p>Don't read unused columns from the Delta table. When set to `true`, this option instructs the connector to avoid reading columns from the Delta table that are not used in any view definitions. To be skipped, the columns must be either nullable or have default values. This can improve ingestion performance, especially for wide tables.</p><p>Note: The simplest way to exclude unused columns is to omit them from the Feldera SQL table declaration. The connector never reads columns that aren't declared in the SQL schema. Additionally, the SQL compiler emits warnings for declared but unused columns—use these as a guide to optimize your schema.</p>|
| `skip_unused_columns` (<b>DEPRECATED</b>) | bool | false | <p>This property is deprecated. Use the [table-level `skip_unused_columns` property](/sql/grammar#ignoring-unused-columns) instead.</p><p>Don't read unused columns from the Delta table. When set to `true`, this option instructs the connector to avoid reading columns from the Delta table that are not used in any view definitions. To be skipped, the columns must be either nullable or have default values. This can improve ingestion performance, especially for wide tables.</p><p>Note: The simplest way to exclude unused columns is to omit them from the Feldera SQL table declaration. The connector never reads columns that aren't declared in the SQL schema. Additionally, the SQL compiler emits warnings for declared but unused columns—use these as a guide to optimize your schema.</p>|
| `max_concurrent_readers` | integer| 6 | <p>Maximum number of concurrent object store reads performed by all Delta Lake connectors.</p><p>This setting is used to limit the number of concurrent reads of the object store in a pipeline with a large number of Delta Lake connectors. When multiple connectors are simultaneously reading from the object store, this can lead to transport timeouts.</p><p>When enabled, this setting limits the number of concurrent reads across all connectors. This is a global setting that affects all Delta Lake connectors, and not just the connector where it is specified. It should therefore be used at most once in a pipeline. If multiple connectors specify this setting, they must all use the same value.</p><p>The default value is 6.</p>|

[*]: Required fields
Expand Down
2 changes: 1 addition & 1 deletion docs.feldera.com/docs/sql/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ for details.
The `append_only` annotation on a table instructs Feldera that the table will
only receive `INSERT` updates.

See the [Time Series Analysis Guide](/tutorials/time-series#append-only-tables)
See the [Time Series Analysis Guide](/tutorials/time-series#append_only-tables)
for details.

## `emit_final` views
Expand Down
4 changes: 2 additions & 2 deletions docs.feldera.com/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@
"openapi-to-postmanv2": "^5.4.0",
"lodash": "^4.17.23",
"serialize-javascript": "^7.0.3",
"gray-matter/js-yaml": "3.13.1",
"json-refs/js-yaml": "3.13.1",
"gray-matter/js-yaml": "3.14.2",
"json-refs/js-yaml": "3.14.2",
"*/js-yaml": "4.1.1"
}
}
Loading
Loading