commit,author,date,change summary,change details 44e41c12ab25e36c202f58e068ced262eadc8d16,Lakshmi Narayanan Sreethar,Tue Sep 5 21:03:21 2023 +0530,Fix segfault in set_integer_now_func,"When an invalid function oid is passed to set_integer_now_func, it finds out that the function oid is invalid but before throwing the error, it calls ReleaseSysCache on an invalid tuple causing a segfault. Fixed that by removing the invalid call to ReleaseSysCache. Fixes #6037 " e66a40038e3c84fb1a68da67ad71caf75c64a027,Bharathy,Sat Sep 2 09:24:31 2023 +0530,Fix server crash on UPDATE of compressed chunk,UPDATE query with system attributes in WHERE clause causes server to crash. This patch fixes this issue by checking for system attributes and handle cases only for segmentby attributes in fill_predicate_context(). Fixes #6024 c6a930897e9f9e9878db031cc7fb6ea79d721a74,Jan Nidzwetzki,Tue Aug 29 21:13:51 2023 +0200,Use Debian Bookworm for 32-bit tests,"So far, we have used Debian Buster (10) for our 32-bit tests. This distribution is EOL in ~1 year and contains an old LLVM version (7.0). LLVM 7 contains a few bugs that break the JIT functionality of PostgreSQL (missing mixed-sign 64-bit operands on 32-bit architectures / failed to resolve name __mulodi4). This patch changes the used Distribution for 32-bit tests to Debian Bookworm (12 / LLVM 14). Since the PostgreSQL download server no longer offers 32-bit Debian packages, PostgreSQL is built from source. " 8e941b80ae1b0e0b6affe5431454cdc637628d99,Lakshmi Narayanan Sreethar,Mon Aug 28 23:19:22 2023 +0530,Fix incorrect row count in EXPLAIN ANALYZE INSERT .. ON CONFLICT output,INSERT ... ON CONFLICT statements record few metrics in the ModifyTable node's instrument but they get overwritten by hypertable_modify_explain causing wrong output in EXPLAIN ANALYZE statments. Fix it by saving the metrics into HypertableModify node before replacing them. Fixes #6014 caada43454e25d3098744fa6b675ac7d07390550,Lakshmi Narayanan Sreethar,Tue May 30 20:32:29 2023 +0530,PG16: Fix concurrent update issues with MERGE.,PG16 commit postgres/postgres@9321c79c fixes an issue with concurrent update issues in MERGE. This patch adapts that fix into the MERGE support for hypertables as well. postgres/postgres@9321c79c e4facda540286b0affba47ccc63959fefe2a7b26,Sven Klemm,Tue Aug 29 18:13:24 2023 +0200,Add compatibility layer for _timescaledb_internal functions,With timescaledb 2.12 all the functions present in _timescaledb_internal were moved into the _timescaledb_functions schema to improve schema security. This patch adds a compatibility layer so external callers of these internal functions will not break and allow for more flexibility when migrating. 77dc6ed42c6d0b65dd971566c2184e9bd7008e68,Jan Nidzwetzki,Thu Aug 31 09:56:38 2023 +0200,Fix non-deterministic cagg_insert isolation test,"One permutation of the cagg_insert isolation test contains an assumption about the execution order of three processes after a lock is released. However, this behavior is non-deterministic. This PR makes the assumption explicit and adds proper markers to the isolation test to make the output deterministic. " 08231c8aacd17152f315ad36d95c031fb46073aa,Jan Nidzwetzki,Tue Aug 29 14:47:57 2023 +0200,Export is_decompress_chunk_path / is_gapfill_path,This patch adds the 'ts_' prefix to the function names of is_decompress_chunk_path and is_gapfill_path and makes them available for use by other parts of TimescaleDB. fa04a067e0dce3a09b229cf2e6c127984cb9896f,Jan Nidzwetzki,Wed Aug 30 23:49:34 2023 +0200,Fix an invalid SPI result use after free,This PR fixes the invalid use of an SPI result after SPI_finish is called and the result is freed. e3437786ad7e3b5f152167ce0081e546178b8a12,Sven Klemm,Wed Aug 30 09:15:34 2023 +0200,Make multinode tests conditional,Since multinode tests take a long time to run we dont want to run them in CI on individual PRs but only on nightly runs. 3b6dc7dc013572f91db181ccbbe4854f596714dd,Sven Klemm,Tue Aug 29 12:52:06 2023 +0200,Move partialize functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - finalize_agg_ffunc(internal,text,name,name,name[],bytea,anyelement) - finalize_agg_sfunc(internal,text,name,name,name[],bytea,anyelement) - partialize_agg(anyelement) - finalize_agg(text,name,name,name[][],bytea,anyelement) " 623381ce99978b7f05f32ec1f5c117345ef6cd8e,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue Aug 29 12:39:17 2023 +0200,Reread the catalog data after locking the chunk,"The compression status can change, and this is prevented by locking, so to keep our data consistent, we should reread the chunk metadata after we have locked it. Currently we have some code in place that masks this inconsistency by rereading the metadata in another place. This code is removed. " a9751ccd5eb030026d7b975d22753f5964972389,Sven Klemm,Tue Aug 29 10:49:47 2023 +0200,Move partitioning functions to _timescaledb_functions schema,To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - get_partition_for_key(val anyelement) - get_partition_hash(val anyelement) b2a91494a11d8b82849b6f11f9ea6dc26ef8a8cb,Sven Klemm,Mon Aug 28 23:26:23 2023 +0200,Move ddl_internal functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - chunk_constraint_add_table_constraint(_timescaledb_catalog.chunk_constraint) - chunk_drop_replica(regclass,name) - chunk_index_clone(oid) - chunk_index_replace(oid,oid) - create_chunk_replica_table(regclass,name) - drop_stale_chunks(name,integer[]) - health() - hypertable_constraint_add_table_fk_constraint(name,name,name,integer) - process_ddl_event() - wait_subscription_sync(name,name,integer,numeric) " 4516df285c962f801722019868fc0a982ed43a57,Jan Nidzwetzki,Tue Aug 29 08:02:48 2023 +0200,Make up/downgrade test deterministic,"Two queries in post.continuous_aggs.v3.sql had no ORDER BY specification. Therefore, the query output was not deterministic. This patch adds the missing ORDER BY. " 6576d969b319dac8e7fd08a9cf4cfc8197b34d1d,Sven Klemm,Mon Aug 28 15:32:54 2023 +0200,Move log invalidation functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - cagg_watermark(integer) - cagg_watermark_materialized(integer) - hypertable_invalidation_log_delete(integer) - invalidation_cagg_log_add_entry(integer,bigint,bigint) - invalidation_hyper_log_add_entry(integer,bigint,bigint) - invalidation_process_cagg_log(integer,integer,regtype,bigint,bigint,integer[],bigint[],bigint[]) - invalidation_process_cagg_log(integer,integer,regtype,bigint,bigint,integer[],bigint[],bigint[],text[]) - invalidation_process_hypertable_log(integer,integer,regtype,integer[],bigint[],bigint[]) - invalidation_process_hypertable_log(integer,integer,regtype,integer[],bigint[],bigint[],text[]) - materialization_invalidation_log_delete(integer) " 28c7457faf9b909ea89b26b61cfa5a8428e2c23c,Sven Klemm,Mon Aug 28 08:38:26 2023 +0200,Move scheduler functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - restart_background_workers() - stop_background_workers() - start_background_workers() - alter_job_set_hypertable_id(integer,regclass) " 01e480d5d668f7fbfc81800d4aad6c5ee61ba227,James Guthrie,Thu Aug 10 13:23:01 2023 +0200,Account for uncompressed rows in 'create_compressed_chunk',"`_timescaledb_internal.create_compressed_chunk` can be used to create a compressed chunk with existing compressed data. It did not account for the fact that the chunk can contain uncompressed data, in which case the chunk status must be set to partial. Fixes #5946 " a323547e691f7ca9ad5d0b3924071a51b0a365cd,Fabrízio de Royes Mello,Sun Aug 27 13:20:04 2023 +0200,Move cagg_migrate functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - cagg_migrate_create_plan(_timescaledb_catalog.continuous_agg,text,boolean,boolean) - cagg_migrate_execute_copy_data(_timescaledb_catalog.continuous_agg,_timescaledb_catalog.continuous_agg_migrate_plan_step) - cagg_migrate_execute_copy_policies(_timescaledb_catalog.continuous_agg,_timescaledb_catalog.continuous_agg_migrate_plan_step) - cagg_migrate_execute_create_new_cagg(_timescaledb_catalog.continuous_agg,_timescaledb_catalog.continuous_agg_migrate_plan_step) - cagg_migrate_execute_disable_policies(_timescaledb_catalog.continuous_agg,_timescaledb_catalog.continuous_agg_migrate_plan_step) - cagg_migrate_execute_drop_old_cagg(_timescaledb_catalog.continuous_agg,_timescaledb_catalog.continuous_agg_migrate_plan_step) - cagg_migrate_execute_enable_policies(_timescaledb_catalog.continuous_agg,_timescaledb_catalog.continuous_agg_migrate_plan_step) - cagg_migrate_execute_override_cagg(_timescaledb_catalog.continuous_agg,_timescaledb_catalog.continuous_agg_migrate_plan_step) - cagg_migrate_execute_plan(_timescaledb_catalog.continuous_agg) - cagg_migrate_execute_refresh_new_cagg(_timescaledb_catalog.continuous_agg,_timescaledb_catalog.continuous_agg_migrate_plan_step) - cagg_migrate_plan_exists(integer) - cagg_migrate_pre_validation(text,text,text) Co-authored-by: Fabrízio de Royes Mello " e02b1f348eb4c48def00b7d5227238b4d9d41a4a,Sven Klemm,Sun Aug 27 13:20:04 2023 +0200,Simplify schema move update script,Use dynamic sql to create the ALTER FUNCTION statements for those functions that may not exist in previous versions. 184e8398182fb3972137b4faaa64d3a08836dfd9,Sven Klemm,Wed Aug 23 20:59:47 2023 +0200,Move policy functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - policy_compression_check(jsonb) - policy_compression_execute(integer,integer,anyelement,integer,boolean,boolean) - policy_compression(integer,jsonb) - policy_job_error_retention_check(jsonb) - policy_job_error_retention(integer,jsonb) - policy_recompression(integer,jsonb) - policy_refresh_continuous_aggregate_check(jsonb) - policy_refresh_continuous_aggregate(integer,jsonb) - policy_reorder_check(jsonb) - policy_reorder(integer,jsonb) - policy_retention_check(jsonb) - policy_retention(integer,jsonb) " 6fb3c3f3f43fe70b7c5034ddfb11451df802890c,Lakshmi Narayanan Sreethar,Wed Jul 5 23:49:07 2023 +0530,PG16: Handle updates to make_restrictinfo function,"While fixing a bug in filtering of ""cloned"" outer-join quals, PG16 adds 3 new parameters to the make_restrictinfo function. Updated the compat function to handle this change. This patch also cleans up the variants of make_restrictinfo from other versions to make it clear which args are passed. postgres/postgres@991a3df227 " 9425402264624f78fd052c700f53f7a736230007,Lakshmi Narayanan Sreethar,Thu May 18 17:05:21 2023 +0530,PG16: Handle removed EquivalenceClass members,PG16 removes the outerjoin_delayed mechanism and RestrictInfo.nullable_relids. postgres/postgres@b448f1c8 postgres/postgres@3bef56e1 3a493a47d1b5cc1a148c0040d531bf132f567e80,Lakshmi Narayanan Sreethar,Thu Aug 24 20:11:44 2023 +0530,Remove unused functions,Removed the unused functions ts_make_pathkey_from_sortop and ts_make_pathkey_from_sortinfo. 040d45510430d13a9e1eda0ecb02981dfa1be39c,Lakshmi Narayanan Sreethar,Sat May 13 12:20:42 2023 +0530,PG16: Datum macros are now inline functions,PG16 converted the *GetDatum() and DatumGet*() macros to inline functions. This doesn't affect most of our code but a few places like struct object initialisations cannot use inline functions. This commit updates all the places affected by this change. postgres/postgres@c8b2ef05 0da18a93b5bd00c6b25ff3c485b2228d371b3326,Sven Klemm,Tue Aug 22 14:15:13 2023 +0200,Move chunk functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - calculate_chunk_interval(int, bigint, bigint) - chunk_status(regclass) - chunks_in(record, integer[]) - chunk_id_from_relid(oid) - show_chunk(regclass) - create_chunk(regclass, jsonb, name, name, regclass) - set_chunk_default_data_node(regclass, name) - get_chunk_relstats(regclass) - get_chunk_colstats(regclass) - create_chunk_table(regclass, jsonb, name, name) - freeze_chunk(regclass) - unfreeze_chunk(regclass) - drop_chunk(regclass) - attach_osm_table_chunk(regclass, regclass) " 5bba74a2ec083728f8e93e09d03d102568fd72b5,Fabrízio de Royes Mello,Mon Aug 7 19:49:47 2023 -0300,Relax strong table lock when refreshing a CAGG,When refreshing a Continuous Aggregate we take a table lock on _timescaledb_catalog.continuous_aggs_invalidation_threshold when processing the invalidation logs (the first transaction of the refresh Continuous Aggregate procedure). It means that even two different Continuous Aggregates over two different hypertables will wait each other in the first phase of the refreshing procedure. Also it lead to problems when a pg_dump is running because it take an AccessShareLock on tables so Continuous Aggregate refresh execution will wait until the pg_dump finish. Improved it by relaxing the strong table-level lock to a row-level lock so now the Continuous Aggregate refresh procedure can be executed in multiple sessions with less locks. Fix #3554 0f3d39574bf696e6627471ee16b5c26a12fb436c,Sven Klemm,Wed Aug 23 13:06:49 2023 +0200,Remove _timescaledb_internal.get_time_type,This function was used in an old version of a cagg informational view that was removed but the function itself was left in. 3373d43143ab8b17c49fdffd34fd35d551d33185,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Aug 16 12:12:42 2023 +0200,Don't look up entire Chunk struct for compressed chunks,It's not needed. Also add them to baserel cache. 373c55662ca5f8a2993abf9b2aa7f5f4006b3229,Konstantina Skovola,Mon Aug 7 16:36:17 2023 +0300,Fix ordered append for partially compressed chunks,"In the exclusive presence of partially compressed chunks, this optimization was not applied because no pathkeys were supplied. Additionally, this patch makes sure that if applicable, the `enable_decompression_sorted_merge` optimization is chosen for the path, since it is more beneficial due to the ability to push down the sort below DecompressChunk. " d1246ee2f30c171491c3f5b9efe0626314c37c13,Erik Nordström,Wed Aug 23 10:18:48 2023 +0200,Exclude PRs from bugs project board,"An `issue_comment` event is triggered by comments on both issues and PRs. However, the bugs project workflow did not filter out pull requests when the `issue_comment` event happened, so pull requests were also added to the bugs project board. Add a conditional that filters out pull requests so that only issues are added to the board. " e47832b51ac7b18e7441de3a1eeffe579057d8e2,Sven Klemm,Wed Aug 23 09:07:14 2023 +0200,Bump pgspot version to 0.6.0,pgspot 0.6.0 has a bugfix for function signature tracking to no longer consider default values as part of the function signature. cf04496e4b4237440274eb25e4e02472fc4e06fc,Sven Klemm,Tue Aug 22 12:01:19 2023 +0200,Move utility functions to _timescaledb_functions schema,To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - generate_uuid() - get_git_commit() - get_os_info() - tsl_loaded() 183362e17baf71ae4f663709a9cb412a986aa4d8,Sven Klemm,Tue Aug 22 09:37:40 2023 +0200,Move size_util functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - relation_size(regclass) - data_node_hypertable_info(name, name, name) - data_node_chunk_info(name, name, name) - hypertable_local_size(name, name) - hypertable_remote_size(name, name) - chunks_local_size(name, name) - chunks_remote_size(name, name) - range_value_to_pretty(bigint, regtype) - get_approx_row_count(regclass) - data_node_compressed_chunk_stats(name, name, name) - compressed_chunk_local_stats(name, name) - compressed_chunk_remote_stats(name, name) - indexes_local_size(name, name) - data_node_index_size(name, name, name) - indexes_remote_size(name, name, name) " 3db692296028ea3c7407150a76aaa1feed6c5b4e,Mats Kindahl,Mon Aug 21 17:35:19 2023 +0200,Call eq_func correctly in time_bucket_gapfill,"The equality comparison function is called using `DirectFunctionCall2Coll`, which do not set the `fcinfo->flinfo` when calling the PostgreSQL function. Since `array_eq` uses `fcinfo->flinfo->fn_extra` for caching, and `flinfo` is null, this causes a crash. Fix this issue by using `FunctionCall2Coll` instead, which sets `fcinfo->flinfo` before calling the PostgreSQL function. Fixes #5981 " 4256009e4cfe3daebe199540ab8c32af34836e01,Sven Klemm,Mon Aug 21 18:55:35 2023 +0200,Move dist_internal functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - set_dist_id(uuid) - set_peer_dist_id(uuid) - validate_as_data_node() - show_connection_cache() - ping_data_node(name, interval) - remote_txn_heal_data_node(oid) " e99832727ac3f8aebd146015158490b28754ac1a,Jan Nidzwetzki,Mon Aug 21 14:49:35 2023 +0200,Place data in first/last function in correct mctx,"So far, the ts_bookend_deserializefunc() function has allocated the deserialized data in the current memory context. This data could be removed before the aggregation is finished. This patch moves the data into the aggregation memory context. " 09dd20d7f73938473881b0c0d1fe2e1bb4659161,Lakshmi Narayanan Sreethar,Fri Jun 2 19:36:26 2023 +0530,PG16: Align GUC variables initial value with boot values,"Any value hardcoded to a GUC variable will be overwritten by the boot_value when the GUC mechanism starts up. PG16 makes this more clear by checking that if a hardcoded value exists, it is same as the boot_value. The server asserts in debug builds if values are not equal. Commit 09636092 already fixes most of our code to align the hardcoded and boot_values for various GUC variables. This patch updates only the ts_guc_max_cached_chunks_per_hypertable initialisation which had an initial value different from the boot_value. postgres/postgres@a73952b79 " ba06c6eb2aac3be5ec1145b899cb747c82cca02a,Lakshmi Narayanan Sreethar,Fri Jun 2 19:02:35 2023 +0530,PG16: Include guc header to use GetConfigOptionByName,postgres/postgres@0a20ff54f5 cf0f9b5bd8def79c346bf5462abe733bfeb4e192,Lakshmi Narayanan Sreethar,Thu Jun 1 23:29:18 2023 +0530,PG16: Replace float8in_internal_opt_error with float8in_internal,PG16 updated float8in_internal_opt_error() function to use soft error reporting and renamed it to float8in_internal(). Updated the code to use the new mechanism. postgres/postgres@ccff2d20e 0a66bdb8d36a1879246bd652e4c28500c4b951ab,Sven Klemm,Sun Aug 20 22:47:10 2023 +0200,Move functions to _timescaledb_functions schema,"To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the following functions: - to_unix_microseconds(timestamptz) - to_timestamp(bigint) - to_timestamp_without_timezone(bigint) - to_date(bigint) - to_interval(bigint) - interval_to_usec(interval) - time_to_internal(anyelement) - subtract_integer_from_now(regclass, bigint) " a640d7ddf16f289c9472c2699952df1122642cd1,Sven Klemm,Mon Aug 21 10:10:53 2023 +0200,Fix psql \if expression,"The expression part of the psql \if cannot contain actual SQL expression and is instead much more limited. A valid value is any unambiguous case-insensitive match for one of: true, false, 1, 0, on, off, yes, no. See https://www.postgresql.org/docs/current/app-psql.html " 0dd06e919f7cee9d7f7672b5244e7122d0c66436,Sven Klemm,Thu Aug 17 09:59:25 2023 +0200,Move get_create_command into _timescaledb_functions schema,To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for the get_create_command function. 56ea8b4de93cefc38e002202d8ac96947dcbaa77,Sven Klemm,Thu Apr 13 13:16:14 2023 +0200,Move trigger functions to _timescaledb_functions schema,To increase schema security we do not want to mix our own internal objects with user objects. Since chunks are created in the _timescaledb_internal schema our internal functions should live in a different dedicated schema. This patch make the necessary adjustments for our trigger functions. 2080c3c0f0d1fd8b90e9fb57491eaf936131f03f,Konstantina Skovola,Thu Aug 17 14:15:21 2023 +0300,Post-release fixes for 2.11.2,Bumping the previous version and adding tests for 2.11.2 beb3b39599940082a7b0f4bf0d1d76e4500e5ebb,Lakshmi Narayanan Sreethar,Thu Jun 1 23:01:46 2023 +0530,PG16: vacuum_set_xid_limits is now vacuum_get_cutoffs,PG16 refactors how VACUUM passes around its XID cutoffs. A new dedicated struct is now used by VACUUM to maintain the XID/MXID cutoffs such as FreezeLimit and OldestXmin. The vacuum_set_xid_limits() function is also now replaced with a a new vacuum_get_cutoffs() function that uses the new struct. postgres/postgres@4ce3afb8 f92cab8c2a81e5e0d7643e6fbaeed1a371b7bf07,Lakshmi Narayanan Sreethar,Thu Jun 1 23:17:57 2023 +0530,PG16: Rename append_pathkeys to append_pathkeys_custom,Renamed append_pathkeys() to ts_append_pathkeys to prevent conflict with upstream changes postgres/postgres@1349d279 3438636a05936f4d20f05b42ac6e6e20d8ca736f,Lakshmi Narayanan Sreethar,Wed Jul 5 22:46:44 2023 +0530,PG16: Macro HeapKeyTest is now an inline function,postgres/postgres@4eb3b112 8d2dc760b7738d772f4d43e7ed8b32f9c7ed11e8,Lakshmi Narayanan Sreethar,Thu Jun 1 23:13:56 2023 +0530,"PG16: When removing a relation from the query, drop its RelOptInfo.","PG16 removes the notion of ""dead relation"" and instead recommends deleting the relation's RelOptInfo from the planner's data structures when it is no longer needed. postgres/postgres@e9a20e45 " 82eeb6ec2a19f30d917ca8e51303e217d201b883,Lakshmi Narayanan Sreethar,Thu May 18 21:35:56 2023 +0530,"PG16: Node tags T_Join, T_Plan and T_Scan have been removed","Node tags T_Join, T_Plan and T_Scan have been removed in PG16 as those nodes are of abstract type and never directly instantiated. postgres/postgres@251154be postgres/postgres@8c73c11a " b7b9a9b3ef0d1541388617db9e9b9237423eb4f1,Lakshmi Narayanan Sreethar,Tue May 30 22:12:39 2023 +0530,PG16: Rename ri_RootToPartitionMap to ri_RootToChildMap,postgres/postgres@fb958b5d d088b3a5d906f18f554b3322ec6d5cd00987cb4c,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue Aug 15 23:18:13 2023 +0200,Do not add broken compressed join clauses,"We used to add join clauses that referenced a compressed column at the level of the compressed scan, and later remove them. This is wrong and useless, just don't add them. " 22a2f49a2f09ef6b5809bf75bd93d016434e48f6,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Fri Aug 11 13:22:30 2023 +0200,Fix filtering of the redundant decompress chunk clauses,"We just had a bad cast inside is_redundant_derived_clause before, because it doesn't work with IndexClauses. Filtering didn't work as a result. " 3a27669c846dd641b5054281729a8a71c576257d,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue Aug 15 14:28:58 2023 +0200,Simplify compressed pathkey lookup,"When looking up a pathkey for compressed scan, we used to do a lot of work, including a quadratic lookup through all the equivalence members, to always arrive at the same canonical pathkey we started from. Just remove this useless code for a significant planning speedup. This uncovers two bugs in parameterization of decompressed paths and generation of equivalence members for segmentby columns, fix them as well. " 04ce1bc498f8a2f7732a50bf673de175db97fd04,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Aug 10 15:32:40 2023 +0200,Use cached Chunk struct when considering compressed paths,"Full catalog lookups for a Chunk are expensive, avoiding them speeds up the planning. " fb617e415058473457e26ab8e42b6637f0f4cf74,Sven Klemm,Tue Aug 15 09:08:49 2023 +0200,Bump Postgres versions used in CI,"Bump postgres version used to 13.12, 14.9 and 15.4 " 154bbbb01a14046c639c1b978dd1b4e004a1f3cc,Jan Nidzwetzki,Tue Jul 4 15:46:53 2023 +0200,Perform startup chunk exclusion in parallel leader,"The parallel version of the ChunkAppend node uses shared memory to coordinate the plan selection for the parallel workers. If the workers perform the startup exclusion individually, it may choose different subplans for each worker (e.g., due to a ""constant"" function that claims to be constant but returns different results). In that case, we have a disagreement about the plans between the workers. This would lead to hard-to-debug problems and out-of-bounds reads when pstate->next_plan is used for subplan selection. With this patch, startup exclusion is only performed in the parallel leader. The leader stores this information in shared memory. The parallel workers read the information from shared memory and don't perform startup exclusion. " 1102d34f42daf24a6fdbbbcae2c6e46feb50ca8c,Mats Kindahl,Thu Aug 10 13:53:57 2023 +0200,Remove telemetry isolation test,"The telemetry isolation test `telemetry_iso` does not test anything and does not seem to work, so it is removed. The debug waitpoint was taken in the same session, so the waitpoint was not waited on. " 71b0168ab72b15fee539ecef3886a17d09e7d042,Mats Kindahl,Wed Mar 22 14:45:21 2023 +0100,Add debug utilities to debug builds,"This will move the definitions of `debug_waitpoint_enable`, `debug_waitpoint_disable`, and `debug_waitpoint_id` to always be defined for debug builds and modify existing tests accordingly. This means that it is no longer necessary to generate isolation test files from templates (in most cases), and it will be straightforward to use these functions in debug builds. The debug utilities can be disabled by setting the option `ENABLE_DEBUG_UTILS` to `OFF`. " 4bd704f3fcc1093641a8e772fe53bb0d633020ec,Lakshmi Narayanan Sreethar,Fri Jul 28 07:52:10 2023 +0530,Further code cleanup after PG12 removal,Removed PG12 specific code guarded by the PG13_LT and PG13_GE macros. 2cb42a62f91b0aa81e9411f44936a63f6ad9c8aa,Konstantina Skovola,Thu Aug 10 15:50:13 2023 +0300,Remove test_status calls from telemetry test,"Due to the postman-echo endpoint redirecting http requests to https, we get an unexpected 301 response in the tests, leading to repeated test failures. This commit removes these function calls. " 9a2dfbfb83efdd94340c87d90a7893c7e53bd2da,Jan Nidzwetzki,Wed Jul 12 22:29:25 2023 +0200,Improved parallel DecompressChunk worker selection,"This PR improves the way the number of parallel workers for the DecompressChunk node are calculated. Since 1a93c2d482b50a43c105427ad99e6ecb58fcac7f, no partial paths for small relations are generated, which could cause a fallback to a sequential plan and a performance regression. This patch ensures that for all relations, a partial path is created again. " 44eab9cf9bef34274c88efd37a750eaa74cd8044,Konstantina Skovola,Wed Aug 9 15:26:03 2023 +0300,Release 2.11.2,This release contains bug fixes since the 2.11.1 release. We recommend that you upgrade at the next available opportunity. **Features** * #5923 Feature flags for TimescaleDB features **Bugfixes** * #5680 Fix DISTINCT query with JOIN on multiple segmentby columns * #5774 Fixed two bugs in decompression sorted merge code * #5786 Ensure pg_config --cppflags are passed * #5906 Fix quoting owners in sql scripts. * #5912 Fix crash in 1-step integer policy creation **Thanks** * @mrksngl for submitting a PR to fix extension upgrade scripts * @ericdevries for reporting an issue with DISTINCT queries using segmentby columns of compressed hypertable b96d1709219a9ba6b502970437d42e5dcd70643c,Lakshmi Narayanan Sreethar,Wed Aug 9 10:54:29 2023 +0100,"Revert ""PG16: Use new function to check vacuum permission""",This reverts commit 8b0ab4164 as the commit that introduced the new function has been reverted upstream. postgres/postgres@95744599 b5b46a3e581b222f679c2d4aa15944646d8190d9,noctarius aka Christoph Engelbert,Wed Aug 9 13:28:54 2023 +0200,Make logrepl markers for (partial) decompressions (#5805),"Added logical replication messages (PG14+) as markers for (partial) decompression events (mutual compression), which makes it possible to differentiate inserts happening as part of the decompression vs actual inserts by the user, and filter the former out of the event stream. While some tools may be interested in all events, synching the pure ""state"" (without internal behavior) is required for others. As of now this PR is missing tests. I wonder if anyone has a good idea how to create an automatic test for it." a9505b40951ff6eaac330df61767dbf4ca01e84c,Lakshmi Narayanan Sreethar,Thu Jul 20 16:24:43 2023 +0300,PG16: Replace pg_class_ownercheck() with object_ownercheck,PG16 replaces pg_foo_ownercheck() functions with a common object_ownercheck() function. Added a new compat function for pg_class_ownercheck() function affected by this change and replaced all its callers. postgres/postgres@afbfc029 22ea5771ad1a7928439b74f6cf3cd53872163c02,Lakshmi Narayanan Sreethar,Thu Jul 20 16:24:32 2023 +0300,PG16: Make aclcheck function calls compatible with PG16,PG16 replaced most of the aclcheck functions with a common object_aclcheck function. Updated the various aclcheck calls in the code to use the new function when compiled with PG16. postgres/postgres@c727f511 b2b3acf6ac1f580c0c6737386f6360b912eb6c8f,Lakshmi Narayanan Sreethar,Mon May 22 21:22:27 2023 +0530,PG16: No need to pass create_new_ph flag to find_placeholder_info,"PG16 also optimized the PlaceFolderInfo lookups to perform in constant time, so there is no need to do an additional cheap/quick test using bms_overlap to see if the PHV might be evaluated in the outer rels. postgres/postgres@6569ca439 postgres/postgres@b3ff6c742 " 8abe14807229e691f5b75d37a6e465a58c3de886,Lakshmi Narayanan Sreethar,Wed Aug 2 20:36:38 2023 +0100,PG16: stringToQualifiedNameList requires escontext parameter,postgres/postgres@858e776c84f 592da23633e49dd3ad97922d64cdcfc22d219d90,Nikhil Sontakke,Fri Aug 4 14:08:03 2023 +0530,Fix assert in debug wait points,Need to ensure that we should try to take a lock only if a valid transaction is around. Otherwise assert is hit due to an error within an error. Fixes #5917 2eb0a3883b47529dbf182475f6b81e8180d33114,Lakshmi Narayanan Sreethar,Thu Jul 20 17:20:38 2023 +0300,PG16: Handle DefineIndex's new parameter,"PG16 adds a new parameter to DefineIndex, total_parts, that takes in the total number of direct and indirect partitions of the relation. Updated all the callers to pass either the actual number if it is known or -1 if it is unknown at that point. postgres/postgres@27f5c712 " 3af0d282ea71d9a8f27159a6171e9516e62ec9cb,Lakshmi Narayanan Sreethar,Wed Aug 2 20:24:14 2023 +0100,PG16: ExecInsertIndexTuples requires additional parameter,"PG16 adds a new boolean parameter to the ExecInsertIndexTuples function to denote if the index is a BRIN index, which is then used to determine if the index update can be skipped. The fix also removes the INDEX_ATTR_BITMAP_ALL enum value. Adapt these changes by updating the compat function to accomodate the new parameter added to the ExecInsertIndexTuples function and using an alternative for the removed INDEX_ATTR_BITMAP_ALL enum value. postgres/postgres@19d8e23 " 8a2b6a03e0a7da92e5cdfc0f6802f93c7cc80820,Mats Kindahl,Mon Aug 7 09:28:09 2023 +0200,Add weird user names to update test,Since we want to be able to handle update of weird user names we add some to the update tests and create policies on them. This will create jobs with the strange name as owner. 07762ea4cedefc88497f0d1f8712d1515cdc5b6e,Sven Klemm,Mon Aug 7 18:31:40 2023 +0200,Test timescaledb debian 12 packages in CI, 7aeed663b9c0f337b530fd6cad47704a51a9b2ec,Dmitry Simonenko,Thu Aug 3 14:30:23 2023 +0300,Feature flags for TimescaleDB features,This PR adds several GUCs which allow to enable/disable major timescaledb features: - enable_hypertable_create - enable_hypertable_compression - enable_cagg_create - enable_policy_create 5cf354e2469ee7e43248bed382a4b49fc7ccfecd,Markus Engel,Mon Jul 31 11:28:25 2023 +0200,Fix quoting owners in sql scripts.,"When referring to a role from a string type, it must be properly quoted using pg_catalog.quote_ident before it can be casted to regrole. Fixed this, especially in update scripts. " 2863daf3df83c63ee36c0cf7b66c522da5b4e127,Dmitry Simonenko,Thu Aug 3 14:36:39 2023 +0300,Support CREATE INDEX ONLY ON main table,This PR adds support for CREATE INDEX ONLY ON clause which allows to create index only on the main table excluding chunks. Fix #5908 52ed394d4d2c13e693ff280c658c56f3eeb46f74,Lakshmi Narayanan Sreethar,Thu Jul 20 17:28:59 2023 +0300,PG16: Remove recursion-marker values in enum AlterTableType,PG16 removed the recursion-marker values used to handle certain subcommands during an ALTER TABLE execution and provides an alternative flag. Removed the references to the recursion-marker values from timescaledb code. postgres/postgres@840ff5f4 28612ebc3cbb6e2eeb995b25203b12974ef88127,Konstantina Skovola,Tue Aug 1 14:59:07 2023 +0300,Fix crash in 1-step integer policy creation,"Previously when a retention policy existed on the underlying hypertable, we would get a segmentation fault when trying to add a Cagg refresh policy, due to passing a bool instead of pointer to bool argument to function `ts_jsonb_get_int64_field` in a particular code path. Fixed by passing the expected pointer type. Fixes #5907 " b2773aa344c6f8fe8600830bfac417364676582d,Nikhil Sontakke,Tue Aug 1 18:33:02 2023 +0530,Fix crash in COPY from program returning error,Reset the errcallback appropriately so that the ereport in case of a PROGRAM returning error can work correctly. 0d127f6dcc8827e28528854991cc0c22ee204141,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue Aug 1 10:04:10 2023 +0200,Clean up compressed batch handling in DecompressChunk node,"* Remove unneeded data from batch states to use less memory * keep only the compressed column data because only for them we have to do something per row, other columns don't change * Adjust batch memory context size so that the bulk decompression results fit into it. * Determine whether we're going to use bulk decompression for each column at planning time, not at execution time. * Introduce ""batch queue"" to unify control flow for normal and batch sorted merge decompression * In batch sorted merge, compare batches on scan slot, not on projected slot. This avoids keeping the second slot in batches, and projection can be performed after we find the top batch. * this requires some custom code to build sort infos relative to scan tuple, not to targetlist. * Return a reference for the current top batch scan tuple as a result of DecompressChunk exec, don't copy it out. It is guaranteed to live until the next exec, which is the usual lifetime guarantee. This is needed to prepare for vectorized filters. " d5268c36fbd23fa2a93c0371998286e8688247bb,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Fri Jul 28 13:35:05 2023 +0200,Fix SQLSmith workflow,The build was failing because it was picking up the wrong version of Postgres. Remove it. ee2ddf889e07ac20cff55013e2bf7020f99e65e1,Mats Kindahl,Tue Jul 25 16:32:06 2023 +0200,Check unique indexes when enabling compression,"When enabling compression on a table, there is a check of unique and primary key *constraints*, but no check if there is just a unique index. This means that it is possible to create a compressed table without getting a warning that you should include the columns in the index into the segmentby field, which can lead to suboptimal query times. This commit adds a check for the unique index as well and ensure that a similar warning is printed as for a unique constraint. Fixes #5892 " 61c288ec5eb966a9b4d8ed90cd026ffc5e3543c9,Lakshmi Narayanan Sreethar,Tue Jul 25 16:11:35 2023 +0530,Fix broken CI after PG12 removal,The commit cdea343cc updated the gh_matrix_builder.py script but failed to import PG_LATEST variable into the script thus breaking the CI. Import that variable to fix the CI tests. e5691bee11e4f41e4beef1eae5782912b83b2c94,Lakshmi Narayanan Sreethar,Fri Jul 7 21:50:14 2023 +0530,Cleanup PG12 specific code from source and test files,"Removed the PG12 specific macros and all the now, dead code. Also updated the testcases which had workarounds in place to make them compatible with PG12. " ac33d04aa8f792e9076cdbb1162a5a367e713bf1,Lakshmi Narayanan Sreethar,Fri Jul 7 14:53:01 2023 +0530,Remove update files for PG12,Removed the update files that were used only for PG12. c3a9f90fdd972b41ad63c97a2dba352a27073361,Lakshmi Narayanan Sreethar,Fri Jul 7 14:14:43 2023 +0530,Merge PG12 specific testfiles,Merged testfiles that were split out due to their output differing only in PG12. 7936e8015bdd3fda9071b11939d6fece108a65fb,Lakshmi Narayanan Sreethar,Fri Jul 7 01:02:34 2023 +0530,Remove PG12 specific test output files, 81b520d3b5132cb483128837a0e5dc89b98ff308,Lakshmi Narayanan Sreethar,Thu Jul 6 22:37:44 2023 +0530,Remove support for PG12,Remove support for compiling TimescaleDB code against PG12. PG12 specific macros and testfiles will be removed in a followup patch. cdea343cc9b8c3547791028ae1c228d3fdaaed10,Lakshmi Narayanan Sreethar,Thu Jul 6 21:49:23 2023 +0530,Remove PG12 support from github workflows, 906bd38573a4752f6f3ec94e925683b5444f924c,Mats Kindahl,Thu Jul 13 14:32:55 2023 +0200,Add job exit status and runtime to log,"When a job finishes execution, either because of an error or a success, this commit will print out the execution time of the job in the log together with a message about what job that finished. For continuous aggregate refreshes, the number of rows deleted from and inserted into the materialization table will be printed. " 36e71000132cf3a5430849c11ff99e910ef81207,Jan Nidzwetzki,Wed Jul 12 22:29:25 2023 +0200,Fix duplicates on partially compressed chunk reads,"When the uncompressed part of a partially compressed chunk is read by a non-partial path and the compressed part by a partial path, the append node on top could process the uncompressed part multiple times because the path was declared as a partial path and the append node assumed it could be executed in all workers in parallel without producing duplicates. This PR fixes the declaration of the path. " 1bd527375d0dbe3735cc6efb297d09ff57824fe0,Rafia Sabih,Wed Jul 5 14:33:20 2023 +0200,Rectify interval calculation,"For continuous aggregates with variable bucket size, the interval was wrongly manipulated in the process. Now it is corrected by creating a copy of interval structure for validation purposes and keeping the original structure untouched. Fixes #5734 " 4c3d64aa988ad667ff737f20a807058d6fb754cc,noctarius aka Christoph Engelbert,Wed Jul 12 20:22:14 2023 +0200,Support CAGG names in chunk_detailed_size (#5839),"This patch adds support to pass continuous aggregate names to `chunk_detailed_size` to align it to the behavior of other functions such as `show_chunks`, `drop_chunks`, `hypertable_size`." 963d4eefbff3c4eedb851efd5a51418f14ce1820,noctarius aka Christoph Engelbert,Wed Jul 12 20:21:27 2023 +0200,Make `set_chunk_time_interval` CAGGs aware (#5852),"This patch adds support to pass continuous aggregate names to the `set_chunk_time_interval` function to align it with functions, such as `show_chunks`, `drop_chunks`, and others. It reuses the previously existing function to find a hypertable or resolve a continuous aggregate to its underlying hypertable found in chunk.c. It, however, moves the function to hypertable.c and exports it from here. There is some discussion if this functionality should stay in chunk.c, though, it feels wrong in that file now that it is exported." 88aaf23ae37fe7f47252b87325eb570aa417c607,noctarius aka Christoph Engelbert,Wed Jul 12 14:53:40 2023 +0200,Allow Replica Identity (Alter Table) on CAGGs (#5868),"This commit is a follow up of #5515, which added support for ALTER TABLE ... REPLICA IDENTITY (FULL | INDEX) on hypertables. This commit allows the execution against materialized hypertables to enable update / delete operations on continuous aggregates when logical replication in enabled for them." eaa1206b7f01672b95ea45486bcb7602499ffd25,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Jul 6 18:14:08 2023 +0200,Improvements for bulk decompression,* Restore default batch context size to fix a performance regression on sorted batch merge plans. * Support reverse direction. * Improve gorilla decompression by computing prefix sums of tag bitmaps during decompression. 7657efe019bb020af095cd9ef3d577cb7bddd7d0,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Jul 6 15:56:28 2023 +0200,Cache the libfuzzer corpus between CI runs,This might help us find something interesting. Also add deltadelta/int8 fuzzing and make other minor improvements. 490bc916afac6182a0537dba6dcf8c07b2735ff1,Jan Nidzwetzki,Tue Jul 4 15:50:17 2023 +0200,Warn if result of ts_set_flags_32 is not used,"The ts_set_flags_32 function takes a bitmap and flags and returns an updated bitmap. However, if the returned value is not used, the function call has no effect. An unused result may indicate the improper use of this function. This patch adds the qualifier pg_nodiscard to the function which triggers a warning if the returned value is not used. " 06d20b1829e7a1afe392e50db17f92370ab0a9f8,Konstantina Skovola,Wed Jun 7 15:13:09 2023 +0300,Enable altering job schedule type through `alter_job`,"In #4664 we introduced fixed schedules for jobs. This was done by introducing additional parameters fixed_schedule, initial_start and timezone for our add_job and add_policy APIs. These fields were not updatable by alter_job so it was not possible to switch from one type of schedule to another without dropping and recreating existing jobs and policies. This patch adds the missing parameters to alter_job to enable switching from one type of schedule to another. Fixes #5681 " b9a58dd5c49da409669caa47da3db8edbcbcea44,Jan Nidzwetzki,Fri Jun 30 10:31:03 2023 +0200,Exclude workflow changes from being backported,"The backport script for the PRs does not have the permission to backport PRs which include workflow changes. So, these PRs are excluded from being automatically backported. Failed CI run: https://github.com/timescale/timescaledb/actions/runs/5387338161/ jobs/9780701395 > refusing to allow a GitHub App to create or update workflow > `.github/workflows/xxx.yaml` without `workflows` permission) " 9bbf5218890fc8c4435b8cd1cc06ca310b243da7,Jan Nidzwetzki,Fri Jun 30 09:45:42 2023 +0200,Remove Ubuntu Kinetic check on ARM64,We stopped to build packages for Ubuntu Kinetic on ARM64 due to the limited support of PostgreSQL versions and the EOL of Kinetic in a few weeks. This patch removes the check for up-to-date packages for this version. a7be1cc06a7f0324f5ac841d3898659352aaa96c,Jan Nidzwetzki,Fri Jun 30 08:22:44 2023 +0200,Fixed the ordering of merge_changelogs.sh script,"The CHANGELOG.MD file contains the sections features, bugfixes, and thanks. This patch adjusts the script merge_changelogs.sh to produce the sections in the same order. " 8a581010957968cc569f5fea43fd6ab7dd3b80a6,Jan Nidzwetzki,Thu Jun 29 11:11:15 2023 +0200,Post-release fixes for 2.11.1,Bumping the previous version and adding tests for 2.11.1. 8ae2da6260c7de68808db918371f8aacafce8332,Jan Nidzwetzki,Tue Jun 27 12:36:29 2023 +0200,Release 2.11.1,This release contains bug fixes since the 2.11.0 release. We recommend that you upgrade at the next available opportunity. **Features** * #5679 Teach loader to load OSM extension **Bugfixes** * #5705 Scheduler accidentally getting killed when calling `delete_job` * #5742 Fix Result node handling with ConstraintAwareAppend on compressed chunks * #5750 Ensure tlist is present in decompress chunk plan * #5754 Fixed handling of NULL values in bookend_sfunc * #5798 Fixed batch look ahead in compressed sorted merge * #5804 Mark cagg_watermark function as PARALLEL RESTRICTED * #5807 Copy job config JSONB structure into current MemoryContext * #5824 Improve continuous aggregate query chunk exclusion **Thanks** * @JamieD9 for reporting an issue with a wrong result ordering * @xvaara for reporting an issue with Result node handling in ConstraintAwareAppend 118526e6aed1afa5e559ff43d16835807572e1eb,Sven Klemm,Tue Jun 27 13:11:24 2023 +0200,Improve continuous aggregate query chunk exclusion,This patch changes the time_bucket exclusion in cagg queries to distinguish between < and <=. Previously those were treated the same leading to failure to exclude chunks when the constraints where exactly at the bucket boundary. e2e7e5f286b2282d17440961a1efa043ba054824,Erik Nordström,Fri Mar 31 14:22:35 2023 +0200,Make hypertables support replica identity,"Add support for setting replica identity on hypertables via ALTER TABLE. The replica identity is used in logical replication to identify rows that have changed. Currently, replica identity can only be altered on hypertables via the root; changing it directly on chunks will raise an error. " 33a3e10f486e82df0db369c5ec2b80702576d9ec,Jan Nidzwetzki,Fri Jun 16 22:50:17 2023 +0200,Fixed batch look ahead in compressed sorted merge,"In decompress_sorted_merge_get_next_tuple it is determine how many batches need to be opened currently to perform a sorted merge. This is done by checking if the first tuple from the last opened batch is larger than the last returned tuple. If a filter removes the first tuple, the first into the heap inserted tuple from this batch can no longer be used to perform the check. This patch fixes the wrong batch look ahead. Fixes: #5797 " da20d071cf88699cafb03f732b3e851bdc286759,Sven Klemm,Thu Jun 22 10:19:05 2023 +0200,Copy job config JSONB structure into current MemoryContext,The job config jsonb can be a nested structure of elements that all need to reside in the same memory context as the other job values. To ensure this we copy the structure on assignment. f1726790224e5444a2a2cb8c66155e4fb0b54e95,Jan Nidzwetzki,Sun Jun 25 21:55:24 2023 +0200,Added perltidy make target,"This patch introduces the make target 'perltidy' to format Perl files with perltidy. In addition, calling perltidy is added to 'make format'. " fb0df1ae4e65a815c61533380f2a5ebdfc5fe1ca,Ante Kresic,Tue Jun 20 15:41:30 2023 +0200,Insert into indexes during chunk compression,"If there any indexes on the compressed chunk, insert into them while inserting the heap data rather than reindexing the relation at the end. This reduces the amount of locking on the compressed chunk indexes which created issues when merging chunks and should help with the future updates of compressed data. " 81d4eb5cfb2c0e4949cbd109e7331539a41db152,Zoltan Haindrich,Fri May 19 12:04:25 2023 +0000,Add Ensure-s to reduce crashes in unexpected cases,It could happen that the Chunk is dropped in the middle of processing another command. The test bgw_db_scheduler_fixed can crash due to that reason. By making sure that the system will error out instead of failing in an assertion could help avoid the situation in which postmaster drop out all clients in these cases. 81e2f35d4b8d52ed3381cff846556611a8974cf9,Jan Nidzwetzki,Tue Jun 20 12:58:31 2023 +0200,Mark cagg_watermark as PARALLEL RESTRICTED,"This patch marks the function cagg_watermark as PARALLEL RESTRICTED. It partially reverts the change of c0f2ed18095f21ac737f96fe93e4035dbfeeaf2c. The reason is as follows: for transaction isolation levels < REPEATABLE READ it can not be ensured that parallel worker reads the same watermark (e.g., using read committed isolation level: worker A reads the watermark, the CAGG is refreshed and the watermark changes, worker B reads the newer watermark). The different views on the CAGG can cause unexpected results and crashes (e.g., the chunk exclusion excludes different chunks in worker A and in worker B). In addition, a correct snapshot is used when the watermark is read from the CAGG and a TAP test is added, which detects inconsistent watermark reads. Co-authored-by: Fabrízio de Royes Mello Co-authored-by: Zoltan Haindrich " a22e732c02474ce6b3ee29c4e81049c60daddc90,Konstantina Skovola,Fri Jun 23 15:35:15 2023 +0300,Fix flaky test bgw_db_scheduler_fixed,"The flakiness was due to two inserts falling into the same chunk instead of distinct ones, so inserted data further apart to ensure they fall in different chunks. " d223000036db55bdc8ef5e576a906f334272de37,Zoltan Haindrich,Wed Jun 14 15:16:21 2023 +0000,Chunk_create must add existing table or fail,Earlier this function have completed successfully if the requested range already existed - regardless an existing table was supplied or not. b2132f00a7b2b5a52af8f577d75fe54fab232443,Zoltan Haindrich,Tue Jun 13 12:10:07 2023 +0000,Make validate_chunk_status accept Chunk as argument,This makes the calls to this method more straightforward and could help to do better checks inside the method. 8b0ab416437df5d6adf2ce58126dd2754c8ff48b,Lakshmi Narayanan Sreethar,Thu May 18 16:38:53 2023 +0530,PG16: Use new function to check vacuum permission,postgres/postgres@b5d63824 d96e72af607f797b56fd3e0d8e51f6b229b9cebf,Lakshmi Narayanan Sreethar,Thu May 18 00:07:00 2023 +0530,PG16: Rename RelFileNode references to RelFileNumber or RelFileLocator,postgres/postgres@b0a55e4 933285e64675febc5608add9cc1ad2d700518593,Lakshmi Narayanan Sreethar,Sat May 13 00:36:53 2023 +0530,PG16: Remove MemoryContextContains usage,Remove the usage of MemoryContextContains as it has been removed in PG16. postgres/postgres@9543eff 1eb7e38d2df37da7674b2207155484838671d040,Konstantina Skovola,Thu Jun 1 12:30:34 2023 +0300,Enable ChunkAppend for space partitioned partial chunks,This is a follow-up patch for timescale#5599 which handles space partitioned hypertables. c48f905f780025bf3c92de0d0dd161108a4116a1,Bharathy,Thu Jun 15 19:02:23 2023 +0530,Index scan support for UPDATE/DELETE.,"During UPDATE/DELETE on compressed hypertables, we do a sequential scan which can be improved by supporting index scans. In this patch for a given UPDATE/DELETE query, if there are any WHERE conditions specified using SEGMENT BY columns, we use index scan to fetch all matching rows. Fetched rows will be decompressed and moved to uncompressed chunk and a regular UPDATE/DELETE is performed on the uncompressed chunk. " 77318dced8a47aae4b2baddba71c829105e8319d,Jan Nidzwetzki,Tue Jun 13 08:25:45 2023 +0200,Fix broken download links,"The download links for several platforms are broken. This patch removes the links for the individual platforms and adds a link that points to the self-hosted install docs instead (as proposed by the docs team, see the discussion in #5762). Fixes: #5762 " f05b7f8105a6d7ef05abdfb0b21cb48824135c72,Jan Nidzwetzki,Thu Jun 15 12:44:53 2023 +0200,Fixed the naming of the Windows GitHub action,"The ignored workflow for windows-build-and-test does not set the name of the actions properly. Therefore, these actions use the default naming. For example, 'Regression Windows / build (15, windows-2022, Debug)'. However, our CI expects names like 'PG15 Debug windows-2022' in the required checks. This PR corrects the name of the jobs. " 4273a27461fab3d23d19bc81b18209e83bb34662,Valery Meleshkin,Wed Jun 14 17:55:51 2023 +0200,Ensure pg_config --cppflags are passed,"CMAKE_CPP_FLAGS is not a thing at all. Furthermore, CMAKE_CXX_FLAGS is not passed to a C compiler. pg_config uses CPPGLAGS for all includes, and needs to be passed into CMAKE_C_FLAGS as well. " 14d08576fb7d12b2d5edf4bfa3ea2722d79f7764,Sotiris Stamokostas,Wed Jun 14 17:57:32 2023 +0300,Allow flaky-test label,With this PR we allow issues with flaky-test label to be added to our bugs board. e302aa2ae97bbd682b1d5d1324e57a77131401d0,Sven Klemm,Fri Jun 2 09:26:39 2023 +0200,Fix handling of Result nodes below Sort nodes in ConstraintAwareAppend,With PG 15 Result nodes can appear between Sort nodes and DecompressChunk when postgres tries to adjust the targetlist. 9c7ae3e8a983ff1a19645c3d2dc0508ae8c69550,Jan Nidzwetzki,Mon Jun 12 15:52:06 2023 +0200,Fixed two bugs in decompression sorted merge code,"SQLSmith found two bugs in the compression sorted merge code. * The unused_batch_states are not initialized properly. Therefore, non-existing unused batch states can be part of the BMS. This patch fixes the initialization. * For performance reasons, We reuse the same TupleDesc across all TupleTableSlots. PostgreSQL sometimes uses TupleDesc data structures with active reference counting. The way we use the TupleDesc structures collides with the reference counting of PostgreSQL. This patch introduces a private TupleDesc copy without reference counting. " 4dce87a1c4a53d080676bfff042e24aa5b822cec,Lakshmi Narayanan Sreethar,Wed May 17 19:37:42 2023 +0530,PG16: Refactor handling of PGDLLEXPORT macro definition,"PG16 defines the PGDLLEXPORT macro as a proper visibility attribute. In the previous versions from PG12 to PG16, the PGDLLEXPORT was always defined as an empty macro. Considering all this, the code has been now updated to skip defining PGDLLEXPORT if it has been already defined properly. If not, the macro is redefined without any additional checks. postgres/postgres@089480c " 0f1fde8d31dd44180820409ea2ca334b9188a01e,Lakshmi Narayanan Sreethar,Wed May 17 21:49:00 2023 +0530,Mark PG16 as a supported version,Note that this change in combination with -DEXPERIMENTAL=ON cmake flag will just allow us to compile timescaledb code with PG16 and this doesn't mean PG16 is supported by the extension. 7df16ee560ec79fd0c4b1590189755bad25585f7,Sotiris Stamokostas,Mon Jun 12 11:50:21 2023 +0300,Renamed need-more-info label,We plan to rename the need-more-info label to waiting-for-author. This PR performs the needed adjustments in our GitHub actions. 8b10a6795c4d696b6c59ab546d733decf2a68352,Sotiris Stamokostas,Fri Jun 9 15:26:18 2023 +0300,Compression test changes for PG14.0,We have changed the compression test by disabling parallel append in some test cases because the regression test was falling only in PG14.0 but not in PG14.8 or any other PostgreSQL version f26e656c0f9ad7eb27c5de2232fef0f9154d80d5,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Jun 7 15:55:53 2023 +0200,Bulk decompression of compressed batches,"Add a function to decompress a compressed batch entirely in one go, and use it in some query plans. As a result of decompression, produce ArrowArrays. They will be the base for the subsequent vectorized computation of aggregates. As a side effect, some heavy queries to compressed hypertables speed up by about 15" ". Point queries with LIMIT 1 can regress by up to 1 ms. If the absolute highest performace is desired for such queries, bulk decompression can be disabled by a GUC. ",,,, c96870c91baf9a035f6c1308d2ab953d65ab73c2,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Jun 7 15:54:48 2023 +0200,Update gorilla and deltadelta fuzzing corpuses, 769646bdb61d74f5cb026b598de24072cc6c4d8c,Zoltan Haindrich,Tue Jun 6 15:11:40 2023 +0000,Fix issues with scripts/test_update_smoke.sh,"The test was failing on first run by leaving a database behind as a sideeffect. Between two steps the extension was dropped; without a proper cleanup. A non-existent sql function was called during cleanup. This patch also removes the ""debug mode"" and every execution will leave the logs/etc in the /tmp directory for further inspection. " b8e674c137f54e293e868ce747f19983a72e0e60,Jan Nidzwetzki,Wed Jun 7 09:05:58 2023 +0200,Fixed handling of NULL values in bookend_sfunc,"In the function bookend_sfunc values are compared. If the first processed value is a NULL value, it was copied into the state of the sfunc. A following comparison between the NULL value of the state and a non-NULL value could lead to a crash. This patch improves the handling of NULL values in bookend_sfunc. " f2eac72e2bb320a4866766f79b181fe78856c8a1,Jan Nidzwetzki,Tue Jun 6 11:59:39 2023 +0200,Ensure tlist is present in decompress chunk plan,"In PostgreSQL < 15, CustomScan nodes are projection capable. The planner invokes create_plan_recurse with the flag CP_IGNORE_TLIST. So, the target list of a CustomScan node can be NIL. However, we rely on the target list to derive information for sorting. This patch ensures that the target list is always populated before the sort functions are called. Fixes: #5738 " ac7090653eb020c2392bc29b653339b581d56bcf,Zoltan Haindrich,Thu Jun 1 10:56:34 2023 +0000,Ensure PR number is referenced in the .unreleased files,Adds a simple check to ensure that the PR number is present at least once in the added changelog file. Also fixes an earlier PR which introduced a typo. 4cef387f850df09fd060760ffe7437a2b9edf21e,Fabrízio de Royes Mello,Wed May 31 14:49:38 2023 -0400,Replace heap_endscan to table_endscan, 1a93c2d482b50a43c105427ad99e6ecb58fcac7f,Sotiris Stamokostas,Wed May 3 22:20:44 2023 +0300,Improve parallel workers for decompression,"So far, we have set the number of desired workers for decompression to 1. If a query touches only one chunk, we end up with one worker in a parallel plan. Only if the query touches multiple chunks PostgreSQL spins up multiple workers. These workers could then be used to process the data of one chunk. This patch removes our custom worker calculation and relies on PostgreSQL logic to calculate the desired parallelity. Co-authored-by: Jan Kristof Nidzwetzki " 10cab43e6c3348a1353a682a01bdf8a70173aca8,Jan Nidzwetzki,Thu Apr 6 13:12:13 2023 +0200,Enable compressed merge append for partial chunks,This patch enables the compressed merge optimization (see #5530) also for partially compressed chunks. c507f31069fccd3755a4138b907cbe997d34d15f,Dipesh Pandit<70561403+pdipesh02@users.noreply.github.com>,Thu Jun 1 19:34:47 2023 +0530,Internal Server Error when loading Explorer tab (#5723)," Internal Server Error when loading Explorer tab (SDC #995) This is with reference to a weird scenarios where chunk table entry exist in timescaledb catalog but it does not exist in PG catalog. The stale entry blocks executing hypertable_size function on the hypertable. The changes in this patch are related to improvements suggested for hypertable_size function which involves: 1. Locking the hypertable in ACCESS SHARE mode in function hypertable_size to avoid risk of chunks being dropped by another concurrent process. 2. Joining the hypertable and inherited chunk tables with ""pg_class"" to make sure that a stale table without an entry is pg_catalog is not included as part of hypertable size calculation. 3. An additional filter (schema_name) is required on pg_class to avoid calculating size of multiple hypertables with same in different schema. NOTE: With this change calling hypertable_size function will require select privilege on the table. Disable-check: force-changelog-file" ecc1b7b11a2aa9d483b14e240128632047af1b46,Konstantina Skovola,Fri Apr 21 11:51:44 2023 +0300,Enable ChunkAppend for partial chunks,"This patch enables ChunkAppend optimization for partially compressed chunks on hypertables without space partitioning, allowing for more efficient processing of LIMIT order by queries. A follow-up patch is required to handle space partitioned hypertables. " d6030a32d87b2b2a3caa8026faf007b8e6b532e5,epgts,Wed May 31 13:18:48 2023 -0500,Teach loader to load OSM extension (#5679), b38c920266b33c5b83c85279c9b36f4131b9ea83,Bharathy,Sat May 27 10:00:37 2023 +0530,MERGE support on hypertables,"This patch does following: 1. Planner changes to create ChunkDispatch node when MERGE command has INSERT action. 2. Changes to map partition attributes from a tuple returned from child node of ChunkDispatch against physical targetlist, so that ChunkDispatch node can read the correct value from partition column. 3. Fixed issues with MERGE on compressed hypertable. 4. Added more testcases. 5. MERGE in distributed hypertables is not supported. 6. Since there is no Custom Scan (HypertableModify) node for MERGE with UPDATE/DELETE on compressed hypertables, we don't support this. Fixes #5139 " ab2cccb6e2b10008c2604af96be71ed8709ecb22,Zoltan Haindrich,Fri May 12 06:30:41 2023 +0000,Post-release 2.11.0,Adjust the upgrade/downgrade scripts and add the tests. (cherry picked from commit d5fea0a842cbd38d2d72db16e9e67f1c9b1ccf36) 70d0704368226a1a5b604b49b7c21c4a4c9131aa,Rafia Sabih,Thu May 4 11:01:41 2023 +0200,Refactor continuous aggregate code, 29154b29d11901ee4b98312ca6245e87216a5877,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed May 24 21:57:54 2023 +0200,Ignore the telemetry test for the time begin,It's the fifths run and it still fails in some jobs. 6589f43160980475c04fcc2a6e4e2b985a111357,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed May 24 10:55:35 2023 +0200,Compression fuzzing in CI,"This serves as a way to exercise the decompression fuzzing code, which will be useful when we need to change the decompression functions. Also this way we'll have a check in CI that uses libfuzzer, and it will be easier to apply it to other areas of code in the future. " 54d4c3de6bbb52421b15f6035db662f683059c86,Fabrízio de Royes Mello,Mon May 22 16:35:45 2023 -0300,Introduce utility function ts_get_relation_relid,"In several places of our code base we use a combination of `get_namespace_oid` and `get_relname_relid` to return the Oid of a schema qualified relation, so refactored the code to encapsulate this behavior in a single function. " f2743648dff26f61f305aa8579bcac806e61a2a2,Eric Gillespie,Tue Mar 28 13:01:56 2023 -0500,Update emacs configuration,"Use postgres for sql-mode, which makes it easier to connect to local postgresql with C-c C-z and also sets up postgresql extensions for font-lock. Copy the 4-space tab-width setting to diff-mode so that, for example, diffs of lines containing 18 TABs don't display with 144-space indent. " f14ff40d6305f139aaabc3e22cbc2fb5fca55bf2,Sven Klemm,Tue May 23 10:55:52 2023 +0200,Bump PG version used in CI,"Use PG 12.15, 13.11, 14.8 and 15.3 in CI. " a31c9b9f8cdfe8643499b710dc983e5c5d6457e4,Sven Klemm,Mon May 22 11:34:06 2023 +0200,Increase number of sqlsmith loops in nightly CI,To improve coverage with sqlsmith we run it for longer in the scheduled nightly run. dd9f01ae9b5b7a4cf3f71ea2b78d69f511e58608,Konstantina Skovola,Mon May 22 11:03:01 2023 +0300,Fix occasional scheduler quit with `delete_job`,"Previously it was possible to send pg_cancel to the scheduler instead of the background worker for the job. That was because we attempted to take an advisory lock on the job id, which was assumed to be held by the background worker for the job. However, it is possible either for the job's worker or for the scheduler to be the one holding the lock. The scheduler takes this lock after the job's worker has exited, to update the job's scheduled status. This patch adds a check to determine if the background worker holding the lock is the scheduler. If it's the scheduler, wait for the lock to be released without canceling the worker. Fixes #5711, #5224 " d9849325d0d0f81a13db1e41aa56f8b567945e72,Rafia Sabih,Thu Apr 27 15:01:38 2023 +0200,Improve test suite,Add more regression tests for Continuous aggregates with joins. 38ee7f49b0cddc5f7fe697321135b28680471ff7,Maheedhar PV,Thu May 18 18:00:07 2023 +0530,Every PR to have its own changelog,"The changes in this commit 1. workflow action to check if the PR has its own changelog file in "".unreleased/"" folder. 2. script to merge the individual changelog entries that can be copied into the CHANGELOG.md file. 3. script to check the format of the lines in the change log file. 4. script to delete the individual changelogs, post release. " 2bd18b8e44e6bc6d2ef57aebc2a3e22ba91059b3,Mats Kindahl,Wed May 17 13:07:02 2023 +0200,Use environment variables for workflows,Remove GitHub variables from locations where they are expanded in place. See https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-an-intermediate-environment-variable f58d8c20c2562a97ad8dec7dfc738fff29749138,Zoltan Haindrich,Fri May 12 06:30:41 2023 +0000,Release 2.11.0, 19dd7bbd7a09de25af7c233a5923ac7eaef809de,Konstantina Skovola,Fri May 12 17:47:06 2023 +0300,Fix DISTINCT query with JOIN on multiple segmentby columns,"Previously when adding equivalence class members for the compressed chunk's variables, we would only consider Vars. This led us to ignore cases where the Var was wrapped in a RelabelType, returning inaccurate results. Fixed the issue by accepting Vars with RelabelType for segmentby equivalence class. Fixes #5585 " fb65086b5542a871dc3d9757724e886dca904ef6,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue May 16 18:33:48 2023 +0200,Add a ubsan suppression for overflow in histogram(),"It is in postgres code, and doesn't lead to bugs. " 8ff0648fd0768b9229853f933056855420ad82ee,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue May 16 14:43:18 2023 +0200,Fix ubsan failure in gorilla decompression,Also add more tests 936d751037381cbbb3a59ac9da36cdaa0dd4904a,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Mon May 15 17:52:37 2023 +0200,Add AddressSanitizer instrumentation for memory contexts,"Use manual poison/unpoison at the existing Valgrind hooks, so that AddressSanitizer sees palloc/pfree as well, not only the underlying mallocs which are called much less often. Fix some out-of-bound reads found with this instrumentation. " f58500a32c5abc6bb3459310cb7990cca37c316b,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue May 16 14:47:17 2023 +0200,Add clang-tidy changes to git blame ignore list, 030bfe867df02dffa4f5c0fc8a0909075def6f4a,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Mon May 15 17:28:35 2023 +0200,Fix errors in decompression found by fuzzing,"For deltadelta and gorilla codecs, add various length and consistency checks that prevent segfaults on incorrect data. " a7321199a4184aaa39c9c5b56b412497d62da1dc,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Mon May 15 17:23:27 2023 +0200,Enable branch-level code coverage,Helps to check the test coverage for various complex conditions in the decompression code. 62b6bc5f7fede92d6e1fa039ffa65e344eb325fb,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Mon May 15 17:23:01 2023 +0200,Add deltadelta int8 fuzzing corpus, 451b982a74c9bfa6d2a5509811c2ea89a4d079ca,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Mon May 15 17:22:32 2023 +0200,Add gorilla-float8 fuzzing corpus, 3947c011244050878ebb091b560e19ea4c9d8378,Mats Kindahl,Fri Apr 28 10:38:43 2023 +0200,Support sending telemetry event reports,Add table `_timescaledb_catalog.telemetry_event` table containing events that should be sent out with telemetry reports. The table will be truncated after reporting being generated. 2d71a5bca9a21490819d239aeee8b2758a54ec1d,Bharathy,Thu May 11 17:55:27 2023 +0530,Fix leak during concurrent UPDATE/DELETE,"When updating and deleting the same tuple while both transactions are running at the same time, we end up with reference leak. This is because one of the query in a transaction fails and we take error path. However we fail to close the table. This patch fixes the above mentioned problem by closing the required tables. Fixes #5674 " 656daf45f6596e5d663082a8673e705359c01534,Mats Kindahl,Tue May 9 18:58:12 2023 +0200,Fix subtransaction resource owner,"When executing a subtransaction using `BeginInternalSubTransaction` the memory context switches from the current context to `CurTransactionContext` and when the transaction is aborted or committed using `ReleaseCurrentSubTransaction` or `RollbackAndReleaseCurrentSubTransaction` respectively, it will not restore to the previous memory context or resource owner but rather use `TopTransactionContext`. Because of this, both the memory context and the resource owner will be wrong when executing `calculate_next_start_on_failure`, which causes `run_job` to generate an error when used with the telemetry job. This commit fixes this by saving both the resource owner and the memory context before starting the internal subtransaction and restoring it after finishing the internal subtransaction. Since the `ts_bgw_job_run_and_set_next_start` was incorrectly reading the wrong result from the telemetry job, this commit fixes this as well. Note that `ts_bgw_job_run_and_set_next_start` is only used when running the telemetry job, so it does not cause issues for other jobs. " abb6762450fb90aae5536641fef85cfd3c75b510,Erik Nordström,Wed May 3 14:06:25 2023 +0200,Reduce memory usage for distributed analyze,Use a per-tuple memory context when receiving chunk statistics from data nodes. Otherwise memory usage is proportional to the number of chunks and columns. 96d2acea30e6cc27ab624e5a155f6b57767ac411,Erik Nordström,Wed May 3 14:05:47 2023 +0200,Cleanup PGresults on transaction end,Fix a regression due to a previous change in c571d54c. That change unintentionally removed the cleanup of PGresults at the end of transactions. Add back this functionality in order to reduce memory usage. f250eaa631920c209673fd0babfe29af9b9778dd,Fabrízio de Royes Mello,Fri May 5 12:14:04 2023 -0300,Remove FK from continuous_agg_migrate_plan,During the `cagg_migrate` execution if the user set the `drop_old` parameter to `true` the routine will drop the old Continuous Aggregate leading to an inconsistent state because the catalog code don't handle this table as a normal catalog table so the records are not removed when dropping a Continuous Aggregate. The same problem will happen if you manually drop the old Continuous Aggregate after the migration. Fixed it by removing the useless Foreign Key and also adding another column named `user_view_definition` to the main plan table just to store the original user view definition for troubleshooting purposes. Fixed #5662 ab224789922f0e3b88bf57f5b43f0a5c258243cc,Ante Kresic,Tue May 9 14:45:15 2023 +0200,Fix DML decompression issues with bitmap heap scan,"Bitmap heap scans are specific in that they store scan state during node initialization. This means they would not pick up on any data that might have been decompressed during a DML command from the compressed chunk. To avoid this, we update the snapshot on the node scan state and issue a rescan to update the internal state. " bd36afe2f3a5bc437d58a88a85b19b2253c0a926,shhnwz,Wed May 10 11:17:38 2023 +0530,Fixed Coverity Scan Warnings,Unused Value reported during coverity scan link: https://scan4.scan.coverity.com/reports.htm#v56957/p12995 8e69a9989f9e9893807e7ea2403ab1c267895396,Ante Kresic,Thu Apr 27 11:59:01 2023 +0200,Ignore multinode tests from PR CI runs,"dist_move_chunk, dist_param, dist_insert and remote_txn create a lot of friction due to their flakiness. Ignoring them until we can fix them. " 3dc6824eb5c3384c4e8fda78a3e8288dcc3f8da0,Fabrízio de Royes Mello,Wed May 3 16:01:34 2023 -0300,Add GUC to enable/disable DML decompression, 6782beb1504bc421f973269f029c9854e17c7650,Ante Kresic,Thu May 4 11:42:13 2023 +0200,Fix index scan handling in DML decompression,We need to use the correct qualifiers for index scans since the generic scan qualifiers are not populated in this case. 8ca17e704c8f9320f360bab7eaf3622391d686bd,Dmitry Simonenko,Thu May 4 14:54:27 2023 +0300,Fix ALTER TABLE SET with normal tables,Running ALTER TABLE SET with multiple SET clauses on a regular PostgreSQL table produces irrelevant error when timescaledb extension is installed. Fix #5641 9259311275ded2e562e5222bc0853509dc2dd206,Sven Klemm,Wed May 3 09:42:38 2023 +0200,Fix JOIN handling in UPDATE/DELETE on compressed chunks,When JOINs were present during UPDATE/DELETE on compressed chunks the code would decompress other hypertables that were not the target of the UPDATE/DELETE operations and in the case of self-JOINs potentially decompress chunks not required to be decompressed. 769f9fe609df950a82d6697a08f33675da5c8f28,Bharathy,Wed May 3 22:37:08 2023 +0530,Fix segfault when deleting from compressed chunk,"During UPDATE/DELETE on compressed hypertables, we iterate over plan tree to collect all scan nodes. For each scan nodes there can be filter conditions. Prior to this patch we collect only first filter condition and use for first chunk which may be wrong. In this patch as and when we encounter a target scan node, we immediatly process those chunks. Fixes #5640 " 90f585ed7fbe51568749c96dc650f6666a4a20af,Fabrízio de Royes Mello,Fri Apr 28 09:16:33 2023 -0300,Fix CoverityScan deference after null check,Don't need to check NULL for `direct_query->jointree` because we don't allow queries without FROM clause in Continuous Aggregate definition. CoverityScan link: https://scan4.scan.coverity.com/reports.htm#v54116/p12995/fileInstanceId=131745632&defectInstanceId=14569562&mergedDefectId=384045 6e65172cd821c5509da3b7f41904a5b2f1916d16,Konstantina Skovola,Fri Apr 21 15:29:13 2023 +0300,Fix tablespace for compressed hypertable and corresponding toast,"If a hypertable uses a non default tablespace, the compressed hypertable and its corresponding toast table and index is still created in the default tablespace. This PR fixes this unexpected behavior and creates the compressed hypertable and its toast table and index in the same tablespace as the hypertable. Fixes #5520 " df32ad4b7920970a75a2b83dd3587fdda07b8302,Jan Nidzwetzki,Thu Apr 6 13:12:13 2023 +0200,Optimize compressed chunk resorting,"This patch adds an optimization to the DecompressChunk node. If the query 'order by' and the compression 'order by' are compatible (query 'order by' is equal or a prefix of compression 'order by'), the compressed batches of the segments are decompressed in parallel and merged using a binary heep. This preserves the ordering and the sorting of the result can be prevented. Especially LIMIT queries benefit from this optimization because only the first tuples of some batches have to be decompressed. Previously, all segments were completely decompressed and sorted. Fixes: #4223 Co-authored-by: Sotiris Stamokostas " cc9c3b343113fc9a43f095720c1d2f2596a23b03,Fabrízio de Royes Mello,Wed Apr 26 20:18:19 2023 -0300,Post-release 2.10.3,Adjust the upgrade/downgrade scripts and add the tests. ed8ca318c056ee8b01197a5740a25a358a034894,Nikhil Sontakke,Tue Apr 25 20:27:31 2023 +0530,Quote username identifier appropriately,Need to use quote_ident() on the user roles. Otherwise the extension scripts will fail. Co-authored-by: Mats Kindahl 2ce4bbc432e5af21c3a65971e1951db242e4520f,Bharathy,Fri Apr 28 07:22:32 2023 +0530,Enable continuous_aggs tests on all PG version., 1d092560f45bbec887a8cb098f338a789250196d,Zoltan Haindrich,Fri Apr 14 14:24:42 2023 +0000,Fix on-insert decompression after schema changes,On compressed hypertables 3 schema levels are in use simultaneously * main - hypertable level * chunk - inheritance level * compressed chunk In the build_scankeys method all of them appear - as slot have their fields represented as a for a row of the main hypertable. Accessing the slot by the attribut numbers of the chunks may lead to indexing mismatches if there are differences between the schemes. Fixes: #5577 be2879438490d5f3c4aeafd446b6c60afcefd6a5,Mats Kindahl,Tue Apr 25 14:28:10 2023 +0200,Enable run_job() for telemetry job,"Since telemetry job has a special code path to be able to be used both from Apache code and from TSL code, trying to execute the telemetry job with run_job() will fail. This code will allow run_job() to be used with the telemetry job to trigger a send of telemetry data. You have to belong to the group that owns the telemetry job (or be the owner of the telemetry job) to be able to use it. Closes #5605 " d5a286174d93aff24af3e3a911b84420177912a6,Fabrízio de Royes Mello,Tue Apr 25 17:33:56 2023 -0300,Changelog for 2.10.3, 8a95d1b9ee017bee2df3a2e57b0b949b50de1247,Fabrízio de Royes Mello,Wed Apr 26 18:05:05 2023 -0300,Add missing matrix in ignored 32bits regression workflow, e140cc702c35ecd810ed434ca81c18d008de31fc,Fabrízio de Royes Mello,Wed Apr 26 16:18:05 2023 -0300,Add missing matrix in ignored regression workflow, 002b6e879abaeecbde295775d7c46292bcb535bf,Fabrízio de Royes Mello,Wed Apr 26 14:37:25 2023 -0300,Fix CI ignored regression workflows,"We defined some paths to ignore regression test workflows when, for example, we change the CHANGELOG.md and others. But in fact it was not happening because we didn't define a proper name in the fake regression workflow that fake to the CI that some required status passed. Fixed it defining a proper regression name like we do for the regular regression workflow. " 3c8d7cef77d34d88ace3cd0a58fbb7640965d5c0,Fabrízio de Royes Mello,Wed Apr 26 09:02:57 2023 -0300,Fix cagg_repair for the old CAgg format,In commit 4a6650d1 we fixed the cagg_repair running for broken Continuous Aggregates with JOINs but we accidentally removed the code path for running against the old format (finalized=false) leading us to a dead code pointed out by CoverityScan: https://scan4.scan.coverity.com/reports.htm#v54116/p12995/fileInstanceId=131706317&defectInstanceId=14569420&mergedDefectId=384044 Fixed it by restoring the old code path for running the cagg_repair for Continuous Aggregates in the old format (finalized=false). d3730a4f6ac89612d749d10843def181f7830635,Mats Kindahl,Tue Apr 25 18:01:48 2023 +0200,Add permission checks to run_job(),"There were no permission checks when calling run_job(), so it was possible to execute any job regardless of who owned it. This commit adds such checks. " 4a6650d170cc07dbec0afe01bb3685a2cf9d9a37,Fabrízio de Royes Mello,Fri Apr 21 16:34:49 2023 -0300,Fix broken CAgg with JOIN repair function,The internal `cagg_rebuild_view_definition` function was trying to cast a pointer to `RangeTblRef` but it actually is a `RangeTblEntry`. Fixed it by using the already existing `direct_query` data struct to check if there are JOINs in the CAgg to be repaired. 910663d0be132aa05100233ebf7e97c66da7297f,Ante Kresic,Mon Apr 24 06:29:11 2023 +0200,Reduce decompression during UPDATE/DELETE,"When updating or deleting tuples from a compressed chunk, we first need to decompress the matching tuples then proceed with the operation. This optimization reduces the amount of data decompressed by using compressed metadata to decompress only the affected segments. " 3bf58dac02c1804578eac6eae95aec849c7f72b8,Lakshmi Narayanan Sreethar,Mon Apr 24 20:35:08 2023 +0530,Update windows package link in github action, 44dc042bb3784e5f6610504db7d8741f095be045,Bharathy,Mon Apr 24 19:40:23 2023 +0530,Fixed transparent decompress chunk test which seem to be flaky., 2f194e6109d186a4c61352afaa40381db617e437,Jan Nidzwetzki,Mon Apr 24 12:26:28 2023 +0200,Make compression metadata column names reusable,Move the creation of metadata column names for min/max values to separate functions to make the code reusable. c54d8bd9464cd2cf410f37968edeb1c438950547,Jan Nidzwetzki,Fri Apr 21 15:00:10 2023 +0200,Add missing order by to compression_ddl tests,"Some queries in compression_ddl had no order by. Therefore the output order was not defined, which led to flaky tests. " 583c36e91e115302431accb086eeb996f52411db,Ante Kresic,Thu Apr 20 09:00:21 2023 +0200,Refactor compression code to reduce duplication, 744b44cc52f1c2a3ae329657cdfdf29868df1015,Sven Klemm,Tue Apr 18 11:59:11 2023 +0200,Fix parameterization in DecompressChunk path generation,All children of an append path are required to have the same parameterization so we have to reparameterize when the selected path does not have the right parameterization. 23b3f8d7a6c6f9e5549fcf18cbf70937f02bcc57,Ante Kresic,Thu Apr 20 12:54:28 2023 +0200,Block unique idx creation on compressed hypertable,"This block was removed by accident, in order to support this we need to ensure the uniqueness in the compressed data which is something we should do in the future thus removing this block. " 12f3131f9e784a03136d046f9b2fd840145ee6f0,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Apr 20 09:40:34 2023 +0400,Post-release 2.10.2,Adjust the upgrade/downgrade scripts and add the tests. a0df8c8e6df75b952fef9f371d689f2f064397e5,Zoltan Haindrich,Fri Apr 14 14:24:42 2023 +0000,Fix on-insert decompression for unique constraints,Inserting multiple rows into a compressed chunk could have bypassed constraint check in case the table had segment_by columns. Decompression is narrowed to only consider candidates by the actual segment_by value. Because of caching - decompression was skipped for follow-up rows of the same Chunk. Fixes #5553 a49fdbcffba03e4af99d6f578e9f457c24aa8fad,Ante Kresic,Tue Apr 18 14:52:33 2023 +0200,Reduce decompression during constraint checking,"When inserting into a compressed chunk with constraints present, we need to decompress relevant tuples in order to do speculative inserting. Usually we used segment by column values to limit the amount of compressed segments to decompress. This change expands on that by also using segment metadata to further filter compressed rows that need to be decompressed. " 28d9db1af95f6a1ee9302edab79cb730d32099d2,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Fri Apr 14 13:52:23 2023 +0400,Changelog for 2.10.2, ## 2.10.2 (2023-04-20) **Bugfixes** * #5410 Fix file trailer handling in the COPY fetcher * #5446 Add checks for malloc failure in libpq calls * #5233 Out of on_proc_exit slots on guc license change * #5428 Use consistent snapshots when scanning metadata * #5499 Do not segfault on large histogram() parameters * #5470 Ensure superuser perms during copy/move chunk * #5500 Fix when no FROM clause in continuous aggregate definition * #5433 Fix join rte in CAggs with joins * #5556 Fix duplicated entries on timescaledb_experimental.policies view * #5462 Fix segfault after column drop on compressed table * #5543 Copy scheduled_jobs list before sorting it * #5497 Allow named time_bucket arguments in Cagg definition * #5544 Fix refresh from beginning of Continuous Aggregate with variable time bucket * #5558 Use regrole for job owner * #5542 Enable indexscan on uncompressed part of partially compressed chunks **Thanks** * @nikolaps for reporting an issue with the COPY fetcher * @S-imo-n for reporting the issue on Background Worker Scheduler crash * @geezhu for reporting issue on segfault in historgram() * @mwahlhuetter for reporting the issue with joins in CAggs * @mwahlhuetter for reporting issue with duplicated entries on timescaledb_experimental.policies view * @H25E for reporting error refreshing from beginning of a Continuous Aggregate with variable time bucket 5633960f8b0e4675e342b1721bde954fdbe938e7,Konstantina Skovola,Tue Apr 11 12:33:11 2023 +0300,Enable indexscan on uncompressed part of partially compressed chunks,"This was previously disabled as no data resided on the uncompressed chunk once it was compressed, but this is not the case anymore with partially compressed chunks, so we enable indexscan for the uncompressed chunk again. Fixes #5432 Co-authored-by: Ante Kresic " ca472ebb0d3e290d029bd9beeda1647547302920,shhnwz,Wed Apr 12 10:59:26 2023 +0530,Fixed transparent decompress chunk,Transparent decompress chunk was added into to ignore list due to the side effect of #5118. This issue is to fix the flaky nature of the test. 9a64385f34db737cc174be5ee8ed560bd82b1663,Mats Kindahl,Thu Apr 13 13:28:25 2023 +0200,Use regrole for job owner,"Instead of using a user name to register the owner of a job, we use regrole. This allows renames to work properly since the underlying OID does not change when the owner name changes. We add a check when calling `DROP ROLE` that there is no job with that owner and generate an error if there is. " 20db884bd72b1798844176d34bb106b211f4e30e,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Mon Apr 17 19:23:02 2023 +0400,Increase remote tuple and startup costs,"Our cost model should be self-consistent, and the relative values for the remote tuple and startup costs should reflect their real cost, relative to costs of other operations like CPU tuple cost. For example, now remote costs are set even lower than the parallel tuple and startup cost. Contrary to that, their real world cost is going to be an order of magnitude higher or more, because parallel tuples are sent through shared memory, and remote tuples are sent over the network. Increasing these costs leads to query plan improvements, e.g. we start to favor the GROUP BY pushdown in some cases. " b16bf3b100c5a9ce693ef5fe49208421445e78c5,Fabrízio de Royes Mello,Mon Apr 17 11:57:40 2023 -0300,Fix post repair tests,Commit 3f9cb3c2 introduced new repair tests for broken Continuous Aggregates with JOIN clause but in the post.repair.sql we not properly calling the post.repair.cagg_joins.sql because a wrong usage of psql `if` statement. d32224a914877f8f9342cca63859ca1d76d9bddb,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Fri Apr 14 13:18:15 2023 +0400,Add labels to backported PRs,Makes it easier to check what was done when looking at the release milestone. 90e54def8a55e833f7e6bd5408d585ce244c8bd0,Sven Klemm,Fri Apr 14 12:22:54 2023 +0200,Improve interpolate error message on datatype mismatch,Include information about the expected and the returned datatype in the error details of interpolate. a3d778f7a0313fca4ad86f64f9f32158011c8a0e,Fabrízio de Royes Mello,Tue Apr 4 15:39:38 2023 -0300,Add CI check for missing gitignore entries,Whenever we create a template sql file (*.sql.in) we should add the respective .gitignore entry for the generated test files. So added a CI check to check for missing gitignore entries for generated test files. a383c8dd4ff6cc932dfafe00b7a1a44d9efb6740,Lakshmi Narayanan Sreethar,Tue Apr 11 16:44:13 2023 +0530,Copy scheduled_jobs list before sorting it,"The start_scheduled_jobs function mistakenly sorts the scheduled_jobs list in-place. As a result, when the ts_update_scheduled_jobs_list function compares the updated list of scheduled jobs with the existing scheduled jobs list, it is comparing a list that is sorted by job_id to one that is sorted by next_start time. Fix that by properly copying the scheduled_jobs list into a new list and use that for sorting. Fixes #5537 " b10139ba4899d88053c98130881b8fe84cc7ccf6,Lakshmi Narayanan Sreethar,Thu Apr 13 20:20:34 2023 +0530,Update bgw_custom testcase,Added a few cleanup steps and updated a test logic to make the testcase runs more stable. b136bb554baf08bfc7f648a9d5f8ec59402230a3,Maheedhar PV,Thu Apr 13 21:52:20 2023 +0530,Run dist_move_chunk as a solo test in PG15.2,Running the test dist_move_chunk in parallel can cause a timeouts(deadlock) in other tests running in parallel on PG15.2. Force the test to run solo. ca240c33ca2d6f445984f4f43c38d571c1bfac83,Maheedhar PV,Wed Apr 12 20:26:48 2023 +0530,Build failure on windows,"The cmake call to ""find_package(OpenSSL)"" on windows may set the variable OPENSSL_LIBRARIES to a non-standard list which might include non path items such as ""debug"", ""optimized"". These non-standard elements in the list would cause the link failure with error ""LNK1104"". Fix: Check and retain only valid paths in OPENSSL_LIBRARIES list. closes#407 " 464d20fb4103ea99ac048c116f0b386594dff143,Ante Kresic,Thu Mar 30 12:46:44 2023 +0200,Propagate vacuum/analyze to compressed chunks,"With recent changes, we enabled analyze on uncompressed chunk tables for compressed chunks. This change includes analyzing the compressed chunks table when analyzing the hypertable and its chunks, enabling us to remove the generating stats when compressing chunks. " 3f9cb3c27ac828901842c70e17451beec754bbb8,Rafia Sabih,Tue Mar 14 16:37:02 2023 +0100,Pass join related structs to the cagg rte,"In case of joins in the continuous aggregates, pass the required structs to the new rte created. These values are required by the planner to finally query the materialized view. Fixes #5433 " 09565acae497274dd48ef8231eda2845aabab1e7,Fabrízio de Royes Mello,Wed Apr 12 12:17:41 2023 -0300,Fix timescaledb_experimental.policies duplicates,Commit 16fdb6ca5e introduced `timescaledb_experimental.policies` view to expose the Continuous Aggregate policies but the current JOINS over our catalog are not accurate. Fixed it by properly JOIN the underlying catalog tables to expose the correct information without duplicates about the Continuous Aggregate policies. Fixes #5492 f6c8468ee630bdad03cdb433ddb6be8daa1709c0,Fabrízio de Royes Mello,Tue Apr 11 16:58:13 2023 -0300,Fix timestamp out of range refreshing CAgg,"When refreshing from the beginning (window_start=NULL) of a Continuous Aggregate with variable time bucket we were getting a `timestamp out of range` error. Fixed it by setting `-Infinity` when passing `window_start=NULL` when refreshing a Continuous Aggregate with variable time bucket. Fixes #5474, #5534 " 3cc8a4ca34ec76f4de66349af3194851acb7c228,Mats Kindahl,Wed Apr 12 12:52:41 2023 +0200,Fix error message for continuous aggregates,"Several error messages for continuous aggregates are not following the error message style guidelines at https://www.postgresql.org/docs/current/error-style-guide.html In particular, they do not write the hints and detailed messages as full sentences. " f0623a8c3891f5a2c28c5da8d3abfa545e5f59d1,Sven Klemm,Wed Apr 12 10:56:35 2023 +0200,Skip Ordered Append when only 1 child node is present,"This is mostly a cosmetic change. When only 1 child is present there is no need for ordered append. In this situation we might still benefit from a ChunkAppend node here due to runtime chunk exclusion when we have non-immutable constraints, so we still add the ChunkAppend node in that situation even with only 1 child. " 0595ff0888f2ffb8d313acb0bda9642578a9ade3,Sven Klemm,Tue Apr 11 22:01:14 2023 +0200,Move type support functions into _timescaledb_functions schema, 2d7eb18f249e898e63fc90bf02012af9ce2ccded,Sven Klemm,Thu Apr 6 14:50:27 2023 +0200,Drop unused SQL functions,"This patch drops the following internal SQL functions which were unused: _timescaledb_internal.is_main_table(regclass); _timescaledb_internal.is_main_table(text, text); _timescaledb_internal.hypertable_from_main_table(regclass); _timescaledb_internal.main_table_from_hypertable(integer); _timescaledb_internal.time_literal_sql(bigint, regtype); " 84b6783a199c37bff6b36a3c0940915e5ca569ce,Ante Kresic,Thu Apr 6 14:00:49 2023 +0200,Fix chunk status when inserting into chunks,"While executing compression operations in parallel with inserting into chunks (both operations which can potentially change the chunk status), we could get into situations where the chunk status would end up inconsistent. This change re-reads the chunk status after locking the chunk to make sure it can decompress data when handling ON CONFLICT inserts correctly. " 54074f1fd4ab7acb3e4d2e2becd82b844b570e67,Ante Kresic,Fri Mar 31 11:36:17 2023 +0200,Add more compression DML isolation tests,Verify that insertion into compressed chunks does not block each other if the chunks is already partially compressed. Also check that using the RETURNING clause works the same. dc5bf3b32ee39e121ca9a8888124460b8f1eee1e,Ante Kresic,Tue Mar 14 12:44:44 2023 +0100,Test compression DML with physical layout changes,These tests try to verify that changing physical layout of chunks (either compressed or uncompressed) should yield consistent results. They also verify index mapping on compressed chunks is handled correctly. 9a466ca185ba3df611cbdb460b7b4aa906fcdf2f,Fabrízio de Royes Mello,Mon Apr 10 14:33:48 2023 -0300,Silence WARNING after extension update,Commit 8afdddc2da added the first step for deprecating the old format of Continuous Aggregate but just for PostgreSQL 15 and later versions. During the extension update we emit a message about the deprecation but this has being emited even if the user is using PostgreSQL versions before 15. Fixed it by emiting the WARNING just when PostgreSQL version is greater or equal to 15. 04f43335dea11e9c467ee558ad8edfc00c1a45ed,Sven Klemm,Thu Apr 6 13:00:00 2023 +0200,Move aggregate support function into _timescaledb_functions,"This patch moves the support functions for histogram, first and last into the _timescaledb_functions schema. Since we alter the schema of the existing functions in upgrade scripts and do not change the aggregates this should work completely transparently for any user objects using those aggregates. " 3814a3f351a0691681e3123631a99e3e5b45aae6,Konstantina Skovola,Thu Apr 6 09:00:55 2023 +0300,Properly format license error hint,Commit 57fde383b3dddd0b52263218e65a0135981c2d34 changed the messaging but did not format the error hint correctly. This patch fixes the error hint. Fixes #5490 8c77be6c68b3f53143eb0faa8b49bbf15674e284,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Apr 6 18:10:15 2023 +0400,Look up compressed column metadata only at planning time,"Now we look them up again at execution time, which adds up for tables with a large number of chunks. This gives about 15" " speedup (100 mcs) on a small query on a table from tests with 50 chunks: `select id, ts, value from metric_compressed order by id, ts limit 100;` ",,,, df70f3e050e817460e8a089a3a8fc90828d5d6c5,Konstantina Skovola,Tue Apr 4 17:25:19 2023 +0300,Remove unused variable in tsl_get_compressed_chunk_index_for_recompression,Commit 72c0f5b25e569015aacb98cc1be3169a1720116d introduced an unused variable. This patch removes it. 975e9ca1662ac4dd050b6fb296e51d507a64a773,Zoltan Haindrich,Fri Mar 17 16:32:33 2023 +0000,Fix segfault after column drop on compressed table,"Decompression produces records which have all the decompressed data set, but it also retains the fields which are used internally during decompression. These didn't cause any problem - unless an operation is being done with the whole row - in which case all the fields which have ended up being non-null can be a potential segfault source. Fixes #5458 #5411 " feef9206facc5c5f506661de4a81d96ef059b095,Sven Klemm,Fri Mar 31 08:22:57 2023 +0200,Add _timescaledb_functions schema,Currently internal user objects like chunks and our functions live in the same schema making locking down that schema hard. This patch adds a new schema _timescaledb_functions that is meant to be the schema used for timescaledb internal functions to allow separation of code and chunks or other user objects. 6440bb3477eef183459272cb90f6ffa8bf30b682,Fabrízio de Royes Mello,Tue Apr 4 17:31:33 2023 -0300,Remove unused function,Remove unused function `invalidation_threshold_htid_found`. 1fb058b1992bd89b6d69e910b42d5cae5ee46212,Bharathy,Wed Apr 5 16:47:52 2023 +0530,Support UPDATE/DELETE on compressed hypertables.,"This patch does following: 1. Executor changes to parse qual ExprState to check if SEGMENTBY column is specified in WHERE clause. 2. Based on step 1, we build scan keys. 3. Executor changes to do heapscan on compressed chunk based on scan keys and move only those rows which match the WHERE clause to staging area aka uncompressed chunk. 4. Mark affected chunk as partially compressed. 5. Perform regular UPDATE/DELETE operations on staging area. 6. Since there is no Custom Scan (HypertableModify) node for UPDATE/DELETE operations on PG versions < 14, we don't support this feature on PG12 and PG13. " c2941a3f9a95417c77b15db900b61bffc89cf36c,Sven Klemm,Mon Apr 3 13:09:06 2023 +0200,Fix windows package test,Use the windows packages from the github release for package testing. 2e6c6b5c58c5e139f8a34a0de30dd2590908f654,Erik Nordström,Wed Mar 8 16:58:05 2023 +0100,Refactor and optimize distributed COPY,"Refactor the code path that handles remote distributed COPY. The main changes include: * Use a hash table to lookup data node connections instead of a list. * Refactor the per-data node buffer code that accumulates rows into bigger CopyData messages. * Reduce the default number of rows in a CopyData message to 100. This seems to improve throughput, probably striking a better balance between message overhead and latency. * The number of rows to send in each CopyData message can now be changed via a new foreign data wrapper option. " c6b9f5097886d0d33bb88f2aee1d427b0c69b969,Ildar Musin,Mon Apr 3 16:08:15 2023 +0200,Fix OSM chunks exclusion from append paths,OSM chunks have their own fdw_private which conflicts with checks in the MergeAppend code path causing segfaults. This commit fixes this by returning early when there is an OSM chunk in the MergeAppendPath. 517dee9f6bf9c1a94b7dd05f1ea0057e4f27f5b6,Nikhil Sontakke,Mon Mar 27 13:40:52 2023 +0530,Add test for superuser chunk copy/move,Add isolation test case to check that the chunk object created during chunk copy/move operation on the destination datanode always has superuser credentials till the end of the operation. ff5959f8f9c1d40dfe8b36a18cbd28d78ac77dca,Rafia Sabih,Tue Mar 28 12:17:06 2023 +0200,Handle when FROM clause is missing in continuous aggregate definition,It now errors out for such a case. Fixes #5500 cb81c331ae4a9fa233106439ef5f73c02b702065,Konstantina Skovola,Tue Mar 28 16:27:00 2023 +0300,Allow named time_bucket arguments in Cagg definition,Fixes #5450 98218c1d079231a9aa469b37ddd0ed39e77c2adb,Rafia Sabih,Wed Feb 8 11:54:28 2023 +0100,Enable joins for heirarchical continuous aggregates,"The joins could be between a continuous aggregate and hypertable, continuous aggregate and a regular Postgres table, and continuous aggregate and a regular Postgres view. " 777c599a345242e69e0e9d505897992c73bf98fc,Mats Kindahl,Tue Mar 28 09:27:52 2023 +0200,Do not segfault on large histogram() parameters,There is a bug in `width_bucket()` causing an overflow and subsequent NaN value as a result of dividing with `+inf`. The NaN value is interpreted as an integer and hence generates an index out of range for the buckets. This commit fixes this by generating an error rather than segfaulting for bucket indexes that are out of range. 22841abdf0d2c7b853386d8594d2a317df5f1e07,Konstantina Skovola,Mon Mar 27 11:19:40 2023 +0300,Update community license related errors,Update the error message printed when attempting to use a community license feature with apache license installed. Fixes #5438 a51d21efbe3ec0a16c986d2e349c7a29e06b2b0c,Erik Nordström,Thu Mar 16 16:45:28 2023 +0100,Fix issue creating dimensional constraints,"During chunk creation, the chunk's dimensional CHECK constraints are created via an ""upcall"" to PL/pgSQL code. However, creating dimensional constraints in PL/pgSQL code sometimes fails, especially during high-concurrency inserts, because PL/pgSQL code scans metadata using a snapshot that might not see the same metadata as the C code. As a result, chunk creation sometimes fail during constraint creation. To fix this issue, implement dimensional CHECK-constraint creation in C code. Other constraints (FK, PK, etc.) are still created via an upcall, but should probably also be rewritten in C. However, since these constraints don't depend on recently updated metadata, this is left to a future change. Fixes #5456 " 72c0f5b25e569015aacb98cc1be3169a1720116d,Konstantina Skovola,Mon Feb 13 13:02:51 2023 +0200,Rewrite recompress_chunk in C for segmentwise processing,"This patch introduces a C-function to perform the recompression at a finer granularity instead of decompressing and subsequently compressing the entire chunk. This improves performance for the following reasons: - it needs to sort less data at a time and - it avoids recreating the decompressed chunk and the heap inserts associated with that by decompressing each segment into a tuplesort instead. If no segmentby is specified when enabling compression or if an index does not exist on the compressed chunk then the operation is performed as before, decompressing and subsequently compressing the entire chunk. " 7e43f45ccb8b30800db5f19af6ab024d37efea2b,Nikhil Sontakke,Tue Mar 21 16:30:21 2023 +0530,Ensure superuser perms during copy/move chunk,"There is a security loophole in current core Postgres, due to which it's possible for a non-superuser to gain superuser access by attaching dependencies like expression indexes, triggers, etc. before logical replication commences. To avoid this, we now ensure that the chunk objects that get created for the subscription are done so as a superuser. This avoids malicious dependencies by regular users. " 38fcd1b76b14ddd73f64853a1bf6efca5e17fb4c,Fabrízio de Royes Mello,Fri Mar 17 17:47:34 2023 -0300,Improve Realtime Continuous Aggregate performance,"When calling the `cagg_watermark` function to get the watermark of a Continuous Aggregate we execute a `SELECT MAX(time_dimension)` query in the underlying materialization hypertable. The problem is that a `SELECT MAX(time_dimention)` query can be expensive because it will scan all hypertable chunks increasing the planning time for a Realtime Continuous Aggregates. Improved it by creating a new catalog table to serve as a cache table to store the current Continous Aggregate watermark in the following situations: - Create CAgg: store the minimum value of hypertable time dimension data type; - Refresh CAgg: store the last value of the time dimension materialized in the underlying materialization hypertable (or the minimum value of materialization hypertable time dimension data type if there's no data materialized); - Drop CAgg Chunks: the same as refresh cagg. Closes #4699, #5307 " 699fcf48aa71189e581c27777715f89824b56268,shhnwz,Fri Dec 23 11:07:51 2022 +0530,Stats improvement for Uncompressed Chunks,During the compression autovacuum use to be disabled for uncompressed chunk and enable after decompression. This leads to postgres maintainence issue. Let's not disable autovacuum for uncompressed chunk anymore. Let postgres take care of the stats in its natural way. Fixes #309 5c07a57a0246cf81eb55d5f563825cd20512c7cf,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Mar 22 12:43:22 2023 +0400,Simplify control flow in decompress_chunk_exec,"No functional changes, mostly just reshuffles the code to prepare for batch decompression. Also removes unneeded repeated column value stores and ExecStoreTuple, to save 3-5" execution time on some queries. ,,,, 63b416b6b0065e6e178913a916a2f366f4f0d1a4,Erik Nordström,Fri Mar 10 13:15:02 2023 +0100,Use consistent snapshots when scanning metadata,"Invalidate the catalog snapshot in the scanner to ensure that any lookups into `pg_catalog` uses a snapshot that is consistent with the snapshot used to scan TimescaleDB metadata. This fixes an issue where a chunk could be looked up without having a proper relid filled in, causing an assertion failure (`ASSERT_IS_VALID_CHUNK`). When a chunk is scanned and found (in `chunk_tuple_found()`), the Oid of the chunk table is filled in using `get_relname_relid()`, which could return InvalidOid due to use of a different snapshot when scanning `pg_class`. Calling `InvalidateCatalogSnapshot()` before starting the metadata scan in `Scanner` ensures the pg_catalog snapshot used is refreshed. Due to the difficulty of reproducing this MVCC issue, no regression or isolation test is provided, but it is easy to hit this bug when doing highly concurrent COPY:s into a distributed hypertable. " 7d6cf90ee783656b8230e6ed8b35b47126729423,Fabrízio de Royes Mello,Sun Mar 19 18:30:48 2023 -0300,Add missing gitignore entry,Pull request #4827 introduced a new template SQL test file but missed to add the properly `.gitignore` entry to ignore generated test files. cc51e20e87f3ce9426b88790149cd7321cc4853e,Bharathy,Mon Mar 20 21:59:38 2023 +0530,Add support for ON CONFLICT DO UPDATE for compressed hypertables,This patch fixes execution of INSERT with ON CONFLICT DO UPDATE by removing error and allowing UPDATE do happen on the given compressed hypertable. 8cccc375fbf3bd8e0025f596583d3f09db489e9d,Konstantina Skovola,Tue Mar 14 14:23:09 2023 +0200,Add license information to extension description,Fixes #5436 736c20fb7194f8e057fb5101e19deeba8c78946b,Konstantina Skovola,Mon Mar 20 14:53:37 2023 +0200,Hardcode PG14-15 versions for chocolatey, 2570ab11109df25d1839042ec46a0b404cdd0bea,syvb,Wed Mar 15 20:34:05 2023 -0400,Add new clangd cache location to gitignore, 67ff84e8f2e380b8466152b8054ca4d30a17a3ed,Mats Kindahl,Wed Mar 15 14:20:41 2023 +0100,Add check for malloc failure in libpq calls,"The functions `PQconndefaults` and `PQmakeEmptyPGresult` calls `malloc` and can return NULL if it fails to allocate memory for the defaults and the empty result. It is checked with an `Assert`, but this will be removed in production builds. Replace the `Assert` with an checks to generate an error in production builds rather than trying to de-reference the pointer and cause a crash. " 790b322b242351b151bfbd46706d42dc61e05595,Zoltan Haindrich,Mon Mar 13 14:04:00 2023 +0000,Fix DEFAULT value handling in decompress_chunk,The sql function decompress_chunk did not filled in default values during its operation. Fixes #5412 827684f3e2718c09e93914d60b88e79488374b33,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Mar 15 17:27:32 2023 +0400,Use prepared statements for parameterized data node scans,"This allows us to avoid replanning the inner query on each new loop, speeding up the joins. " 03a799b874a1a5d6accb40a2b49ed53a77b48e0a,Sven Klemm,Tue Mar 14 20:21:35 2023 +0100,Mention that new status values need handling in downgrade script,When adding new status values we must make sure to add special handling for these values to the downgrade script as previous versions will not know how to deal with those. f8022eb33253dbe91bc643c317b787e03be37e1d,Dmitry Simonenko,Mon Mar 13 17:15:31 2023 +0200,Add additional tests for compression with HA,Make sure inserts into compressed chunks work when a DN is down Fix #5039 65562f02e88e814b4823be40fd27bee245b130ae,Sven Klemm,Sun Jan 29 21:53:50 2023 +0100,Support unique constraints on compressed chunks,This patch allows unique constraints on compressed chunks. When trying to INSERT into compressed chunks with unique constraints any potentially conflicting compressed batches will be decompressed to let postgres do constraint checking on the INSERT. With this patch only INSERT ON CONFLICT DO NOTHING will be supported. For decompression only segment by information is considered to determine conflicting batches. This will be enhanced in a follow-up patch to also include orderby metadata to require decompressing less batches. c02cb76b38ee3ecbf33a8fe0d6a5864454ee2b67,Sven Klemm,Sun Feb 5 19:23:14 2023 +0100,Don't reindex relation during decompress_chunk,Reindexing a relation requires AccessExclusiveLock which prevents queries on that chunk. This patch changes decompress_chunk to update the index during decompression instead of reindexing. This patch does not change the required locks as there are locking adjustments needed in other places to make it safe to weaken that lock. 20ea406616d557bb93de5c93710656bce0a5de8f,Sven Klemm,Sun Mar 12 21:12:59 2023 +0100,Add utility function to map attribute numbers,This patch adds a function ts_map_attno that can be used to map the attribute number from one relation to another by column name. 356a20777c6f1aab234c5604f0f97de8077b4890,Jan Nidzwetzki,Fri Mar 10 14:34:56 2023 +0100,Handle user-defined FDW options properly,"This patch changes the way user-defined FDW options (e.g., startup costs, per-tuple costs) are handled. So far, these values were retrieved in apply_fdw_and_server_options() but reset to default values afterward. " 5e0391392aca031c601f79e885452bb82ad909d1,Maheedhar PV,Mon Mar 13 05:18:18 2023 +0530,Out of on_proc_exit slots on guc license change,"Problem: When the guc timescaledb.license = 'timescale' is set in the conf file and a SIGHUP is sent to postgress process and a reload of the tsl module is triggered. This reload happens in 2 phases 1. tsl_module_load is called which will load the module only if not already loaded and 2.The ts_module_init is called for every ts_license_guc_assign_hook irrespective of if it is new load.This ts_module_init initialization function also registers a on_proc_exit function to be called on exit. The list of on_proc_exit methods are maintained in a fixed array on_proc_exit_list of size MAX_ON_EXITS (20) which gets filled up on repeated SIGHUPs and hence an error. Fix: The fix is to make the ts_module_init() register the on_proc_exit callback, only in case the module is reloaded and not in every init call. Closes #5233 " e92d5ba748de9ba7f26221ebb9717c7db3fcca5d,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Mar 9 14:34:28 2023 +0400,Add more tests for compression,"Unit tests for different data sequences, and SQL test for float4. " f5db023152afed21ac5ea413b5e5ae12185a23be,Jan Nidzwetzki,Thu Mar 9 14:19:19 2023 +0100,Track file trailer only in debug builds,The commit 96574a7 changes the handling of the file_trailer_received flag. It is now only used in asserts and not in any other kind of logic. This patch encapsulates the file_trailer_received in a USE_ASSERT_CHECKING macro. 217ba014a7fc8e864a1dbd1b4af8634db67ed507,Sven Klemm,Wed Mar 8 23:03:51 2023 +0100,Use version checks to decide about RelationGetSmgr backporting,Use explicit version checks to decide whether to define backported RelationGetSmgr function or rely on the function being available. This simplifies the cmake code a bit and make the backporting similar to how we handle this for other functions. 7b8177aa74de886b2e88a0fad41bc8ab9feb1e87,Jan Nidzwetzki,Tue Mar 7 13:32:02 2023 +0100,Fix file trailer handling in the COPY fetcher,"The copy fetcher fetches tuples in batches. When the last element in the batch is the file trailer, the trailer was not handled correctly. The existing logic did not perform a PQgetCopyData in that case. Therefore the state of the fetcher was not set to EOF and the copy operation was not correctly finished at this point. Fixes: #5323 " a854b2760f66cc640bf2fe71683b0e4595dfbdde,Sven Klemm,Wed Mar 8 13:27:13 2023 +0100,Simplify ts_indexing_relation_has_primary_or_unique_index,Rely on postgres functionality for index column tracking instead of rolling our own. f54dd7b05df420df1fd3318bd3f768305c619c20,Bharathy,Wed Mar 8 18:16:29 2023 +0530,Fix SEGMENTBY columns predicates to be pushed down,WHERE clause with SEGMENTBY column of type text/bytea non-equality operators are not pushed down to Seq Scan node of compressed chunk. This patch fixes this issue. Fixes #5286 c76a0cff688afe970d256c31862e63207d29b2ba,Erik Nordström,Fri Feb 24 12:38:16 2023 +0100,Add parallel support for partialize_agg(),"Make `partialize_agg()` support parallel query execution. To make this work, the finalize node need combine the individual partials from each parallel worker, but the final step that turns the resulting partial into the finished aggregate should not happen. Thus, in the case of distributed hypertables, each data node can run a parallel query to compute a partial, and the access node can later combine and finalize these partials into the final aggregate. Esssentially, there will be one combine step (minus final) on each data node, and then another one plus final on the access node. To implement this, the finalize aggregate plan is simply modified to elide the final step, and to reserialize the partial. It is only possible to do this at the plan stage; if done at the path stage, the PostgreSQL planner will hit assertions that assume that the node has certain values (e.g., it doesn't expect combine Paths to skip the final step). " c13ed17fbce56129136239ad369ac5e5f1502821,Bharathy,Tue Mar 7 20:11:42 2023 +0530,Fix DELETE command tag,DELETE on hypertables always reports 0 as affected rows. This patch fixes this issue. f680b995299c2900a1883f4a94128b89d5b61db5,Sven Klemm,Tue Mar 7 11:29:52 2023 +0100,Fix assertion in calculate_chunk_interval for negative target size,When called with negative chunk_target_size_bytes calculate_chunk_interval will throw an assertion. This patch adds error handling for this condition. Found by sqlsmith. 00321dba410dbfd3984c45401941d1f5989e05de,Sven Klemm,Tue Mar 7 09:42:59 2023 +0100,2.10.1 Post-release adjustments,Add 2.10.1 to update test scripts and adjust the downgrade versioning. 5a3cacd06f95270569082cf14ba55b9092426c17,Konstantina Skovola,Fri Feb 10 10:21:47 2023 +0200,Fix sub-second intervals in hierarchical caggs,"Previously we used date_part(""epoch"", interval) and integer division internally to determine whether the top cagg's interval is a multiple of its parent's. This led to precision loss and wrong results in the case of intervals with sub-second components. Fixed by using the `ts_interval_value_to_internal` function to convert intervals to appropriate integer representation for division. Fixes #5277 " 00b566dfe478c11134bcf1e7bcf38943e7fafe8f,Fabrízio de Royes Mello,Mon Mar 6 15:51:03 2023 -0300,Remove unused functions,We don't use `ts_catalog_delete[_only]` functions anywhere and instead we rely on `ts_catalog_delete_tid[_only]` functions so removing it from our code base. d386aa1def48bb2a3321beffe3bb565548666765,Sven Klemm,Fri Mar 3 08:44:49 2023 +0100,Release 2.10.1,This release contains bug fixes since the 2.10.0 release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * #5159 Support Continuous Aggregates names in hypertable_(detailed_)size * #5226 Fix concurrent locking with chunk_data_node table * #5317 Fix some incorrect memory handling * #5336 Use NameData and namestrcpy for names * #5343 Set PortalContext when starting job * #5360 Fix uninitialized bucket_info variable * #5362 Make copy fetcher more async * #5364 Fix num_chunks inconsistency in hypertables view * #5367 Fix column name handling in old-style continuous aggregates * #5378 Fix multinode DML HA performance regression * #5384 Fix Hierarchical Continuous Aggregates chunk_interval_size **Thanks** * @justinozavala for reporting an issue with PL/Python procedures in the background worker * @Medvecrab for discovering an issue with copying NameData when forming heap tuples. * @pushpeepkmonroe for discovering an issue in upgrading old-style continuous aggregates with renamed columns * @pushpeepkmonroe for discovering an issue in upgrading old-style continuous aggregates with renamed columns 5cd2c038796fb302190b080c90e5acddbef4b8d1,Sven Klemm,Sat Mar 4 10:18:34 2023 +0100,Simplify windows-build-and-test-ignored.yaml,Remove code not needed for the skip workflow of the windows test. 8a2f1f916aeee0dabc0eb653f2483ba5e8d43f74,Sven Klemm,Sat Mar 4 09:06:28 2023 +0100,Fix windows package test,Chocolatey has all the postgres versions we need available so we can reenable previously disabled tests. But the recent packages seem to have different versioning schema without a suffix. 830c37b5b0d2ab920e12e288000a4cdb0f2d5896,Dmitry Simonenko,Mon Mar 6 17:42:36 2023 +0200,Fix concurrent locking with chunk_data_node table,"Concurrent insert into dist hypertable after a data node marked as unavailable would produce 'tuple concurrently deleted` error. The problem occurs because of missing tuple level locking during scan and concurrent delete from chunk_data_node table afterwards, which should be treated as `SELECT … FOR UPDATE` case instead. Based on the fix by @erimatnor. Fix #5153 " 4c0075010dbf57d6d300feda74bfa60cb76355ba,Ildar Musin,Thu Mar 2 17:13:18 2023 +0100,Add hooks for hypertable drops,To properly clean up the OSM catalog we need a way to reliably track hypertable deletion (including internal hypertables for CAGGS). 474b09bdfc1f7bf2fe9039c07d6a97f34b9bd495,Sven Klemm,Fri Feb 24 21:08:45 2023 +0100,Use pgspot 0.5.0 in CI, 32046832d3af53d2432bb1b2c7c9bdf5f88fcfb8,Fabrízio de Royes Mello,Thu Mar 2 16:09:49 2023 -0300,Fix Hierarchical CAgg chunk_interval_size,When a Continuous Aggregate is created the `chunk_interval_size` is defined my the `chunk_interval_size` of the original hypertable multiplied by a fixed factor of 10. The problem is currently when we create a Hierarchical Continuous Aggregate the same factor is applied and it lead to an exponential `chunk_interval_size`. Fixed it by just copying the `chunk_interval_size` from the base Continuous Aggregate for an Hierachical Continuous Aggreagate. Fixes #5382 1423b55d1888b73c40d3d326fa63f8647fd8a413,Nikhil Sontakke,Thu Mar 2 13:24:31 2023 +0530,Fix perf regression due to DML HA,"We added checks via #4846 to handle DML HA when replication factor is greater than 1 and a datanode is down. Since each insert can go to a different chunk with a different set of datanodes, we added checks on every insert to check if DNs are unavailable. This increased CPU consumption on the AN leading to a performance regression for RF > 1 code paths. This patch fixes this regression. We now track if any DN is marked as unavailable at the start of the transaction and use that information to reduce unnecessary checks for each inserted row. " a6ff7ba6cc15b280a275e5acd315741ec9c86acc,Mats Kindahl,Tue Feb 28 12:04:17 2023 +0100,Rename columns in old-style continuous aggregates,"For continuous aggregates with the old-style partial aggregates renaming columns that are not in the group-by clause will generate an error when upgrading to a later version. The reason is that it is implicitly assumed that the name of the column is the same as for the direct view. This holds true for new-style continous aggregates, but is not always true for old-style continuous aggregates. In particular, columns that are not part of the `GROUP BY` clause can have an internally generated name. This commit fixes that by extracting the name of the column from the partial view and use that when renaming the partial view column and the materialized table column. " e2e7ae304521b74ac6b3f157a207da047d44ab06,Sven Klemm,Fri Mar 3 11:22:06 2023 +0100,Don't run sanitizer test on individual PRs,Sanitizer tests take a long time to run so we don't want to run them on individual PRs but instead run them nightly and on commits to master. 9574dd8e748486d9c365c199c94b4b535a7dd250,Sven Klemm,Fri Mar 3 08:11:49 2023 +0100,Adjust ARM64 package test,We only package timescaledb for ubuntu kinetic on PG14 because there are no prebuilt postgres packages available for the other postgres versions. 6be14423d5c51282d5c0d6f9c0b57a4e7123a4b7,Pallavi Sontakke,Fri Mar 3 15:52:34 2023 +0530,Flag test space_constraint.sql.in for release run (#5380),"It was incorrectly flagged as requiring a debug build. Disable-check: force-changelog-changed" 386d31bc6e51913a02b0f0a0f964b36068bb9925,Erik Nordström,Fri Feb 24 18:25:45 2023 +0100,Make copy fetcher more async,"Make the copy fetcher more asynchronous by separating the sending of the request for data from the receiving of the response. By doing that, the async append node can send the request to each data node before it starts reading the first response. This can massively improve the performance because the response isn't returned until the remote node has finished executing the query and is ready to return the first tuple. " 750e69ede1a3161b5a650a7a0dfe1dd9cf1fe1b0,Sotiris Stamokostas,Thu Mar 2 09:54:55 2023 +0200,Renamed size_utils.sql,Renamed: tsl/test/sql/size_utils.sql tsl/test/expected/size_utils.out To: tsl/test/sql/size_utils_tsl.sql tsl/test/expected/size_utils_tsl.out because conflicting with test/sql/size_utils.sql 7887576afaa113bc6e45f5007a6c6bca21b2108d,Jan Nidzwetzki,Thu Feb 23 20:23:32 2023 +0100,Handle MERGE command in reference join pushdown,"At the moment, the MERGE command is not supported on distributed hypertables. This patch ensures that the join pushdown code ignores the invocation by the MERGE command. " e6f6eb3ab8c95b2531401324b97d345ec6fb8ff2,shhnwz,Sun Feb 26 22:18:02 2023 +0530,Fix for inconsistent num_chunks,Different num_chunks values reported by timescaledb_information.hypertables and timescaledb_information.chunks. View definition of hypertables was not filtering dropped and osm_chunks. Fixes #5338 2f7e0433a99d48455e65bcc7e6f63f547b243c7f,gayyappan,Fri Oct 28 16:32:43 2022 -0400,Create index fails if hypertable has foreign table chunk,We cannot create indexes on foreign tables. This PR modifies process_index_chunk to skip OSM chunks 56ce7907d91ba8b5ec80ff2d79c32ceaaae65eaa,Bharathy,Mon Feb 27 09:23:07 2023 +0530,Backport MERGE command specific postgresql code,This patch backports following: 1. Refactor ExecInsert/Delete/Update Backported commit 25e777cf8e547d7423d2e1e9da71f98b9414d59e 2. Backport all MERGE related interfaces and its implementations. Backported commit 7103ebb7aae8ab8076b7e85f335ceb8fe799097c 152ef02d74fbe376e307399564bcb48b23a18150,Fabrízio de Royes Mello,Fri Feb 24 09:58:28 2023 -0300,Fix uninitialized bucket_info variable,The `bucket_info` variable is initialized by `caggtimebucketinfo_init` function called inside the following branch: `if (rte->relkind == RELKIND_RELATION || rte->relkind == RELKIND_VIEW)` If for some reason we don't enter in this branch then the `bucket_info` will not be initialized leading to an uninitialized variable when returning `bucket_info` at the end of the `cagg_validate_query` function. Fixed it by initializing with zeros the `bucket_info` variable when declaring it. Found by coverity scan. 0118e6b9520a4c45363e2c92d4664a91dd77e786,noctarius aka Christoph Engelbert,Mon Jan 9 17:20:38 2023 +0100,Support CAGG names in hypertable_(detailed_)size,This small patch adds support for continuous aggregates to the `hypertable_detailed_size` (and with that `hypertable_size`). It adds an additional check to see if a continuous aggregate exists if a hypertable with the given regclass name isn't found. c8c50dad7eca4f7425bfb9980b872a1c44201ceb,Maheedhar PV,Tue Feb 21 23:03:46 2023 +0530,Post-release fixes for 2.10.0,Bumping the previous version and adding tests for 2.10.0 e0be9eaa281527ee1cc8569ce025cfe64dca8574,Jan Nidzwetzki,Tue Jan 24 09:48:07 2023 +0100,Allow pushdown of reference table joins,"This patch adds the functionality that is needed to perform distributed, parallel joins on reference tables on access nodes. This code allows the pushdown of a join if: * (1) The setting ""ts_guc_enable_per_data_node_queries"" is enabled * (2) The outer relation is a distributed hypertable * (3) The inner relation is marked as a reference table * (4) The join is a left join or an inner join " f12a361ef7de9566113ea79617d49d62597b4bd2,Dmitry Simonenko,Tue Feb 21 19:17:20 2023 +0200,Add timeout argument to the ping_data_node(),This PR introduces a timeout argument and a new logic to the timescale_internal.ping_data_node() function which allows to handle io timeouts for nodes being unresponsive. Fix #5312 09766343997aa903f9d6a1ab14bcfc49a0045864,Sven Klemm,Mon Feb 20 13:31:19 2023 +0100,Set name for COPY insert buffer hash table,Having the hash table named makes debugging easier as the name is used for the MemoryContext used by the hash table. 330bb8f4af761f3277b4b62269dca9c50e882bac,Jan Nidzwetzki,Thu Feb 16 15:38:58 2023 +0100,Added coccinelle rule to find strlcpy on NameData,NameData is a fixed-size type of 64 bytes. Using strlcpy to copy data into a NameData struct can cause problems because any data that follows the initial null-terminated string will also be part of the data. 8a51a76d00b0003ad56bfa39c208da4332bf6042,Mats Kindahl,Mon Feb 20 12:25:50 2023 +0100,Fix changelog message for NameData issue, 0746517c77d4412909776e009143fbaed5c71f0e,Oleg Tselebrovskiy,Tue Feb 14 13:17:04 2023 +0700,Fix some incorrect memory handling,"While running TimescaleDB under valgrind I've found two cases of incorrect memory handling. Case 1: When creating timescaledb extension, during the insertion of metadata there is some junk in memory that is not zeroed before writing there. Changes in metadata.c fix this. Case 2: When executing GRANT smth ON ALL TABLES IN SCHEMA some_schema and deconstructing this statement into granting to individual tables, process of copying names of those tables is wrong. Currently, you aren't copying the data itself, but an address to data on a page in some buffer. There's a problem - when the page in this buffer changes, copied address would lead to wrong data. Changes in process_utility.c fix this by allocating memory and then copying needed relname there. Fixes #5311 " 7e43c702ba2663d56feba1043431d2232d8437c8,Jan Nidzwetzki,Mon Feb 20 09:23:41 2023 +0100,Increase timeout for PostgreSQL in upgrade tests,"The upgrade and downgrade tests are running PostgreSQL in Docker containers. The function 'wait_for_pg' is used to determine if PostgreSQL is ready to accept connections. In contrast to the upgrade tests, the downgrade tests use more relaxed timeout values. The upgrade tests sometimes fail because PostgreSQL cannot accept connections within the configured time range. This patch applies the more relaxed timeout values also to the upgrade script. " 0cbd7407a66c84365f8969d1a197572a57325442,Mats Kindahl,Fri Feb 17 13:16:52 2023 +0100,Get PortalContext when starting job,"When executing functions, SPI assumes that `TopTransactionContext` is used for atomic execution contexts and `PortalContext` is used for non-atomic contexts. Since jobs need to be able to commit and start transactions, they are executing in a non-atomic context hence `PortalContext` will be used, but `PortalContext` is not set when starting the job. This is not a problem for PL/PgSQL executor, but for other executors (such as PL/Python) it would be. This commit fixes the issue by setting the `PortalContext` variable to the portal context created for the portal and restores it (to NULL) after execution. Fixes #5326 " 83fc20f1954c5ff0d22f790cd8ee11309724445a,Jan Nidzwetzki,Fri Feb 17 16:24:45 2023 +0100,Change bgw_main field length to BGW_MAXLEN,"The content of bgw_main is copied into BackgroundWorker. bgw_function_name, which has a length of BGW_MAXLEN. This patch ensures that the size of these fields has the same length. " 91b4a66eb9ae5feea57b94707b4a8e44b6505e9d,Maheedhar PV<58430377+mahipv@users.noreply.github.com>,Mon Feb 20 11:06:05 2023 +0530,Release 2.10.0 (#5324),"This release contains new features and bug fixes since the 2.9.3 release. This release is high priority for upgrade. We strongly recommend that you upgrade as soon as possible. **Features** * #5241 Allow RETURNING clause when inserting into compressed chunks * #5245 Manage life-cycle of connections via memory contexts * #5246 Make connection establishment interruptible * #5253 Make data node command execution interruptible * #5243 Enable real-time aggregation for continuous aggregates with joins * #5262 Extend enabling compression on a continuous aggregrate with 'compress_segmentby' and 'compress_orderby' parameters **Bugfixes** * #4926 Fix corruption when inserting into compressed chunks * #5218 Add role-level security to job error log * #5214 Fix use of prepared statement in async module * #5290 Compression can't be enabled on continuous aggregates when segmentby/orderby columns need quotation * #5239 Fix next_start calculation for fixed schedules" ce85546b5633ba2654fc3c26c658a9381e7ce73d,Sven Klemm,Mon Feb 13 08:35:41 2023 +0100,Fix ABI test,Pin the OpenSSL version we compile against to OpenSSL 1.1 in backwards ABI test because OpenSLL 3 is not available across all supported versions. c7f46393e7406791f49edababe9f058995b152a0,Fabrízio de Royes Mello,Fri Feb 17 19:07:52 2023 -0300,Change usage of term nested to hierarchical,"To don't make developers confused the right name for Continuous Aggregates on top of another Continuous Aggregates is `Hierarchical Continuous Aggregates`, so changed the usage of term `nested` for `hierarchical`. " d50de8a72d1c294323ac4f0599ca1e5e1b89546f,Nikhil Sontakke,Thu Feb 2 10:38:32 2023 +0200,Fix uninitialized `bucket_info.htpartcolno` warning,Found by coverity. 38b71d0e703caa568a38a854e041a916c5601057,Mats Kindahl,Wed Feb 15 15:32:12 2023 +0100,Use NameData and namestrcpy for names,"Using `strlcpy` to copy variables holding PostgreSQL names can cause issues since names are fixed-size types of length 64. This means that any data that follows the initial null-terminated string will also be part of the data. Instead of using `const char*` for PostgreSQL names, use `NameData` type for PostgreSQL names and use `namestrcpy` to copy them rather than `strlcpy`. " 09636092714bbff8689d8d37c3965b99bee0db26,Jacob Champion,Mon Feb 6 09:05:33 2023 -0800,Align GUC initializations with boot values,"As of upstream a73952b795 (16devel), custom GUCs have their initialized values checked against the configured boot value. If they differ, an assertion is tripped. So I've gone through the list and aligned the two sides, adding defaults where they did not previously exist for consistency. To allow for dynamic boot values, the new assertion logic allows the initialized value to be zero, in which case any boot value will be accepted. ts_guc_max_open_chunks_per_insert relies on the value of work_mem, so I've removed its (explicit) initializer entirely. " 917f7804a73ce93bbd704061011516a14123af0e,Jacob Champion,Wed Feb 1 16:41:16 2023 -0800,Remove obsolete usage of AssertArg(),Upstream HEAD (16devel) removed this in commit b1099eca8f. Assert() is identical. f21bf6faacb3c4f9cb2a2af3f76ea3db80cdb949,Jacob Champion,Wed Feb 1 17:00:19 2023 -0800,Fix misuse of CStringGetDatum,"OidFunctionCall1() returns a Datum, not const char *. " 20e468f40c98b45c29452a0a8c490fabf820670d,Jacob Champion,Fri Jan 27 15:14:17 2023 -0800,Fix use of TextDatumGetCString(),"TextDatumGetCString() was made typesafe in upstream HEAD (16devel), so now the compiler catches this. As Tom puts it in ac50f84866: ""TextDatumGetCString(PG_GETARG_TEXT_P(x))"" is formally wrong: a text* is not a Datum. Although this coding will accidentally fail to fail on all known platforms, it risks leaking memory if a detoast step is needed, unlike ""TextDatumGetCString(PG_GETARG_DATUM(x))"" which is what's used elsewhere. " fd66f5936abb6f9271a713e15002be3c4854fb39,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Feb 9 12:55:07 2023 +0400,Warn about mismatched chunk cache sizes,"Just noticed abysmal INSERT performance when experimenting with one of our customers' data set, and turns out my cache sizes were misconfigured, leading to constant hypertable chunk cache thrashing. Show a warning to detect this misconfiguration. Also use more generous defaults, we're not supposed to run on a microwave (unlike Postgres). " 48d4f41d95dccf99751bec04b95d05f9ad4d6f45,Konstantina Skovola,Tue Feb 14 15:07:03 2023 +0200,Run apt package tests against Ubuntu 22.10 (Kinetic Kudu), 9d3866a50efecbed3604622b7b3b981bea0200c3,Zoltan Haindrich,Fri Feb 10 13:22:09 2023 +0000,Accept all compression options on caggs,"Enable to properly handle 'compress_segmentby' and 'compress_orderby' compression options on continous aggregates. ALTER MATERIALIZED VIEW test_table_cagg SET ( timescaledb.compress = true, timescaledb.compress_segmentby = 'device_id' ); Fixes #5161 " d00c1f37213dced5c443baf6fdfc1d298281779d,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Mon Feb 13 11:29:58 2023 +0400,Fix some errors in processing of code coverage data,"We were using mismatched compiler and gcov, which led to either segfaults or errors like ""GCOV_TAG_COUNTER_ARCS mismatch"". Add some cmake code that tries to find the gcov that matches the compiler. This should hopefully fix some of the mysterious missing coverage problems that we've been experiencing for a while. " 9ec11d8af6e56dcf994c7e70c66ae40233099230,Sven Klemm,Fri Feb 10 12:04:32 2023 +0100,"Bump PG version used in CI to 12.14, 13.10, 14.7 and 15.2", ef25fb9ec79d8cbe7d75fdc24fe1e3221f383d15,Sven Klemm,Fri Feb 10 12:10:36 2023 +0100,Add dist_ref_table_join generated test files to .gitignore, 484a4ea3fc5acb58468f19d445b38c7aaf38fb7b,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Fri Feb 10 20:02:46 2023 +0400,Fix our codecov repository yaml,Codecov keeps complaining that it's invalid. ece15d66a438971dcd517d0575c03305e27874dd,Rafia Sabih,Mon Jan 30 14:53:51 2023 +0530,Enable real time aggregation for caggs with joins, 348796f9d9dcfc01716e78886ac1cb3de8623f8f,Konstantina Skovola,Fri Jan 27 17:06:59 2023 +0200,Fix next_start calculation for fixed schedules,"This patch fixes several issues with next_start calculation. - Previously, the offset was added twice in some cases. This is fixed by this patch. - Additionally, schedule intervals with month components were not handled correctly. Internally, time_bucket with origin is used to calculate the next start. However, in the case of month intervals, the timestamp calculated for a bucket is always aligned on the first day of the month, regardless of origin. Therefore, previously the result was aligned with origin by adding the difference between origin and its respective time bucket. This difference was computed as a fixed length interval in terms of days and time. That computation led to incorrect computation of next start occasionally, for example when a job should be executed on the last day of a month. That is fixed by adding an appropriate interval of months to initial_start and letting Postgres handle this computation properly. Fixes #5216 " 756ef68d0af0b3a9a5e7616b1bb7e1ac993c6f98,Sven Klemm,Tue Feb 7 10:57:36 2023 +0100,Fix compression_hypertable ordering reliance,The hypertable_compression test had on implicit reliance on the ordering of tuples when querying the materialized results. This patch makes the ordering explicit in this test. caf79e0f5e3b789f70361f4fd3093afcea3a1da0,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Feb 2 16:28:25 2023 +0400,"When backporting, fetch the main branch with large depth",Apparently in some cases we're getting a shallow repo in the Github Actions. 063a9dae295672fbbfa4087cf524cd35705afe17,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Feb 8 15:08:35 2023 +0400,Improve cost model for data node scans,"1) Simplify the path generation for the parameterized data node scans. 1) Adjust the data node scan cost if it's an index scan, instead of always treating it as a sequential scan. 1) Hard-code the grouping estimation for distributed hypertable, instead of using the totally bogus per-column ndistinct value. 1) Add the GUC to disable parameterized data node scan. 1) Add more tests. " cad2440b58926817912809ef48626bb1b5ca77cd,Zoltan Haindrich,Tue Feb 7 09:53:09 2023 +0000,Compression can't be enabled on caggs,The continuous aggregate creation failed in case segmentby/orderby columns needed quotation. 1eb8aa3f14d1f5c2198b44150944295fbc253c93,Lakshmi Narayanan Sreethar,Tue Feb 7 13:24:09 2023 +0530,Post-release fixes for 2.9.3,Bumping the previous version and adding tests for 2.9.3. 4cb76bc0530d3f0c86aa3ea7e96212de0af5adf7,Rafia Sabih,Thu Jan 26 12:46:53 2023 +0530,Cosmetic changes to create.c, 8132908c97ae469ace776b6cf55bbe3dd8bf8971,Sven Klemm,Sun Feb 5 13:13:08 2023 +0100,Refactor chunk decompression functions,Restructure the code inside decompress_chunk slightly to make core loop reusable by other functions. fb3ad7d6c6dd52813c703f11de6692708b883b5a,Lakshmi Narayanan Sreethar,Fri Feb 3 11:05:05 2023 +0530,Release 2.9.3,This release contains bug fixes since the 2.9.2 release. This release is high priority for upgrade. We strongly recommend that you upgrade as soon as possible. **Bugfixes** * #4804 Skip bucketing when start or end of refresh job is null * #5108 Fix column ordering in compressed table index not following the order of a multi-column segment by definition * #5187 Don't enable clang-tidy by default * #5255 Fix year not being considered as a multiple of day/month in hierarchical continuous aggregates * #5259 Lock down search_path in SPI calls 206056ca12c611b2038b35bfa0eac43372a563c2,Erik Nordström,Fri Feb 3 12:14:13 2023 +0100,Fix dist_hypertable test,A previous change accidentally broke the dist_hypertable test so that it prematurely exited. This change restores the test so that it executes properly. b81033b835bd22802cf59ae5dde5c9bc137d19f5,Erik Nordström,Thu Feb 2 16:50:19 2023 +0100,Make data node command execution interruptible,"The function to execute remote commands on data nodes used a blocking libpq API that doesn't integrate with PostgreSQL interrupt handling, making it impossible for a user or statement timeout to cancel a remote command. Refactor the remote command execution function to use a non-blocking API and integrate with PostgreSQL signal handling via WaitEventSets. Partial fix for #4958. Refactor remote command execution function " c4d8f3530787ee21f8a80176d0b11b18ba585d9e,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Feb 1 17:08:22 2023 +0400,Improve automated backports,"A follow-up for the review comments in the previous PR. 1. Create one backport PR per one source PR, even with multiple commits. 1. Add a comment to the source PR if we fail to backport it for some reason. " 6bc89802161656b00928dbf5fffc08a0e2fdfffb,Konstantina Skovola,Tue Jan 31 10:38:58 2023 +0200,Fix year not multiple of day/month in nested CAgg,"Previously all intervals were converted to seconds using ""epoch"" with date_part. However, this treats a year as 365.25 days to account for leap years, leading to the unexpected situation that a year is not a multiple of a day or a month. Fixed by treating month-only intervals as multiples of 30 days. Fixes #5231 " 9133319081aef92705f1405087822fc281d215d4,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Feb 1 20:38:12 2023 +0400,Fix pylint again,Apparently a new version is out and it complains about Exception being too general. 739fd00bb9184832d6527e253cc44ca1a47585ca,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Thu Jan 26 18:58:22 2023 +0400,Add a workflow for automated backporting,"1. Find the latest release branch 1. For each commit in main and not in release branch (compared by title), find the corresponding PR. 1. If the PR fixes an issue labeled ""bug"", and neither the PR nor the issue are labeled ""no-backport"", cherry-pick the commits from the PR onto the release branch, and create a PR with these changes. " 2a47462fbc930ac6230a83d07100c8a21d040b70,Sven Klemm,Wed Feb 1 07:59:14 2023 +0100,Remove SECURITY DEFINER from get_telemetry_report,We should not broadly make functions security definer as that increases the attack surface for our extension. Especially for our telemetry we should strive to only run it with the minimum required privileges. 44cd71a602ba96029001de6e97a1b44488730080,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Feb 1 13:15:39 2023 +0400,Fix the python code style check,"For an unknown reason, pip started to install an older version of prospector which is incompatible with the current pylint. Require the new prospector version explicitly. " d8f19e57a04d17593df5f2c694eae8775faddbc7,Sven Klemm,Wed Feb 1 08:34:20 2023 +0100,Bump version of setup-wsl github action,The currently used version pulls in Node.js 12 which is deprecated on github. https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/ 789bb26dfbf1aaf85163e5ddfc70fa6dae0894fb,Sven Klemm,Tue Jan 31 13:59:48 2023 +0100,Lock down search_path in SPI calls, f75a51def79796ff7fef58ec950c859fe4e71618,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue Jan 31 15:55:55 2023 +0400,Run yamllint in CI,Helps find errors in GitHub workflows. c0f2ed18095f21ac737f96fe93e4035dbfeeaf2c,Fabrízio de Royes Mello,Mon Jan 30 17:11:22 2023 -0300,Mark cagg_watermark parallel safe,"The `cagg_watermark` function perform just read-only operations so is safe to make it parallel safe to take advantage of the Postgres parallel query. Since 2.7 when we introduced the new Continuous Aggregate format we don't use partials anymore and those aggregate functions `partialize_agg` and `finalize_agg` are not parallel safe, so make no sense don't take advantage of Postgres parallel query for realtime Continuous Aggregates. " e6173d12414735641f76bd783c9dad89f7b8c3c0,Fabrízio de Royes Mello,Mon Jan 30 18:24:10 2023 -0300,Remove unused function prototype, d489ed6f3225931d400ac7b664dbb003e76a7f0b,Erik Nordström,Tue Jan 24 18:18:22 2023 +0100,Fix use of prepared statement in async module,"Broken code caused the async connection module to never send queries using prepared statements. Instead, queries were sent using the parameterized query statement instead. Fix this so that prepared statements are used when created. " 1a3e7ad7d19f2530f25bd30752957c04ac8fa82e,Lakshmi Narayanan Sreethar,Mon Jan 30 12:19:03 2023 +0530,Run dist_move_chunk as a solo test in PG15,"When run in a parallel group, the dist_move_chunk test can get into a deadlock with another test running a 'DROP DATABASE' command. So, mark it as a solo test to disallow it from running in a parallel group. Closes #4972 " 03b740cd709b5e1887352f3ca1d879773ed8a923,Lakshmi Narayanan Sreethar,Mon Jan 30 11:46:11 2023 +0530,Enable telemetry_stats testcase,The telemetry_stats testcase was accidentally disabled by PR #5162. 5d12a3883d80f026252eaac8416143de6cd5653d,Erik Nordström,Mon Jan 30 11:57:05 2023 +0100,Make connection establishment interruptible,"Refactor the data node connection establishment so that it is interruptible, e.g., by ctrl-c or `statement_timeout`. Previously, the connection establishment used blocking libpq calls. By instead using asynchronous connection APIs and integrating with PostgreSQL interrupt handling, the connection establishment can be canceled by an interrupt caused by a statement timeout or a user. Fixes #2757 " 01ea255f2f3226178b42cff597c427ec66c62442,Sven Klemm,Mon Jan 30 13:36:55 2023 +0100,Rename variable in ts_chunk_dispatch_get_chunk_insert_state,The variable new_chunk was misleading since the chunk could be either a new chunk or an existing chunk. b229b3aefd01049ee6fd046f762ad7a9fcb2a6e2,Sven Klemm,Mon Jan 30 10:31:53 2023 +0100,Small decompress_chunk refactor,Refactor the decompression code to move the decompressor initialization into a separate function. cce0e18c3665fec327797b2d469eafb411ee3050,Erik Nordström,Mon Jan 30 11:14:51 2023 +0100,Manage life-cycle of connections via memory contexts,"Tie the life cycle of a data node connection to the memory context it is created on. Previously, a data node connection was automatically closed at the end of a transaction, although often a connection needs to live beyond a single transaction. For example, a connection cache is maintained for data node connections, and, for such cases, a flag was set on a connection to avoid closing it automatically. Instead of tying connections to transactions, they are now life-cycle managed via memory contexts. This simplifies the handling of connections and avoids having to create exceptions to closing connections at transaction end. " 872128438f16cccb60e570a49f9ab51791964a91,Sven Klemm,Mon Jan 30 12:37:33 2023 +0100,Update checkout version in changelog-check,Update checkout to not pull in Node.js 12 which is deprecated on GitHub. https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. 5661ff15236ea6bba8125160eba71538910f1cb9,Mats Kindahl,Wed Jan 25 08:38:56 2023 +0100,Add role-level security to job error log,"Since the job error log can contain information from many different sources and also from many different jobs it is important to ensure that visibility of the job error log entries is restricted to job owners. This commit extend the view `timescaledb_information.job_errors` with role-based checks so that a user can only see entries for jobs that she has permission to view and restrict the permissions to `_timescaledb_internal.job_errors` so that users only can view the job error log through the view. A special case is added so that the superuser and the database owner can see all log entries, even if there is no associated job id with the log entry. Closes #5217 " c89fb2550dce9c35c7b89cfa33c8c813a579f610,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Wed Jan 25 17:45:00 2023 +0400,Run sqlsmith on all commits in main,This would help to gather more statistics. We have some rare assertion failures that happen with SQLSmith like #4185 21a3f8206c0de98932867096637c7d1e3d04d925,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Fri Jan 27 18:46:09 2023 +0400,Run python linter and formatter in CI,Helps find some errors and cosmetic problems. 334864127db53edd711d1053a26aaeeb346f6d33,Sven Klemm,Mon Jan 30 00:15:51 2023 +0100,Stop blocking RETURNING for compressed chunks,Recent refactorings in the INSERT into compressed chunk code path allowed us to support this feature but the check to prevent users from using this feature was not removed as part of that patch. This patch removes the blocker code and adds a minimal test case. 043092a97fc7348f2ddfa08edd758723de93993c,Rafia Sabih,Fri Oct 7 16:25:46 2022 +0200,Fix timestamp out of range,"When start or end for a refresh job is null, then it gives an error while bucketing because start and end are already min and max timestamp value before bucketing. Hence, skip bucketing for this case. Partly fixes #4117 " 684637a172b3a05a7500425cd8ce82cf7aad6092,Bharathy,Tue Jan 24 20:16:12 2023 +0530,Post-release fixes for 2.9.2,Bumping the previous version and adding tests for 2.9.2 552950d221b332c6e0cbc2682e39022f2b972f32,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Fri Jan 20 16:52:00 2023 +0400,Save SQLSmith results to the CI DB,Mostly we are interested in stacktraces and failed queries here. a67b90e977194f3e55c93ed6b3f5d2a671d503c1,Rafia Sabih,Mon Oct 24 13:05:55 2022 +0200,Allow joins in continuous aggregates,Enable the support of having join in the query used for creating the continuous aggregates. It has follwoing restrictions- 1. Join can involve only one hypertable and one normal table 2. Join should be a inner join 3. Join condition can only be equality 9ae3ae33b758fb05ee07c90078d0a20790e196fa,Jan Nidzwetzki,Wed Jan 18 14:47:48 2023 +0100,Add scan plan logic for remote joins,This patch adds the missing functionality to create scan plans for remote joins. Most of the code is a backport from PG Upstream. f211294c613a40a56604a7be6ac0d1663ea49958,Bharathy,Mon Jan 23 15:05:30 2023 +0530,Release 2.9.2,This release contains bug fixes since the 2.9.1 release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * #5114 Fix issue with deleting data node and dropping the database on multi-node * #5133 Fix creating a CAgg on a CAgg where the time column is in a different order of the original hypertable * #5152 Fix adding column with NULL constraint to compressed hypertable * #5170 Fix CAgg on CAgg variable bucket size validation * #5180 Fix default data node availability status on multi-node * #5181 Fix ChunkAppend and ConstraintAwareAppend with TidRangeScan child subplan * #5193 Fix repartition behavior when attaching data node on multi-node d2254cb5c5a8e267abcd8503c4b3f17596d1004c,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Mon Jan 16 19:04:55 2023 +0400,Don't enable clang-tidy by default,"This is aimed at developers. If we enable it by default, it confuses our users and slows down the build for them. " 014b7b9664f828c30b98dd36a563ac6190f08337,Konstantina Skovola,Fri Jan 20 09:33:30 2023 +0200,Move .gitattributes to root and remove build-13,Commit effc8efe148c4ec0048bd7c1dfe0ca01df2afdc9 accidentally placed .gitattributes inside a build-13 directory instead of the root of the project. This commit removes build-13 and fixes the structure. 28dbeaa2ca98c6f86c4b8cee6241841df80855cd,Jan Nidzwetzki,Tue Jan 17 10:36:44 2023 +0100,Add cost estimation for remote joins,This patch adds the missing functionality to estimate the costs of remote joins. Most of the code is a backport from PG Upstream. effc8efe148c4ec0048bd7c1dfe0ca01df2afdc9,Konstantina Skovola,Fri Jun 3 18:26:52 2022 +0300,Add workflow to check for CHANGELOG update, 167625984094c4a62770521d60feeaa1b7acc8d6,Erik Nordström,Tue Jan 17 14:10:01 2023 +0100,Fix repartition behavior when attaching data node,"When attaching a data node and specifying `repartition=>false`, the current number of partitions should remain instead of recalculating the partitioning based on the number of data nodes. Fixes #5157 " 19065bbdf39d2536a227041331adf24ffb38ffdc,Jan Nidzwetzki,Tue Jan 17 08:46:21 2023 +0100,Introduce a FDW option to mark reference tables,"With this patch, the ability to mark reference tables (tables that exist on all data nodes of a multi-node installation) via an FDW option has been added. " 9a2cbe30a102a9d027f832b6c288b0b9adddf62a,Bharathy,Wed Jan 18 10:32:56 2023 +0530,"Fix ChunkAppend, ConstraintAwareAppend child subplan","When TidRangeScan is child of ChunkAppend or ConstraintAwareAppend node, an error is reported as ""invalid child of chunk append: Node (26)"". This patch fixes the issue by recognising TidRangeScan as a valid child. Fixes: #4872 " 8f4fa8e4cca73f11d3892ce6afde04ca104465d2,Mats Kindahl,Mon Jan 16 09:24:32 2023 +0100,Add build matrix to Windows and Linux builds,"Build matrix is missing from the ignore workflows for the Windows and Linux builds, so this commit adds them. " 6aa3d6e2fc7d68b4ef4946ad2b8b186202c946ae,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue Jan 17 13:36:41 2023 +0400,Don't run the APT package test on PRs,We run them nightly on main. 1d3d81491b9116dc00d609ba9e900c1f91064eca,Jan Nidzwetzki,Tue Jan 17 09:07:43 2023 +0100,Update issue workflow actions and used token,This patch updates the used issue workflow actions and ensures that the 'secrets.ORG_AUTOMATION_TOKEN' is used by all actions. 5c897ff75dd014d00484507ee96eb2730fda596c,Dmitry Simonenko,Mon Jan 16 15:20:25 2023 +0200,Fix default data node availability status,"Function alter_data_node() return uninitialized value for ""available"" option when it is not presented in the option list. Fix #5154 " dbe89644b5abf9d5a15f43d8d260b676ea8af0a1,Sven Klemm,Mon Jan 16 10:15:15 2023 +0100,Remove no longer used compression code,The recent refactoring of INSERT into compression chunk made this code obsolete but forgot to remove it in that patch. 73df496c75c206745d44bab2088bf4c6c661ec51,Fabrízio de Royes Mello,Tue Jan 10 15:19:18 2023 -0300,Fix CAgg on CAgg variable bucket size validation,"Previous attempt to fix it (PR #5130) was not entirely correct because the bucket width calculation for interval width was wrong. Fixed it by properly calculate the bucket width for intervals using the Postgres internal function `interval_part` to extract the epoch of the interval and execute the validations. For integer widths use the already calculated bucket width. Fixes #5158, #5168 " ca9d508edeada0deba58beda5f1735631af2b21b,Mats Kindahl,Fri Jan 13 11:33:38 2023 +0100,Add missing ignore files,The workflow ignore files for 32-bit Linux builds and Windows was missing from #5156 so these are added here. 1e7b9bc558b1e6f9d036cf10baabc214f8eeb8d6,Erik Nordström,Thu Dec 22 10:24:42 2022 +0100,Fix issue with deleting data node and dropping database,"When deleting a data node with the option `drop_database=>true`, the database is deleted even if the command fails. Fix this behavior by dropping the remote database at the end of the delete data node operation so that other checks fail first. " 4118a72575d4a1d493ec0884e659e6a7437fd3a0,Fabrízio de Royes Mello,Wed Jan 11 17:29:32 2023 -0300,Remove parallel safe from partialize_agg,Previous PR #4307 mark `partialize_agg` and `finalize_agg` as parallel safe but this change is leading to incorrect results in some cases. Those functions are supposed work in parallel but seems is not the case and it is not evident yet the root cause and how to properly use it in parallel queries so we decided to revert this change and provide correct results to users. Fixes #4922 f36db10826f53b48f6cacd5083342aa79fc8df87,Mats Kindahl,Mon Jan 9 13:53:58 2023 +0100,Do not run regress workflows on benign changes,"If only documentation is changed, the full regression check workflow will still be executed, so this commit will instead skip running the regression workflows if there are only changes to files that will not affect the success of the workflow. " 396bc6def7c50523367f5ca12f688cd6d508d321,Jan Nidzwetzki,Tue Jan 3 15:59:02 2023 +0100,Add join functionality to the deparser code,This patch adds the missing functionality to handle joins to the deparser. Most of the code is a backport from PG Upstream. cfd34f2752213de6cf7b52b0e5ffffb573883ed8,Sven Klemm,Tue Jan 10 12:14:41 2023 +0100,Restructure chunk_dispatch,This patch adjusts the code layout for chunk_dispatch to be similar to the other custom nodes. All the files related to chunk_dispatch are moved into dedicated nodes/chunk_dispatch directory. 06eca172bda323779ba2105814507494e4d81303,Lakshmi Narayanan Sreethar,Tue Jan 10 15:22:48 2023 +0530,Fix telemetry_stats test failure in PG15,The telemetry_stats testcase uses random() with seed(1) to generate the column values on which the hypertable is partitioned. The Postgres commit postgres/postgres@3804539e48 updates the random() implementation to use a better algorithim causing the test to generate a different set of rows in PG15. Due to this the test failed in PG15 as the distrubution stats of the tuples have now changed. Fixed that by creating separate test outputs for PG15 and other releases. Fixes #5037 7d3d260afb1f47ff0e7be5a49bfeb715198a1e79,Lakshmi Narayanan Sreethar,Tue Jan 10 13:40:22 2023 +0530,Skip auto assigning reviewers for draft PRs,Reviewers should be assigned only when the PR is ready for review. 7a6101a441ca4ad02018ffddd225e7abdea4385f,Sven Klemm,Tue Jan 3 22:24:04 2023 +0100,Bump pgspot version used in CI, cd48553de5464505df6006b987cf7970bb48fe76,Fabrízio de Royes Mello,Mon Dec 26 18:03:07 2022 -0300,Fix CAgg on CAgg variable bucket size validation,During the creation of a CAgg on top of another CAgg we check if the bucket width is multiple of the parent and for this arithmetic we made an assumption that picking just the `month` part of the `interval` for variable bucket size was enought. This assumption was wrong for bucket sizes that doesn't have the month part (i.e: using timezones) leading to division by zero error because in that case the `month` part of an `interval` is equal to 0 (zero). Fixed it by properly calculating the bucket width for variable sized buckets. Fixes #5126 73c97524a0130e728eda2d58f82cba05ddad854a,Fabrízio de Royes Mello,Wed Dec 28 15:56:49 2022 -0300,Fix CAgg on CAgg different column order,Creating a CAgg on a CAgg where the time column is in a different order of the original hypertable was raising the following exception: `ERROR: time bucket function must reference a hypertable dimension column` The problem was during the validation we're initializing internal data structure with the wrong hypertable metadata. Fixes #5131 41d6a1f142a80fbb781c77feafba7d75e6707f25,Fabrízio de Royes Mello,Thu Jan 5 16:02:11 2023 -0300,Fix adding column with NULL constraint,"Adding new column with NULL constraint to a compressed hypertable is raising an error but it make no sense because NULL constraints in Postgres does nothing, I mean it is useless and exist just for compatibility with other database systems: https://www.postgresql.org/docs/current/ddl-constraints.html#id-1.5.4.6.6 Fixed it by ignoring the NULL constraints when we check for `ALTER TABLE .. ADD COLUMN ..` to a compressed hypertable. Fixes #5151 " 1751efbaea57be75d813fbf93a0c9b4ec8a7d994,Jan Nidzwetzki,Thu Dec 1 09:57:09 2022 +0100,Improve the PR workflow actions,"This patch includes two changes to the PR handling workflow: (1) It changes the trigger for the workflow to pull_request_target. So, PRs can now also be assigned to reviewers if the PR is opened from external sources. (2) A workflow has been added that automatically assigns PRs to the author. " 2c70dc622e4ca36deb3b1afa3ae51ebe96f4be5d,Alexander Kuzmenkov<36882414+akuzm@users.noreply.github.com>,Tue Jan 3 15:41:14 2023 +0400,Run all the tests on every commit in main,"If we test every commit in master, we can allow GitHub to merge the PRs automatically without requiring a manual rebase on the current master. These rebases are a real time sink. " 64e8e31c6e2a33e5c17bafab7ded90dc23588c32,Sven Klemm,Mon Jan 2 09:33:24 2023 +0100,Improve ASSERT_IS_VALID_CHUNK macro,Currently when ASSERT_IS_VALID_CHUNK fails it is impossible to tell which of the conditions fails without opening the coredump in debugger as all the conditions are ANDed in a single Assert. This patch splits the conditions into individual Asserts so you can immediately see from stacktrace which condition failed. 3a8d294d5862a9c2c3244b0754affab4472e5756,Bharathy,Mon Jan 2 15:30:22 2023 +0530,SELECT from partial compressed chunks crashes,"SELECT from partially compressed chunk crashes due to reference to NULL pointer. When generating paths for DecompressChunk, uncompressed_partial_path is null which is not checked, thus causing a crash. This patch checks for NULL before calling create_append_path(). Fixes #5134 " 1eacb35107eebafb703677c2192936b9cd43541a,Sven Klemm,Thu Dec 22 13:53:04 2022 +0100,Show postgres log in apt package test, b92f36d765e08e3df826f594467c13cd2b935e2d,Sven Klemm,Fri Dec 23 17:56:04 2022 +0100,Add 2.9.1 to update test scripts, 93667df7d883af7f64bea60719902d705333253c,Sven Klemm,Fri Dec 23 07:14:25 2022 +0100,Release 2.9.1,This release contains bug fixes since the 2.9.0 release. This release is high priority for upgrade. We strongly recommend that you upgrade as soon as possible. **Bugfixes** * #5072 Fix CAgg on CAgg bucket size validation * #5101 Fix enabling compression on caggs with renamed columns * #5106 Fix building against PG15 on Windows * #5117 Fix postgres server restart on background worker exit * #5121 Fix privileges for job_errors in update script 0a3615fc706ab4dc986a443eedda8e7412120ace,Konstantina Skovola,Fri Dec 23 12:38:59 2022 +0200,Fix privileges for job_errors table in update script, cdf8676689549a3240126c603283b6012add3f13,Konstantina Skovola,Thu Dec 22 21:23:51 2022 +0200,Fix postgres server restart on background worker exit,Fixed by removing the croak signal handler that was introduced to enable the silent exit of the telemetry job. ca609696d64e3877c9796983de7b3a31dee44fc6,Sven Klemm,Thu Dec 22 10:07:52 2022 +0100,Fix Windows package test, b1314e63f2ff6151ab5becfb105afa3682286a4d,Sven Klemm,Thu Dec 22 12:03:35 2022 +0100,Fix RPM package test for PG15 on centos 7,Installing PG15 on Centos 7 requires the EPEL repository to satisfy the dependencies. 4527f51e7c0e5e923fcf37e4a15d5ee424b4f83b,Sven Klemm,Sat Nov 5 11:13:25 2022 +0100,Refactor INSERT into compressed chunks,This patch changes INSERTs into compressed chunks to no longer be immediately compressed but stored in the uncompressed chunk instead and later merged with the compressed chunk by a separate job. This greatly simplifies the INSERT-codepath as we no longer have to rewrite the target of INSERTs and on-the-fly compress leading to a roughly 2x improvement on INSERT rate into compressed chunk. Additionally this improves TRIGGER-support for INSERTs into compressed chunks. This is a necessary refactoring to allow UPSERT/UPDATE/DELETE on compressed chunks in follow-patches. bd20afccbbdccc1e4929b0f04838d996cd4c0e3a,Sven Klemm,Tue Dec 20 18:03:57 2022 +0100,Add test for PG15 windows package, 1d5167233650c7fda45009bd09e3bc0948b8e473,Sven Klemm,Tue Dec 20 11:38:48 2022 +0100,Skip package downgrade test for PG15,Since we currently only have 1 released version that supports PG15 we cannot test downgrade with PG15. a4806827dca4c87c44e85e29704584144bd2912e,Lakshmi Narayanan Sreethar,Wed Dec 7 13:22:42 2022 +0530,Enable PG15 in various CI checks,This commit enables PG15 in the following workflows: - Regression Linux - ABI test - Memory tests - Coverity - SQLSmith - Additional cron tests Co-authored-by: Bharathy Satish 024b1e1f30db0c58b49eae04ff0b50055b191734,Fabrízio de Royes Mello,Fri Dec 9 16:01:50 2022 -0300,Fix CAgg on CAgg bucket size validation,The bucket size of a Continuous Aggregate should be greater or equal to the parent Continuous Aggregate because there are many cases where you actually want to roll up on another dimension. c5e496a554e9f4d04578f39669108554c22c918d,Bharathy,Tue Dec 20 19:50:39 2022 +0530,Fix column ordering in compressed table index.,"When defining compression segment by parameter using multiple columns, the parameter ordering is not respected for index creation. This patch fixes the issue by maintaining the same order in which user has defined columns in segment by clause. Fixes #5104 " 7d1b74a8c6b0e07c7cd156943f48fc9be16876dc,Sven Klemm,Mon Dec 19 23:51:57 2022 +0100,Use rand() instead of random(),Use rand() instead of random() cause the latter is not available on Windows and postgres stopped backporting it with PG15. Ideally we switch to the crypto functions added in PG15 but that requires a bit more work and this is the minimal change required to get it to build against PG15 on Windows. b74f563a3364d5d65304cd576f7e8d5574287f8d,Sven Klemm,Mon Dec 19 20:42:42 2022 +0100,Run regression tests on pg15 windows, 08bb21f7e69b636a736635e081bf16b1083eacb1,Sven Klemm,Thu Dec 15 16:42:46 2022 +0100,2.9.0 Post-release adjustments,Add 2.9.0 to update test scripts and adjust downgrade scripts for 2.9.0. Additionally adjust CHANGELOG to sync with the actual release CHANGELOG and add PG15 to CI. 27310470bee4790c66d2c75b9e0b58e14145fed4,Alexander Kuzmenkov,Thu Dec 1 21:18:34 2022 +0400,Allow AsyncAppend under IncrementalSort,We forgot to add a case for it. c0e9bb4a30b9d1b15d057992e803f975f2c22358,Sven Klemm,Sat Dec 17 08:57:21 2022 +0100,Fix enabling compression on caggs with renamed columns,On caggs with realtime aggregation changing the column name does not update all the column aliases inside the view metadata. This patch changes the code that creates the compression configuration for caggs to get the column name from the materialization hypertable instead of the view internals. Fixes #5100 317f8f1a9964df67960394145c83be1e1f9d1460,Fabrízio de Royes Mello,Fri Dec 2 11:39:00 2022 -0300,Refactor CAggs on CAggs regression tests,When CAggs on CAggs was introduced in commit 3749953 the regression tests was splited into 6 (six) different test suites. Simplyfied it grouping tests and reduced to just 2 (two) different test suites. It save resources and time because each suite test spawn it own Postgres instance. 854b67774904e1d244aef2dcd49e533e33d52167,Fabrízio de Royes Mello,Fri Dec 16 14:07:09 2022 -0300,Fix broken postgres install on Windows CI,The last minor versions for PG14 (14.6) and PG15 (15.1) were unlisted by chocolatey maintainers due to some issues. Fixed it by hardcoding the 14.5 until the packages become available again. 4694c7d7986f9c6f2d61201c94273e980f45f224,Fabrízio de Royes Mello,Mon Dec 5 14:31:30 2022 -0300,Refactor CAgg migrate regression tests,When CAggs migration was introduced in commit e34218ce the regression tests was splited into 6 (six) different test suites. Simplyfied it grouping tests and reduced to just 2 (two) different test suites. It save resources and time because each suite test spawn it own Postgres instance. e5843dddd3e97fb1fafc82336335abff0696dca1,Mats Kindahl,Mon Dec 5 13:26:51 2022 +0100,Link development and design documentation,"The tree contains a lot of design and architecture documents, but they are not linked together, so this commits adds a few additional README and build a basic structure for the documentation. " 601b37daa84c33917157d9a57a07ca3b5832b630,shhnwz,Thu Oct 13 20:38:05 2022 +0530,Index support for compress chunk,"It allows to override tuplesort with indexscan if compression setting keys matches with Index keys. Moreover this feature has Enable/Disable Toggle. To Disable from the client use the following command, SET timescaledb.enable_compression_indexscan = 'OFF' " cbf51803dd38b1e05a066ce4636ec97fb1057464,Ante Kresic,Thu Dec 15 09:59:42 2022 +0100,Fix index att number calculation,Attribute offset was used by mistake where attribute number was needed causing wrong values to be fetched when scanning compressed chunk index. 3f9e3379a93ee2657d48c26c3d8a1dbe530b441f,Ante Kresic,Wed Dec 14 18:53:00 2022 +0100,Allow BitmapHeapScans on non-parameterized paths,"Planner can decide to use BitmapHeapScans exclusively when scanning compressed chunks. Since we filter out such scans due to previously reported issues, this can lead to no query plan devised when querying compressed chunks. This change allows Bitmap scans on compressed chunks only when it is not parameterized. Fixes #5090 " 3b3846b0ffc2b9d4ab4de505ba50b15044d40a62,Lakshmi Narayanan Sreethar,Tue Dec 13 23:09:24 2022 +0530,Fix assertion failure in cursor_fetcher_rewind,"The cursor_fetcher_rewind method assumes that the data node cursor is rewind either after eof or when there is an associated request. But the rewind can also occur once the server has generated required number of rows by joining the relation being scanned with another regular relation. In this case, the fetch would not have reached eof and there will be no associated requests as the rows would have been already loaded into the cursor causing the assertion in cursor_fetcher_rewind to fail. Fixed that by removing the Assert and updating cursor_fetcher_rewind to discard the response only if there is an associated request. Fixes #5053 " 940626b1d4d6458cfa10559fd56fe57f2117288b,Jan Nidzwetzki,Wed Dec 14 10:00:10 2022 +0100,Fix Git permission issue during CI build,"The new permissions checks to fix CVE-2022-29187 in Git caused some issues in our CI pipeline. This patch adds the checkout directory to Git's ""safe.directory"" setting. " 558688c86f2e02cc8b5721e58294dbe552364772,Mats Kindahl,Tue Dec 13 09:41:56 2022 +0100,Reset baserel cache on invalid hypertable cache,"When popping the hypertable cache stack, it might happen that the hypertable cache was invalidated between the push and the pop. In that case, the baserel cache can contain invalid entries pointing to the now popped hypertable cache, so we reset the baserel cache. Fixes #4795 " df16815009b6353383c720e364e1b3d2c82f8867,Matvey Arye,Wed Dec 7 21:14:06 2022 -0500,Fix memory leak for compression with merge chunks,The RelationInitIndexAccessInfo call leaks cache memory and seems to be unnecessary. 13ac2fc63aa4c21b0f731a74c6197caa0b3d6515,Ildar Musin,Mon Nov 14 17:10:40 2022 +0100,Add a GUC variable to enable/disable OSM for SELECTs, dd65a6b43676459668f100ffdf58ab112ff911d9,Bharathy,Mon Dec 12 16:37:27 2022 +0530,Fix segfault after second ANALYZE,"Issue occurs in extended query protocol mode only where every query goes through PREPARE and EXECUTE phase. First time ANALYZE is executed, a list of relations to be vaccumed is extracted and saved in a list. This list is referenced in parsetree node. Once execution of ANALYZE is complete, this list is cleaned up, however reference to the same is not cleaned up in parsetree node. When second time ANALYZE is executed, segfault happens as we access an invalid memory location. Fixed the issue by restoring the actual value in parsetree node once ANALYZE completes its execution. Fixes #4857 " d92739099b7135712f15d6e0c5e9240d5b430109,Jan Nidzwetzki,Thu Dec 8 15:56:29 2022 +0100,Reduce test group size in sanitizer runs,"When the sanitizer is active, the tests require a lot of memory. If they are run in large parallel groups, out-of-memory situations can occur. This patch reduces the size of parallel groups to 5 when the sanitizer is active. " a01e483bf3552ead468bf6156d734a17f0007d4c,Alexander Kuzmenkov,Thu Dec 8 18:40:35 2022 +0400,More gdb output in CI,Print locals and arguments. c76dfa0acbd19b33c8ef43e84c88a6f69e6ae8ff,Jan Nidzwetzki,Wed Dec 7 20:59:00 2022 +0100,Improve Sanitizer checks,This patch contains two changes to the Sanitizer checks: (1) All logfiles of the Sanitizer will be uploaded to the CI database. (2) The Sanitizer checks are executed on every PR. 323d41b53b2880a7b47b776eb882330790bf530d,Jan Nidzwetzki,Wed Dec 7 22:37:19 2022 +0100,Ensure dist_hypertable is executed as solo test,"The `dist_hypertable` test needs a lot of memory, especially when the sanitizer is enabled. This patch runs this test as a `SOLO_TEST`. This ensures that PostgreSQL does not run into an out-of-memory situation. " 5fd9170b0a4697fc55ffd32275d2348d42b26d6a,Jan Nidzwetzki,Wed Dec 7 17:52:53 2022 +0100,Correct sanitizer log directory,"So far, we have treated the 'log_path' setting of the sanitizer like a file. In fact, this value is used as a prefix for the created log file. Since we expected the exact file name when uploading the sanitizer output, this file was not found and we lost the messages of the sanitizer. This PR changes the behavior. We now treat the setting as a prefix and upload all files created in a new sanitizer output folder. " bfed42c2d371322b7f5bcddfbf43d09042296379,Bharathy,Tue Dec 6 09:50:31 2022 +0530,Fix remote_txn on PG15,"In remote_txn, testcases which kill remote processes on data nodes, tend to rollback transactions and as part of the process, WARNINGS/ERROR are reported to client. Client however reports WARNINGS/ERROR in different order intermittently. This is an issue specific to psql utility. Placing psql in gdb and trying to diagnise the problem does not reproduce the issue. This patch fixes the tests by not reporting WARNINGS. Fixes #4837 " fd42fe76fa37d1dce9e1aa3d6fe6709c797310c5,Erik Nordström,Tue Nov 15 11:52:34 2022 +0100,Read until EOF in COPY fetcher,"Ensure the COPY fetcher implementation reads data until EOF with `PQgetCopyData()`. Also ensure the malloc'ed copy data is freed with `PQfreemem()` if an error is thrown in the processing loop. Previously, the COPY fetcher didn't read until EOF, and instead assumed EOF when the COPY file trailer is received. Since EOF wasn't reached, it required terminating the COPY with an extra call to the (deprecated) `PQendcopy()` function. Still, there are cases when a COPY needs to be prematurely terminated, for example, when querying with a LIMIT clause. Therefore, distinguish between ""normal"" end (when receiving EOF) and forceful end (cancel the ongoing query). " cd4509c2a35d86f6b5d4a2dc8701ef116693cf9a,Sachin,Wed Nov 30 12:32:58 2022 +0000,Release 2.9.0,This release adds major new features since the 2.8.1 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Hierarchical Continuous Aggregates (aka Continuous Aggregate on top of another Continuous Aggregate) * Improve `time_bucket_gapfill` function allowing specifying timezone to bucket * Use `alter_data_node()` to change the data node configuration. This function introduces the option to configure the availability of the data node. This release also includes several bug fixes. **Features** * #4476 Batch rows on access node for distributed COPY * #4567 Exponentially backoff when out of background workers * #4650 Show warnings when not following best practices * #4664 Introduce fixed schedules for background jobs * #4668 Hierarchical Continuous Aggregates * #4670 Add timezone support to time_bucket_gapfill * #4678 Add interface for troubleshooting job failures * #4718 Add ability to merge chunks while compressing * #4786 Extend the now() optimization to also apply to CURRENT_TIMESTAMP * #4820 Support parameterized data node scans in joins * #4830 Add function to change configuration of a data nodes * #4966 Handle DML activity when datanode is not available * #4971 Add function to drop stale chunks on a datanode **Bugfixes** * #4663 Don't error when compression metadata is missing * #4673 Fix now() constification for VIEWs * #4681 Fix compression_chunk_size primary key * #4696 Report warning when enabling compression on hypertable * #4745 Fix FK constraint violation error while insert into hypertable which references partitioned table * #4756 Improve compression job IO performance * #4770 Continue compressing other chunks after an error * #4794 Fix degraded performance seen on timescaledb_internal.hypertable_local_size() function * #4807 Fix segmentation fault during INSERT into compressed hypertable * #4822 Fix missing segmentby compression option in CAGGs * #4823 Fix a crash that could occur when using nested user-defined functions with hypertables * #4840 Fix performance regressions in the copy code * #4860 Block multi-statement DDL command in one query * #4898 Fix cagg migration failure when trying to resume * #4904 Remove BitmapScan support in DecompressChunk * #4906 Fix a performance regression in the query planner by speeding up frozen chunk state checks * #4910 Fix a typo in process_compressed_data_out * #4918 Cagg migration orphans cagg policy * #4941 Restrict usage of the old format (pre 2.7) of continuous aggregates in PostgreSQL 15. * #4955 Fix cagg migration for hypertables using timestamp without timezone * #4968 Check for interrupts in gapfill main loop * #4988 Fix cagg migration crash when refreshing the newly created cagg **Thanks** * @jflambert for reporting a crash with nested user-defined functions. * @jvanns for reporting hypertable FK reference to vanilla PostgreSQL partitioned table doesn't seem to work * @kou for fixing a typo in process_compressed_data_out * @xvaara for helping reproduce a bug with bitmap scans in transparent decompression * @byazici for reporting a problem with segmentby on compressed caggs * @tobiasdirksen for requesting Continuous aggregate on top of another continuous aggregate * @xima for reporting a bug in Cagg migration 29f35da905e99dace4b5cf87b2653a5b5c7e25ee,Sachin,Thu Dec 1 14:37:07 2022 +0000,Fix Github CI failures,"Not specifying alpine version causes libssl version to change, which in turn cause error in downgrade tests as well as ABI tests. This commit also fixes shellcheck failures. Some failing windows tests are addd to ignore list. Co-authored-by: Lakshmi Narayanan Sreethar Co-authored-by: Alexander Kuzmenkov Signed-off-by: Sachin " 1a806e2fde2e6148cd7ffd02752ed59478d426cd,Sven Klemm,Tue Nov 29 10:41:47 2022 +0100,Check for presence of RelationGetSmgr,RelationGetSmgr was backported by upstream to the STABLE branches but is not yet available in any released version so we cannot use pg version to determine presence of RelationGetSmgr. 09c0ba713691f69cdc96c992c44f1654181e9838,Mats Kindahl,Thu Nov 24 11:03:26 2022 +0100,Do not spam log with telemetry problems,"The telemetry process runs on a regular basis and usually does not make a lot of noise, but in a few particular cases, it writes entries to the log unnecessarily. If the telemetry server cannot be contacted, it will print a warning in the log that the server cannot be contacted. Since it is nothing wrong with the system and the telemetry process will try to re-connect at a later time, it is unnecessary to print as a warning. If the telemetry response is malformed, a warning is printed. This is also unnecessary since there is nothing wrong with the system, there is nothing the user can do about it, and this warning can be largely ignored. If the hard-coded telemetry scheme is incorrect, a warning will be printed. This should not normally happen, and if it happens on a running server, there is nothing that can be done to eliminate the error message and the message is unnecessary. When the telemetry job exits, a standard termination message is printed in the log. Although harmless, it is mostly confusing and provide no value to the user. If the telemetry process is attempting to connect, or is connected, to the telemetry server, the telemetry server will wait until the connection gets a timeout before shutting down. This is unnecessary since there is no critical problem in aborting the connection and doing a direct shutdown. This commit turns those warnings into notices and installs a signal handler so that the telemetry job exits silently and abort any outstanding connections. Fixes #4028 " 558da2c5c698aa4237f3ebc35dd12e7c511c2a90,Sven Klemm,Sat Nov 26 11:18:27 2022 +0100,Use RelationGetSmgr instead of rd_smgr,rd_smgr should not be accessed directly but RelationGetSmgr should be used instead. Accessing it directly can lead to segfaults when parallel relcache flushes are happening. https://github.com/postgres/postgres/commit/f10f0ae420ee62400876ab34dca2c09c20dcd030 2d0087a0e7c2be92cd5f4a1e47098d73c5ecde30,Sven Klemm,Sun Nov 27 00:00:37 2022 +0100,Fix segfault in cagg creation,When trying to create cagg on top of any relation that is a neither a hypertable nor a continuous aggregate the command would segfault. This patch changes the code to handle this case gracefully and error out when trying to create a cagg on top of a relation that is not supported. Found by coverity. 35c91204987ccb0161d745af1a39b7eb91bc65a5,Fabrízio de Royes Mello,Thu Nov 24 13:19:36 2022 -0300,Add Hierarchical Continuous Aggregates validations,Commit 3749953e introduce Hierarchical Continuous Aggregates (aka Continuous Aggregate on top of another Continuous Aggregate) but it lacks of some basic validations. Validations added during the creation of a Hierarchical Continuous Aggregate: * Forbid create a continuous aggregate with fixed-width bucket on top of a continuous aggregate with variable-width bucket. * Forbid incompatible bucket widths: - should not be equal; - bucket width of the new continuous aggregate should be greater than the source continuous aggregate; - bucket width of the new continuous aggregate should be multiple of the source continuous aggregate. 83b13cf6f73a74656dde9cc6ec6cf76740cddd3c,Sven Klemm,Fri Nov 25 08:27:45 2022 +0100,Use packaged postgres for sqlsmith and coverity CI,The sqlsmith and coverity workflows used the cache postgres build but could not produce a build by themselves and therefore relied on other workflows to produce the cached binaries. This patch changes those workflows to use normal postgres packages instead of custom built postgres to remove that dependency. 3b94b996f248291caa37de206ef65dd4cc3a56c2,Sven Klemm,Mon Nov 21 17:36:55 2022 +0100,Use custom node to block frozen chunk modifications,This patch changes the code that blocks frozen chunk modifications to no longer use triggers but to use custom node instead. Frozen chunks is a timescaledb internal object and should therefore not be protected by TRIGGER which is external and creates several hazards. TRIGGERs created to protect internal state contend with user-created triggers. The trigger created to protect frozen chunks does not work well with our restoring GUC which we use when restoring logical dumps. Thirdly triggers are not functional for any internal operations but are only working in code paths that explicitly added trigger support. ce778faa117f503231b3cd62d94d6718670ac257,Mats Kindahl,Fri Nov 25 16:10:40 2022 +0100,Updating scheduled run,Updating scheduled run to avoid original creator from being notified. 4a30e5969bf310402f468f2792c48370b144dc3f,Konstantina Skovola,Wed Nov 16 13:42:12 2022 +0200,Fix flaky bgw_db_scheduler_fixed test,"Apply date_trunc to last_successful_finish. Commit 20cdd9ca3ed0c2d62779c4fc61d278a489b4460a mostly fixed the flakiness, but date_trunc was not applied to the last_successful_finish so we still got some flaky runs. " c92e29ba3a64d99173afa97ac6d8910dae49760c,Nikhil Sontakke,Fri Nov 11 19:14:21 2022 +0530,Fix DML HA in multi-node,"If a datanode goes down for whatever reason then DML activity to chunks residing on (or targeted to) that DN will start erroring out. We now handle this by marking the target chunk as ""stale"" for this DN by changing the metadata on the access node. This allows us to continue to do DML to replicas of the same chunk data on other DNs in the setup. This obviously will only work for chunks which have ""replication_factor"" > 1. Note that for chunks which do not have undergo any change will continue to carry the appropriate DN related metadata on the AN. This means that such ""stale"" chunks will become underreplicated and need to be re-balanced by using the copy_chunk functionality by a micro service or some such process. Fixes #4846 " 26e3be1452e47be35795982626c4f549d45158d1,Dmitry Simonenko,Thu Nov 24 18:33:58 2022 +0200,Test dist caggs with an unavailable data node,Add additional test cases to ensure caggs functionality on distributed hypertable during data node being unavailable. Fix #4978 826dcd2721d0f406672da3aca466a130a230e498,Dmitry Simonenko,Thu Nov 24 15:13:17 2022 +0200,Ensure nodes availability using dist restore point,Make sure that a data node list does not have unavailable data nodes when using create_distributed_restore_point() API. Fix #4979 7bfd28a02fb3e4431a8d21c3d87e2ef6d4f807fb,Bharathy,Thu Nov 24 17:50:36 2022 +0530,Fix dist_fetcher_type test on PG15, 5813173e07af7d798e6d29ec10123e361c13ded3,Dmitry Simonenko,Wed Nov 23 16:54:59 2022 +0200,Introduce drop_stale_chunks() function,This function drops chunks on a specified data node if those chunks are not known by the access node. Call drop_stale_chunks() automatically when data node becomes available again. Fix #4848 bdae647f0a73a3941541dbf3b2a0cda0b6f9ecfa,Alexander Kuzmenkov,Tue Nov 22 18:56:11 2022 +0400,Add i386 check results to database,Also add some more gdb commands to give us more context. 26db8666376541f8929a0bfa2cc637092fe1ed99,Alexander Kuzmenkov,Tue Nov 22 18:54:30 2022 +0400,Fix GITHUB_OUTPUT on Windows,"We have to add it to WSLENV and translate it as a path, so that it properly passes the WSL <-> native process boundary. " 40297f1897ab73d5b9b7d960b5ae83338f14e1bf,Konstantina Skovola,Tue Nov 22 15:23:24 2022 +0200,Fix TRUNCATE on hierarchical caggs,"When truncating a cagg that had another cagg defined on top of it, the nested cagg would not get invalidated accordingly. That was because we were not adding a corresponding entry in the hypertable invalidation log for the materialization hypertable of the base cagg. This commit adds an invalidation entry in the table so that subsequent refreshes see and properly process this invalidation. Co-authored-by: name " 35fa891013bcba87515b026e7e7b0abb728f1a54,Fabrízio de Royes Mello,Tue Nov 22 17:35:00 2022 -0300,Add missing gitignore entry,Pull request #4998 introduced a new template SQL test file but missed to add the properly `.gitignore` entry to ignore generated test files. e84a6e2e6523de90b7b41a9bcf80120b85844894,Fabrízio de Royes Mello,Wed Nov 16 14:59:56 2022 -0300,Remove the refresh step from CAgg migration,We're facing some weird `portal snapshot` issues running the `refresh_continuous_aggregate` procedure called from other procedures. Fixed it by ignoring the Refresh Continuous Aggregate step from the `cagg_migrate` and warning users to run it manually after the execution. Fixes #4913 7bc6e56cb7aa24e4142ddb97b51f58754bb46c58,Lakshmi Narayanan Sreethar,Mon Nov 21 14:47:42 2022 +0530,Fix plan_hashagg test failure in PG15,Updated the expected output of plan_hashagg to reflect changes introduced by postgres/postgres@4b160492. 639a5018a3bad4499bce0e1215c24270514234d5,Sven Klemm,Tue Nov 22 11:37:45 2022 +0100,Change time of scheduled CI run,Since we now use the date as a part of the cache key to ensure no stale cache entries hiding build failures we need to make sure we have a cache entry present before workflows that depend on cache are run. 48d9733fda44b869d95c3d3158f8423ab87b64ad,Konstantina Skovola,Mon Nov 21 18:43:43 2022 +0200,Add telemetry for caggs on top of caggs,Commit #4668 introduced hierarchical caggs. This patch adds a field `num_caggs_nested` to the telemetry report to include the number of caggs defined on top of other caggs. fd84bf42a5c691d6e9f3e928f95830b81a85f17a,Jan Nidzwetzki,Tue Nov 22 08:53:46 2022 +0100,Use Ensure in get_or_add_baserel_from_cache,"This patch changes an Assert in get_or_add_baserel_from_cache to an Ensure. Therefore, this check is also performed in release builds. This is done to detect metadata corruptions at an early stage. " a5b8c9b084aa6edd5eb8aeb38cc0ab16ecfd8ac4,Fabrízio de Royes Mello,Fri Nov 18 16:26:05 2022 -0300,Fix caggs on caggs tests on PG15,PR #4668 introduced the Hierarchical Continuous Aggregates (aka Continuous Aggregate on top of another Continuous Aggregate) but unfortunately we miss to fix the regression tests on PG15. 89cede81bdae16f82125e75fdd4e891ad0157ed5,Bharathy,Mon Nov 21 13:31:45 2022 +0530,Fix PG15 specific tests., 3b5653e4ccdb0dfbb7cef6e55c7fb49447bfd8e7,Fabrízio de Royes Mello,Fri Nov 18 16:35:02 2022 -0300,Ignore trailing whitespaces changes in git blame, a4356f342f1732857a1d8057f71219b50f1919b2,Fabrízio de Royes Mello,Wed Nov 16 17:00:40 2022 -0300,Remove trailing whitespaces from test code, b1742969d09114cd5138abe16f5c70dfbf864e4f,Fabrízio de Royes Mello,Wed Nov 16 16:34:53 2022 -0300,Add SQL test files to trailing whitespace CI check,In commit 1f807153 we added a CI check for trailing whitespaces over our source code files (.c and .h). This commit add SQL test files (.sql and .sql.in) to this check. 3749953e9704e45df8f621607989ada0714ce28d,Fabrízio de Royes Mello,Wed Oct 5 18:45:40 2022 -0300,Hierarchical Continuous Aggregates,"Enable users create Hierarchical Continuous Aggregates (aka Continuous Aggregates on top of another Continuous Aggregates). With this PR users can create levels of aggregation granularity in Continuous Aggregates making the refresh process even faster. A problem with this feature can be in upper levels we can end up with the ""average of averages"". But to get the ""real average"" we can rely on ""stats_aggs"" TimescaleDB Toolkit function that calculate and store the partials that can be finalized with other toolkit functions like ""average"" and ""sum"". Closes #1400 " fd11479700a1d91f78b20ba57cfddd43836fb72a,Jan Nidzwetzki,Wed Nov 2 11:32:48 2022 +0100,Speed up get_or_add_baserel_from_cache operation,"Commit 9f4dcea30135d1e36d1c452d631fc8b8743b3995 introduces the get_or_add_baserel_from_cache function. It contains a performance regression, since an expensive metadata scan (ts_chunk_get_hypertable_id_by_relid) is performed even when it could be avoided. " 380464df9bb35784853f186b9e268cd9d50e442b,Jan Nidzwetzki,Fri Oct 28 16:09:52 2022 +0200,Perform frozen chunk status check via trigger,"The commit 9f4dcea30135d1e36d1c452d631fc8b8743b3995 introduces frozen chunks. Checking whether a chunk is frozen or not has been done so far in the query planner. If it is not possible to determine which chunks are affected by a query in the planner (e.g., due to a cast in the WHERE condition), all chunks are checked. This leads (1) to an increased planning time and (2) to the situation that a single frozen chunk could reject queries, even if the frozen chunk is not addressed by the query. " 7c32ceb0737b0d959eac0d866447918c4453844a,Lakshmi Narayanan Sreethar,Thu Nov 17 18:13:41 2022 +0530,Fix perl test import in PG15,Removed an invalid import from 007_healthcheck.pl test. Also enabled all the perl tests and a couple of others in PG15. b9ca06d6e3d6a0d8ca924917fddb3e78e6f14a64,gayyappan,Mon Nov 14 09:56:24 2022 -0500,Move freeze/unfreeze chunk to tsl,Move code for freeze and unfreeze chunk to tsl directory. bfa641a81c2db82366baf7848c0723ee6eae7f90,Bharathy,Thu Nov 17 18:44:09 2022 +0530,INSERT .. SELECT on distributed hypertable fails on PG15,INSERT .. SELECT query containing distributed hypertables generates plan with DataNodeCopy node which is not supported. Issue is in function tsl_create_distributed_insert_path() where we decide if we should generate DataNodeCopy or DataNodeDispatch node based on the kind of query. In PG15 for INSERT .. SELECT query timescaledb planner generates DataNodeCopy as rte->subquery is set to NULL. This is because of a commit in PG15 where rte->subquery is set to NULL as part of a fix. This patch checks if SELECT subquery has distributed hypertables or not by looking into root->parse->jointree which represents subquery. Fixes #4983 1e3200be7db529560d12a0a2323ed962e8614301,Sachin,Mon Nov 7 08:50:22 2022 +0000,USE C function for time_bucket() offset,Instead of using SQL UDF for handling offset parameter added ts_timestamp/tz/date_offset_bucket() which will handle offset 839e42dd0c1d132a230760681e135706d3b7c971,Lakshmi Narayanan Sreethar,Thu Nov 10 02:02:18 2022 +0530,Use async API to drop database from delete_data_node,PG15 introduced a ProcSignalBarrier mechanism in drop database implementation to force all backends to close the file handles for dropped tables. The backend that is executing the drop database command will emit a new process signal barrier and wait for other backends to accept it. But the backend which is executing the delete_data_node function will not be able to process the above mentioned signal as it will be stuck waiting for the drop database query to return. Thus the two backends end up waiting for each other causing a deadlock. Fixed it by using the async API to execute the drop database command from delete_data_node instead of the blocking remote_connection_cmdf_ok call. Fixes #4838 1b65297ff7ed21ae2b338a88cf16d151903178f4,Alexander Kuzmenkov,Wed Nov 16 13:32:26 2022 +0400,Fix memory leak with INSERT into compressed hypertable,We used to allocate some temporary data in the ExecutorContext. 7e4ebd131f0e6716bdea3447ff5e801eb71314bd,Alexander Kuzmenkov,Tue Nov 15 19:18:13 2022 +0400,Escape the quotes in gdb command, 676d1fb1f1f82f34ae5cdad96022796682215f27,Alexander Kuzmenkov,Tue Nov 15 18:03:05 2022 +0400,Fix const null clauses in runtime chunk exclusion,"The code we inherited from postgres expects that if we have a const null or false clause, it's going to be the single one, but that's not true for runtime chunk exclusion because we don't try to fold such restrictinfos after evaluating the mutable functions. Fix it to also work for multiple restrictinfos. " f3a3da780440ec204a4c979bc663aa298c80c0d3,Mats Kindahl,Wed Nov 2 22:29:11 2022 +0100,Take advisory lock for job tuple,"Job ids are locked using an advisory lock rather than a row lock on the jobs table, but this lock is not taken in the job API functions (`alter_job`, `delete_job`, etc.), which appears to cause a race condition resulting in addition of multiple rows with the same job id. This commit adds an advisory `RowExclusiveLock` on the job id while altering it to match the advisory locks taken while performing other modifications. Closes #4863 " 51e5f319184050c36599a0517fdf4b2a6a23fd62,Ante Kresic,Thu Nov 10 11:56:01 2022 +0100,Update compress chunk interval on compressed data,Compress chunk interval is set using an ALTER TABLE statement. This change makes it so you can update the compress chunk interval while keeping the rest of the compression settings intact. Updating it will only affect chunks that are compressed and rolled up after the change. 8b6eb9024f6995e4615098e9c166aabb4c77f2e5,Sven Klemm,Fri Nov 11 22:19:01 2022 +0100,Check for interrupts in gapfill main loop,Add CHECK_FOR_INTERRUPTS() macro to gapfill main loop. 87756bcff955cb3916c88ef19c3992d5c25183a7,Sven Klemm,Thu Nov 10 19:13:26 2022 +0100,Bump postgres versions used in CI,"Use PG 12.13, 13.9 and 14.6 in our CI " 2f237e6e57e5ac66c126233d66969a1f674ffaa4,Sven Klemm,Thu Nov 10 19:10:49 2022 +0100,Add Enterprise Linux 9 packages to RPM package test, 33531212b233027d10ba7887ef9754dad14c35b2,Lakshmi Narayanan Sreethar,Mon Nov 14 19:30:33 2022 +0530,Disable dist_move_chunk test in PG15,The dist_move_chunk causes the CI to hang when compiled and run with PG15 as explained in #4972. Also fixed schema permission issues in data_node and dist_param tests. 8afdddc2da6020e56ba9128d8f36d3cb5b7775f6,Bharathy,Tue Nov 15 06:18:15 2022 +0530,Deprecate continuous aggregates with old format,"This patch will report a warning when upgrading to new timescaledb extension, if their exists any caggs with partial aggregates only on release builds. Also restrict users from creating cagss with old format on timescaledb with PG15. " b085833fdac4d6ffeafef320227e5e8c53f23a1b,Mats Kindahl,Fri Nov 11 10:34:59 2022 +0100,Print errors in release builds for jobs,Old assertions checking integrety of metadata for jobs will print error message in release builds instead of continuing executing with bad metadata. 121631c70fdf08c285ac9201d0767ed99e4891c8,Alexander Kuzmenkov,Mon Nov 14 18:09:54 2022 +0400,Support parameterized data node scans in joins,"This allows us to perform a nested loop join of a small outer local table to an inner distributed hypertable, without downloading the entire hypertable to the access node. " 9964ba8ba6440a7df9f7799f880fb144db1858a6,Alexander Kuzmenkov,Mon Nov 14 18:21:10 2022 +0400,Remove accidental debug output,Was added in # 4890 0d30155b26da1711f7f0b824f1fa7a7302ebf268,Alexander Kuzmenkov,Mon Nov 14 16:59:59 2022 +0400,Upload test results into a database,This will help us find the flaky tests or the rare failures. 0360812e3ccc627ab6e1cb6aab8e95f925956593,Alexander Kuzmenkov,Wed Nov 9 20:01:58 2022 +0400,Simplify llvm configuration for linux/macos builds,Set it only in the matrixbuilder. feb09c54e9da991baec6ecb11d9bd661b20a121b,Alexander Kuzmenkov,Wed Nov 9 20:03:49 2022 +0400,Rebuild cached PG daily and on config changes,Otherwise it's easy to break these builds and not notice it until much later. 141e114ccb5207bd5ba5f99ee7d46d1736bad899,Mats Kindahl,Mon Nov 14 12:38:45 2022 +0100,Fix race in bgw_db_scheduler_fixed,"When deleting a job in the test, the job does not necessarily terminate immediately, so wait for a log entries from the job before checking the jobs table. Fixed #4859 " e2b7c76c9c1d53edf15ea3d4d01d666a675a7c5f,Markos Fountoulakis,Thu Nov 10 10:55:22 2022 +0200,Disable MERGE when using hypertables,Fixes #4930 Co-authored-by: Lakshmi Narayanan Sreethar 9e276c58ee7b99ae479f4a544de12bbbcc84c211,Fabrízio de Royes Mello,Fri Nov 11 14:42:22 2022 -0300,"Revert ""Upload test results into the database""",This reverts commit 252cefb509153fadcb32741a27ec3fa977487049 because it broke our CI globally. 6ae192631edb3c5a852b0ac2d0441e5b3e2b8a12,Fabrízio de Royes Mello,Thu Nov 10 09:56:09 2022 -0300,Fix CAgg migration with timestamp without timezone,It was a leftover from the original implementation where we didn't add tests for time dimension using `timestamp without timezone`. Fixed it by dealing with this datatype and added regression tests. Fixes #4956 252cefb509153fadcb32741a27ec3fa977487049,Alexander Kuzmenkov,Wed Nov 9 20:06:32 2022 +0400,Upload test results into the database,This will help us find the flaky tests or the rare failures. f13214891cf47fed0db6183ce6b56d9273865de3,Erik Nordström,Fri Oct 14 11:09:53 2022 +0200,Add function to alter data nodes,"Add a new function, `alter_data_node()`, which can be used to change the data node's configuration originally set up via `add_data_node()` on the access node. The new functions introduces a new option ""available"" that allows configuring the availability of the data node. Setting `available=>false` means that the node should no longer be used for reads and writes. Only read ""failover"" is implemented as part of this change, however. To fail over reads, the alter data node function finds all the chunks for which the unavailable data node is the ""primary"" query target and ""fails over"" to a chunk replica on another data node instead. If some chunks do not have a replica to fail over to, a warning will be raised. When a data node is available again, the function can be used to switch back to using the data node for queries. Closes #2104 " fe6731cead8625ed71c3feb9ed5a17679bffe837,Sven Klemm,Sat Nov 5 16:39:06 2022 +0100,Fix compress_segmentby in isolation tests,compress_segmentby should never be on a column with random() values as that will result in very inefficient compression as the batches will only have 1 tuple each. 4f9eef3211166b22424beb51a87dfe7c329b60c7,Jan Nidzwetzki,Thu Nov 10 14:13:47 2022 +0100,Print correct variables in downgrade test script,The downgrade script has printed a message in which the same variable is used for the upgrade and the downgrade version. This patch corrects the output and uses the correct variables. 6ad28248f364d2b78ecb9495c37bc13a99969eb0,Alexander Kuzmenkov,Wed Nov 9 19:59:15 2022 +0400,Change the flaky check to use output files,"This is simpler, accounts for both sql and isolation tests, changes in included files, and versioned tests. " e4ba2bcf560568ae68f3775c058f0a8d7f7c0501,Sven Klemm,Wed Nov 9 09:29:36 2022 +0100,Remove debian 9 from packages tests.,Debian 9 is EOL since July 2022 so we won't build packages for it anymore and can remove it from CI. 1f807153085b37c4c48f955ddbcc575bec10c6d2,Mats Kindahl,Tue Nov 8 13:59:18 2022 +0100,Check for trailing whitespace,"As a result of editing, trailing whitespace is often resulting and since some editors automatically remove trailing whitespace this creates diffs with more changed lines than necessary. Add a check that files do not have trailing whitespace and fail if there are. " 9744b4f3bc9859bed7a434143b46c9ae298999bf,Sven Klemm,Tue Nov 1 10:25:46 2022 +0100,Remove BitmapScan support in DecompressChunk,We don't want to support BitmapScans below DecompressChunk as this adds additional complexity to support and there is little benefit in doing so. This fixes a bug that can happen when we have a parameterized BitmapScan that is parameterized on a compressed column and will lead to an execution failure with an error regarding incorrect attribute types in the expression. bfef3173bc89b8470fbc423ceb5a49ee138c82e3,Fabrízio de Royes Mello,Mon Nov 7 17:06:13 2022 -0300,Refactor CAgg migration code to use job API,The current implementation update the jobs table directly and to make it consistent with other parts of the code we changed it to use the `alter_job` API instead to enable and disable the jobs during the migration. This refactoring is related to #4863. 2a64450651a62114b2de3811f772eac44232ee9c,Bharathy,Mon Nov 7 21:42:22 2022 +0530,Add new tests to gitignore list,"Since new tests specific to PG15 were added, these tests which generated .sql files needs to be added to .gitnore " 3a9688cc97bb726d31b5c381ece9e1a5da756cc9,Bharathy,Mon Nov 7 20:05:30 2022 +0530,Extra Result node on top of CustomScan on PG15,"On PG15 CustomScan by default is not projection capable, thus wraps this node in Result node. THis change in PG15 causes tests result files which have EXPLAIN output to fail. This patch fixes the plan outputs. Fixes #4833 " b95576550cae05c0db1b520b5f0ae1668312e949,Mats Kindahl,Fri Nov 4 15:01:57 2022 +0100,Add printout for multiple jobs with same job_id,"We have a rare condition where a debug build asserts on more than one job with the same job id. Since it is hard to create a reproduction, this commit adds a printout for those conditions and print out all the jobs with that job id in the postgres log. Part of #4863 " 3059290beaa4e722e77dada386a9dd29d59c51df,Sven Klemm,Fri Nov 4 20:44:53 2022 +0100,Add new chunk state CHUNK_STATUS_COMPRESSED_PARTIAL,A chunk is in this state when it is compressed but also has uncompressed data in the uncompressed chunk. Individual tuples can only ever exist in either area. This is preparation patch to add support for uncompressed staging area for DML operations. 5b0bff384bd85db1dd727050e1782ac618a4b553,Sven Klemm,Fri Nov 4 20:35:49 2022 +0100,Improve InvalidOid coccinelle check,The initial version of the check did not include a detailed message about the code failure in the CI output and did not check for expressions with operands in wrong order. f289ef8828c71ee5cb62bcc30fce92b607912aa5,Sven Klemm,Fri Nov 4 20:40:58 2022 +0100,Remove unused function ts_chunk_is_uncompressed_or_unordered, 12745c880668825c8e3b966c43e3d4cdcb4d2900,Bharathy,Mon Nov 7 09:09:04 2022 +0530,Fix error: variable not found in subplan target list on PG15,"On PG15 new flag CUSTOMPATH_SUPPORT_PROJECTION is introduced. This flag tells if a planner node is projection capable or not. CustomScan created in TimescaleDB by default is not projection capable, this causes CustomScan node to be wrapped around Result node. Update query on a hypertable has a logic which is based on assumption that ""ModifyTable"" plan nodes lefttree should be CustomScan node. With PG15 this assumption is broken which causes ""ERROR: variable not found in subplan target list"". Fixes #4834 " 6c73b61b998293f750ed3392f8afa35b97a63b66,Fabrízio de Royes Mello,Thu Nov 3 15:10:34 2022 -0300,Fix orphan jobs after CAgg migration,When using `override => true` the migration procedure rename the current cagg using the suffix `_old` and rename the new created with suffix `_new` to the original name. The problem is the `copy polices` step was executed after the `override` step and then we didn't found the new cagg name because it was renamed to the the original name leading the policy orphan (without connection with the materialization hypertable). Fixed it by reordering the steps executin the `copy policies` before the `override` step. Also made some ajustments to properly copy all `bgw_job` columns even if this catalog table was changed. Fixes #4885 1847f64a2f766c5ff2936fa39c568fb4f82ae702,Alexander Kuzmenkov,Wed Nov 2 11:12:37 2022 +0400,Fix sanitizer builds with -Wclobbered,They use GCC 10 which has some other set of false positives. c54cf3ea56de9e05852ea54f90c55c5a221fa1bb,Konstantina Skovola,Fri Oct 14 10:43:18 2022 +0300,Add job execution statistics to telemetry,"This patch adds two new fields to the telemetry report, `stats_by_job_type` and `errors_by_sqlerrcode`. Both report results grouped by job type (different types of policies or user defined action). The patch also adds a new field to the `bgw_job_stats` table, `total_duration_errors` to separate the duration of the failed runs from the duration of successful ones. " f1535660b04251cfb6cdfd4435cfa5185420f1f7,Fabrízio de Royes Mello,Tue Nov 1 19:07:51 2022 -0300,Honor usage of OidIsValid() macro,Postgres source code define the macro `OidIsValid()` to check if the Oid is valid or not (comparing against the `InvalidOid` type). See `src/include/c.h` in Postgres source three. Changed all direct comparisons against `InvalidOid` for the `OidIsValid` call and add a coccinelle check to make sure the future changes will use it correctly. 7dd45cf348576f3afec90fe2ee282e97dda0ed26,Fabrízio de Royes Mello,Fri Oct 28 15:29:10 2022 -0300,Fix failure resuming a CAgg migration,"Trying to resume a failed Continuous Aggregate raise an exception that the migration plan already exists, but this is wrong and the expected behaviour should be resume the migration and continue from the last failed step. " 08791cad4316b7cb567176fb1ea46f28a6f66fd8,Alexander Kuzmenkov,Thu Nov 3 13:41:00 2022 +0400,Disable llvm on macos,I accidentally re-enabled it when adding the flaky check. 2475c1b92ff9e566cb3858924ca153aa14ab4a25,Ante Kresic,Fri Sep 16 15:45:07 2022 +0200,Roll up uncompressed chunks into compressed ones,"This change introduces a new option to the compression procedure which decouples the uncompressed chunk interval from the compressed chunk interval. It does this by allowing multiple uncompressed chunks into one compressed chunk as part of the compression procedure. The main use-case is to allow much smaller uncompressed chunks than compressed ones. This has several advantages: - Reduce the size of btrees on uncompressed data (thus allowing faster inserts because those indexes are memory-resident). - Decrease disk-space usage for uncompressed data. - Reduce number of chunks over historical data. From a UX point of view, we simple add a compression with clause option `compress_chunk_time_interval`. The user should set that according to their needs for constraint exclusion over historical data. Ideally, it should be a multiple of the uncompressed chunk interval and so we throw a warning if it is not. " 8d1755bd78008dd894510b38fa609458b1beb9f8,Sutou Kouhei,Wed Nov 2 11:29:29 2022 +0900,Fix a typo in process_compressed_data_out(), c06b647680dabac74cd027735ad7176f855c24a0,Bharathy,Tue Nov 1 19:39:38 2022 +0530,pg_dump on PG15 does not log messages with log level set to PG_LOG_INFO.,"Version 15 pg_dump program does not log any messages with log level < PG_LOG_WARNING to stdout. Whereas this check is not present in version 14, thus we see corresponding tests fail with missing log information. This patch fixes by supressing those log information, so that the tests pass on all versions of postgresql. Fixes #4832 " 840f144e09640fcf981706eab30c387ce59a1fd1,Alexander Kuzmenkov,Thu Oct 27 18:04:27 2022 +0400,Enable and fix -Wclobbered,The one in job_stat.c could probably lead to errors. 1cc8c15cad938b432995217bcdaada5030a34b9f,Alexander Kuzmenkov,Thu Oct 27 15:59:31 2022 +0400,Do not clobber the baserel cache on UDF error,The baserel cache should only be allocated and freed by the top-level query. 20cdd9ca3ed0c2d62779c4fc61d278a489b4460a,Sven Klemm,Tue Nov 1 12:22:59 2022 +0100,Fix bgw_db_scheduler_fixed flakyness,Depending on date boundaries the number of chunks produced for the cagg hypertable was not constant resulting on flaky tests on certain days. 3d30f07bf48d5e0e186535ede438d8e4003a7664,Sven Klemm,Tue Nov 1 11:11:48 2022 +0100,Swap lookup order for clang-format,Look for the binary with exact version before looking for the generic name to prevent failure when clang-format is lower then required version but clang-format-14 exists. d51fefb74405a519c51dd0f10a89d75c816399c4,Alexander Kuzmenkov,Mon Oct 31 18:23:14 2022 +0400,Add the Flaky Check,It runs new or changed tests multiple times to find flakiness. e08e0a59db5f8cee24350f79971a0271ee4269d6,gayyappan,Tue Oct 25 10:43:33 2022 -0400,Add hook for chunk creation,"After data is tiered using OSM, we cannot insert data into the same range. Need a callback that can be invoked by timescaledb to check for range overlaps before creating a new chunk " 5b2d9d5a106d6dc381d16758780893d5bcbc35dd,Alexander Kuzmenkov,Fri Oct 28 17:06:46 2022 +0400,Enable -Wnewline-eof,"It enforces the newline at end of file, which is required by the C standard. " c48b1231a884bebe061624281f30934037885c9b,gayyappan,Thu Oct 27 18:47:18 2022 -0400,Allow foreign tables in hypetable modify path,OSM chunks are foreign tables. Modify assert to allow updates/deletes on hypertables with foreign table chunks. dfbf030af7ceec8bf6b41ae02e87414bc72bd318,Jan Nidzwetzki,Thu Oct 27 16:26:22 2022 +0200,Remove no-activity label on issue activity,"So far, only the ""need-more-info"" label was removed when there was activity on an issue. This PR ensures that the ""no-activity"" label is also removed. " 85f5efdc8f86630039faeaa3523a2ddbe16f1974,Alexander Kuzmenkov,Thu Oct 27 19:00:01 2022 +0400,Fix -Wsign-compare with PG 15,"It changed the type of Var.varno from Index to int. I'm starting to wonder if it was a good idea to enable this warning, but maybe we can give it the last try. " d8e892a658d9b3a3071764be745465c331f86d52,Alexander Kuzmenkov,Wed Oct 26 17:25:22 2022 +0400,Save postgres logs on Windows in CI,We don't save them currently. 8d1e165d7f2cd660fe5ee82d34463a4f9aa420a6,Fabrízio de Royes Mello,Wed Oct 26 16:44:59 2022 -0300,Refactor Continuous Aggregate catalog code,Get rid of `GETSTRUCT` to fill the form data and use `heap_deform_tuple` instead. This is necessary specially when you have variable lenght fields and/or fields that accept NULL values. This refactoring will be specially usefull in a following PR for Nested Continuous Aggregates where we'll add a new metadata to the catalog that can accept NULL values. Also refactor the rename view and schema code paths improving the readability and maintainability. 9b157d5438049dc2a267dd3a5ed57ef33b8820f4,Alexander Kuzmenkov,Thu Oct 27 15:17:26 2022 +0400,Don't use docker for clang-format,It's not something we normally do for the developer tools. 313845a88245312fdb9d19f26c29d2e027808a3e,Alexander Kuzmenkov,Thu Oct 27 15:08:09 2022 +0400,Enable -Wextra,Our code mostly has warnings about comparison with different signedness. 864da20cee764b33b149444d561af4f60653eb64,Alexander Kuzmenkov,Fri Oct 21 17:04:41 2022 +0400,Build on Ubuntu 22.04,It has newer GCC which should detect more warnings. 2e7d7ee960d2a0506ff415e1f4f50b74a51df632,Fabrízio de Royes Mello,Wed Oct 26 11:08:24 2022 -0300,Remove useless message from tsl_cagg_try_repair,The PR #3899 introduced a new function named `tsl_cagg_try_repair` to try to fix buggy Continuous Aggregates that lead to segfault on a select query. It added an INFO message when skipping the check for Continuous Aggregate that don't have partials and it's annoying during the upgrade/downgrade the extension specially if you have many Continuous Aggregate. Remove this message because it's completely useless. da9af2c05d1be2b487c2d3f873a0de49bdb2740f,Alexander Kuzmenkov,Fri Oct 21 22:39:38 2022 +0400,Do not cache the classify_relation result,"It depends on the context, not only on the relation id. The same chunk can be expanded both as a child of hypertable and as an independent table. " 498b8af261f7c5e93852448db55ca418e86ae6e2,Dmitry Simonenko,Wed Oct 26 13:00:21 2022 +0300,Block multi-statement DDL command in one query,Ensure that queries involving several distributed DDL commands in one query string are blocked. Fix #4818 cef8c462dbfa46e68896976e9bc1e7ecd5e46162,Sven Klemm,Sun Oct 23 16:38:44 2022 +0200,Fix postgres version assert in telemetry,The telemetry code that reads the postgres version was not updated when support for older postgres version was dropped so we introduce a new macro PG_MAJOR_MIN which is the oldest pg major version we support. f05545883bc3bd338936446d7007fc53759a0cf8,Jan Nidzwetzki,Mon Oct 24 12:11:15 2022 +0200,Add a Coccinelle test for PG 12.3+ ereport syntax,"This PR adds a Coccinelle test for ereport(..) calls that use the PG 12.3+ syntax (postgres/postgres@a86715451653c730d637847b403b0420923956f7). We had some of these calls in the past, which we had to fix afterward ( see #4733, #4809, #4871). This Coccinelle patch detects such calls and reports them in the CI run. " fabb01fdaf611c2deb66ce53e133a0ca3efb3e0c,Konstantina Skovola,Wed Oct 19 08:58:35 2022 +0300,Add telemetry for fixed schedule jobs,Previous commit #4664 introduced the ability to execute background jobs on a fixed schedule. This commit updates our telemetry data to include the number of jobs scheduled to execute on a fixed schedule vs the number registered to execute on a drifting schedule. 06f2e57c50fc54b899565fc156aac7ba72b91e03,Fabrízio de Royes Mello,Fri Oct 21 15:38:13 2022 -0300,Fix ereport call in job schedule for PG 12.0,"Since PG 12.3 the `ereport` syntax changed and the commit 54ed0d introduced and `ereport` call that works just using newer PG versions. Changed the `ereport` call a PG 12.0 compatible syntax. CI failure: https://github.com/timescale/timescaledb/actions/runs/3292571583/jobs/5428090238 Related PRs: #4733, #4809 " 39c9921947eec033081003d4ee4b74604476a9c6,Alexander Kuzmenkov,Fri Oct 21 18:43:26 2022 +0400,Fix flaky copy_memory_usage tests,"The changes from e555eea lead to flakiness. They are a leftover of earlier version and probably not needed anymore. The original version is also still flaky on Windows, so use linear regression to tell if the memory usage is increasing. Verified to still fail on 2.7.x " 25628e037a225857eee2f17f1d2a3a88c9317727,Alexander Kuzmenkov,Tue Oct 18 18:04:37 2022 +0400,Use non-refcounted tupdesc for multi-insert buffers,"We don't need reference counting there, and it spends a lot of CPU in ResourceOwner. This slightly improves COPY performance on some data sets. " 84b2fef6ef166a5bad2b9d8fafcf0590259f8473,Mats Kindahl,Thu Oct 20 13:37:26 2022 +0200,Fix GitHub output action,To avoid untrusted logged data to use `set-state` and `set-output` workflow commands without the intention of the workflow author GitHub have introduced a new set of environment files to manage state and output. This commit changes the existing uses of `set-output` to use the new environment files instead. See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ 4b05402580b9ae8e843acaf48f1559c7b261e032,Erik Nordström,Thu Sep 1 17:26:41 2022 +0200,Add health check function,"A new health check function _timescaledb_internal.health() returns the health and status of the database instance, including any configured data nodes (in case the instance is an access node). Since the function returns also the health of the data nodes, it tries hard to avoid throwing errors. An error will fail the whole function and therefore not return any node statuses, although some of the nodes might be healthy. The health check on the data nodes is a recursive (remote) call to the same function on those nodes. Unfortunately, the check will fail with an error if a connection cannot be established to a node (or an error occurs on the connection), which means the whole function call will fail. This will be addressed in a future change by returning the error in the function result instead. " 23c01c44e5c259cbc1091a1f3492758a27dcaad3,Jan Nidzwetzki,Fri Oct 21 09:24:17 2022 +0200,Remove an unused function in the copy code,Since e555eea9dbc05f4c09cf0d7e23b814054a459d19 the function TSCopyMultiInsertInfoIsEmpty is no longer used. This patch removes the unused code from src/copy.c. e555eea9dbc05f4c09cf0d7e23b814054a459d19,Jan Nidzwetzki,Mon Oct 17 11:57:28 2022 +0200,Fix performance regressions in the copy code,"In 8375b9aa536a619a5ac2644e0dae3c25880a4ead, a patch was added to handle chunks closes during an ongoing copy operation. However, this patch introduces a performance regression. All MultiInsertBuffers are deleted after they are flushed. In this PR, the performance regression is fixed. The most commonly used MultiInsertBuffers survive flushing. The 51259b31c4c62b87228b059af0bbf28caa143eb3 commit changes the way the per-tuple context is used. Since this commit, more objects are stored in this context. The size of the context was used to set the tuple size to PG < 14. The extra objects in the context lead to wrong (very large) results and flushes almost after every tuple read. The cache synchronization introduced in 296601b1d7aba7f23aea3d47c617e2d6df81de3e is reverted. With the current implementation, `MAX_PARTITION_BUFFERS` survive the flash. If `timescaledb.max_open_chunks_per_insert` is lower than `MAX_PARTITION_BUFFERS` , a buffer flush would be performed after each tuple read. " 40a6c4cf87edb3ca261cebcf4b2c6af11fcc6f0e,Erik Nordström,Sat Oct 15 16:20:34 2022 +0200,Fix unused sort in dimension partition lookup,"Dimension partition lookups use binary search to find the partition to place a chunk in. However, in the code, an array of partitions might not be sorted because the sort happened on a copy of the array instead of the main array. This change fixes the issue to ensure the array is sorted and binary search works properly. " f862212c8ca19b1af56c7608a68f22b7dd0c985e,Alexander Kuzmenkov,Tue Oct 18 21:12:35 2022 +0400,Add clang-tidy warning readability-inconsistent-declaration-parameter-name,Mostly cosmetic stuff. Matched to definition automatically with --fix-notes. 73c3d02ed2472b9c92e118c67a01749802595950,Alexander Kuzmenkov,Tue Oct 18 21:20:06 2022 +0400,Enable clang-tidy warning readability-misleading-indentation, 05ba1cf22f0dc9232069b566dd23c3edb2cbaee4,Alexander Kuzmenkov,Tue Oct 18 20:53:38 2022 +0400,Add clang-tidy warning readability-suspicious-call-argument,"Helps find accidentally swapped arguments, like in the recent epoll_ctl() error. " 276d3a331dc4e248056424230ad953041866106d,Mats Kindahl,Fri May 20 20:55:39 2022 +0200,Add macro to assert or error,"For some unexpected conditions, we have a check and an error that is generated. Since this always generate an error, it is more difficult to find the bug if the error is generated rather than an assert fired generating a core dump. Similarly, some asserts can occur in production builds and will lead to strange situations triggering a crash. For those cases we should instead generate an error. This commit introduces a macro `Ensure` that will result in an assert in debug builds, but an error message in release build. This macro should only be used for conditions that should not occur during normal runtime, but which can happen is odd corner-cases in release builds and therefore warrants an error message. It also replaces some existing checks with such errors to demonstrate usage. " f55aaf06dd2bf8d1d9296b19bb5f7eea541f05af,Nikhil Sontakke,Tue Oct 4 15:55:54 2022 +0530,Add hook for ssl options,External components like timescaledb_cloudutils might want to add additional options or do additional ssl related processing. They can do so by implementing a hook and then assigning it to a timescaledb variable to allow timescale to invoke it as appropriate. d218715d5c064c88f0e8d56070d762a95c3b65ab,Bharathy,Wed Oct 19 17:35:02 2022 +0530,Fix tests which fail on PG15,"When TimescaleDB 2.9.0 compiled against PG15, many tests fail due to permissions on default public schema not getting propagated to data nodes, although connected user has required GRANTS on access nodes. This patch fixes failing tests by explicitly granting required permissions for connected user or role after data nodes are added. " 080011d767a6e61d09dc83b4d58f8be7dc9ecb31,Alexander Kuzmenkov,Tue Oct 18 20:26:45 2022 +0400,Speed up the dist_copy tests,"In some cases we can use less chunks, less data, and not truncate tables. " 702ac53c0ac4fe339e87130622ddaaa63366f03f,Fabrízio de Royes Mello,Tue Oct 18 17:30:00 2022 -0300,Bump codecov github action version,Leftover from previous commit 8950ab where we bumped some github action versions to run on Node16 instead of Node12 (in deprecation). 54ed0d5c05df990674a6fc9857e1120cdcc9b338,Konstantina Skovola,Thu Aug 25 13:58:08 2022 +0300,Introduce fixed schedules for background jobs,"Currently, the next start of a scheduled background job is calculated by adding the `schedule_interval` to its finish time. This does not allow scheduling jobs to execute at fixed times, as the next execution is ""shifted"" by the job duration. This commit introduces the option to execute a job on a fixed schedule instead. Users are expected to provide an initial_start parameter on which subsequent job executions are aligned. The next start is calculated by computing the next time_bucket of the finish time with initial_start origin. An `initial_start` parameter is added to the compression, retention, reorder and continuous aggregate `add_policy` signatures. By passing that upon policy creation users indicate the policy will execute on a fixed schedule, or drifting schedule if `initial_start` is not provided. To allow users to pick a drifting schedule when registering a UDA, an additional parameter `fixed_schedule` is added to `add_job` to allow users to specify the old behavior by setting it to false. Additionally, an optional TEXT parameter, `timezone`, is added to both add_job and add_policy signatures, to address the 1-hour shift in execution time caused by DST switches. As internally the next start of a fixed schedule job is calculated using time_bucket, the timezone parameter allows using timezone-aware buckets to calculate the next start. " 8950abe0ee071549fbcd2d52ce544532ef3e7844,Fabrízio de Royes Mello,Mon Oct 17 15:22:30 2022 -0300,Bump github action versions,All github actions that run on Node12 are deprecated so bumped github action versions to run on Node16. https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/ 043bd55c0bf7ef372da6ad3e7874be030b10781c,Fabrízio de Royes Mello,Thu Oct 13 20:05:04 2022 -0300,Miss segmentby compression option in CAGGs,"Timescale 2.7 released a new version of Continuous Aggregate (#4269) that store the final aggregation state instead of the byte array of the partial aggregate state, offering multiple opportunities of optimizations as well a more compact form. This new version also removes the unecessary `chunk_id` column from the materialization hypertable and consequently the re-aggregation in the user view. It means the user view that query the materialization hypertable don't have a GROUP BY clause anymore that was problematic for query performance. Before 2.7 when users turn compression ON we infer compression options `segmentby` and `orderby` based on the GROUP BY clause and time bucket respectively. With the new version without a GROUP BY clause in the user view the inferetion for the 'segmentby' compression option stopped to work. Fixed it by changing the code to the compression on the new version of Continuous Aggregate (aka finals form) behave the same as the old version. Fix #4816 " bde337e92d32e7d20f5c5e75fb0ccbc3e50e8262,Alexander Kuzmenkov,Mon Oct 17 19:41:20 2022 +0300,Fix the flaky pg_dump test,It was frequently failing on Windows. Sort by what is actually printed. 0e32656b54ca2221ea1a2da6b941e8055f13c7da,Bharathy,Mon Oct 17 20:53:34 2022 +0530,Support for PG15.,"As part of this patch, added and fixed some of the regress checks which fail on PG15. " d5c25e8914513694bb20dad22146c39c16d908a7,Markos Fountoulakis,Mon Oct 17 16:06:15 2022 +0300,Adjust partition pruning to support PG15,PostgreSQL 15 introduced a Bitmapset for tracking non-pruned partitions for performance purposes. Adjust our code for expanding hypertable chunks to support this. https://github.com/postgres/postgres/commit/475dbd0b718 066bcbed6d18a8c8a29a96b5ac607d9939ddc860,Alexander Kuzmenkov,Thu Oct 13 20:25:11 2022 +0300,Rename row-by-row fetcher to COPY fetcher,"This name better reflects its characteristics, and I'm thinking about resurrecting the old row-by-row fetcher later, because it can be useful for parameterized queries. " 38878bee1671a0288c562e8a4e0934bb4b06143a,Bharathy,Wed Oct 12 09:20:09 2022 +0530,Fix segementation fault during INSERT into compressed hypertable.,"INSERT into compressed hypertable with number of open chunks greater than ts_guc_max_open_chunks_per_insert causes segementation fault. New row which needs to be inserted into compressed chunk has to be compressed. Memory required as part of compressing a row is allocated from RowCompressor::per_row_ctx memory context. Once row is compressed, ExecInsert() is called, where memory from same context is used to allocate and free it instead of using ""Executor State"". This causes a corruption in memory. Fixes: #4778 " 8f5698f49d11d41dfe7b97c63e98b543e6d5de7a,Sven Klemm,Tue Oct 11 00:05:10 2022 +0200,Show information about OOM killer in CI,Include OOM kill event logs into error printout. Previously these would only be visible by inspecting the postgres log and looking for killed by signal 9. e33bd89727cd058d09d353c2595965da613b97ae,Markos Fountoulakis,Tue Oct 11 13:43:02 2022 +0300,Adjust TAP tests permissions,"Starting with PG15, default permissions on the public schema is restricted for any non-superuser non-owner. Adjust TAP tests so as to not fail with ""permission denied for schema public"". https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b073c3cc " cc7ea8efd10ba5ade6cbf0e9af988ade06d24011,Sven Klemm,Mon Oct 10 23:35:23 2022 +0200,Fix ereport call in dist_copy for PG 12.0,ereport in PG 12.0 requires extra parenthesis around the auxiliary function calls. 459c365794fd63c7708067e37b38a0a6769bcea6,Alexander Kuzmenkov,Mon Oct 10 17:34:59 2022 +0300,Ignore clang-format-14 changes in git blame, fbe4d3c1df1d82dca4e21bd09292ddaa7d3368d8,Alexander Kuzmenkov,Mon Oct 10 14:20:44 2022 +0300,Fix a warning with clang 14,Mark the variable as used for asserts only. 7758f5959c8ed64499ab0e6bb66c30464b11dd81,Alexander Kuzmenkov,Tue Oct 4 12:14:37 2022 +0300,Update .clang-format for version 14,The only configuration we're missing is the newline for braces after case labels. The rest of the differences looks like bugs/omissions of the version 8 that we use now. Require clang-format-14 in cmake and use it in the CI check. We can't support versions earlier than 14 because they have some formatting differences that can't be configured. 30596c0c478a168344a550b35ca480c5ad5a4a9f,Alexander Kuzmenkov,Mon Oct 10 14:36:04 2022 +0300,Batch rows on access node for distributed COPY,"Group the incoming rows into batches on access node before COPYing to data nodes. This gives 2x-5x speedup on various COPY queries to distributed hypertables. Also fix the text format passthrough, and prefer text transfer format for text input to be able to use this passthrough. It saves a lot of CPU on the access node. " 7600896a66e55fd933e81433f1aceecc7ddfc1c6,Markos Fountoulakis,Thu Oct 6 16:25:34 2022 +0300,Fix libpq disconnect for PG15,Make sure the FATAL error message before the data node disconnects is not lost when using PG15. https://github.com/postgres/postgres/commit/618c1670 2defb2b0b374b150dc1e678c03f621fb2f59ecf8,Sven Klemm,Sat Oct 8 16:55:42 2022 +0200,Improve job_crash_log test,Older versions seens to have problems when undef is passed to poll_query_until so we change the call to pass explicit query instead. efbd8a94b2bcf54e0772dc533949555bd98eaeb7,Sven Klemm,Sat Oct 8 22:39:49 2022 +0200,Increase timeout to wait for cluster start,By default pg_isready only waits for 3 seconds before giving up which is occasionally not enough in the windows tests. This patch bumps the timeout up to 30 seconds which should be plenty to have the cluster start up under all circumstances. e0bbd4042acf1bdcfdcc4cac54c3c3e5bcbdd8e4,Fabrízio de Royes Mello,Thu Sep 22 18:29:36 2022 -0300,Fix missing upgrade/downgrade tests DDL validation,Recently we fixed a DDL error (#4739) after upgrading to 2.8.0 version that surprisly the CI upgrade/dowgrade tests didn't complained during the development of the feature (#4552). Fixed it by adding an specific query in the `post.catalog.sql` script to make sure we'll check all the constraints of our internal tables and catalog. 2f739bb3286bca4db6fda9e71ba6e74c7f22ecff,Jan Nidzwetzki,Wed Oct 5 14:49:12 2022 +0200,Post-release fixes for 2.8.1,Bumping the previous version and adding tests for 2.8.1. ea1014fa61dff6ae9bfb3e8acde08f72e85fc00f,Konstantina Skovola,Thu Oct 6 15:55:12 2022 +0200,Fix flaky bgw_custom test, 45a8c0b5cf60ba8dbbb1c0236a51b6afa96517be,Sven Klemm,Wed Oct 5 23:02:15 2022 +0200,Improve formatting when printing coredump information,Add a newline between query and stacktrace when printing coredump information to make copying the query easier. d602c1fe2eb50cdf3684fd7454d231f6ad83c3ac,Sven Klemm,Wed Oct 5 23:01:27 2022 +0200,Fix typo in homebrew workflow, d2f0c4ed202fbf0b6cd3161197ad032fed70ace3,Sven Klemm,Tue Oct 4 14:14:31 2022 +0200,Fix update script handling of bgw_job_stat,"Update scripts should not use ADD/DROP/RENAME and always rebuild catalog tables to ensure the objects are identical between new install, upgrade and downgrade. " a76f76f4ee6ee1d5437fe31cbd551e0d6f51393f,Fabrízio de Royes Mello,Tue Oct 4 14:20:23 2022 -0300,Improve size utils functions and views performance,Changed queries to use LATERAL join on size functions and views instead of CTEs and it eliminate a lot of unnecessary projections and give a chance for the planner to push-down predicates. Closes #4775 8cda0e17ec89452f1d536f8b1d66f939c891ea4e,Sven Klemm,Sat Oct 1 21:19:28 2022 +0200,Extend the now() optimization to also apply to CURRENT_TIMESTAMP,The optimization that constifies certain now() expressions before hypertable expansion did not apply to CURRENT_TIMESTAMP even though it is functionally similar to now(). This patch extends the optimization to CURRENT_TIMESTAMP. 12b7b9f665865fde19c0a6460006147a3e545095,Jan Nidzwetzki,Thu Sep 29 14:50:15 2022 +0200,Release 2.8.1,"This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * #4454 Keep locks after reading job status * #4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * #4671 Fix a possible error while flushing the COPY data * #4675 Fix bad TupleTableSlot drop * #4676 Fix a deadlock when decompressing chunks and performing SELECTs * #4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * #4694 Change parameter names of cagg_migrate procedure * #4698 Do not use row-by-row fetcher for parameterized plans * #4711 Remove support for procedures as custom checks * #4712 Fix assertion failure in constify_now * #4713 Fix Continuous Aggregate migration policies * #4720 Fix chunk exclusion for prepared statements and dst changes * #4726 Fix gapfill function signature * #4737 Fix join on time column of compressed chunk * #4738 Fix error when waiting for remote COPY to finish * #4739 Fix continuous aggregate migrate check constraint * #4760 Fix segfault when INNER JOINing hypertables * #4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column " b259191dfe27bc13cb33dbc2bd327cf825ded400,Alexander Kuzmenkov,Mon Oct 3 19:36:20 2022 +0300,Add COSTS OFF to make test more stable,For some reason the cost estimates are different on my machine. 8a5e59b0978209d5913cdffcb6c5f69487ac1273,Konstantina Skovola,Wed Oct 5 09:01:32 2022 +0200,Fix flaky bgw_custom test,"The test was flaky because the scheduler was launching the scheduled jobs, therefore their next_start field could differ depending on whether they had finished executing. Fixed by not selecting the next_start field in alter_job calls, when it is not of interest. Fixes #4719 " 33e4e554d4a18284bb2b1f79398aa9bb5ff0cc88,Jan Nidzwetzki,Tue Oct 4 16:57:04 2022 +0200,Ensure that internal users don't trigger workflows,"This patch adds a membership check to the 'Waiting for Engineering' workflow. The check ensures that the workflow is only triggered by external users. Without this check, when someone from the database engineering team responded to an issue, the ""need-more-info"" flag was removed and the issue was erroneously moved to the ""waiting for engineering"" column. " f1c6fd97a3ca38aecae518efc2b2cda649dc745d,Bharathy,Tue Oct 4 20:20:23 2022 +0530,Continue compressing other chunks after an error,"When a compression_policy is executed by a background worker, the policy should continue to execute even if compressing or decompressing one of the chunks fails. Fixes: #4610 " ea5038f26387c47b6dba538f521f99cea3430870,Dmitry Simonenko,Tue Oct 4 10:06:06 2022 +0300,Add connection cache invalidation ignore logic,"Calling `ts_dist_cmd_invoke_on_data_nodes_using_search_path()` function without an active transaction allows connection invalidation event happen between applying `search_path` and the actual command execution, which leads to an error. This change introduces a way to ignore connection cache invalidations using `remote_connection_cache_invalidation_ignore()` function. This work is based on @nikkhils original fix and the problem research. Fix #4022 " fa9d07f6a81f3f5109718c0232cdee60367278dc,Sven Klemm,Fri Sep 30 10:05:19 2022 +0200,Add link to community forum to issue template chooser, eac3132b67f8ef2f377ff7e58c40ce144a2d847e,Sven Klemm,Sat Oct 1 09:17:24 2022 +0200,Migrate CI summary slack bot to eng-database, 7a6ce372e7727c43e0e077b31eb51ae75f6457f8,Sven Klemm,Sat Oct 1 15:06:55 2022 +0200,Adjust pgspot CI check,Change the pgspot check to use downgrade_to_version instead of update_from_version. We use downgrade_to_version instead of update_from_version because when the release PR for a new version has been merged to main but the version is not tagged yet update_from will not exist yet. In all other situations update_from_version and downgrade_to_version should point to the same version. f7c769c684d1cafe0b73c405b109f035fde57c61,Rafia Sabih,Wed Sep 28 15:33:26 2022 +0200,Allow manual index creation in CAggs,"The materialised hypertable resides in the _timescaledb.internal schema which resulted in permission error at the time of manual index creation by non super user. To solve this, it now switches to timescaledb user before index creation of CAgg. Fixes #4735 " 9bd772de250308946b0d7ec6bb71f47ec1ee583a,Konstantina Skovola,Tue Sep 6 16:25:09 2022 +0300,Add interface for troubleshooting job failures,"This commit gives more visibility into job failures by making the information regarding a job runtime error available in an extension table (`job_errors`) that users can directly query. This commit also adds an infromational view on top of the table for convenience. To prevent the `job_errors` table from growing too large, a retention job is also set up with a default retention interval of 1 month. The retention job is registered with a custom check function that requires that a valid ""drop_after"" interval be provided in the config field of the job. " 1d4b9d6977098e74ad10888c2b44e8faf187458b,Sven Klemm,Thu Sep 29 08:59:06 2022 +0200,Fix join on time column of compressed chunk,Do not allow paths that are parameterized on a compressed column to exist when creating paths for a compressed chunk. 940187936c93b6d78e702ded75d79d914f8fea96,Sven Klemm,Tue Sep 27 19:49:42 2022 +0200,Fix segfault when INNER JOINing hypertables,This fixing a segfault when INNER JOINing 2 hypertables that are ordered by time. d833293a5244092aa680a2ee293b61a678be0246,Sven Klemm,Wed Sep 28 05:08:37 2022 +0200,Add missing ORDER BY clause to dist_move_chunk, cc110a33a2e785e8619f82a72ec48745c75208eb,Ante Kresic,Mon Sep 26 14:53:01 2022 +0200,Move ANALYZE after heap scan during compression,"Depending on the statistics target, running ANALYZE on a chunk before compression can cause a lot of random IO operations for chunks that are bigger than the number of pages ANALYZE needs to read. By moving that operation after the heap is loaded into memory for sorting, we increase the chance of hitting cache and reducing disk operations necessary to execute compression jobs. " 9c819882f317f5d3d53e0bd040c6e7eac2c4bee2,Ante Kresic,Mon Sep 26 14:35:23 2022 +0200,Increase memory usage for compression jobs,"When compressing larger chunks, compression sort tends to use temporary files since memory limits (`work_mem`) are usually pretty small to fit all the data into memory. On the other hand, using `maintenance_work_mem` makes more sense since its generally safer to use a larger value without impacting general resource usage. " c0e193dd810d11e66d75a987b7b1291e6fe7a9f9,Mats Kindahl,Fri Jun 17 13:10:46 2022 +0200,Keep locks after reading job status,"When reading the job status table `bgw_job_stat` and after that updating it, locks where released after the read, allowing a competing session to update the job status and trigger a concurrent update error either in a session doing the update or in the scheduler. Since the scheduler does not recover after aborting with an error, this caused the background worker subsystem to stop and not start new jobs. This commit fixes this by upgrading `RowExclusiveLock` to `ShareRowExclusiveLock` to ensure that not two sessions tries to update the row at the same time, remove an initial speculative lock that are taken when a job status row can be added, and also keeps the lock until the end of the transaction to prevent other sessions to update. Since these updating transactions are short, it should not cause other threads to block long. Fixes #4293 " f6dd55a19161d7d64d224069adbe0ab48c10ea29,Bharathy,Tue Sep 27 19:33:15 2022 +0530,Hypertable FK reference to partitioned table,"Consider a hypertable which has a foreign key constraint on a referenced table, which is a parititioned table. In such case, foreign key constraint is duplicated for each of the partitioned table. When we insert into a hypertable, we end up checking the foreign key constraint multiple times, which obviously leads to foreign key constraint violation. Instead we only check foreign key constraint of the parent table of the partitioned table. Fixes #4684 " 244b3e637c2a290a0dfc3262339ce1b190f63c08,Mats Kindahl,Mon Sep 26 16:05:46 2022 +0200,Move perltidyrc to root,"To allow perltidy to be used from editors and IDEs, the perltidyrc file is moved from `scripts/perltidyrc` to `.perltidyrc`. This will allow editors and IDEs to quickly find the style file in the root of the repository, where `perltidy` normally searches for the style file. The workflow is then updated to use the new location, and the two options `--backup-file-extension` and `--backup-and-modify-in-place` are moved to the workflow file to allow editors and IDEs to use whatever method they find useful to process the file. " 6011c1446e9ddcd7f1c3ac22a3580be5523ea36b,Alexander Kuzmenkov,Wed Sep 21 16:10:25 2022 +0300,Fix error when waiting for remote COPY to finish,"Pass proper flags to WaitLatchOrSocket, and fix the swapped up arguments. " 9d0d159ac10ee7784754437ed0fa26f01de0cccf,Markos Fountoulakis,Tue Sep 20 15:15:47 2022 +0300,Port perl tests to support PG15,Port the timescaledb perl tests and the corresponding infrastructure to support the new perl namespace introduced in PG15. https://github.com/postgres/postgres/commit/b3b4d8e6 893faf8a6b096f98a01435d36cf88f385a827b4c,Fabrízio de Royes Mello,Thu Sep 15 16:37:06 2022 -0300,Fix Continuous Aggregate migration policies,After migrate a Continuous Aggregate from the old format to the new using `cagg_migrate` procedure we end up with the following problems: * Refresh policy is not copied from the OLD to the NEW cagg; * Compression setting is not copied from the OLD to the NEW cagg. Fixed it by properly copying the refresh policy and setting the `timescaledb.compress=true` flag to the new CAGG. Fix #4710 2529ae3f68504b6375b159f36c9c51a7d708d6e7,Sven Klemm,Fri Sep 16 23:12:47 2022 +0200,Fix chunk exclusion for prepared statements and dst changes,The constify code constifying TIMESTAMPTZ expressions when doing chunk exclusion did not account for daylight saving time switches leading to different calculation outcomes when timezone changes. This patch adds a 4 hour safety buffer to any such calculations. 217f514657bcf008a9973b9fbffb6cd414b5d98d,Fabrízio de Royes Mello,Wed Sep 21 10:15:32 2022 -0300,Fix continuous aggregate migrate check constraint,Instances upgraded to 2.8.0 will end up with a wrong check constraint in catalog table `continuous_aggregate_migrate_plan_step`. Fixed it by removing and adding the constraint with the correct checks. Fix #4727 4cb38350c67a2e60c8725934b402b53c812c22ee,Jan Nidzwetzki,Thu Sep 22 13:29:52 2022 +0200,Change project management GH action,"In af8e3c6b12035dd45bc7bf61e9420489cfa56eaa, an GitHub action to assign issues to a project column has been introduced. However, this action does not work with organization-wide projects. This patch replaces the used project management action with a more recent one. " de30d190e47c2874d27f3a5406efc5654fb1c3fc,Jan Nidzwetzki,Tue Sep 6 16:24:20 2022 +0200,Fix a deadlock in chunk decompression and SELECTs,"This patch fixes a deadlock between chunk decompression and SELECT queries executed in parallel. The change in a608d7db614c930213dee8d6a5e9d26a0259da61 requests an AccessExclusiveLock for the decompressed chunk instead of the compressed chunk, resulting in deadlocks. In addition, an isolation test has been added to test that SELECT queries on a chunk that is currently decompressed can be executed. Fixes #4605 " 5600fc06d65eb366cb304b8723eb1e953501c0fd,Jan Nidzwetzki,Wed Sep 21 10:29:22 2022 +0200,Updated the text of the stalebot,This PR improves the text of the stalebot and makes it clearer who is addressed by the message. 7508d6663967816633beeeba55696d8c33e106c0,Sven Klemm,Mon Sep 19 17:57:32 2022 +0200,Fix gapfill function signature,Very recent compilers will warn about function pointers with empty argument list. While currently in C func() means a function with an unspecified argument list the next version of the C standard will change this to mean func(void). db66a194b28aca11850aef5c50f837fa47f21d43,Jan Nidzwetzki,Tue Sep 20 13:53:07 2022 +0200,Fix an ereport syntax error when using PG 12.0,"Since PG 12.3, the syntax of the ereport() function has changed slightly. The change 97a603fe5c5ba25aa6e4e596fd7da7a401051d7b introduces an ereport() call that only works using newer PG versions. This PR changes the ereport() call to a PG 12.0 compatible syntax. " b451f0c521ee5e8c3f06c633cc455f0b4b111bd8,Jan Nidzwetzki,Mon Sep 19 14:01:39 2022 +0200,Update the name of the automation token,We now have an organization-wide token for automation. This patch changes the name in existing Github actions accordingly. a4d9c9fd6995f00b24edacde1eb6248aa35d9b08,Jan Nidzwetzki,Mon Sep 19 09:34:56 2022 +0200,Fix CI coredump information print,"The patch #4714 introduces the extraction of the running query from a coredump by calling gdb with two commands. These commands are separated by a newline character. This character may not always be correctly converted to a new line before being processed by gdb, resulting in the following error: Invalid character '\' in expression. This PR ensures the proper handling of the newline character. " af8e3c6b12035dd45bc7bf61e9420489cfa56eaa,Mats Kindahl,Wed Sep 14 14:02:15 2022 +0200,Update procedural.yml,Fix the 'Waiting for Engineering' job. Issues that contain the label 'need-more-info' that receive an issue_comment are automatically moved to the 'Waiting for Engineering' column on the bug board and the 'need-more-info' is removed. 97a603fe5c5ba25aa6e4e596fd7da7a401051d7b,Konstantina Skovola,Thu Sep 15 15:57:28 2022 +0300,Remove support for procedures as custom checks,Procedures doing their own transaction handling could lead to errors or even crashes. Fixes #4703 042735f1f5d9b253e479075866d8b9cc24eaaa60,Markos Fountoulakis,Fri Sep 16 12:33:21 2022 +0300,Fix vacuum_set_xid_limits_compat() macro,Fix an error in the PG15 portion of the vacuum_set_xid_limits_compat() preprocessor macro. d00a55772cd1762e295673a3d8f5ee0b58f9b964,Bharathy,Sat Sep 17 07:39:04 2022 +0530,error compressing wide table,"Consider a compressed hypertable has many columns (like more than 600 columns). In call to compress_chunk(), the compressed tuple size exceeds, 8K which causes error as ""row is too big: size 10856, maximum size 8160."" This patch estimates the tuple size of compressed hypertable and reports a warning when compression is enabled on hypertable. Thus user gets aware of this warning before calling compress_chunk(). Fixes #4398 " ffd9dfb7ebd05e2bc8d619ac87b4d5937dae0f23,Sven Klemm,Thu Sep 15 20:54:05 2022 +0200,Fix assertion failure in constify_now,The code added to support VIEWs did not account for the fact that varno could be from a different nesting level and therefore not be present in the current range table. 85d0e16a982ab3110b39b0c713faa421dd82fd81,Sven Klemm,Fri Sep 16 13:02:29 2022 +0200,Fix flaky pg_dump test,Use DROP DATABASE WITH(FORCE) to drop the database in pg_dump test since occasionally there would still be connections to the database leading to test failures. Unfortunately PG12 does not support that syntax so we have to drop without that option on PG12. 1cb3edddc1cc6057cf5cce74fbb24cfd4db1e215,Sven Klemm,Fri Sep 16 11:24:35 2022 +0200,Fix bgw_db_scheduler regresscheck configuration,Recent refactoring changed bgw_db_scheduler to also be run on non-Debug builds. Adjust the configuration so it is only included for debug builds. 3e1ce8c34aae49b5e6dc5188a154f1986e56fe13,Sven Klemm,Thu Sep 15 22:18:22 2022 +0200,Include running query in CI coredump information,Pretty print the active query during a coredump when printing the stacktrace for a coredump. 7aadf1332fb21e8b61ff6a996e7c1d08c913cee9,Nikhil Sontakke,Mon Sep 5 18:17:14 2022 +0530,Fix some clang compile warnings,Local compilation of timescaledb fails on MaCOSx with clang version 11.0.0. Fixes for a couple of warnings. fee27484cec7a01c2a98c16133e364f089689181,Alexander Kuzmenkov,Thu Sep 15 20:13:15 2022 +0300,Do not use row-by-row fetcher for parameterized plans,"We have to prepare the data node statement in this case, and COPY queries don't work with prepared statements. " 424f6f7648e88907b9373ef44ec851787db21080,Sven Klemm,Mon Sep 12 08:50:04 2022 +0200,Remove database port from test output,Don't include the used database ports into test output as this will lead to failing tests when running against a local instance or against a preconfigured cloud instance. 1642750e47aaa953015f050f9f850d3f2b41b550,Sven Klemm,Tue Sep 13 12:38:21 2022 +0200,Remove multiple PG configurations from regresscheck-t,Using multiple different configurations in a single target will not work when running against a local instance or when running against a preconfigured cloud instance. With recent adjustments to the test cleanup this should not be needed anymore and if we really need different configuration we should make it a separate target to make it compatible with instances configured outside of pg_regress. cea1a21c01d76f0e421fcfd86e2ae116f92e1943,Sven Klemm,Tue Sep 13 20:42:23 2022 +0200,Improve SKIPS handling in regression tests,This patch changes the regression test schedule generation to only completely rebuild the schedule when TESTS is specified. In all other cases the existing schedule is amended. This means any parallelity present in the cmake generated schedule will be kept. 063bfcfef912ad74ec86d8e9d0c0f15fff73b87d,Sven Klemm,Tue Sep 13 23:05:30 2022 +0200,Fix pgspot update script check,Previously the pgspot call used in CI did not properly check the update script. Unsafe function creations for functions with changed signatures could go undetected in the update script. 3b3681858d0a29676098212acabb37ea4549e82a,Konstantina Skovola,Mon Sep 12 14:44:38 2022 +0300,Remove test case causing bgw_custom crash,"Patch #4425 introduced regression test failures, namely, a crash in function `ts_bgw_job_update_by_id`. The failures are due to the COMMIT statement in the custom check procedure. This patch removes that particular test case from bgw_custom. " 0144c75b3f1b932bac4cf28b432c7fb179307b6f,Sven Klemm,Tue Sep 13 22:24:32 2022 +0200,Remove flaky test from bgw_db_scheduler,"The out of background worker test in bgw_db_scheduler is flaky and fails very often, especially in the 32bit environment and on windows. This patch removes that specific test from bgw_db_scheduler. If we want to test this specific part of the scheduler this should be better rewritten in an isolation test. " 88e2f24ea3e09f4c39256d27b558d997d0367738,Mats Kindahl,Mon Sep 12 13:04:51 2022 +0200,Add automation for waiting on author,If an issue is labeled with `needs-more-info` it is automatically considered as waiting for author and we should move it to the column for that. 02ad4f6b763a52a206fab9e231ab023d62360e60,Fabrízio de Royes Mello,Mon Sep 12 16:26:55 2022 -0300,Change parameter names of cagg_migrate procedure,"Removed the underline character prefix '_' from the parameter names of the procedure `cagg_migrate`. The new signature is: cagg_migrate( IN cagg regclass, IN override boolean DEFAULT false, IN drop_old boolean DEFAULT false ) " 6ecefff93ec2991505af56442b751c6c2925e303,Fabrízio de Royes Mello,Fri Sep 9 14:26:01 2022 -0300,Add CAGG migration permission tests,Timescale 2.8 released a migration path from the old format of Continuous Aggregate to the new format (#4552). Unfortunately it lacks of proper tests when a non-superuser execute the migration. For a non-superuser execute the migration properly it requires SELECT/INSERT/UPDATE permissions in some catalog objects: * _timescaledb_catalog.continuous_agg_migrate_plan * _timescaledb_catalog.continuous_agg_migrate_plan_step * _timescaledb_catalog.continuous_agg_migrate_plan_step_step_id_seq Improved the regression tests to cover the lack of permissions in the catalog objects for non-superusers. d2baef3ef361b4fa0d54fcbfd8e0b49cb4fa1975,Sven Klemm,Mon Sep 5 23:29:28 2022 +0200,Fix planner chunk exclusion for VIEWs,Allow planner chunk exclusion in subqueries. When we decicde on whether a query may benefit from constifying now and encounter a subquery peek into the subquery and check if the constraint references a hypertable partitioning column. Fixes #4524 b869f91e256df56461021dfe836c69a4c45836e3,Bharathy,Mon Sep 12 17:25:58 2022 +0530,Show warnings during create_hypertable().,"The schema of base table on which hypertables are created, should define columns with proper data types. As per postgres best practices Wiki (https://wiki.postgresql.org/wiki/Don't_Do_This), one should not define columns with CHAR, VARCHAR, VARCHAR(N), instead use TEXT data type. Similarly instead of using timestamp, one should use timestamptz. This patch reports a WARNING to end user when creating hypertables, if underlying parent table, has columns of above mentioned data types. Fixes #4335 " 4e47302c2c2a923c8ed30e34d3486a942767059b,Alexander Kuzmenkov,Fri Sep 9 18:45:06 2022 +0300,Speed up chunk search by restriction clauses,"We don't have to look up the dimension slices for dimensions for which we don't have restrictions. Also sort chunks by ids before looking up the metadata, because this gives more favorable table access patterns (closer to sequential). This fixes a planning time regression introduced in 2.7. " a26a5974dce08e1146bc7aa35eac7e1f6f1716e2,Sven Klemm,Sat Sep 10 10:20:08 2022 +0200,Improve space constraint exclusion datatype handling,This patch adjusts the operator logic for valid space dimension constraints to no longer look for an exact match on both sides of the operator but instead allow mismatched datatypes. Previously a constraint like `col = value` would require `col` and `value` to have matching datatype with this change `col` and `value` can be different datatype as long as they have equality operator in btree family. Mismatching datatype can happen commonly when using int8 columns and comparing them with integer literals. Integer literals default to int4 so the datatypes would not match unless special care has been taken in writing the constraints and therefore the optimization would never apply in those cases. f27e62734141ed7fca30401f22a5ad1ead86b9e2,Sven Klemm,Fri Sep 9 13:57:16 2022 +0200,Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries,Since we do not use our own hypertable expansion for SELECT FOR UPDATE queries we need to make sure to add the extra information necessary to get hashed space partitions with the native postgres inheritance expansion working. 6de979518d286a83783f68172fbf1623f8873e26,Sven Klemm,Thu Sep 8 18:10:10 2022 +0200,Fix compression_chunk_size primary key,"The primary key for compression_chunk_size was defined as chunk_id, compressed_chunk_id but other places assumed chunk_id is actually unique and would error when it was not. Since it makes no sense to have multiple entries per chunk since that reference would be to a no longer existing chunk the primary key is changed to chunk_id only with this patch. " 8e4dcddad62ac9f70933a8d61ab02c9c687e0c4e,Alexander Kuzmenkov,Thu Sep 8 12:18:06 2022 +0300,Make the copy_memory_usage test less flaky,Increase the failure threshold. d65cad94e3dadd8ec4423575940061c8daada550,Sven Klemm,Mon Sep 5 13:22:51 2022 +0200,Reorganize gapfill header files,Combine gapfill header files to only have gapfill.h and gapfill_internal.h and make C files globally unique. b34b91f18bcaf3d6d997a6053e0e23dc44108fc3,Sven Klemm,Mon Sep 5 10:49:10 2022 +0200,Add timezone support to time_bucket_gapfill,"This patch adds a new time_bucket_gapfill function that allows bucketing in a specific timezone. You can gapfill with explicit timezone like so: `SELECT time_bucket_gapfill('1 day', time, 'Europe/Berlin') ...` Unfortunately this introduces an ambiguity with some previous call variations when an untyped start/finish argument was passed to the function. Some queries might need to be adjusted and either explicitly name the positional argument or resolve the type ambiguity by casting to the intended type. " 698084c0e1b5e2f6079db355367270f9f4f96a34,Markos Fountoulakis,Tue Sep 6 15:53:00 2022 +0300,Fix bad TupleTableSlot drop,PostgreSQL 15 exposed a use after free bug that went undetected in previous versions. 533a138ae22ddabf8a576d23dcb8f88a842fb6e5,Alexander Kuzmenkov,Mon Sep 5 13:51:42 2022 +0300,Fix a possible error while flushing the COPY data,"Apparently, pqFlush can report a success status, but the underlying socket can become invalid at the same time. Check for this. In passing, also change to the proper error reporting functions that use the connection error information, where needed. " 0d5049cf15275b159890e1ee4d435a94e59f9f4a,Konstantina Skovola,Mon Sep 5 09:36:30 2022 +0200,Fix max backoff on MAX_CONSECUTIVE_FAILURES, 83eb635e53ea1332b96f402141c1b3f7e3e9fab2,Mats Kindahl,Fri Sep 2 10:46:32 2022 +0200,Fix coverity warning about potential overflow,"There is a potential overflow in shifting in the code. The shift count should be less than 20, so this should be safe, but adding a cast to ensure that types match later usage and also extending the assert to capture bugs. " fe8d823bef3561d3db8b56f2cefb7d4dd9de0be8,Sven Klemm,Tue Jul 12 12:44:59 2022 +0200,Add compat macros for PG15 signature changes,PG15 refactors INSERT/UPDATE/DELETE code and changes the signatures of some of the functions. https://github.com/postgres/postgres/commit/25e777cf 9eef2e70f27366602f4da9ae89e44eeccba7bd08,Sven Klemm,Thu Sep 1 11:08:17 2022 +0200,Don't error when compression metadata is missing,Since the process that adjusts the compressed chunk statistics gets run during VACUUM we do not want to throw any errors in that process to let the VACUUM succeed. 0d7da62251cff811ea531c3d54522ed6f3021e06,Sven Klemm,Tue Jul 12 12:15:51 2022 +0200,Adjust reorder code for PG15,"PG15 removes ""recheck"" argument from check_index_is_clusterable and consolidate VACUUM xid cutoff logic. https://github.com/postgres/postgres/commit/b940918d https://github.com/postgres/postgres/commit/efa4a946 " 5c8b0b52b4dfadb8d8ab4c5c5d42c2f75a24728c,Sven Klemm,Thu Sep 1 12:00:39 2022 +0200,Adjust ts_get_node_name for PG15,PG15 removes T_Expr nodeTag because it's an abstract type. https://github.com/postgres/postgres/commit/85399291 39a700a4de4090a76268181faca19ea3d689e9ea,Sven Klemm,Thu Sep 1 17:35:34 2022 +0200,Fix changelog entries added in wrong place,Commit ed212b44 added the changelog entries to the bottom of the changelog instead of the top. 3722b0bf23e3f10c8c053a1fd21dd01ebbcc1012,Sven Klemm,Wed Aug 31 21:55:12 2022 +0200,Add 2.8.0 to update tests,Add 2.8.0 to update tests and adjust the downgrade script files. fca9078d6c7592d2df6e0e40428688401aa47110,Konstantina Skovola,Wed Aug 31 19:52:53 2022 +0300,Exponentially backoff when out of background workers,"The scheduler detects the following three types of job failures: 1.Jobs that fail to launch (due to shortage of background workers) 2.Jobs that throw a runtime error 3.Jobs that crash due to a process crashing In cases 2 and 3, additive backoff is applied in calculating the next start time of a failed job. In case 1 we previously retried to launch all jobs that failed to launch simultaneously. This commit introduces exponential backoff in case 1, randomly selecting a wait time in [2, 2 + 2^f] seconds at microsecond granularity. The aim is to reduce the collision probability for jobs that compete for a background worker. The maximum backoff value is 1 minute. It does not change the behavior for cases 2 and 3. Fixes #4562 " ed212b4442ca112d281dc5331e4dc8e677e6fae9,Bharathy,Mon Aug 29 10:01:56 2022 +0530,GROUP BY error when setting compress_segmentby with an enum column,When using a custom ENUM data type for compressed hypertable on the GROUP BY clause raises an error. Fixed it by generating scan paths for the query by checking if the SEGMENT BY column is a custom ENUM type and then report a valid error message. Fixes #3481 1d4f90b1d31e7603f00d78d03e1992195e74865b,Sven Klemm,Wed Aug 31 17:28:18 2022 +0200,Pass PlannerInfo by reference to can_exclude_constraints_using_clauses,PlannerInfo is 608 bytes and should be passed by reference and not by value. Found by coverity. 21673f2df2703bc7d7d8012c31ea593940c3a4fd,Sven Klemm,Wed Aug 31 17:08:27 2022 +0200,Fix copy-paste error in ts_bgw_job_insert_relation,Found by coverity. f432d7f93169a5c806f5c9da628513c69ca8eed2,Sven Klemm,Thu Aug 25 18:32:34 2022 +0200,Release 2.8.0,"This release adds major new features since the 2.7.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * time_bucket now supports bucketing by month, year and timezone * Improve performance of bulk SELECT and COPY for distributed hypertables * 1 step CAgg policy management * Migrate Continuous Aggregates to the new format **Features** * #4188 Use COPY protocol in row-by-row fetcher * #4307 Mark partialize_agg as parallel safe * #4380 Enable chunk exclusion for space dimensions in UPDATE/DELETE * #4384 Add schedule_interval to policies * #4390 Faster lookup of chunks by point * #4393 Support intervals with day component when constifying now() * #4397 Support intervals with month component when constifying now() * #4405 Support ON CONFLICT ON CONSTRAINT for hypertables * #4412 Add telemetry about replication * #4415 Drop remote data when detaching data node * #4416 Handle TRUNCATE TABLE on chunks * #4425 Add parameter check_config to alter_job * #4430 Create index on Continuous Aggregates * #4439 Allow ORDER BY on continuous aggregates * #4443 Add stateful partition mappings * #4484 Use non-blocking data node connections for COPY * #4495 Support add_dimension() with existing data * #4502 Add chunks to baserel cache on chunk exclusion * #4545 Add hypertable distributed argument and defaults * #4552 Migrate Continuous Aggregates to the new format * #4556 Add runtime exclusion for hypertables * #4561 Change get_git_commit to return full commit hash * #4563 1 step CAgg policy management * #4641 Allow bucketing by month, year, century in time_bucket and time_bucket_gapfill * #4642 Add timezone support to time_bucket **Bugfixes** * #4359 Create composite index on segmentby columns * #4374 Remove constified now() constraints from plan * #4416 Handle TRUNCATE TABLE on chunks * #4478 Synchronize chunk cache sizes * #4486 Adding boolean column with default value doesn't work on compressed table * #4512 Fix unaligned pointer access * #4519 Throw better error message on incompatible row fetcher settings * #4549 Fix dump_meta_data for windows * #4553 Fix timescaledb_post_restore GUC handling * #4573 Load TSL library on compressed_data_out call * #4575 Fix use of `get_partition_hash` and `get_partition_for_key` inside an IMMUTABLE function * #4577 Fix segfaults in compression code with corrupt data * #4580 Handle default privileges on CAggs properly * #4582 Fix assertion in GRANT .. ON ALL TABLES IN SCHEMA * #4583 Fix partitioning functions * #4589 Fix rename for distributed hypertable * #4601 Reset compression sequence when group resets * #4611 Fix a potential OOM when loading large data sets into a hypertable * #4624 Fix heap buffer overflow * #4627 Fix telemetry initialization * #4631 Ensure TSL library is loaded on database upgrades * #4646 Fix time_bucket_ng origin handling * #4647 Fix the error ""SubPlan found with no parent plan"" that occurred if using joins in RETURNING clause. **Thanks** * @AlmiS for reporting error on `get_partition_hash` executed inside an IMMUTABLE function * @Creatation for reporting an issue with renaming hypertables * @janko for reporting an issue when adding bool column with default value to compressed hypertable * @jayadevanm for reporting error of TRUNCATE TABLE on compressed chunk * @michaelkitson for reporting permission errors using default privileges on Continuous Aggregates * @mwahlhuetter for reporting error in joins in RETURNING clause * @ninjaltd and @mrksngl for reporting a potential OOM when loading large data sets into a hypertable * @PBudmark for reporting an issue with dump_meta_data.sql on Windows * @ssmoss for reporting an issue with time_bucket_ng origin handling " ae6773fca6f285a3f0f56cda27bff1d08d8af596,Alexander Kuzmenkov,Tue Aug 23 18:30:20 2022 +0300,Fix joins in RETURNING,"To make it work, it is enough to properly pass the parent of the PlanState while initializing the projection in RETURNING clause. " 1fa8373931aeb98d175ac7988f1705b4a1c647d0,Sven Klemm,Sun Aug 28 19:18:16 2022 +0200,Fix segfaults in policy check functions, c697700addbe50027ca744c5aabca4181c89c0e3,Dmitry Simonenko,Mon Aug 29 16:55:18 2022 +0300,Add hypertable distributed argument and defaults,This PR introduces a new `distributed` argument to the create_hypertable() function as well as two new GUC's to control its default behaviour: timescaledb.hypertable_distributed_default and timescaledb.hypertable_replication_factor_default. The main idea of this change is to allow automatic creation of the distributed hypertables by default. e34218ce2963358a500f6bc315aace0fad29c450,Fabrízio de Royes Mello,Tue Jul 5 13:39:14 2022 +0200,Migrate Continuous Aggregates to the new format,"Timescale 2.7 released a new version of Continuous Aggregate (#4269) that store the final aggregation state instead of the byte array of the partial aggregate state, offering multiple opportunities of optimizations as well a more compact form. When upgrading to Timescale 2.7, new created Continuous Aggregates are using the new format, but existing Continuous Aggregates keep using the format they were defined with. Created a procedure to upgrade existing Continuous Aggregates from the old format to the new format, by calling a simple procedure: test=# CALL cagg_migrate('conditions_summary_daily'); Closes #4424 " c43307387e6e96765a98deca041b47f57e3c3928,Matvey Arye,Fri Jul 29 15:08:53 2022 -0400,Add runtime exclusion for hypertables,"In some cases, entire hypertables can be excluded at runtime. Some Examples: WHERE col @> ANY(subselect) if the subselect returns empty set WHERE col op (subselect) if the op is a strict operator and the subselect returns empty set. When qual clauses are not on partition columns, we use the old chunk exclusion, otherwise we try hypertable exclusion. Hypertable exclusion is executed once per hypertable. This is cheaper than the chunk exclusion that is once-per-chunk. " 706a3c0e50777819c6a1caea5f29e9fa8354b34d,Alexander Kuzmenkov,Wed Aug 24 17:14:15 2022 +0300,Enable statement logging in the tests,"Remove 'client_min_messages = LOG' where not needed, and add the 'LOG: statement' output otherwise. " 5d934baf1de8b7eace6384128f0942d4798d3798,Sven Klemm,Mon Aug 22 08:16:14 2022 +0200,Add timezone support to time_bucket,"This patch adds a new function time_bucket(period,timestamp,timezone) which supports bucketing for arbitrary timezones. " dc145b7485171f7d38ba563efecd0c311a00bdd2,Konstantina Skovola,Tue Jun 28 16:19:32 2022 +0300,Add parameter check_config to alter_job,"Previously users had no way to update the check function registered with add_job. This commit adds a parameter check_config to alter_job to allow updating the check function field. Also, previously the signature expected from a check was of the form (job_id, config) and there was no validation that the check function given had the correct signature. This commit removes the job_id as it is not required and also checks that the check function has the correct signature when it is registered with add_job, preventing an error being thrown at job runtime. " e0f3e17575902b995fb10ffb70a47f4418aa3c00,Mats Kindahl,Fri Jun 3 13:39:20 2022 +0300,Use new validation functions,"Old patch was using old validation functions, but there are already validation functions that both read and validate the policy, so using those. Also removing the old `job_config_check` function since that is no longer use and instead adding a `job_config_check` that calls the checking function with the configuration. " 7c55d0d5dcdd10e87d3479bf3f7ed4b317f6a045,gayyappan,Thu Aug 18 14:36:15 2022 -0400,Modify OSM chunk's constraint info in chunk catalog,"The OSM chunk registers a dummy primary dimension range in the TimescaleDB catalog. Use the max interval of the dimension instead of the min interval i.e use range like [Dec 31 294246 PST, infinity). Otherwise, policies can try to apply the policy on an OSM chunk. Add test with policies for OSM chunks " bc85fb1cf04a05bd5fd32427464aa5d56e7468fc,Alexander Kuzmenkov,Wed Aug 24 10:30:06 2022 +0300,Fix the flaky dist_ddl test,Add an option to hide the data node names from error messages. f4ba6e3c8e37a23017c93d1921f8a0a1afa528b3,Sven Klemm,Mon Aug 22 07:21:09 2022 +0200,Fix time_bucket_ng origin handling,This patch makes time_bucket_ng work correctly with origin values that are after the timestamp value to be bucketed. c3a344dc0c188f44c1b3e0fef3e7d2160404898c,Markos Fountoulakis,Wed Aug 24 11:03:50 2022 +0300,Fix Windows Packages workflow, 0786226e43ae80b992fb7c43a4117679cc2ce4cc,Jan Nidzwetzki,Thu Aug 18 14:41:07 2022 +0200,Ensure TSL library is loaded on database upgrades,"This patch ensures that the TSL library is loaded when the database is upgraded and post_update_cagg_try_repair is called. There are some situations when the library is not loaded properly (see #4573 and Support-Dev-Collab#468), resulting in the following error message: ""[..] is not supported under the current ""timescale"" license HINT: Upgrade your license to 'timescale'"" " 51259b31c4c62b87228b059af0bbf28caa143eb3,Alexander Kuzmenkov,Tue Aug 23 18:52:15 2022 +0300,Fix OOM in large INSERTs,"Do not allocate various temporary data in PortalContext, such as the hyperspace point corresponding to the row, or the intermediate data required for chunk lookup. " 3acfbd02c13a35f3d66231251b701965ab6c7619,Joshua Lockerman,Mon Aug 22 11:56:20 2022 -0400,Schema-qualify function telemetry,"This commit unconditionally schema-qualifies the function telemetry. In the default setting `format_procedure()` only schema-qualifies things that aren't on the search path. This could cause to much variation across users. Further, it'd be nice to tell if functions in-use are from an experimental schema or a main one. " 1c0bf4b7776d26991fb2aa0322d2dd165bfecef1,Sven Klemm,Sun Aug 21 19:09:48 2022 +0200,Support bucketing by month in time_bucket_gapfill, c488fcdbc9931c271361d0e4997a2b60e6eddf19,Sven Klemm,Sun Aug 21 18:56:45 2022 +0200,"Allow bucketing by month, year, century in time_bucket","This patch allows bucketing by month for time_bucket with date, timestamp or timestamptz. When bucketing by month the interval must only contain month components. When using origin together with bucketing by month only the year and month components are honoured. To bucket by month we get the year and month of a date and convert that to the nth month since origin. This allows us to treat month bucketing similar to int bucketing. During this process we ignore the day component and therefore only support bucketing by full months. " 82fc2cac699901749084c96c5f38ebf07e5e428a,Dmitry Simonenko,Mon Aug 22 17:49:57 2022 +0300,Fix rename for distributed hypertable,"Fix ALTER TABLE RENAME TO command execution on a distributed hypertable, make sure data node list is set and command is executed on the data nodes. Fix #4491 " 90cace417e86d995d7c28356d9ded2dbbf6c035d,Dmitry Simonenko,Mon Aug 22 17:19:43 2022 +0300,Load TSL library on compressed_data_out call,"A call to `compressed_data_out` from a replication worker would produce a misleading error saying that your license is ""timescale"" and you should upgrade to ""timescale"" license, even if you have already upgraded. As a workaround, we try to load the TSL module it in this function. It will still error out in the ""apache"" version as intended. We already had the same fix for `compressed_data_in` function. " c643173b8b440b1d1fe6170be60907d428953f13,gayyappan,Thu Aug 18 11:43:55 2022 -0400,Filter out osm chunks from chunks information view,Modify timescaledb_information.chunks view to filter out osm chunks. We do not want user to invoke chunk specific operations on OSM chunks. 0ffb7dadedcf2fa7bf83a22cc7196e8db6672336,Sven Klemm,Fri Aug 19 08:53:59 2022 +0200,Improve windows package test,Change the windows package test to use explicit postgres versions and test both minimum and maximum supported postgres minor version. 3e03afb5ba1cac271d906b75fa3ad5b8e493a886,Sven Klemm,Fri Aug 19 11:17:24 2022 +0200,Fix ABI test,Recent CI settings refactoring made in commit 0c6f4e24 broke the ABI test. This patch fixes the test and adds setting for the minimum ABI compatible version. 210246135eb66fff5f498f2201da026ab3e9e45f,Sven Klemm,Fri Aug 19 13:18:50 2022 +0200,Make cron test use latest supported PG version, 1f6d69720d1efd53ec42df5fca5e54a85898a5c6,Sven Klemm<31455525+svenklemm@users.noreply.github.com>,Thu Aug 18 09:20:38 2022 +0200,Add link to forum to README,This also replaces appveyor badge with link to Windows GitHub actions. 324201bb7fab6ecd7dc46766368c3ee65ebc7194,Sven Klemm,Thu Aug 18 09:14:50 2022 +0200,Enable Windows workflow in nightly CI run,Enable the Windows CI workflow in nightly runs. In nightly runs the CI run will also test release configuration in addition to the debug configuration run on PRs. This patch also removes the hard coded postgres version numbers from the workflow and reads it from ci settings. 6beda28965dc980a4555165b3d6a1ee368a2547b,gayyappan,Thu Aug 4 23:53:30 2022 -0400,Modify chunk exclusion to include OSM chunks,OSM chunks manage their ranges and the timescale catalog has dummy ranges for these dimensions. So the chunk exclusion logic cannot rely on the timescaledb catalog metadata to exclude an OSM chunk. 847919a05f014f7ff2d7e6ccd27c316ad8ba64cf,gayyappan,Wed Aug 3 16:49:29 2022 -0400,Add osm_chunk field to chunk catalog table,Setting this field to true indicates that this is an OSM chunk. 8f920b393a6cfbe45428221af49e16214a4155df,Fabrízio de Royes Mello,Wed Aug 17 18:44:57 2022 -0300,Add missing gitignore entry,Pull request #4416 introduced a new template SQL test file but missed to add the properly gitgnore entry to ignore generated test files. 4c9e89c80e5dc8576ded919d934e91beb4638bba,Sven Klemm,Fri Aug 12 16:15:51 2022 +0200,Migrate windows tests to GitHub Actions,This patch changes our windows test to run in GitHub actios instead of appveyor. This allows us to drop appveyor and have all CI workflows in one place. This also improves our windows testing as appveyor was only running on PG12 and this patch changes the windows tests to run on all supported PG versions. This patch also changes the windows tests to use the same configuration as the other tests. Since our test infrastructure heavily relies on shell scripting the test runner cannot run natively on Windows. We use WSL instead to get a unix environment to execute the test runner. 30785726168aed99706a56a6327959c5618841ed,Sven Klemm,Mon Aug 15 20:50:46 2022 +0200,Fix regresschecklocal-t,Commit 05dd4787d1 changed regresscheck to run with multiple configurations but did not make similar adjustments to regresschecklocal-t. b5a16c4e0ba21c714c8838ffa22e200ff06886b8,Joshua Lockerman,Tue Aug 16 14:56:02 2022 -0400,Fix telemetry initialization,"We cannot rely on the planner initializing the shared hashmap reference since there's no guarantee we'll actually run SPI code in the BGW before we marshal the request. This commit adds initialization checking to `ts_function_telemetry_read()` itself to ensure it will always read from an initialized state if possible. As this code run very infrequently, it currently checks every time and does not skip if it once saw a lack of entries. This is unlikely to matter. It would be nice to test this more explicitly, but that will likely have to wait until we have better reporting about BGW internals. For now I've relied on manual testing. Look at the telemetry report for the DB with UUID `d412018f-a435-492f-9ab6-b09d480ab1c1` received at timestamp `2022-08-16 18:56:53+00` for an example. " 5c96d25058e26874376f0d5d473469438234c7f1,Sven Klemm,Mon Aug 15 17:51:23 2022 +0200,Clean up multinode databases in tests,Not cleaning up created databases will prevent multiple regresschecklocal runs against the same instance cause it will block recreating the test users as they are still referenced in those databases. 9c6433e6edf3ba7fa4ea9207220da3e0db8e87e8,Markos Fountoulakis,Tue Aug 16 19:10:07 2022 +0300,Handle TRUNCATE TABLE on chunks,"Make truncating a uncompressed chunk drop the data for the case where they reside in a corresponding compressed chunk. Generate invalidations for Continuous Aggregates after TRUNCATE, so as to have consistent refresh operations on the materialization hypertable. Fixes #4362 " fc865de6e0d3890b0e4c8cae5db24942fcee1304,Fabrízio de Royes Mello,Tue Aug 16 15:05:28 2022 -0300,Change usage of scanint8 to pg_strtoint64,"PostgreSQL 15 changed `scanint8` to `pg_strtoint64`, so added it to the compatibility layer and use just of pg_strtoint64 in source code. postgres/postgres@cfc7191dfea330dd7a71e940d59de78129bb6175 " e97fa59839e20108840e3ee206ca2f4a69333f39,Fabrízio de Royes Mello,Tue Aug 16 12:21:13 2022 -0300,Remove unused header file,The Postgres header utils/int8.h is not necessary and it was removed in version 15 by commit postgres/postgres@cfc7191dfea330dd7a71e940d59de78129bb6175 d3d30c054766c5f1a3e9e978e383d8d28314efae,Markos Fountoulakis,Tue Aug 16 17:51:57 2022 +0300,Fix heap buffer overflow,Fix heap buffer overflow when C-strings are shorter than NameData types. Fixes #4571 d256758fd771e603d81ef99431daaa541de033d3,Sven Klemm,Tue Aug 16 08:21:35 2022 +0200,Disable local line in pg_hba.conf on Windows,Windows does not support unix domain socket connections so we comment out `local` lines in pg_hba.conf. On PG12 on Windows this will prevent server start otherwise. 060cd893270f206cff1fa547801ef4786125c2d7,Markos Fountoulakis,Tue Aug 16 12:37:42 2022 +0300,Fix identical branches,The same code is executed because the 'then' and 'else' branches are identical. Discovered by Coverity CID 380009. f92fe0cdd449008e758923f77979505516a35d26,Sven Klemm,Mon Aug 15 14:30:50 2022 +0200,Make test settings explicit,Our tests assume timezone to be US/Pacific and will fail with different values. 7bd25e093165c3b354b7bbcac4c5ed60fd58ffe6,Sven Klemm,Tue Aug 16 07:41:46 2022 +0200,Explicitly set hash table memory context,We require hash_create to have an explicit memory context so they are not accidentally created in TopMemoryContext leading to memory leaks if unintended. ee4423c18181846d30d8910a02ff13ffeb79323a,Sven Klemm,Fri Aug 12 09:29:12 2022 +0200,Fix coccinelle hash_create script,Previously the hash_create coccinelle script would not detect hash_create calls that declared the return variable in the same statement or that used the return value of hash_create as return value. 8c5a7597659669793a91bae6f3538bd721f29f45,Sven Klemm,Mon Aug 15 18:45:20 2022 +0200,Add missing ORDER BY clauses to cagg_policy test, 03defb33ef1d54ac7585e1eb6ee6caabaf4b27dc,Joshua Lockerman,Thu Aug 11 14:43:11 2022 -0400,Fix telemetry BGW memory leak,Applies the same fix as PR 4534 to the hash table created in the telemetry background worker. a3cfc091e89962e8860faf5a1db651d349dec5ac,Joshua Lockerman,Wed Aug 10 10:18:41 2022 -0400,Re-enable telemetry tests,They should be functioning after 2.7.2 a6cda9c9f051b8526a94dabeeb3f2055bf9072cb,Sven Klemm,Mon Aug 15 10:28:19 2022 +0200,Fix chunk_utils_internal test,Change chunk_utils_internal test to not use oid but instead use the role name. Using the oid can lead to failing tests when oid assignment is different especially when run with regresschecklocal-t. 131773a902580ffefebd7a6807b29dc4627807bc,Sven Klemm,Thu Aug 11 17:57:24 2022 +0200,Reset compression sequence when group resets,The sequence number of the compressed tuple is per segment by grouping and should be reset when the grouping changes to prevent overflows with many segmentby columns. 0c6f4e24587bfb7ab56642a6df75172ee4472845,Sven Klemm,Thu Aug 11 11:12:52 2022 +0200,Centralize CI settings,This patch adds a new settings file for common github action settings. Instead of repeating latest pg versions in every github workflow we can read the settings from this central file. 1c4c2bfd8ad5e69459775e92b2792d3b4043be76,Sven Klemm,Mon Aug 15 10:01:28 2022 +0200,Bump pgspot version used in CI to 0.3.3,Use pgspot 0.3.3 in CI and remove the obsolote check for 2.6.1. 29937fd57c86f02465820fed85552d9a61d75051,Sven Klemm,Thu Aug 11 10:10:58 2022 +0200,"Bump PG version used in CI to 14.5,13.8 and 12.12", 500c2259992ca6ea166ae7cd698e0dfcf76f77ac,Fabrízio de Royes Mello,Fri Aug 5 15:19:24 2022 -0300,Handle properly default privileges on CAggs,If a default privilege is configured and applied to a given Continuous Aggregate during it creation just the user view has the ACL properly configured but the underlying materialization hypertable no leading to permission errors. Fixed it by copying the privileges from the user view to the materialization hypertable during the Continous Aggregate creation. Fixes #4555 16fdb6ca5e3a42c120a45c7dc635e03c12f4b2e9,Rafia Sabih,Thu Jul 7 21:07:54 2022 +0200,Checks for policy validation and compatibility,"At the time of adding or updating policies, it is checked if the policies are compatible with each other and to those already on the CAgg. These checks are: - refresh and compression policies should not overlap - refresh and retention policies should not overlap - compression and retention policies should not overlap Co-authored-by: Markos Fountoulakis " 088f688780d2c015cc3e807573381acc7f2f2839,Rafia Sabih,Mon May 9 14:55:07 2022 +0300,Miscellaneous,-Add infinity for refresh window range Now to create open ended refresh policy use +/- infinity for end_offset and star_offset respectivly for the refresh policy. -Add remove_all_policies function This will remove all the policies on a given CAgg. -Remove parameter refresh_schedule_interval -Fix downgrade scripts -Fix IF EXISTS case Co-authored-by: Markos Fountoulakis bca65f46975f61e36e2ce57dc1ae7d80d6cccf05,Rafia Sabih,Wed May 4 15:56:14 2022 +0530,1 step CAgg policy management,"This simplifies the process of adding the policies for the CAggs. Now, with one single sql statements all the policies can be added for a given CAgg. Similarly, all the policies can be removed or modified via single sql statement only. This also adds a new function as well as a view to show all the policies on a continuous aggregate. " be429eb3d9419440c56f33f7e32422ebd697fe29,Sven Klemm,Thu Aug 11 19:43:02 2022 +0200,Regenerate SSL certs,The SSL certs used for testing expired today. This patch adds new certificates that last for 100 years. 95cc330e0cd7d81778864601b89082b32212ebf1,gayyappan,Mon Aug 8 17:49:25 2022 -0400,Add inherited check constraints to OSM chunk,"When a table is added to an inheritance hierrachy, PG checks if all check constraints are present on this table. When a OSM chunk is added as a child of a hypertable with constraints, make sure that all check constraints are replicated on the child OSM chunk as well. " d843fdd3235d03fe104bc01ccc7918d188c315fb,Sven Klemm,Mon Aug 8 20:59:05 2022 +0200,Disable build_info test on appveyor,The build_info seems to be very flaky on appveyor since a couple days and often timing out instead of completing successfully. This patch temporarily disables that test on appveyor. a6107020e6e90387d390115de347224f58961178,Sven Klemm,Fri Aug 5 14:54:20 2022 +0200,Fix segfaults in compression code with corrupt data,Sanity check the compression header for sane algorithm before using it as index into an array. Previously this would result in a segfault and could happen with corrupted compressed data. ea7a9db97258f8804c956f75d1d72055f0f352da,Fabrízio de Royes Mello,Mon Aug 8 10:36:07 2022 -0300,Add primnodes.h to ts_get_node_name function,In partitioning.c the function `resolve_function_argtype` deal with some primitive nodes in order to resolve the argument type for hashing functions. Refactoring the code a bit and added the primitive nodes (primnodes.h) to the `ts_get_node_name` and improve the error message of `resolve_function_argtype` showing the name of the node type instead of the code when an unsupported node type is detected. 5c129be60fc0d23d9903b44953511b28c67a5ba7,Fabrízio de Royes Mello,Sun Aug 7 15:20:57 2022 -0300,Fix partitioning functions,When executing `get_partition_{hash|for_key}` inside an IMMUTABLE function we're getting the following error: `ERROR: unsupported expression argument node type 112` This error is because the underlying `resolve_function_argtype` was not dealing with `T_Param` node type. Fixed it by dealing properly with `T_Param` node type returning the `paramtype` for the argument type. Fixes #4575 d35ea0f997bf815f652c2a4713924e1b51709012,Fabrízio de Royes Mello,Sun Aug 7 13:04:00 2022 -0300,Fix assertion in GRANT .. ON ALL TABLES IN SCHEMA,When working on a fix for #4555 discovered that executing `{GRANT|REVOKE} .. ON ALL TABLES IN SCHEMA` in an empty schema lead to an assertion because we change the way that command is executed by collecting all objects involved and processing one by one. Fixed it by executing the previous process utility hook just when the list of target objects is not empty. Fixes #4581 13df260089b50c79ed590eee19228f66eb540de2,Sven Klemm,Wed Aug 3 07:54:55 2022 +0200,Dont run isolation tests on PG 12.0 and 13.2,The output format of the isolation tester got changed by pg upstream and backported to earlier versions. This means PG versions before the backport have different output format then latest PG version. We used to add all the isolation tests to the IGNORE list for those PG versions but that is error prone and often forgotten when new isolation tests are added. This patch skips the isolation test on PG versions with incompatible output format. ce94cf126614e305a1bb4bd0f905cfedefb7fe17,Alexander Kuzmenkov,Tue Aug 2 18:01:42 2022 +0300,Use non-blocking data node connections for COPY,"This PR switches the postgres connections to data nodes to non-blocking mode for the duration of the COPY. Other operations on these connections are unaffected. It serves as a preparation for working with multiple connections simultaneously. No functional changes are introduced, but it's done as a separate PR to simplify review and debugging/bisect in case we introduce an error. " 17844a4087a69ec51aa75ef8b815d0109c019a40,Sven Klemm,Tue Aug 2 18:41:20 2022 +0200,Fix ABI test git errors,The most recent versions of the postgres docker image no longer contain git by default. The version available also contains a fix for the git vulnerability that changed the git behaviour to check file ownership. Since we bind mount the source checkout into the build container the user of the checkout is unlikely to match the user inside the container. This patch configures git to skip the owner check for the bind-mounted directory. https://github.blog/2022-04-12-git-security-vulnerability-announced/ bbfdcaccd5807751a8753d5c814ddf44fcd89bb2,Sven Klemm,Tue Aug 2 18:42:57 2022 +0200,Fix cmake when build directory not child of source directory,When the build directory was not a child of the source directory fetching the git information about the current commit would fail leading to this information not being included in the executable which in turn resulted in failing tests. d639cd89858e214709055ec7a5df498e3357e7af,Jan Nidzwetzki,Tue Aug 2 08:30:12 2022 +0200,Change the stalebot close message.,GitHub issues that are closed by a collaborator (or the stalebot) cannot be re-opened by the issue author. This commit changes the close message of the stale bot to reflect this. 025bda6a81e63db42c34287802a35bc11b1001c3,Erik Nordström,Thu Apr 28 08:07:09 2022 +0200,Add stateful partition mappings,"Add a new metadata table `dimension_partition` which explicitly and statefully details how a space dimension is split into partitions, and (in the case of multi-node) which data nodes are responsible for storing chunks in each partition. Previously, partition and data nodes were assigned dynamically based on the current state when creating a chunk. This is the first in a series of changes that will add more advanced functionality over time. For now, the metadata table simply writes out what was previously computed dynamically in code. Future code changes will alter the behavior to do smarter updates to the partitions when, e.g., adding and removing data nodes. The idea of the `dimension_partition` table is to minimize changes in the partition to data node mappings across various events, such as changes in the number of data nodes, number of partitions, or the replication factor, which affect the mappings. For example, increasing the number of partitions from 3 to 4 currently leads to redefining all partition ranges and data node mappings to account for the new partition. Complete repartitioning can be disruptive to multi-node deployments. With stateful mappings, it is possible to split an existing partition without affecting the other partitions (similar to partitioning using consistent hashing). Note that the dimension partition table expresses the current state of space partitions; i.e., the space-dimension constraints and data nodes to be assigned to new chunks. Existing chunks are not affected by changes in the dimension partition table, although an external job could rewrite, move, or copy chunks as desired to comply with the current dimension partition state. As such, the dimension partition table represents the ""desired"" space partitioning state. Part of #4125 " 49b6486dad4458dbab5ead98bd9304208639762c,Sven Klemm,Mon Aug 1 09:52:19 2022 +0200,Change get_git_commit to return full commit hash,This patch changes get_git_commit to always return the full hash. Since different git versions do not agree on the length of the abbreviated hash this made the length flaky. To make the length consistent change it to always be the full hash. 65b5dc900f0ea5e2a76b3a592a4fc25126da6d2b,Dmitry Simonenko,Mon Aug 1 10:36:11 2022 +0300,Support add_dimension() with existing data,"This change allows to create new dimensions even with existing chunks. It does not modify any existing data or do migration, instead it creates full-range (-inf/inf) dimension slice for existing chunks in order to be compatible with newly created dimension. All new chunks created after this will follow logic of the new dimension and its partitioning. Fix: #2818 " 336f4b513f27fb2be6e041cdb3e4c289bf1adf9f,Sven Klemm,Sun Jul 31 17:43:04 2022 +0200,Fix hash_create calls without HASH_CONTEXT flag,This patch fixes callsites that set an explicit memory context in the control structure but do not specify the HASH_CONTEXT flag leading to the hash table being created in TopMemoryContext. This patch also changes call sites that want to create the hash table in TopMemoryContext to be explicit about this. Additionally this patch adds a coccinelle script to detect these errors and prevent adding similar code in the future. 28440b79008230ef8c50da2f8d4640456bba8e02,Fabrízio de Royes Mello,Mon Jun 13 17:25:59 2022 -0300,Enable ORDER BY on Continuous Aggregates,Users often execute TopN like queries over Continuous Aggregates and now with the release 2.7 such queries are even faster because we remove the re-aggregation and don't store partials anymore. Also the previous PR #4430 gave us the ability to create indexes direct on the aggregated columns leading to performance improvements. But there are a noticable performance difference between `Materialized-Only` and `Real-Time` Continuous Aggregates for TopN queries. Enabling the ORDER BY clause in the Continuous Aggregates definition result in: 1) improvements of the User Experience that can use this so commom clause in SELECT queries 2) performance improvements because we give the planner a chance to use the MergeAppend node by producing ordered datasets. Closes #4456 eccd6df7821ed94400f8693300ac29f5ef6ebf58,Sven Klemm,Mon Jul 18 08:10:05 2022 +0200,Throw better error message on incompatible row fetcher settings,When a query has multiple distributed hypertables the row-by-by fetcher cannot be used. This patch changes the fetcher selection logic to throw a better error message in those situations. Previously the following error would be produced in those situations: unexpected PQresult status 7 when starting COPY mode 4ab158d729cb8b1495c9024b6027e6d40022d86e,Rafia Sabih,Tue Jul 19 19:31:08 2022 +0200,Fix gitcommit.h generation,"The commands for generating gitcommit.h need to be executed on every make run and not on cmake run to detect branch switches, commit changes or local modifications. That's why we add the commands in a custom target and run them on every make run. We do the generation part in a temporary file and only overwrite the actual file when the content is different to not trigger unnecessary recompilations. " 08fb8e44894bcae0b0d9b5ccf9349c725fc250e0,Alexander Kuzmenkov,Thu Jul 28 12:51:23 2022 +0300,Add chunks to baserel cache on chunk exclusion,This speeds up the planning by avoiding the repeated lookup of the chunk metadata. 008c6cf37187ed706e3abab67305f60721fd0cae,Sven Klemm,Wed Jul 27 11:06:34 2022 +0200,Fix dump_meta_data for windows,The dump_meta_data sql script used an external call to date to get the current data which does not work on windows. This patch changes the script to use now() instead to get the current time and date. Fixes #3674 6db09c7f2e7aa2000776062fd57244793e3fba92,Sven Klemm,Thu Jul 28 07:12:41 2022 +0200,Fix timescaledb_post_restore GUC handling,In the session timescaledb_post_restore() was called the value for timescaledb.restoring might not be changed because the reset_val for the GUC was still on. We have to use explicit SET in this session to adjust the GUC. a584263179ce7c9ffbf3700fa3323a831963c8d4,Rafia Sabih,Thu Jul 21 19:46:15 2022 +0200,Fix alter column for compressed table,"Enables adding a boolean column with default value to a compressed table. This limitation was occurring due to the internal representation of default boolean values like 'True' or 'False', hence more checks are added for this. Fixes #4486 " 8a308deb85d14fa6bb48bfb1262ba0d1e5e4c2cd,Sven Klemm,Tue Jul 26 22:25:38 2022 +0200,Add a generic memory leak test to CI,"Unlike the existing memory leak test which tests a single INSERT of 100.000.000 rows in a single transaction, this runs 1000000 SELECT, 10K UPDATE and 50K INSERT queries in the same backend. " 5fd13d3875ee9b1338c23460055632f760308ccc,Jan Nidzwetzki,Tue Jul 26 10:11:06 2022 +0200,Trigger bug board assignment on issue comment,"So far, issues were added to the bug board when they are labeled; issues that are not on the board and receive a response were not automatically added. This patch extends the workflow trigger; all bug issues that receive a response are also assigned to the board. " 990824a38365a2b811c858a790ebe093934c540d,Sven Klemm,Mon Jul 25 09:48:58 2022 +0200,Remove unused function and macro,This patch removes ts_chunk_add_foreign_table_as_chunk function which never had an implementation added and the ASSERT_IS_NULL_OR_VALID_CHUNK macro which has no user. 7608cb8719ecf717fa8eba11f1e74efed00eccd4,Sven Klemm,Mon Jul 25 09:30:04 2022 +0200,2.7.2 Post-release,Add 2.7.2 to update tests and adjust downgrade script generation for 2.7.2. 3ea06f4984b4141dba4209ad2d4e431f149a3f55,Sven Klemm,Mon Jul 25 09:43:32 2022 +0200,Remove ubuntu 21.10 (impish) from package tests,Ubuntu 21.10 is EOL and we no longer build packages for it. 661a6507edd0ec9c9c14b9317a6178aa4fd866b5,Sven Klemm,Mon Jul 25 09:47:01 2022 +0200,Fix coccinelle CI action,Switch coccinelle CI action to ubuntu 22.04 because 21.10 is EOL and stopped working. 851fffb1ae723e899d25147fc2dca0f325528d90,Sven Klemm,Thu Jul 21 10:36:16 2022 +0200,Release 2.7.2,"This release is a patch release. We recommend that you upgrade at the next available opportunity. Among other things this release fixes several memory leaks, handling of TOASTed values in GapFill and parameter handling in prepared statements. **Bugfixes** * #4517 Fix prepared statement param handling in ChunkAppend * #4522 Fix ANALYZE on dist hypertable for a set of nodes * #4526 Fix gapfill group comparison for TOASTed values * #4527 Handle stats properly for range types * #4532 Fix memory leak in function telemetry * #4534 Use explicit memory context with hash_create * #4538 Fix chunk creation on hypertables with non-default statistics **Thanks** * @3a6u9ka, @bgemmill, @hongquan, @stl-leonid-kalmaev and @victor-sudakov for reporting a memory leak * @hleung2021 and @laocaixw for reporting an issue with parameter handling in prepared statements " 90c7c652b19a46c07c2a7f63dc1efc9a0d39f18d,Sven Klemm,Thu Jul 21 23:22:31 2022 +0200,Fix chunk creation on hypertables with non-default statistics,When triggering chunk creation on a hypertable with non-default statistics targets by a user different from the hypertable owner the chunk creation will fail with a permission error. This patch changes the chunk table creation to run the attribute modification as the table owner. Fixes #4474 ce9672aee3643eba503fb58ebc5262b6ccdaac98,Sven Klemm,Fri Jul 22 06:29:54 2022 +0200,Fix dist_copy_long test on macOS,On macOS zcat expects the file to end in .Z appending that extension when the supplied filename does not have it. Leading to the following error for the dist_copy_long test: zcat: can't stat: data/prices-10k-random-1.tsv.gz (data/prices-10k-random-1.tsv.gz.Z): No such file or directory This patch changes the dist_copy_long test to use the shell to read the file instead and use input redirection so zcat never sees the filename. 6b0a9937c59d4648297ee753e12b2445e50e14cd,gayyappan,Thu Jul 7 18:14:19 2022 -0400,Fix attach_osm_table_chunk,Add chunk inheritance when attaching a OSM tabel as a chunk of the hypertable ba41a92d9ea966304102406390f58f6dfc45e374,Sven Klemm,Thu Jul 21 18:59:17 2022 +0200,Bump macOS version used in CI to 11,macOS 10.15 is deprecated as github action environment and will be unsupported by end of august. This PR switches our CI to use macOS 11 environment instead. https://github.com/actions/virtual-environments/issues/5583 1f6b0240a361f70d87769c61011edb27080462d7,Alexander Kuzmenkov,Thu Jul 21 15:53:15 2022 +0300,Add a distributed COPY test with more data,Use data sets with 10k and 100k rows with a variety of partitioning settings. This may help to catch some rare corner cases. 296601b1d7aba7f23aea3d47c617e2d6df81de3e,Alexander Kuzmenkov,Tue Jun 28 17:49:23 2022 +0300,Synchronize chunk cache sizes,"Specifically, flush the chunk multi-insert states when the number of it reaches the size of the chunk cache. Otherwise, the chunks corresponding to the multi-insert states go out of cache and have to be looked up again when the multi-insert state is flushed, which leads to a performance hit. " 7c841b8d924b017cf65c3e3ce631a686bd7afa00,Sven Klemm,Wed Jul 20 23:13:10 2022 +0200,Use explicit memory context with hash_create,This changes the hash_create calls in the function cache and in the cagg code to use explicit memory context. Without this hash_create will create the hash table in TopMemoryContext potentially leading to memory leaks. 4ed116b6f6aef5d913df1cce9b80b8eb96af351d,Dmitry Simonenko,Thu Jul 21 11:57:02 2022 +0300,Fix ANALYZE on dist hypertable for a set of nodes,Make sure ANALYZE can be run on a specific set of data nodes assigned to the distributed hypertable Fix #4508 ca3c85ba1e31c29276e8af505da56dc3b272a022,Konstantina Skovola,Wed Jul 20 21:39:05 2022 +0300,Remove CODEOWNERS file to activate pull-review,"In #4525 a workflow was introduced for running pull-review to automate reviewer assignment based on code ownership. However the CODEOWNERS file was not removed. As reviewers will be assigned based on it if it exists, it prevents the pull-review workflow from taking effect. This commit removes CODEOWNERS to allow pull-review to do the reviewer assignment. " 47d6a226f572611b31d393fc54ec2c965515590a,Sven Klemm,Wed Jul 20 19:36:13 2022 +0200,Fix memory leak in function telemetry,hash_create will create the hash table in TopMemoryContext unless explicitly requested to create in different memory context. The function telemetry code did not explicitly request a different context leading to a memory leak. 63a80eec0d23aae3be48bbca4df33f945b3a4a58,Nikhil Sontakke,Tue Jul 19 19:12:37 2022 +0530,Handle stats properly for range types,"For range types, the operator entry in statistics table is invalid. Also, certain range types don't have ""VALUES"" but only have ""NUMBERS"" entries. " 0e9dd3c70bf9dc5fc8f92a9efda171a880696957,Konstantina Skovola,Wed Jul 20 08:55:30 2022 +0300,Add config file and workflow for pull-review,"This commit adds a workflow for running pull-review in the serverless mode, using docker container CLI. " d5619283f3a9d04b926227be2cdf87db8104191a,Sven Klemm,Tue Jul 19 13:04:48 2022 +0200,Fix gapfill group comparison,The gapfill mechanism to detect an aggregation group change was using datumIsEqual to compare the group values. datumIsEqual does not detoast values so when one value is toasted and the other value is not it will not return the correct result. This patch changes the gapfill code to use the correct equal operator for the type of the group column instead of datumIsEqual. 0d175b262e3f34f790318e1c4ea28eae104d6b77,Sven Klemm,Mon Jul 18 07:16:19 2022 +0200,Fix prepared statement param handling in ChunkAppend,This patch fixes the param handling in prepared statements for generic plans in ChunkAppend making those params usable in chunk exclusion. Previously those params would not be resolved and therefore not used for chunk exclusion. Fixes #3719 cfac68ec3d8ed80f65b35f594d6c83b0b7002fae,Sven Klemm,Fri Jul 15 09:21:02 2022 +0200,Ignore compression_chunk_race test in 12.0 and 13.2,The isolation tester in earlier PG versions is lacking features we rely on in our tests so the results of isolation tests are ignored when testing on the earliest version of a major pg version. 91c3820d4e693ed0eec5e75ca74300f73a7bd83c,Jan Nidzwetzki,Mon Jul 18 07:46:12 2022 +0200,Activate a stalebot to close issues,"In #4499, a stalebot configuration was introduced. According to the configuration, the stalebot runs in dry mode. The output of a dry mode run was analyzed and issues with an outdated ""need-more-info"" label have been cleaned up. So, the stalebot can now be activated. " 597b71881ad20f8498938c918e23aa0abe280532,Sven Klemm,Sun Jul 17 13:44:49 2022 +0200,Fix assertion hit in row_by_row_fetcher_close,When executing multinode queries that initialize row-by-row fetcher but never execute it the node cleanup code would hit an assertion checking the state of the fetcher. Found by sqlsmith. d55ceb34c311fe1f8de0f6ad90e32199307e8895,Sven Klemm,Fri Jul 15 12:31:41 2022 +0200,Fix unaligned pointer access,The sanitizer found unaligned pointers. On amd64 this is not a problem but other platforms like arm are more strict regarding pointer alignment. 93bad0098d14bd9be6f6c984ab487efc508ca6a4,Jan Nidzwetzki,Thu Jul 7 10:59:50 2022 +0200,Introduce a stalebot to close issues,"This PR introduces a configuration for the 'actions/stale' stalebot, which is used to automatically flag and close stale issues after a few days of inactivity. Note: The configuration runs in dry-mode to test the behavior of the bot on our repository first. " 5670378e03ce8db6030fe3fe836f6b1b9cde6fca,Mats Kindahl,Thu Jul 7 12:54:04 2022 +0200,Post-release fixes for 2.7.1, daa46222ca6eb4894532bf99f2f899895d6c7e1b,Mats Kindahl,Wed Jun 29 14:16:19 2022 +0200,Release 2.7.1, fdb12f7abe72950e61f20956d781a9fa9c2966af,Nikhil Sontakke,Tue Jul 5 21:04:41 2022 +0530,Handle timescaledb versions aptly in multinode,The current check where we deem a DN incompatible if it's on a newer version is exactly the opposite of what we want it to be. Fix that and also add relevant test cases. 0c03ed954d0933b107319178cd5b04a592c4f1d2,Nikhil Sontakke,Wed Jul 6 18:56:40 2022 +0530,Ignore telemetry test for now,Temporarily ignore to allow the release of 2.7.1 335f298ef756a2e7cb11f4a4fb0f866b853e9ba5,Fabrízio de Royes Mello,Tue Jul 5 14:59:34 2022 +0200,Segfault when executing IMMUTABLE functions,Executing an IMMUTABLE function that has parameters and exception handling block multiple times in the same transaction causes a null pointer segfault when try to reset a non-initialized ts_baserel_info. Fixed it by preventing to reset a non-initialized `ts_baserel_info`. Fixes #4489 a608d7db614c930213dee8d6a5e9d26a0259da61,Jan Nidzwetzki,Wed Jun 29 15:12:46 2022 +0200,Fix race conditions during chunk (de)compression,"This patch introduces a further check to compress_chunk_impl and decompress_chunk_impl. After all locks are acquired, a check is made to see if the chunk is still (un-)compressed. If the chunk was (de-)compressed while waiting for the locks, the (de-)compression operation is stopped. In addition, the chunk locks in decompress_chunk_impl are upgraded to AccessExclusiveLock to ensure the chunk is not deleted while other transactions are using it. Fixes: #4480 " 1bbb6059cba0e75872325fe1e88ffdc8d25d4383,Alexander Kuzmenkov,Thu Jun 30 11:18:58 2022 +0300,Add more tests for distributed INSERT and COPY,"More interleavings of INSERT/COPY, and test with slow recv() to check waiting. " 07c8ed6af3fa01efa8e3943711c5456834acdf4e,Mats Kindahl,Mon Jul 4 09:58:59 2022 +0200,Add workflow to move issue to triage,Add workflow to automatically add all issues opened or labeled to the bug board for triage. 6c20e74674a88164c77e9d6b14dd50aa71d65752,gayyappan,Fri Jun 24 10:41:29 2022 -0400,Block drop chunk if chunk is in frozen state,"A chunk in frozen state cannot be dropped. drop_chunks will skip over frozen chunks without erroring. Internal api , drop_chunk will error if you attempt to drop a chunk without unfreezing it. This PR also adds a new internal API to unfreeze a chunk. " dd4bc4c24601ca39b43f49d4cde32a218f42b8bd,Joshua Lockerman,Thu Jun 2 16:11:44 2022 -0400,Add telemetry about replication,This commit adds telemetry about replication status to our telemetry gatherer. It adds a new sub object `replication` containing two fields: - `is_wal_receiver` is a boolean which is true if-and-only-if the current cluster has a `wal_receiver`. - `num_wal_senders` is the number of `wal_senders` that the current cluster has. 8bf6c887a93d2c013f84647248182d46607a91f3,Nikhil Sontakke,Fri Jun 24 13:19:31 2022 +0530,Better superuser handling for move_chunk,"The current code was assuming the bootstrap superuser for the actual move chunk operation. However, we can make it further flexible by using the logged in credentials if those happen to have superuser privileges. " 0b76a8e5c37a9d0b3d1f9d82c74548c187966a36,Sven Klemm,Fri Jun 24 19:12:46 2022 +0200,Remove travis config,We migrated all our CI jobs to GitHub actions quite a while ago but never removed the travis config file even though we were not using it anymore. 79bf4f53b144ed25569b19b76d0aafbbcc17a15f,gayyappan,Tue Jun 21 17:14:43 2022 -0400,Add api to associate a hypertable with custom jobs,"This PR introduces a new SQL function to associate a hypertable or continuous agg with a custom job. If this dependency is setup, the job is automatically deleted when the hypertable/cagg is dropped. " bd3d2ef884e4a475735567e880f7a524455163c8,Konstantina Skovola,Thu Jun 23 14:20:41 2022 +0300,Make reproduction steps field mandatory, 131f58ee602af8dcdc4b901587cb72beb3f51d2e,gayyappan,Thu Apr 28 17:51:27 2022 -0400,Add internal api for foreign table chunk,Add _timescaledb_internal.attach_osm_table_chunk. This treats a pre-existing foreign table as a hypertable chunk by adding dummy metadata to the catalog tables. 6c38c60b9782797b538129ba92f61268b4da42c6,Markos Fountoulakis,Thu Jun 23 10:59:24 2022 +0300,Repair numeric partial state on the fly,The numeric format changed between PG13 and PG14 to include infinities. As a result the serialized partial state of numeric aggregates also changed format. If a user that has stored partials (e.g. by using Continuous Aggregates) upgrades to PG14 then the partial state deserialization will lead to errors due to the mismatch with the PG14 code. Repair the deserialization process on the fly by appending zeroed plus-infinity and minus-infinity counts for the numeric aggregate state to use. Fixes #4427 93e9d421931a50d443da02c078f1e8ae93388517,Alexander Kuzmenkov,Mon Jun 20 11:33:32 2022 +0300,Use COPY protocol in row-by-row fetcher,This gives about 2x speedup for bulk data transfer queries. e3b2fbdf1513ee8b2db111a3a7c1c0f5d42b1b26,Nikhil Sontakke,Tue Jun 21 20:21:31 2022 +0530,Fix empty bytea handlng with distributed tables,"The ""empty"" bytea value in a column of a distributed table when selected was being returned as ""null"". The actual value on the datanodes was being stored appropriately but just the return code path was converting it into ""null"" on the AN. This has been handled via the use of PQgetisnull() function now. Fixes #3455 " db6447378624aa09745ce0c7762205e12fa33a76,Sven Klemm,Tue Jun 21 08:15:19 2022 +0200,Use PG 14.4 in update test,Switch update/downgrade test to use 14.4. This has been split off the CI PR switching the rest of CI to 14.4 because it usually takes more then a week for the upstream docker images to become available. f400c75b79db73d49245728139ef0c26df3d4551,Jacob Champion,Mon Jun 20 11:18:35 2022 -0500,Tweak commit-msg hook's line character limits,"The commit-msg hook was counting the end-of-line characters in the 50- and 72-character limit, so commit messages had to be wrapped slightly shorter in order to avoid the hook complaint. Strip the EOL characters during the check instead. Since we use universal newline handling during the file read, stripping LF ('\n') should be enough to handle both Windows and Unix development environments. " bab5cc101d451f160f409d5848eab053908ca04d,Jacob Champion,Mon Jun 20 10:52:49 2022 -0500,Update commit-msg hook to Python 3,"This git hook is copied automatically during CMake, and breaks commits immediately afterwards if Python 2 isn't installed. Since Python 2 has been end-of-life for a while now, take this chance to upgrade. " 1221977c9c922accab2255507113bfa83d854216,Jacob Champion,Mon Jun 20 10:48:06 2022 -0500,Inspect pg_config.h for SSL support,"We parse apart the output of `pg_config --configure` to see whether or not OpenSSL is enabled. This is a bit brittle; it has broken in the past with the change from --with-openssl to --with-ssl, and upstream is currently testing a change to the build system (Meson, which doesn't rely on autotools during configuration) that is likely to further interfere with this approach. As an alternative, we can just look at the header files to get this information. USE_OPENSSL is not defined in pg_config.h if SSL support is not compiled in. " 79aa0f637834066f4e7e470bd49bb629bab9fdd7,Dmitry Simonenko,Tue Jun 21 13:38:27 2022 +0300,Do not allow dist ddl during create extension,"This change fixes logic for the extension loading check, by moving it before the distributed_ddl guc check. " 02d4aefb85340dcafea6729141bac033fe3f3ef1,Sven Klemm,Sun Jun 12 15:50:07 2022 +0200,Fix flaky data_node_bootstrap test,Copy collation and chartype before releasing syscache since we need them past the lifetime of the current context. 5c69adfb7e0fcda27d89c1b98eee2f9f94639e5d,Alexander Kuzmenkov,Tue Jun 21 10:56:09 2022 +0300,Add more tests for errors on data nodes,Use a data type with faulty send/recv functions to test various error handling paths. 1a9d775a2c88cee3bd8bb4cf1ddd47cb670a31f3,Nikhil Sontakke,Tue Jun 14 17:29:54 2022 +0530,Fix perms in copy/move chunk,"We mandate that a superuser or a user with REPLICATION privileges can invoke copy_chunk or move_chunk procedures. Internally, many stages are carried out to complete the activity and different stages need different user permissions. To keep things uniform we now switch to the bootstrap superuser in each stage. Care is taken to ensure that the original hypertable ownership is retained on the new chunk post the move operation. " 048d86e7e7a73f30002eeb9f85b25d84c6206cf7,Sven Klemm,Fri Jun 17 21:51:35 2022 +0200,Fix duplicate header guard,The compression code had 2 files using the same header guard. This patch renames the file with floating point helper functions to float_utils.h and renames the other file to compression/api since that more clearly reflects the purpose of the functions. 5cfedda1d0ab379b9745d39a04a8c66293e055f5,Sven Klemm,Mon Jun 20 07:33:32 2022 +0200,Bump pgspot version used in CI to 0.3.2, cb096757fa8abe8eea4d4f1959f641e385ba71ea,Sven Klemm,Thu Jun 16 06:48:17 2022 +0200,Bump Postgres version used in CI to 14.4,Make CI use just released PG 14.4. We skip the version bump for the update script as docker images with 14.4 are nowhere to be seen. 1fbd4116f9c4ce5724d32ca68a205b69624eff28,Pavel Borisov,Wed Jun 15 20:24:50 2022 +0400,Fix tests to be resilient to possible PG planner changes., 42f197e5799b06c7f33198f79a48531c582cdc7b,Fabrízio de Royes Mello,Wed Jun 15 12:01:07 2022 -0300,Explicit constraint names in schema definition,"In `src/ts_catalog/catalog.c` we explicit define some constraints and indexes names into `catalog_table_index_definitions` array, but in our pre-install SQL script for schema definition we don't, so let's be more explicit here and prevent future surprises. " ed55654a32c2b542419bcb57da14f0bda9c55652,Nikhil Sontakke,Thu Jun 16 17:23:42 2022 +0530,Retain hypertable ownership on attach_data_node,If a superuser is used to invoke attach_data_node on a hypertable then we need to ensure that the object created on this data node has the same original ownership permissions. Fixes #4433 d83db1578cec5d3b7393ad0ddd744f416441b805,Konstantina Skovola,Fri Jun 17 09:32:09 2022 +0300,Redirect CI failures report to new channel, 19b3f67b9c5cf92184ba1fce3495df2cc2947e11,Erik Nordström,Fri Jun 3 13:58:44 2022 +0200,Drop remote data when detaching data node,"Add a parameter `drop_remote_data` to `detach_data_node()` which allows dropping the hypertable on the data node when detaching it. This is useful when detaching a data node and then immediately attaching it again. If the data remains on the data node, the re-attach will fail with an error complaining that the hypertable already exists. The new parameter is analogous to the `drop_database` parameter of `delete_data_node`. The new parameter is `false` by default for compatibility and ensures that a data node can be detached without requiring communicating with the data node (e.g., if the data node is not responding due to a failure). Closes #4414 " 56945b37b8920cb7590dd61dc3d8a625e7d650a2,Alexander Kuzmenkov,Wed Jun 1 14:01:40 2022 +0300,Enable ON CONFLICT ON CONSTRAINT for hypertables,It now works since we started to rely on Postgres' arbiter index inference. 1784e83d77bc3004c83af73a0fe6de8307f2583c,Sven Klemm,Sun Jun 12 11:13:10 2022 +0200,Fix segfault in subscription_exec,Add a check for NULL input to subscription_exec as the function currently segfaults on NULL input. Found by sqlsmith. 07c5f7281c0256a57e45e459cf15c944963801ad,Fabrízio de Royes Mello,Thu Jun 9 09:59:55 2022 -0300,Create index on Continuous Aggregates,Timescale 2.7 released a new version of Continuous Aggregate (#4269) that allows users efectivelly create and use indexes in the materialization hypertable. The boring part of it is that users should discover what is the associated materialization hypertable to issue a `CREATE INDEX` statement. Improved it by allowing users to easily create indexes in the materialization hypertable by simple executing a `CREATE INDEX` direct in the Continuous Aggregate. Example: `CREATE INDEX name_of_the_index ON continuous_agregate (column);` 1ac06c71c2bbfd0f878d0078ab9ab01367302c1c,Sven Klemm,Mon Jun 13 08:35:16 2022 +0200,Refactor make_partfunc_call,Refactor make_partfunc_call to only accept fnoid and rettype instead of PartitioningFunc which also gets rid of a coverity warning about that parameter being passed by value instead of reference because that parameter was longer than 128 bytes. 308ce8c47bf36dcb38a729bb6afbe05eaeb4ce9e,Sven Klemm,Sat Jun 11 10:05:06 2022 +0200,Fix various misspellings, eb9f466582762e00ae2aefdece09d658903dfc44,Luigi Servini<39734184+lservini@users.noreply.github.com>,Mon May 9 16:38:44 2022 +0200,Update dump_meta_data.sql,Added `c.dropped is false ` for table _timescaledb_catalog.chunk to skip dropped chunks in the size calculation. f72a277577a76678b8f903230a76567cdbd19718,Fabrízio de Royes Mello,Wed Jun 8 13:54:21 2022 -0300,Fix continuous aggregates deprecated tests,Previous pull request #4269 introduced new format for Continuous Aggregates and we also added regression tests for the `deprecated` version to make sure it will keep working until we decide to completely deprecate and remove the old version. Unfortunately for some deprecated continous aggregates regression tests we miss to set properly the flag `timescaledb.finalized=false`. Fixed it by properly setting the `timecaledb.finalized=false` during the continuous aggregate creation. c6dc0caf1deee6510b6bddd6a0395f0a7368d89f,Rajakavitha Kodhandapani,Tue Jun 7 15:40:58 2022 +0530,Update README.md,fixes: https://github.com/timescale/docs/issues/1162 3c56d3ecebbf476293ff43ded142bc9e5087f6de,Alexander Kuzmenkov,Thu May 26 10:53:48 2022 +0300,Faster lookup of chunks by point,"Don't keep the chunk constraints while searching. The number of candidate chunks can be very large, so keeping these constraints is a lot of work and uses a lot of memory. For finding the matching chunk, it is enough to track the number of dimensions that matched a given chunk id. After finding the chunk id, we can look up only the matching chunk data with the usual function. This saves some work when doing INSERTs. " 216ea65937455dc2194b155e0713093f14a39f96,Sven Klemm,Mon May 23 23:01:46 2022 +0200,Enable chunk exclusion for space dimensions in UPDATE/DELETE,"This patch transforms constraints on hash-based space partitions to make them usable by postgres constraint exclusion. If we have an equality condition on a space partitioning column, we add a corresponding condition on get_partition_hash on this column. These conditions match the constraints on chunks, so postgres' constraint exclusion is able to use them and exclude the chunks. The following transformations are done: device_id = 1 becomes ((device_id = 1) AND (_timescaledb_internal.get_partition_hash(device_id) = 242423622)) s1 = ANY ('{s1_2,s1_2}'::text[]) becomes ((s1 = ANY ('{s1_2,s1_2}'::text[])) AND (_timescaledb_internal.get_partition_hash(s1) = ANY ('{1583420735,1583420735}'::integer[]))) These transformations are not visible in EXPLAIN output as we remove them again after hypertable expansion is done. " ce59820678075875f8db3871caee28b99c635bdd,Sven Klemm,Mon Jun 6 14:57:53 2022 +0200,Fix removal of constified constraints,Commit dcb7dcc5 removed the constified intermediate values used during hypertable expansion but only did so completely for PG14. For PG12 and PG13 some constraints remained in the plan. b6a974e7f3c7439c1ef423ea64431ce2a907e068,Konstantina Skovola,Mon May 23 18:48:09 2022 +0300,Add schedule_interval to policies,Add a parameter `schedule_interval` to retention and compression policies to allow users to define the schedule interval. Fall back to previous default if no value is specified. Fixes #3806 96202a99bdf19f1b459c29d3e12ab42ecf9bc6a5,Sven Klemm,Tue May 31 11:56:43 2022 +0200,Adjust code to PG15 pg_database changes,PG15 changes the type of collate and ctype from name to text. https://github.com/postgres/postgres/commit/54637508 1ef515eb7a5ca0c00f78c552ba9f7c20b67423a5,Sven Klemm,Thu Jun 2 05:55:33 2022 +0200,Add shmem_request_hook,This patch consolidates all shared memory requests in a shmem_request_hook. While there are no strict requirements when to request shared memory for PG < 15 in PG 15 it has to happen in the shmem_request_hook otherwise the request will be blocked. https://github.com/postgres/postgres/commit/4f2400cb f059e00fad24f7438bb4a275b42e56522bebf8a3,Konstantina Skovola,Fri May 27 15:39:31 2022 +0300,Create composite index on segmentby columns,"Previously we created one index per segmentby column, of the form `(col, _ts_meta_sequence_num)`. Compressed data is ordered by segmentby, then by orderby within the segments and lastly by the sequence number of the batch. So if segmentby columns are missing from the index, that index cannot be used to produce ordered output, requiring an extra sort step. A composite index containing all segmentby columns removes the additional sort step and gives better plans. Fixes #4314 " 8f9975d7be856df984763ed837e6b17b36768b4d,Erik Nordström,Mon May 30 16:08:45 2022 +0200,Fix crash during insert into distributed hypertable,"For certain inserts on a distributed hypertable, e.g., involving CTEs and upserts, plans can be generated that weren't properly handled by the DataNodeCopy and DataNodeDispatch execution nodes. In particular, the nodes expect ChunkDispatch as a child node, but PostgreSQL can sometimes insert a Result node above ChunkDispatch, causing the crash. Further, behavioral changes in PG14 also caused the DataNodeCopy node to sometimes wrongly believe a RETURNING clause was present. The check for returning clauses has been updated to fix this issue. Fixes #4339 " 65b0dda97b11224e79f3aa7580b714337718308c,Erik Nordström,Wed Jun 1 17:17:50 2022 +0200,Fix CI coredump handling,The CI task for getting coredumps after crashes of a 32-bit build didn't handle a variable correctly. Fix to avoid errors. cae1f16367ca734b7eca32f7add2f19cc957d537,Sven Klemm,Tue May 31 09:19:11 2022 +0200,Use our implementation of find_em_expr_for_rel for PG15+,PG13 added an implementation of find_em_expr_for_rel to postgres core code. Which is removed again in PG15. This patch adjusts our macros to deal with the removal in PG15. https://github.com/postgres/postgres/commit/f3dd9fe1 f28131cff5b23ede1a316b5d81452918bc815fd9,Konstantina Skovola,Thu May 26 14:39:13 2022 +0300,Don't ask for orderby column if default already set,"When enabling compression on a hypertable, the orderby option can be omitted, which will set the default value of ""time DESC"". However previously, when executing the same command twice not setting orderby, the second time we would get an error that orderby was previously set and must be specified. For example when executing `alter table set (timescaledb.compress, timescaledb.segmentby = '..')` The reason for that error was that it's unclear if no orderby means leave as is, or if it means set the default value. But in the case where orderby is already set to the default value, there is no ambiguity and both cases are equivalent, so the default value can be reset without giving an error. Fixes #4331 " ed948b17721a45301604433c1fd8572ed3fa1901,Alexander Kuzmenkov,Wed Jun 1 15:26:29 2022 +0300,Don't force clang-tidy on,"If the compiler is gcc, clang-tidy might not recognize some of its flags and error out. It will be enabled by default if the compiler is clang. " 533e849c574934eb8f00e254a6467376391622bb,Mats Kindahl,Tue May 24 10:47:57 2022 +0200,Pass parameters to workers as a struct,"Non-functional change. Parameters to workers were passed in as a serialized string, which then needs to be serialized and deserialized using dedicated functions. This commit refactors code to pass parameters to workers as a struct, which is then just copied into the `bgw_extra` field of `BackgroundWorker`. The struct contains simple values and can therefore be copied using memcpy(3c). " 5c0110cbbf644ec79b2d2b0249eb3e112c081c4b,Alexander Kuzmenkov,Fri May 6 14:14:48 2022 +0300,Mark partialize_agg as parallel safe,"Postgres knows whether a given aggregate is parallel-safe, and creates parallel aggregation plans based on that. The `partialize_agg` is a wrapper we use to perform partial aggregation on data nodes. It is a pure function that produces serialized aggregation state as a result. Being pure, it doesn't influence parallel safety. This means we don't need to mark it parallel-unsafe to artificially disable the parallel plans for partial aggregation. They will be chosen as usual based on the parallel-safety of the underlying aggregate function. " 1d0670e703862b284c241ab797404f851b25b5df,Jan Nidzwetzki,Tue May 31 10:33:38 2022 +0200,Fix flaky copy test by generating fixed test data,The copy test is flaky because some test data is generated dynamically based on the current date. This patch changes the data generation to a time series with fixed dates. 1fbe2eb36f3ff4b12023549b76c8c3df0b8a79b9,Sven Klemm,Sat May 28 13:40:10 2022 +0200,Support intervals with month component when constifying now(),"When dealing with Intervals with month component timezone changes can result in multiple day differences in the outcome of these calculations due to different month lengths. When dealing with months we add a 7 day safety buffer. For all these calculations it is fine if we exclude less chunks than strictly required for the operation, additional exclusion with exact values will happen in the executor. But under no circumstances must we exclude too much cause there would be no way for the executor to get those chunks back. " 2715b5564a3be4bb9c3fdb29f81a5c3fb3c0aae6,Sven Klemm,Sun May 29 15:08:55 2022 +0200,Replace pg_atoi with pg_strtoint16/32,"PG 15 removes pg_atoi, so this patch changes all callers to use pg_strtoint16/32. https://github.com/postgres/postgres/commit/73508475 " 12574dc8ecda70d60a3ee0cf4a8489aa23bccb8f,Sven Klemm,Thu May 26 13:34:40 2022 +0200,Support intervals with day component when constifying now(),The initial patch to use now() expressions during planner hypertable expansion only supported intervals with no day or month component. This patch adds support for intervals with day component. If the interval has a day component then the calculation needs to take into account daylight saving time switches and thereby a day would not always be exactly 24 hours. We mitigate this by adding a safety buffer to account for these dst switches when dealing with intervals with day component. These calculations will be repeated with exact values during execution. Since dst switches seem to range between -1 and 2 hours we set the safety buffer to 4 hours. This patch also refactors the tests since the previous tests made it hard to tell the feature was working after the constified values have been removed from the plans. a6b5f9002cf4f3894aa8cbced7f862a73784cada,Alexander Kuzmenkov,Fri May 13 15:49:00 2022 +0300,More clear clang-tidy options,Enable a closed list of checks and treat everything as errors. ecf34132c69e1709cd393eab43b5fcbfd7c201db,Alexander Kuzmenkov,Fri May 13 19:25:18 2022 +0300,Fix clang-tidy warning `readability-redundant-control-flow`, e75274ee7c6eef1dafc9b4f4d9f71e8e88f76813,Alexander Kuzmenkov,Fri May 13 18:22:46 2022 +0300,Fix clang-tidy warning `bugprone-argument-comment`, a3ef0384655d57200e83ad7b13c91a31177b97c1,Alexander Kuzmenkov,Fri May 13 15:46:58 2022 +0300,Fix clang-tidy warning `bugprone-macro-parentheses`, 7aec25d69e45292569fbe27364c2f68d269d416e,Sven Klemm,Wed May 25 20:16:46 2022 +0200,Adjust coccinelle and shellcheck CI config,"Currently coccinelle and shellcheck get run an additional time for every merged commit to master. This patch adjusts the config so they are only run on pull request or on push to prerelease_test instead of push to any branch, similar to how all the other workflows are set up. " c35e9bf611bf52459b1771ac66c0a71a4afe1ad0,Joshua Lockerman,Mon Apr 11 12:45:01 2022 -0400,Function telemetry,"This commit contains extends our telemetry system with function call telemetry. It gathers function call-counts from all queries, and send back counts for those functions that are built in or from our related extensions. " 34bf69544453f0a941f53aa5f2878426bfa5e353,Mats Kindahl,Wed May 25 13:39:16 2022 +0200,Add initializer to auto variable,Compilers are not smart enough to check that `conn` is initialized inside the loop so not initializing it gives an error. Added an initializer to the auto variable to get rid of the error. f0556dc9026342152bf6c34a7d495d5e0b410ee3,Sven Klemm,Wed May 25 10:59:33 2022 +0200,Skip 001_extension test on PG13.2 in CI,The extension test uses the background_psql function which is not present in the 13.2 PostgresNode module. This function is only available in PG 13.5+. https://github.com/postgres/postgres/commit/a9d0a540 0b6a09f027c521dc543b2b3bd3394af593702659,Sven Klemm,Wed May 25 08:10:45 2022 +0200,Add support for SKIPS to provecheck,This patch adds support for skipping individual tests in a provecheck run similar to what we have for our regression checks. dcb7dcc5064a7f843d435482d5c0cf643e7f9021,Sven Klemm,Mon May 23 06:06:58 2022 +0200,Remove constified now() constraints from plan,Commit 35ea80ff added an optimization to enable expressions with now() to be used during plan-time chunk exclusion by constifying the now() expression. The added constified constraints were left in the plan even though they were only required during the hypertable explansion. This patch marks those constified constraints and removes them once they are no longer required. d249954be0407d358ef12590e9a92f2f9926269a,Jan Nidzwetzki,Fri May 20 12:30:50 2022 +0200,Improved buffer management in the copy operator,"The multi-buffer copy optimization creates a multi-insert buffer per ChunkInsertState. However, chunks can be closed. When ts_chunk_dispatch_get_chunk_insert_state is called for a closed chunk again, a new ChunkInsertState is returned. So far, also a new multi-insert buffer has been created. Therefore, multiple batch operations could be executed per chunk, which reduces the efficiency of the optimization. This patch introduces an HTAB that maps from the chunk_id to the multi-insert buffer. Even when a chunk is closed, all tuples for a chunk are stored in the same buffer. " cf9b6267941964fb134c1e2122b91c473a81f7e9,Sven Klemm,Tue May 24 10:42:19 2022 +0200,Post-Release 2.7.0,Adjust version.config and add 2.7.0 to update/downgrade test. 0a682095632ed4c553703e6d27fc7bad6da71c59,Sven Klemm,Wed May 18 12:09:10 2022 +0200,Release 2.7.0,"This release adds major new features since the 2.6.1 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Optimize continuous aggregate query performance and storage * The following query clauses and functions can now be used in a continuous aggregate: FILTER, DISTINCT, ORDER BY as well as [Ordered-Set Aggregate](https://www.postgresql.org/docs/current/functions-aggregate.html#FUNCTIONS-ORDEREDSET-TABLE) and [Hypothetical-Set Aggregate](https://www.postgresql.org/docs/current/functions-aggregate.html#FUNCTIONS-HYPOTHETICAL-TABLE) * Optimize now() query planning time * Improve COPY insert performance * Improve performance of UPDATE/DELETE on PG14 by excluding chunks This release also includes several bug fixes. If you are upgrading from a previous version and were using compression with a non-default collation on a segmentby-column you should recompress those hypertables. **Features** * #4045 Custom origin's support in CAGGs * #4120 Add logging for retention policy * #4158 Allow ANALYZE command on a data node directly * #4169 Add support for chunk exclusion on DELETE to PG14 * #4209 Add support for chunk exclusion on UPDATE to PG14 * #4269 Continuous Aggregates finals form * #4301 Add support for bulk inserts in COPY operator * #4311 Support non-superuser move chunk operations * #4330 Add GUC ""bgw_launcher_poll_time"" * #4340 Enable now() usage in plan-time chunk exclusion **Bugfixes** * #3899 Fix segfault in Continuous Aggregates * #4225 Fix TRUNCATE error as non-owner on hypertable * #4236 Fix potential wrong order of results for compressed hypertable with a non-default collation * #4249 Fix option ""timescaledb.create_group_indexes"" * #4251 Fix INSERT into compressed chunks with dropped columns * #4255 Fix option ""timescaledb.create_group_indexes"" * #4259 Fix logic bug in extension update script * #4269 Fix bad Continuous Aggregate view definition reported in #4233 * #4289 Support moving compressed chunks between data nodes * #4300 Fix refresh window cap for cagg refresh policy * #4315 Fix memory leak in scheduler * #4323 Remove printouts from signal handlers * #4342 Fix move chunk cleanup logic * #4349 Fix crashes in functions using AlterTableInternal * #4358 Fix crash and other issues in telemetry reporter **Thanks** * @abrownsword for reporting a bug in the telemetry reporter and testing the fix * @jsoref for fixing various misspellings in code, comments and documentation * @yalon for reporting an error with ALTER TABLE RENAME on distributed hypertables * @zhuizhuhaomeng for reporting and fixing a memory leak in our scheduler " 26da1b503547a63157e7d974be5b3cdba6e4ac4f,Sven Klemm,Fri May 20 15:34:59 2022 +0200,Show warning for caggs with numeric,Postgres changes the internal state format for numeric aggregates which we materialize in caggs in PG14. This will invalidate the affected columns when upgrading from PG13 to PG14. This patch adds a warning to the update script when we encounter this configuration. 4f58132d37d6a60c3b087d5c34a787228cadbf11,Sven Klemm,Sun May 22 07:27:01 2022 +0200,Fix PG13.2 isolation tests,The deadlock_recompress_chunk isolation test uses syntax not supported by earlier versions of the isolation tester leading to a parse failure when processing that file on PG13.2. This patch skips that particular test on PG13.2. 94ca9c66f420db92ba44384aa10361b39e235b86,Sven Klemm,Fri May 20 21:09:47 2022 +0200,Ignore telemetry isolation test on PG13.2,Commit 7b9d8673 added an isolation test for telemetry but did not add it to the ignore list for earlier postgres versions. PG14 changed the output format for isolation tests which got backported to PG12 and PG13 which makes the output of earlier PG12 and PG13 different from the latest one so we ignore isolation tests on those earlier versions. aa0b36d5bafd30934a7539aa02d5eebc767d6532,Sven Klemm,Fri May 20 17:54:45 2022 +0200,Skip telemetry test on PG12.0,With recent refactorings the telemetry test seems to trigger the same use-after-free bug that got triggered by tablespace test so we skip that test on PG12.0 as well. b845f9423b2cb2905d78b67a2f6671a169748b2e,Sven Klemm,Fri May 20 13:32:02 2022 +0200,Skip tablespace test on PG12.0,The tablespace test causes a segfault on PG12.0 due to an upstream bug in the event trigger handling. 7b9d86735861b925def2bff34f58ea7dbc01038d,Erik Nordström,Thu May 12 12:53:03 2022 +0200,Fix crash and other issues in telemetry reporter,"Make the following changes to the telemetry reporter background worker: - Add a read lock to the current relation that the reporter collects stats for. This lock protects against concurrent deletion of the relation, which could lead to errors that would prevent the reporter from completing its report. - Set an active snapshot in the telemetry background process for use when scanning a relation for stats collection. - Reopen the scan iterator when collecting chunk compression stats for a relation instead of keeping it open and restarting the scan. The previous approach seems to cause crashes due to memory corruption of the scan state. Unfortunately, the exact cause has not been identified, but the change has been verified to work on a live running instance (thanks to @abrownsword for the help with reproducing the crash and testing fixes). Fixes #4266 " 46c95c426c909bf0ffef68a86c1b3724aabcd576,Sven Klemm,Fri May 20 13:04:11 2022 +0200,Ignore pg_dump test on appveyor,On appveyor the pg_dump is flaky and seems to fail every other time. This patch makes appveyor ignore the result of that test. b7472c82ce907490e776d53c76c261db92f855ad,Sven Klemm,Fri May 20 12:41:29 2022 +0200,Remove dead code,Remove noop ternary operator in cagg_rebuild_view_definition. We return if finalized is true on line 2475 so the NIL would never be reached in the ternary operator. Found by coverity. 8c5c7bb4ad6eff5a8c650447c5e7b625389ad40f,Sven Klemm,Thu May 19 20:53:59 2022 +0200,Filter out chunk ids in shared tests,Multinode queries use _timescaledb_internal.chunks_in to specify the chunks from which to select data. The chunk id in regresscheck-shared is not stable and may differ depending on execution order leading to flaky tests. eab4efa323c760e57e3748dfd89bbcf077ab00ac,Sven Klemm,Thu May 19 14:21:53 2022 +0200,Move metrics_dist1 out of shared_setup,The table metrics_dist1 was only used by a single test and therefore should not be part of shared_setup but instead be created in the test that actually uses it. This reduces executed time of regresscheck-shared when that test is not run. 9b91665162df58e8fe40ca80ad269eed3fe328b2,Erik Nordström,Wed May 18 11:56:10 2022 +0200,Fix crashes in functions using AlterTableInternal,"A number of TimescaleDB functions internally call `AlterTableInternal` to modify tables or indexes. For instance, `compress_chunk` and `attach_tablespace` act as DDL commands to modify hypertables. However, crashes occur when these functions are called via `SELECT * INTO FROM ` or the equivalent `CREATE TABLE AS` statement. The crashes happen because these statements are considered process utility commands and therefore sets up an event trigger context for collecting commands. However, the event trigger context is not properly set up to record alter table statements in this code path, thus causing the crashes. To prevent crashes, wrap `AlterTableInternal` with the event trigger functions to properly initialize the event trigger context. " 54d6b41e65729ed28cb84f44cb390ba917a228f2,Dmitry Simonenko,Thu May 19 15:52:12 2022 +0300,Fix move chunk cleanup logic,"Add a new stage ""complete"" in the ""chunk_copy_operation"" to indicate successful move/copy chunk operations. Make the ""cleanup_copy_chunk_operation"" procedure more robust and make it only delete chunk operation entries from the catalog without doing any other unwanted cleanup if called on successful operations. " 8375b9aa536a619a5ac2644e0dae3c25880a4ead,Jan Nidzwetzki,Wed May 18 18:02:38 2022 +0200,Fix a crash in the copy multi-buffer optimization,"This patch solves a crash in the multi-buffer copy optimization, which was introduced in commit bbb2f414d2090efd2d8533b464584157860ce49a. This patch handles closed chunks (e.g., caused by timescaledb.max_open_ chunks_per_insert) properly. The problem is addressed by: 1) Re-reading the ChunkInsertState before the data is stored, which ensures that the underlying table is open. 2) A TSCopyMultiInsertBuffer is deleted after the data of the buffer is flushed. So, operations like table_finish_bulk_insert are executed and the associated chunk can properly be closed. " 43c8e51510b274a648f6919d183d44cfa6d7ce6d,Sven Klemm,Thu May 19 08:03:50 2022 +0200,Fix Var handling for Vars of different level in constify_now,This patch fixes the constify_now optimization to ignore Vars of different level. Previously this could potentially lead to an assertion failure cause the varno of that varno might be bigger than the number of entries in the rangetable. Found by sqlsmith. 5193af739624224b8ef288ec2c71bf78ab08b6bd,Sven Klemm,Wed May 18 18:34:55 2022 +0200,Test attnum stays consistent in update,This patch adds a test for attnum consistency to our update scripts. When attnum between fresh install and updated install is different the updated installation will not be able to correctly process affected catalog tables. f1575bb4c31c2ad3d35a361bcf9c991ee7a37911,Dmitry Simonenko,Wed May 18 18:45:10 2022 +0300,Support moving compressed chunks between data nodes,This change allows to copy or move compressed chunks between data nodes by including compressed chunk into the chunk copy command stages. 11c6813b1d56a0d896b430a98493d7b82c6152dd,Sven Klemm,Wed May 18 12:59:42 2022 +0200,Fix flaky regresscheck-shared,"While we do filter out chunk ids and hypertable ids from the test output, the output was still unstable when those ids switch between single and double digit as that changes the length of the query decorator in EXPLAIN output. This patch removes this decorator entirely from all shared test output. " f266f5cf564fcc5509b91493a39eb201c6f5914a,Fabrízio de Royes Mello,Wed May 11 19:36:58 2022 -0300,Continuous Aggregates finals form,"Following work started by #4294 to improve performance of Continuous Aggregates by removing the re-aggregation in the user view. This PR get rid of `partialize_agg` and `finalize_agg` aggregate functions and store the finalized aggregated (plain) data in the materialization hypertable. Because we're not storing partials anymore and removed the re-aggregation, now is be possible to create indexes on aggregated columns in the materialization hypertable in order to improve the performance even more. Also removed restrictions on types of aggregates users can perform with Continuous Aggregates: * aggregates with DISTINCT * aggregates with FILTER * aggregates with FILTER in HAVING clause * aggregates without combine function * ordered-set aggregates * hypothetical-set aggregates By default new Continuous Aggregates will be created using this new format, but the previous version (with partials) will be supported. Users can create the previous style by setting to `false` the storage paramater named `timescaledb.finalized` during the creation of the Continuous Aggregate. Fixes #4233 " ddd02922c9607b4d81185f7c7e6af8d424ea0328,Nikhil Sontakke,Mon May 9 18:46:10 2022 +0530,Support non-superuser move chunk operations,"The non-superuser needs to have REPLICATION privileges atleast. A new function ""subscription_cmd"" has been added to allow running subscription related commands on datanodes. This function implicitly upgrades to the bootstrapped superuser and then performs subscription creation/alteration/deletion commands. It only accepts subscriptions related commands and errors out otherwise. " 4988dac27349251591595622baa6180c662d59b2,Sven Klemm,Wed May 18 10:59:36 2022 +0200,Fix sqlsmith CI workflow,Commit 3b35da76 changed the setup script for regresscheck-shared to no longer be usable directly by the sqlsmith workflow. This patch set TEST_DBNAME at the top of the script so it is easier to use the script outside of regression check environment. 747b532be6bfa3e93decdcb6f89c9432e5e58b48,Sven Klemm,Wed May 18 04:14:51 2022 +0200,Bump pg version in update test,"Bump PG versions to 12.11, 13.7 and 14.3 in update tests. The update test was skipped with the previous PG version bump because upstream docker images were not yet available which we rely on in the update test. " d1ed09ce98d5f9a6a849e681536ebdc0ef168981,Sven Klemm,Wed May 18 07:26:03 2022 +0200,Fix build failure on PG12.0,Building against PG12.0 failed with test_utils.c:99:5: error: expected ‘;’ before ‘errmsg’ because the errmsg argument was not in parens. 35ea80ffdffcdcf3c81af7881c509f33bbeb9924,Sven Klemm,Mon May 16 13:59:26 2022 +0200,Enable now() usage in plan-time chunk exclusion,"This implements an optimization to allow now() expression to be used during plan time chunk exclusions. Since now() is stable it would not normally be considered for plan time chunk exclusion. To enable this behaviour we convert `column > now()` expressions into `column > const AND column > now()`. Assuming that time always moves forward this is safe even for prepared statements. This optimization works for SELECT, UPDATE and DELETE. On hypertables with many chunks this can lead to a considerable speedup for certain queries. The following expressions are supported: - column > now() - column >= now() - column > now() - Interval - column > now() + Interval - column >= now() - Interval - column >= now() + Interval Interval must not have a day or month component as those depend on timezone settings. Some microbenchmark to show the improvements, I did best of five for all of the queries. -- hypertable with 1k chunks -- with optimization select * from metrics1k where time > now() - '5m'::interval; Time: 3.090 ms -- without optimization select * from metrics1k where time > now() - '5m'::interval; Time: 145.640 ms -- hypertable with 5k chunks -- with optimization select * from metrics5k where time > now() - '5m'::interval; Time: 4.317 ms -- without optimization select * from metrics5k where time > now() - '5m'::interval; Time: 775.259 ms -- hypertable with 10k chunks -- with optimization select * from metrics10k where time > now() - '5m'::interval; Time: 4.853 ms -- without optimization select * from metrics10k where time > now() - '5m'::interval; Time: 1766.319 ms (00:01.766) -- hypertable with 20k chunks -- with optimization select * from metrics20k where time > now() - '5m'::interval; Time: 6.141 ms -- without optimization select * from metrics20k where time > now() - '5m'::interval; Time: 3321.968 ms (00:03.322) Speedup with 1k chunks: 47x Speedup with 5k chunks: 179x Speedup with 10k chunks: 363x Speedup with 20k chunks: 540x " 01c6125de66b142a4a7c676c10c04a182c5f8575,Sven Klemm,Sat May 14 07:20:22 2022 +0200,Reduce repetitions in workflows,Derive postgres major version from full version so we don't have to specify both in matrix. 54f56af3efba2a3735a406ccb0211584055ee440,Sven Klemm,Sat May 14 16:38:13 2022 +0200,Support wildcards in IGNORES,Add support for wildcards in IGNORES for regression tests. a7c8641e04ca01d9e3a7973a7b3c817a430004dd,Sven Klemm,Tue May 17 14:00:07 2022 +0200,Bump postgres versions used in CI,"Bump postgres versions used in CI to 14.3, 13.7 and 12.11. " 952f536636523e98a51c2133cfaaa99a2e2d3e06,Nuno Santos,Fri Feb 11 16:45:31 2022 +0000,Add security policy to repositoy, 79d1d3cb3eec221bfe0fe52d9fd5a70c7369d45f,Konstantina Skovola,Mon May 16 11:51:23 2022 +0300,"Add GUC ""bgw_launcher_poll_time""",This GUC permits configuring the TIMEOUT parameter for the background worker launcher in the loader. Fixes #4217 047d6b175bac4839e90861d25fc25b1446b54bd2,Fabrízio de Royes Mello,Mon May 16 11:47:08 2022 -0300,"Revert ""Pushdown of gapfill to data nodes""",This reverts commit eaf3a38fe9553659e515fac72aaad86cf1a06d1e. 4083e48a1cf62b99e0bc6e40e1ecd454ef5752ec,Fabrízio de Royes Mello,Mon May 16 11:39:53 2022 -0300,"Revert ""Add missing gitignore entry""",This reverts commit 57411719fb1f5e4d5863089bb4b840abea3bc3db. 557023c3a7c5edf6e45c0f6003b75796fdab4b7a,Fabrízio de Royes Mello,Mon May 16 11:35:36 2022 -0300,"Revert ""Ignore flaky ""dist_gapfill_pushdown"" test""",This reverts commit cf83b3b9c2d899fa0156d5c94dc4848b7584e3c9. 3b35da7607072de2d7e8fa29eca80a6258bb6026,Alexander Kuzmenkov,Wed May 11 19:15:32 2022 +0300,More tests for errors when fetching from data nodes,Add a special function that allows to inject these errors. 08a3fca2879a962187ff16709ed6ffadb1d2c057,Jan Nidzwetzki,Mon May 16 08:06:43 2022 +0200,PostgreSQL license file reference added,This patch adds a reference to the PostgreSQL license file. 16d4b3e0b2570622a546625c9a86b9b1752b91b8,Sven Klemm,Sun May 15 16:29:31 2022 +0200,Restructure planner code,This patch moves the apache licensed planner code into it's own subdirectory. c6c64c4021bb475cb488e0c2043b0a8df40baf7b,Alexander Kuzmenkov,Thu May 12 21:58:12 2022 +0300,Remove unused memory allocation in tuplefactory,"It's in a temporary context, so not a memory leak, but it's just not needed. In passing, clean up the tuple factory error handling code, again. " 92f7e5d36164c1fd955c8b9ef5ed5cd4837ae026,Nikhil Sontakke,Thu May 5 18:57:35 2022 +0530,Support op_id in copy/nove chunk API,"Allow users to specify an explicit ""operation_id"" while carrying out a copy/move operation. If it's specified then that is used as the identifier for the copy/move operation. Otherwise, am implicit id as before gets created and used. " 1d504d5153e757207dc57441fa93a96a0b2a4957,Mats Kindahl,Thu May 12 14:28:03 2022 +0200,Remove printouts from signal handlers,"In #4199 existing calls of `ereport` were replaced with calls of `write_stderr` to eliminate the use of signal-unsafe calls, in particular calls to `malloc`. Unfortunately, `write_stderr` contains a call to `vfprintf`, which allocates memory as well, occationally causing servers that are shutting down to become unresponsive. Since the existing signal handlers just called `die` after printing out a useful message, this commit fixes this by using `die` as a signal handler. Fixes #4200 " 356dfa8eb1df807e20af7691d4c0dcb5312f3be5,Fabrízio de Royes Mello,Thu May 12 10:05:10 2022 -0300,Refactor upgrade/downgrade setup sql test scripts,The setup scripts for upgrade/downgrade tests of Continuous Aggregates has too many duplicated code for pre-2.0 tests. Refactor it a bit removing the duplicated code by using `\if \else \endif` psql meta-commands. Also added a properly `round` function to all functions that returns `float8` in SQL scripts because in rare cases it lead to flaky tests. This is part of #4269. 103b0f53bc2722ef00b2f3f3ee049199e65e34f0,Sven Klemm,Thu May 12 11:53:48 2022 +0200,Bump pgspot version used in CI to 0.3.0, efce68ce0de1789b928306933495a9fe731d5c0a,Sven Klemm,Thu May 12 07:56:57 2022 +0200,Remove _PG_fini functions,_PG_fini is meant to be the hook to run code on module unload but this code is not working and has not been working for over 12 years because upstream does not support it. https://github.com/postgres/postgres/commit/ab02d702ef08343fba30d90fdf7df5950063e8c9 bbb2f414d2090efd2d8533b464584157860ce49a,Jan Nidzwetzki,Thu May 5 09:28:00 2022 +0200,Multi-buffer copy optimization backport,"This commit backports the Postgres multi-buffer / bulk insert optimization into the timescale copy operator. If the target chunk allows it (e.g., if no triggers are defined on the hypertable or the chunk is not compressed), the data is stored in in-memory buffers first and then flushed to the chunks in bulk operations. Implements: #4080 " 5d56b1cdbcff50c868b067f156d6a7a77d7dda6f,gayyappan,Wed Apr 6 11:57:33 2022 -0400,Add api _timescaledb_internal.drop_chunk,Add an internal api to drop a single chunk. This function drops the storage and metadata associated with the chunk. Note that chunk dependencies are not affected. e.g. Continuous aggs are not updated when this chunk is dropped. b32ccd03631e40056bdee876db7e524251597225,lijunlong,Tue May 10 05:29:20 2022 -0400,bugfix: memory leak when call list_qsort., c73c5a74b9d40491958c069fd2d6f671845afadd,Alexander Kuzmenkov,Wed May 11 10:25:09 2022 +0300,Avoid extra projection when scanning a compressed chunk,"We don't have to project the result of the underlying scan of DecompressChunk, because the latter can selectively decompress the resulting columns anyway. The projection slows down the things especially when we use JIT, when the projection has to be compiled for each chunk. To avoid this unnecessary work, use the physical targetlist for compressed scan if we can. This commit rewrites the function that determines what we have to decompress, and makes it use the targetlist generated by the planner, instead of building it from scratch. Then we can give it the physical targetlist when possible. " a0e25d9e2447abf17984dd12e5ca6ba04c68f475,Sven Klemm,Wed May 11 01:56:37 2022 +0200,Remove Ubuntu 21.04 from packages tests,Ubuntu 21.04 is EOL since january so we no longer need to support and test it. 9f4dcea30135d1e36d1c452d631fc8b8743b3995,gayyappan,Thu Apr 7 17:10:47 2022 -0400,Add _timescaledb_internal.freeze_chunk API,"This is an internal function to freeze a chunk for PG14 and later. This function sets a chunk status to frozen. Operations that modify the chunk data (like insert, update, delete) are not supported. Frozen chunks can be dropped. Additionally, chunk status is cached as part of classify_relation. " e81e32fe5c56c39b67f2b24942deed26c0552388,Fabrízio de Royes Mello,Mon May 9 10:37:30 2022 -0300,Telemetry Stats for CAggs finals form,Introduced by #4294 and #4269 PRs the default implementation of Continuous Aggregates get rid of `chunk_id` in the materialization hypertable and `partialize_agg`/`finalize_agg` aggregate functions. A new counter named `num_caggs_finalized` was added to telemetry report in this PR to count the number of Continuos Aggregates created in this new format. bb241ffcd18b9f3c279e35a0873e0d91f82670c0,Sven Klemm,Mon May 9 06:41:22 2022 +0200,Add missing ignores for dist_gapfill_pushdown,The patch to ignore result of dist_gapfill_pushdown missed some places that overwrite installcheck_args. 958489fedbae1905eacbde18e3ea6a2b55358921,Sven Klemm,Mon May 9 06:28:50 2022 +0200,Disable downgrade test for ubuntu jammy,ubuntu jammy only has 1 timescaledb version packaged at the moment so downgrade is not possible. 1e8d37b54ebfed36ff117f208064f37c8916a0f3,Fabrízio de Royes Mello,Fri Apr 29 16:03:36 2022 -0300,Remove `chunk_id` from materialization hypertable,First step to remove the re-aggregation for Continuous Aggregates is to remove the `chunk_id` from the materialization hypertable. Also added new metadata column named `finalized` to `continuous_cagg` catalog table in order to store information about the new following finalized version of Continuous Aggregates that will not need the partials anymore. This flag is important to maintain backward compatibility with previous Continuous Aggregate implementation that requires the `chunk_id` to refresh data properly. 6e26a1187a1a6c89156deb387ab425a8b10866bb,Alexander Kuzmenkov,Mon Apr 25 17:53:16 2022 +0300,Use binary format in row-by-row fetcher,"The general idea is to have two types of fetcher: ""fast"" and ""general purpose"". We use the row-by-row fetcher as the ""fast"" one. This commit removes support of text protocol in this fetcher, because it's only needed for some niche types that don't have a binary serialization, and is also slower than binary one. Because the row-by-row fetcher now only understands binary protocol, we must check that the binary serialization is actually available for the participating data types. If not, we have to revert to using the cursor fetcher unless row-by-row was explicitly requested by the user. This happens at execution time, precisely, at creation of TupleFactory, because that's when we look up the conversion functions. The rest of the commit is removing the text protocol support from row-by-row, plus EXPLAIN changes (we don't know the fetcher type at the planning stage anymore, so not showing it). " ede62330ee7c83500d57e03be38d3a736114ad55,James Guthrie,Thu May 5 18:24:50 2022 +0200,Run apt package tests against Ubuntu 22.04 (Jammy Jellyfish), b48a727b77d306f8298479ca19b6810297faaaf7,Konstantina Skovola,Wed May 4 18:44:44 2022 +0300,Fix refresh window cap for cagg refresh,Fixes #4252 e7076f3be1dbd228b8dafe75e1e90ada6c18b667,Sven Klemm,Wed May 4 14:19:58 2022 +0200,Remove unused function prototypes,Commit 01c724b9 added some prototypes to extension.h but did not add an implementation. This patch removes the unused prototypes. 9b5b86c490bf314b9a709a5e0cdbc9fe525ca444,Alexander Kuzmenkov,Mon May 2 18:52:55 2022 +0300,Make the test remote_copy stable by adding ORDER BYs,Currently it depends on the (unspecified) order in which the original rows were inserted. 9b63f4aadd98a7ee57f504a1f6bb80037c772785,Sven Klemm,Wed Apr 27 13:16:09 2022 +0200,Add more descriptive comment to our CI workflows, cf83b3b9c2d899fa0156d5c94dc4848b7584e3c9,Konstantina Skovola,Fri Apr 29 11:44:00 2022 +0300,"Ignore flaky ""dist_gapfill_pushdown"" test","This is a *temporary* change to our workflows to ignore the test because it is flaky and requires many jobs to be restarted before a commit can be merged. Does not address the issue that causes the flakiness, which needs to be investigated. " a4081516cae24f17eeb6241c758d84d90db3d4f6,Sven Klemm,Mon May 2 07:16:55 2022 +0200,Append pg_temp to search_path,Postgres will prepend pg_temp to the effective search_path if it is not present in the search_path. While pg_temp will never be used to look up functions or operators unless explicitly requested pg_temp will be used to look up relations. Putting pg_temp in search_path makes sure objects in pg_temp will be considered last and pg_temp cannot be used to mask existing objects. 9012e2a20d16d5219dde6260127c39cc748b8f87,Alexander Kuzmenkov,Thu Apr 28 12:08:37 2022 +0300,Do not create a memory context for each Chunk,"For some reason, we create a MemoryContext for each Chunk. This context then is almost never used. Just don't do this. " 4f67ed0656be4168b43b2bb465b5c1c426b1853e,Sven Klemm,Fri Apr 29 00:15:45 2022 +0200,Fix unqualified type reference in time_bucket,This was not caught earlier and is not currently caught by CI because the check for unqualified casts is currently only in main branch of pgspot and not yet in the tagged version we use as part of PR checks. 97078dde057cf24f0e93176cf72806d3321a2294,Alexander Kuzmenkov,Fri Apr 22 14:15:18 2022 +0300,Enable DecompressChunk node to skip some input columns,"No functional change. This refactoring adds explicit output attnos to the metadata of column decompression, enabling DecompressChunk node to skip some input columns of the compressed scan. This is ultimately needed to support physical targetlists in compressed scan. " 687e7c7233d898c08aa3b1a461b0b7296beb0a3e,Konstantina Skovola,Thu Apr 21 14:22:58 2022 +0300,"Fix option ""timescaledb.create_group_indexes""","Previously this option was ignored when creating a continuous aggregate, even when explicitly set to true. Fixes #4249 " 3e042bd9dc3389bd476e4f9a1bd9c14571b438e7,gayyappan,Wed Apr 20 13:57:43 2022 -0400,Fix compress_chunk error message,"When we have a hypertable with a cagg defined on it, a call to the compress_chunk with the hypertable's chunk returns an unexpected status error. Fix the error message to return ""compression not enabled"" " aaffc1d5a69e5242067dd7ce81e5096e3765ec85,Mats Kindahl,Fri Apr 22 19:10:45 2022 +0200,Set null vector for insert into compressed table,"As part of inserting into a compressed table, the tuple is materialized, which computes the data size for the tuple using `heap_compute_data_size`. When computing the data size of the tuple, columns that are null are not considered and are just ignored. Columns that are dropped are, however, not explicitly checked and instead the `heap_compute_data_size` rely on these columns being set to null. When reading tuples from a compressed table for insert, the null vector is cleared, meaning that it by default is non-null. Since columns that are dropped are not explicitly processed, they are expected to have a defined value, which they do not have, causing a crash when an attempt to dereference them are made. This commit fixes this by setting the null vector to all null, and the code after will overwrite the columns with proper null bits, except the dropped columns that will be considered null. Fixes #4251 " e3f0fbfeff052c72897c527f6cb71c5e83ee4a39,Mats Kindahl,Mon Apr 25 13:36:13 2022 +0200,Disable 001_extension test on PG12,"PostgreSQL code for PG12 does not have the method `background_psql` available in `PostgresNode` class for use in TAP testing, causing the test `001_extension` to fail. Fixes #4226 " 9fda658075d28b83a5f77fa523f66281648643ba,Sven Klemm,Sat Apr 23 21:34:22 2022 +0200,Fix flaky remote_connection test,"Postgres changed the connection error reporting in the stable branches of PG12, PG13 and PG14 leading to our snapshot tests always failing. This patch removes the checks that now fail with the upstream change. https://github.com/postgres/postgres/commit/ae27b1ac " a1e76d2e84a0b5c53bf8864f51c7cc365be2e231,Alexander Kuzmenkov,Thu Apr 21 16:56:05 2022 +0300,Follow-up for compressed chunk collation #4236,Add a changelog message and add a check for broken chunks to the update script. 730a72880ee4af60b1380c8bd145800fa0d5b133,Alexander Kuzmenkov,Thu Apr 21 11:22:31 2022 +0300,NFC. Don't add whole row var to compressed scan tlist,No functional change. Construction a whole row var with compressed columns doesn't make sense. It has to be constructed from decompressed columns by projection of DecompressChunk node. 368e9bb702574a9fc4de9dbdf712432e12668b64,Sven Klemm,Thu Apr 21 14:05:36 2022 +0200,Add pgspot to CI,Add pgspot to check our installation scripts for following best practices. d137c1aa831b9178cae94d3a91b41ab2a3601f7d,Sven Klemm,Fri Apr 22 09:34:31 2022 +0200,Fix unsafe procedure creation in update script,post_update_cagg_try_repair was created with `CREATE OR REPLACE` instead of CREATE. Additionally the procedure was created in the public schema. This patch adjusts the procedure to be created with `CREATE` and in a timescaledb internal schema. Found by pgspot. ebd232c4fc64c1d59fa75b0707a9317bf0a00ef9,Sven Klemm,Fri Apr 22 11:11:16 2022 +0200,Ignore spelling fixes in git blame,This patch adds the spelling fix commit to the git blame ignore list and adds a thank you to the changelog for the author. The git blame change couldnt be done in the spelling PR because it references the commit hash. 68aec9593c0f37dddbaa4f2e2b34a9ba3f5b11d9,Josh Soref,Mon Mar 14 11:11:06 2022 -0400,Fix various misspellings,"This patch fixes various misspellings of committed, constraint and insufficient in code, comments and documentation. " b0695e788a7589189b334fb4c72a1176c33ba326,Fabrízio de Royes Mello,Wed Apr 20 16:17:14 2022 -0300,Fix flaky distributed grant tests,Commit 9b180915 introduce some tests that are a bit flaky because are checking a DEBUG message from Postgres catcache rehashing. Fixed it by teaching our test runner to ignore this Postgres DEBUG message. eeb9d133170140540eb41ee8dfb54ebc769b7781,Alexander Kuzmenkov,Thu Apr 21 13:27:26 2022 +0300,Avoid compiling extra projection when constifying the tableoid column,"If we didn't constify anything, no need to recompile the projection. It can be costly if we're using JIT. " a2c39b9afedc705513725a9a56b2c5c832e6e13f,Sven Klemm,Thu Apr 21 04:03:45 2022 +0200,Fix logic bug in init_privs query,The query to get the list of saved privileges during extension upgrade had a bug and only applying the classoid restriction for a subset of the entries when it should have been applied to all returned rows leading to a failure during extension update when init privileges for other classoids existed on any of the relevant objects. fca865ced948b8cd605fc8dad55a2d0311984b1e,Sven Klemm,Tue Apr 19 08:49:12 2022 +0200,Mark hypertable parent as dummy rel for UPDATE,When postgres expands an inheritance tree it also adds the parent hypertable as child relation. Since for a hypertable the parent will never have any data we can mark this relation as dummy relation so it gets ignored in later steps. This is only relevant for code paths that use the postgres inheritance code as we don't include the hypertable as child when expanding the hypertable ourself. This is similar to 3c40f924 which did the same adjustment for DELETE. This patch also moves the marking into get_relation_info_hook so it happens a bit earlier and prevents some additional cycles. 6d40c30d10ef43e8428bd6b3a1415bc5c282ea3a,Sven Klemm,Wed Apr 20 13:45:12 2022 +0200,Fix DELETE statement trigger on hypertables for PG14,Commit 3c40f924 accidently broke DELETE statement triggers on PG14 that were only defined on the hypertable itself. This patch fixes the issue and also makes the trigger test no longer pg version specific. 0f70ae87bfb32122e94978b1d12ccb0ccea58ff4,Sven Klemm,Tue Apr 19 04:45:31 2022 +0200,Fix flaky truncate test,Change truncate test to ignore warnings about potentially orphaned files when dropping the test database. This seems to happen quite frequently on appveyor causing the test to be flaky. 3c40f924bedbffeb32236c51d0aecdcaa442368a,Sven Klemm,Mon Apr 18 13:08:00 2022 +0200,Mark hypertable parent as dummy rel,When postgres expands an inheritance tree it also adds the parent hypertable as child relation. Since for a hypertable the parent will never have any data we can mark this relation as dummy relation so it gets ignored in later steps. This is only relevant for code paths that use the postgres inheritance code as we don't include the hypertable as child when expanding the hypertable ourself. 0ab2d39f25d75bec08e724acc9ee5a457b51882e,Alexander Kuzmenkov,Thu Apr 14 19:13:09 2022 +0300,Set correct collation for segmentby columns of compressed chunks,"We don't do this currently, so some queries return the wrong ordering of rows if there is an index on the compressed chunk. The fix only works for the newly created chunks. We could add a migration that corrects the old compressed chunks, but it seems to be too heavy and not to lend itself well to automation -- we'll have to recreate the indexes if there are any. So the old chunks continue to return a wrong result. " 472a68726c04f608421626dfef239be199e8b380,Sven Klemm,Sun Apr 17 19:51:30 2022 +0200,Make hook prefixes consistent,Replace the timescale_ prefix on hooks to timescaledb_ because the latter is the correct prefix and to be consistent with names of our other hooks. fae7e7cfdc51bf0fb5487e96c1a2029b0dca9546,Sven Klemm,Sun Apr 17 19:47:17 2022 +0200,Don't export local reorder function,Change reorder_rel function to not be exported and also removes the timescale_ function name prefix from local functions. fab16f3798d7219532f9c5a6587b52073b0489d5,Markos Fountoulakis,Mon Apr 18 11:31:01 2022 +0300,Fix segfault in Continuous Aggregates,"Add the missing variables to the finalization view of Continuous Aggregates and the corresponding columns to the materialization table. Cover the case of targets that contain Aggref nodes and Var nodes that are outside of the Aggref nodes at the same time. Stop rebuilding the Continuous Aggregate view with ALTER MATERIALIZED VIEW. Attempt to repair the view at post-update time instead, and fail gracefully if it is not possible to do so without raw hypertable schema or data modifications. Stop rebuilding the Continuous Aggregate view when switching realtime aggregation on and off. Instead, manipulate the User View by either: 1. removing the UNION ALL right-hand side and the WHERE clause when disabling realtime aggregation 2. adding the Direct View to the right of a UNION ALL operator and defining WHERE clauses with the relevant watermark checks when enabling realtime aggregation Fixes #3898 " f3ad916912e640b2a3eb006c2a0d49783f63c8c9,Markos Fountoulakis,Thu Apr 14 15:19:49 2022 +0300,Fix Coverity defect 376602,Fix the potential NULL pointer dereference in ts_dimension_slice_scan_iterator_get_by_id(). ca6122e01a12938a4049b193aa422c3477008a7c,Sven Klemm,Sun Apr 17 23:18:54 2022 +0200,Fix downgrade test git failure,Due to a security vulnerability in git recent git versions now check ownership of files and error out if it doesnt match current user. Since we bind mount the source checkout into the build container the user of the checkout is unlikely to match the user inside the container. This patch configures git to skip the owner check for the bind-mounted directory. https://github.blog/2022-04-12-git-security-vulnerability-announced/ bb6dc173a0e23981404746a032e487cfd16f73fb,Fabrízio de Royes Mello,Fri Apr 8 17:10:12 2022 -0300,Set PG14 as default scripts version,Set latest PG14 as the default Postgres version for scripts and some minor fixes. bdaa4607d4fa2322e65f779a24f3fa42f47b5b52,Sven Klemm,Mon Apr 11 21:46:57 2022 +0200,Post release 2.6.1,Add 2.6.1 to update and downgrade tests. 731a39a1226e31efb2d1715156ba2ea2cade3195,Konstantina Skovola,Fri Apr 8 14:59:58 2022 +0300,Fix TRUNCATE error as non-owner on hypertable,"Stop throwing error ""must be owner of hypertable"" when a user with TRUNCATE privilege on the hypertable attempts to TRUNCATE. Previously we had a check that required TRUNCATE to only be performed by the table owner, not taking into account the user's TRUNCATE privilege, which is sufficient to allow this operation. Fixes #4183 " 57411719fb1f5e4d5863089bb4b840abea3bc3db,Fabrízio de Royes Mello,Fri Apr 8 17:01:46 2022 -0300,Add missing gitignore entry,Pull request #4033 introduced a new template SQL test file but missed to add the properly gitgnore entry to ignore generated test files. 4d47271ad3c365a87c74b911ab16bdf13c0846bd,Rafia Sabih,Tue Apr 5 13:02:29 2022 +0200,2.6.1 (2022-04-11) This release is patch release. We recommend that you upgrade at the next available opportunity.,**Bugfixes** * #3974 Fix remote EXPLAIN with parameterized queries * #4122 Fix segfault on INSERT into distributed hypertable * #4142 Ignore invalid relid when deleting hypertable * #4159 Fix ADD COLUMN IF NOT EXISTS error on compressed hypertable * #4161 Fix memory handling during scans * #4186 Fix owner change for distributed hypertable * #4192 Abort sessions after extension reload * #4193 Fix relcache callback handling causing crashes **Thanks** * @abrownsword for reporting a crash in the telemetry reporter * @daydayup863 for reporting issue with remote explain eaf3a38fe9553659e515fac72aaad86cf1a06d1e,Rafia Sabih,Fri Jan 28 10:00:11 2022 +0100,Pushdown of gapfill to data nodes,"Allow the calls of time_bucket_gapfill to be executed at the data nodes for improved query performance. With this, time_bucket_gapfill is pushed to data nodes in the following conditions, 1. when only one data node has all the chunks 2. when space dimension does not overlap across data nodes 3. when group-by matches space dimension " 1b2926c07626280245efddde3fe4103c79d85a2e,Mats Kindahl,Wed Apr 6 17:30:18 2022 +0200,Do not modify aggregation state in finalize,"The function `tsl_finalize_agg_ffunc` modified the aggregation state by setting `trans_value` to the final result when computing the final value. Since the state can be re-used several times, there could be several calls to the finalization function, and the finalization function would be confused when passed a final value instead of a aggregation state transition value. This commit fixes this by not modifying the `trans_value` when computing the final value and instead just returns it (or the original `trans_value` if there is no finalization function). Fixes #3248 " ae50a534852277dc0e2affaaeeedcdac7203b41e,Sven Klemm,Wed Apr 6 11:04:58 2022 +0200,Add chunk exclusion for UPDATE for PG14,"Currently only IMMUTABLE constraints will exclude chunks from an UPDATE plan, with this patch STABLE expressions will be used to exclude chunks as well. This is a big performance improvement as chunks not matching partitioning column constraints don't have to be scanned for UPDATEs. Since the codepath for UPDATE is different for PG < 14 this patch only adds the optimization for PG14. With this patch the plan for UPDATE on hypertables looks like this: Custom Scan (HypertableModify) (actual rows=0 loops=1) -> Update on public.metrics_int2 (actual rows=0 loops=1) Update on public.metrics_int2 metrics_int2_1 Update on _timescaledb_internal._hyper_1_1_chunk metrics_int2 Update on _timescaledb_internal._hyper_1_2_chunk metrics_int2 Update on _timescaledb_internal._hyper_1_3_chunk metrics_int2 -> Custom Scan (ChunkAppend) on public.metrics_int2 (actual rows=0 loops=1) Output: '123'::text, metrics_int2.tableoid, metrics_int2.ctid Startup Exclusion: true Runtime Exclusion: false Chunks excluded during startup: 3 -> Seq Scan on public.metrics_int2 metrics_int2_1 (actual rows=0 loops=1) Output: metrics_int2_1.tableoid, metrics_int2_1.ctid Filter: (metrics_int2_1.""time"" = length(version())) " ff945a7a9449ee3b2b6c1b7cb78f8fa87dbdaa4e,Alexander Kuzmenkov,Mon Apr 4 12:50:22 2022 +0300,Data node scan doesn't support system columns: move this check to an appropriate place,"Before, we would complain that we don't support fetching the system columns with per-data node queries enabled, but still execute the code that fetches it. Don't do this and complain earlier. " a064fd3b483abdd01d1d1b6f86b6a4cc1e6e6b98,Konstantina Skovola,Mon Apr 4 14:23:58 2022 +0300,Add logging for retention policy,Also remove unused code from compression_api. The function policy_compression_get_verbose_log was unused. Moved it to policy_utils and renamed to policy_get_verbose_log so that it can be used by all policies. f2e43900c9248bb869d30496af9a703915f5c01d,Konstantina Skovola,Mon Apr 4 09:22:23 2022 +0300,Update changelog for 4159, a4b151b024ea6398595041dbb2660f27c3465790,Dmitry Simonenko,Mon Apr 4 09:40:36 2022 +0300,Fix owner change for distributed hypertable,Allow ALTER TABLE OWNER TO command to be used with distributed hypertable. Fix #4180 4ee8872177908f3ae48be1491cb098fae577e54d,Alexander Kuzmenkov,Thu Mar 31 17:49:31 2022 +0300,Use virtual tuples in row-by-row fetcher,"We needlessly form/deform the heap tuples currently. Sometimes we do need this when we have row marks and need a ctid (UPDATE RETURNING), but not in this case. The implementation has three parts: 1. Change data fetcher interface to store a tuple into given slot instead of returning a heap tuple. 2. Expose the creation of virtual tuple in tuple factory. 3. Use these facilities in row-by-row fetcher. This gives some small speedup. It will become more important in the future, as other parts of row-by-row fetcher are optimized. " e16908ccd73f212ce75ed646d8fa9b72a37c65bc,Sven Klemm,Fri Apr 1 09:51:39 2022 +0200,Ignore bulk formatting changes in git blame,This patch adds a .git-blame-ignore-revs that contains a list of commits with bulk formatting changes to be ignored with git blame. This file will be used by GitHub but to use it locally you need to tell git about it eg. with the following command: `git config blame.ignoreRevsFile .git-blame-ignore-revs` a154ae56e969a73e1431d6c0a1cac4989f4163ac,Konstantina Skovola,Thu Mar 31 23:26:55 2022 +0300,Fix ADD COLUMN IF NOT EXISTS error on compressed hypertable,"Stop throwing exception with message ""column of relation already exists"" when running the command ALTER TABLE ... ADD COLUMN IF NOT EXISTS ... on compressed hypertables. Fix #4087 " 972afe0096149001d204e87f8cc6addea008df52,Erik Nordström,Fri Mar 25 14:48:35 2022 +0100,Add TAP tests for extension state,"Add a TAP test that checks that the extensions state is updated across concurrent sessions/backends when the extension is ""dropped"" or ""created"". " 01c724b9be2ca0eb04d30b043152f051c9b7ea36,Erik Nordström,Thu Mar 24 09:31:52 2022 +0100,Fix relcache callback handling causing crashes,"Fix a crash that could corrupt indexes when running VACUUM FULL pg_class. The crash happens when caches are queried/updated within a cache invalidation function, which can lead to corruption and recursive cache invalidations. To solve the issue, make sure the relcache invalidation callback is simple and never invokes the relcache or syscache directly or indirectly. Some background: The extension is preloaded and thus have planner hooks installed irrespective of whether the extension is actually installed or not in the current database. However, the hooks need to be disabled as long as the extension is not installed. To avoid always having to dynamically check for the presence of the extension, the state is cached in the session. However, the cached state needs to be updated if the extension changes (altered/dropped/created). Therefore, the relcache invalidation callback mechanism is (ab)used in TimescaleDB to signal updates to the extension state across all active backends. The signaling is implemented by installing a dummy table as part of the extension and any invalidation on the relid for that table signals a change in the extension state. However, as of this change, the actual state is no longer determined in the callback itself, since it requires use of the relcache and causes the bad behavior. Therefore, the only thing that remains in the callback after this change is to reset the extension state. The actual state is instead resolved on-demand, but can still be cached when the extension is in the installed state and the dummy table is present with a known relid. However, if the extension is not installed, the extension state can no longer be cached as there is no way to signal other backends that the state should be reset when they don't know the dummy table's relid, and cannot resolve it from within the callback itself. Fixes #3924 " 9c46a5d5c6e2d5020db5c07de52b2fc15e0e4131,Mats Kindahl,Thu Mar 24 12:46:22 2022 +0100,Abort sessions after extension reload,"If a session is started and loads (and caches, by OID) functions in the extension to use them in, for example, a `SELECT` query on a continuous aggregate, the extension will be marked as loaded internally. If an `ALTER EXTENSION` is then executed in a separate session, it will update `pg_extension` to hold the new version, and any other sessions will see this as the new version, including the session that already loaded the previous version of the shared library. Since the pre-update session has loaded some functions from the old version already, running the same queries with the old named functions will trigger a reload of the new version of the shared library to get the new functions (same name, but different OID), but since this has already been loaded in a different version, it will trigger an error that GUC variables are re-defined. Further queries after that will then corrupt the database causing a crash. This commit fixes this by recording the version loaded rather than if it has been loaded and check that the version did not change after a query has been analyzed (in the `post_analyze_hook`). If the version changed, it will generate a fatal error to force an abort of the session. Fixes #4191 " 915bd032bda0e99c276c719595b0bf92e2e8ce43,Konstantina Skovola,Fri Mar 18 13:22:37 2022 +0200,Fix spelling errors and omissions, 81b71b685ce56388e0851c4362de36781cfcc646,Mats Kindahl,Tue Mar 29 18:16:11 2022 +0200,Remove signal-unsafe calls from signal handlers,Functions `elog` and `ereport` are unsafe to use in signal handlers since they call `malloc`. This commit removes them from signal handlers. Fixes #4200 347b45f109e0a7ce642377715b931fe900333e2f,Sven Klemm,Tue Feb 1 00:32:51 2022 +0100,Add chunk exclusion for DELETE for PG14,"Currently only IMMUTABLE constraints will exclude chunks from a DELETE plan, with this patch STABLE expressions will be used to exclude chunks as well. This is a big performance improvement as chunks not matching partitioning column constraints don't have to be scanned for DELETEs. Additionally this improves usability of DELETEs on hypertables with some chunks compressed. Previously you weren't able to do DELETE on those hypertables which had non-IMMUTABLE constraints. Since the codepath for DELETE is different for PG < 14 this patch only adds the optimization for PG14. With this patch the plan for DELETE on hypertables looks like this: Custom Scan (HypertableModify) (actual rows=0 loops=1) -> Delete on metrics (actual rows=0 loops=1) Delete on metrics metrics_1 Delete on _hyper_5_8_chunk metrics Delete on _hyper_5_11_chunk metrics Delete on _hyper_5_12_chunk metrics Delete on _hyper_5_13_chunk metrics Delete on _hyper_5_14_chunk metrics_2 -> Custom Scan (ChunkAppend) on metrics (actual rows=1 loops=1) Chunks excluded during startup: 4 -> Seq Scan on metrics metrics_1 (actual rows=0 loops=1) Filter: (""time"" > (now() - '3 years'::interval)) -> Bitmap Heap Scan on _hyper_5_14_chunk metrics_2 (actual rows=1 loops=1) Recheck Cond: (""time"" > (now() - '3 years'::interval)) Heap Blocks: exact=1 -> Bitmap Index Scan on _hyper_5_14_chunk_metrics_time_idx (actual rows=1 loops=1) Index Cond: (""time"" > (now() - '3 years'::interval)) " 935684c83a4de3f82dabf1d2aa0ae7bae3a18ca2,Alexander Kuzmenkov,Wed Mar 9 19:31:28 2022 +0300,Cache whether a rel is a chunk in classify_relation,Use a per-query hash table for this. This speeds up the repeated calls to classify_relation by avoiding the costly chunk lookup. ae79ba6eb4d13a2f4bd6140bc2db5ea8e192385c,Alexander Kuzmenkov,Tue Mar 15 18:41:40 2022 +0300,Scan less chunk metadata when planning ForeignModify,"Instead of loading the entire Chunk struct, just look up the data nodes. " fb8dec9fa4f60141680801fba9919bec6f47f877,Rafia Sabih,Wed Feb 2 15:51:34 2022 +0100,Update comments to Postgresql standard style, c1cf067c4f63c5a40fd61a1782b3422d11336090,Erik Nordström,Tue Feb 22 19:03:43 2022 +0100,Improve restriction scanning during hypertable expansion,"Improve the performance of metadata scanning during hypertable expansion. When a hypertable is expanded to include all children chunks, only the chunks that match the query restrictions are included. To find the matching chunks, the planner first scans for all matching dimension slices. The chunks that reference those slices are the chunks to include in the expansion. This change optimizes the scanning for slices by avoiding repeated open/close of the dimension slice metadata table and index. At the same time, related dimension slice scanning functions have been refactored along the same line. An index on the chunk constraint metadata table is also changed to allow scanning on dimension_slice_id. Previously, dimension_slice_id was the second key in the index, which made scans on this key less efficient. " 966c5eb2c2007374371b3880a33c06d8446a2724,Nikhil Sontakke,Thu Mar 17 14:02:31 2022 +0530,Fix remote EXPLAIN with parameterized queries,"In certain multi-node queries, we end up using a parameterized query on the datanodes. If ""timescaledb.enable_remote_explain"" is enabled we run an EXPLAIN on the datanode with the remote query. EXPLAIN doesn't work with parameterized queries. So, we check for that case and avoid invoking a remote EXPLAIN if so. Fixes #3974 Reported and test case provided by @daydayup863 " e101b3ea60760f1091fa19634c1c15982d4768dd,Sven Klemm,Fri Mar 18 16:24:40 2022 +0100,Set minimum required cmake version to 3.10,"cmake > 3.10 is not packaged for some of the platforms we build packages eg old ubuntu and debian version. Currently we modify the CMakeLists.txt in those build environments and set the minimum version to 3.10 already, which proofes that timescaledb builds fine with cmake 3.10. " 566a4ff104924c99ef452e7feccaee2ad31fc47d,Sven Klemm,Fri Mar 18 05:33:35 2022 +0100,Route UPDATE through HypertableModify,Route UPDATE on Hypertables through our custom HypertableModify node. This patch by itself does not make any other changes to UPDATE but is the foundation for other features regarding UPDATE on hypertables. 846878c6bb93fd8481af0d8a547e12897b7f4876,Erik Nordström,Wed Mar 16 11:10:29 2022 +0100,Ensure scan functions use long-lived memory context,"PostgreSQL scan functions might allocate memory that needs to live for the duration of the scan. This applies also to functions that are called during the scan, such as getting the next tuple. To avoid situations when such functions are accidentally called on, e.g., a short-lived per-tuple context, add a explicit scan memory context to the Scanner interface that wraps the PostgreSQL scan API. " b954c00fa8136c87870302ad0297e704f280e2c7,Erik Nordström,Fri Mar 11 17:46:05 2022 +0100,Fix memory handling during scans,"Scan functions cannot be called on a per-tuple memory context as they might allocate data that need to live until the end of the scan. Fix this in a couple of places to ensure correct memory handling. Fixes #4148, #4145 " a759b2b2b9b855975f48d71e6bdf7af1cbf8c917,Sven Klemm,Thu Mar 17 11:32:23 2022 +0100,Show number of chunks excluded in ConstraintAwareAppend EXPLAIN,This patch changes the ConstraintAwareAppend EXPLAIN output to show the number of chunks excluded instead of the number of chunks left. The number of chunks left can be seen from other EXPLAIN output while the actual number of exclusions that happened can not. This also makes the output consistent with output of ChunkAppend. ce3e04a9ec945f8734cc5cdd0a258e8130d5c210,Fabrízio de Royes Mello,Tue Mar 15 15:25:46 2022 -0300,Rename forgotten `master` branch name references, 332dffeebc72cf16afbde802e2f19aead9bd7836,Fabrízio de Royes Mello,Mon Mar 14 14:10:14 2022 -0300,Rename `master` branch to `main`,Following what many communities already did we agreed in renaming the `master` branch to `main`. Resources: - https://sfconservancy.org/news/2020/jun/23/gitbranchname/ - https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue@alap3.anarazel.de Closes #4163 077b2edbc556bbcfb104992effdb6fe6dd89cc3b,Sven Klemm,Tue Mar 15 15:22:14 2022 +0100,Change ChunkAppend file organization,"This patch changes the organization of the ChunkAppend code. It removes all header files except chunk_append/chunk_append.h. It also merges exec.c and explain.c to remove unnecessary function exports, since the code from explain.c was only used by exec.c " cc89f1dc847bf825b5502d72769b931e3502373f,Sven Klemm,Tue Mar 15 13:14:43 2022 +0100,Remove duplicate contain_param functions,This patch exports the contain_param function in planner.c and changes ChunkAppend to use that version instead of having two implementations of that function. f00bdadf0c4d7465411fe7f349a19953ca401498,Erik Nordström,Mon Mar 14 13:47:43 2022 +0100,Trigger Sqlsmith tests manually or by push to branch,"Add workflow events to allow manually running Sqlsmith tests or when pushing to the 'sqlsmith' branch. This is useful when submitting PRs that one wants to run extra checks on, including Sqlsmith. " ab6b90caff19e0193ceb58ba298280daf03a6adb,Sven Klemm,Mon Mar 14 11:44:05 2022 +0100,Reference CVE ID in CHANGELOG,The CVE ID was already referenced in the commit introducing the fix but not in the CHANGELOG. f5fd06cabb044a4b7f1fba694aeb9b1fbec87afc,Mats Kindahl,Fri Mar 4 15:10:09 2022 +0100,Ignore invalid relid when deleting hypertable,"When running `performDeletion` is is necessary to have a valid relation id, but when doing a lookup using `ts_hypertable_get_by_id` this might actually return a hypertable entry pointing to a table that does not exist because it has been deleted previously. In this case, only the catalog entry should be removed, but it is not necessary to delete the actual table. This scenario can occur if both the hypertable and a compressed table are deleted as part of running a `sql_drop` event, for example, if a compressed hypertable is defined inside an extension. In this case, the compressed hypertable (indeed all tables) will be deleted first, and the lookup of the compressed hypertable will find it in the metadata but a lookup of the actual table will fail since the table does not exist. Fixes #4140 " f82df7ca4a6e6521daeb02f83db1548e7ed380ae,Dmitry Simonenko,Fri Mar 11 10:25:38 2022 +0200,Allow ANALYZE command on a data node directly,Allow execution of VACUUM/ANALYZE commands on a data node without enabling timescaledb.enable_client_ddl_on_data_nodes GUC Fix #4157 8f56ced825c2a1df543ac1c7646707c10f400202,Sven Klemm,Mon Mar 7 17:51:59 2022 +0100,Add workflow for running sqlsmith,sqlsmith is a random SQL query generator and very useful for finding bugs in our implementation as it tests complex queries and thereby hits codepaths and interactions between different features not tested in our normal regression checks. 06d837559484c21165cb3429dfdd91c8606973f3,Sven Klemm,Tue Mar 8 11:15:22 2022 +0100,Enhance extension function test,This patch changes the extension function list to include the signature as well since functions with different signature are separate objects in postgres. This also changes the list to include all functions. Even though functions in internal schemas are not considered public API they still need be treated the same as functions in other schemas with regards to extension upgrade/downgrade. This patch also moves the test to regresscheck-shared since we do not dedicated database to run these tests. 33bbdccdcd7ad805c3463ba6426dea0510497ed1,Fabrízio de Royes Mello,Mon Mar 7 11:12:46 2022 -0300,Refactor function `hypertable_local_size`,"Reorganize the code and fix minor bug that was not computing the size of FSM, VM and INIT forks of the parent hypertable. Fixed the bug by exposing the `ts_relation_size` function to the SQL level to encapsulate the logic to compute `heap`, `indexes` and `toast` sizes. " 18afcfd62f3c7235c839eb15235c2318fbeec85c,Fabrízio de Royes Mello,Thu Mar 3 15:14:18 2022 -0300,Refactor function `ts_relation_size`,Current implementation iterate over fork types to calculate the size of each one by calling `pg_relation_size` PostgreSQL function and other calls to calculate indexes and table size (six function calls). Improving it by halving PostgreSQL function calls to calculate the size of the relations (now three function calls). e9fb9acbbbe2c7da2c3769fb7374dda9a7d8ad11,Markos Fountoulakis,Thu Mar 3 11:47:36 2022 +0200,Fix regressions found in nightly CI,Add concurrent_query_and_drop_chunks to ignore-list and fix C compiler warning. 15d33f0624595470cdb3b417deecefe8c12639eb,Mats Kindahl,Thu Nov 25 10:18:10 2021 +0100,Add option to compile without telemetry,"Add option `USE_TELEMETRY` that can be used to exclude telemetry from the compile. Telemetry-specific SQL is moved, which is only included when extension is compiled with telemetry and the notice is changed so that the message about telemetry is not printed when Telemetry is not compiled in. The following code is not compiled in when telemetry is not used: - Cross-module functions for telemetry. - Checks for telemetry job in job execution. - GUC variables `telemetry_level` and `telemetry_cloud`. Telemetry subsystem is not included when compiling without telemetry, which requires some functions to be moved out of the telemetry subsystem: - Metadata handling is moved out of the telemetry module since it is used not only with telemetry. - UUID functions are moved into a separate module instead of being part of the telemetry subsystem. - Telemetry functions are either added or removed when updating from a previous version. Tests are updated to: - Not use telemetry functions to get UUID or Metadata and instead use the moved UUID and metadata functions. - Not include telemetry information in tests that do not require it. - Configuration files do not set telemetry variables when telemetry is not compiled in. - Replaced usage of telemetry functions in non-telemetry tests with other sources of same information. Fixes #3931 " 642a745767c925a12b4b68359c69ff496a276bac,Sven Klemm,Wed Mar 2 12:30:16 2022 +0100,Fix arm64 apt package test,This patch changes the workflow to run apt-get update before installing any packages in case the local package database is outdated and references packages no longer available. b909d4857db23ed38b0cb4a825cefe76f0ef7401,Mats Kindahl,Thu Feb 24 14:49:10 2022 +0100,Fixes to smoke update tests,"Smoke tests where missing critical files and some tests had changed since last run and did not handle update smoke tests, so fixing all necessary issues. " 14deea6bd59c3afa81a4469012911d265242a49a,Erik Nordström,Fri Feb 18 17:05:18 2022 +0100,Improve chunk scan performance,"Chunk scan performance during querying is improved by avoiding repeated open and close of relations and indexes when joining chunk information from different metadata tables. When executing a query on a hypertable, it is expanded to include all its children chunks. However, during the expansion, the chunks that don't match the query constraints should also be excluded. The following changes are made to make the scanning and exclusion more efficient: * Ensure metadata relations and indexes are only opened once even though metadata for multiple chunks are scanned. This avoids doing repeated open and close of tables and indexes for each chunk scanned. * Avoid interleaving scans of different relations, ensuring better data locality, and having, e.g., indexes warm in cache. * Avoid unnecessary scans that repeat work already done. * Ensure chunks are locked in a consistent order (based on Oid). To enable the above changes, some refactoring was necessary. The chunk scans that happen during constraint exclusion are moved into separate source files (`chunk_scan.c`) for better structure and readability. Some test outputs are affected due to the new ordering of chunks in append relations. " 32c1e3aef293bcece63550e09c3dd3496368a687,Erik Nordström,Thu Feb 10 16:42:56 2022 +0100,Allow control of relation open/close in Scanner,"Make the Scanner module more flexible by allowing optional control over when the scanned relation is opened and closed. Relations can then remain open over multiple scans, which can improve performance and efficiency. Closes #2173 " 0f351ff6128e9765f993fe5ed713bd274a3fd2a7,Erik Nordström,Thu Feb 10 15:55:28 2022 +0100,Simplify Scanner by embedding internal state,"As part of adding a scan iterator interface on top of the Scanner module (commit 8baaa98), the internal scanner state that was previously private, was made public. Now that it is public, it makes more sense to make it part of the standard user-facing `ScannerCtx` struct, which also simplifies the code elsewhere. " 91820e26f663eb2210080092b75a290a491424f6,Sven Klemm,Wed Feb 23 17:33:45 2022 +0100,Improve planner error messages regarding nodes,Change error messages when unexpected nodes are encountered to actually show the node name instead of the node id. 3f303c7d42da6e76f610bfb67d017134318cf7ed,Sven Klemm,Wed Feb 23 13:19:34 2022 +0100,Refactor tsl_debug_append_path,This patch splits the node name logic from the child path logic to allow getting a string representation for any postgres node. This adds a new function: const char * ts_get_node_name(Path *path) This patch doesn't add any new callers to the function but it will be used in subsequent patches to produce more user friendly error messages when unexpected node types are encountered during planning. 57368dd98ecba8290008e36c578534612b9e7533,Dmitry Simonenko,Tue Feb 22 13:27:14 2022 +0300,Fix RENAME TO/SET SCHEMA on distributed hypertable,"This PR fixes ON_END logic for distributed DDL execution by removing old leftover check, which marked those commands as unsupported. Fix: #4106 " a4648b11b4979ecc2d5d393585a7ce1a812dff0a,Sven Klemm,Wed Feb 23 02:59:19 2022 +0100,Fix segfault on INSERT in distributed hypertables,When inserting in a distributed hypertable with a query on a distributed hypertable a segfault would occur when all the chunks on the query would get pruned. 58fd0c5cef56e638f8ed03204c399c0e76e9a164,Sven Klemm,Sun Feb 20 22:12:34 2022 +0100,Test APT ARM64 packages,Add tests for ARM64 Debian and Ubuntu packages 37190e8a8aaaecd3a906b092c54527ca987e02e5,Alexander Kuzmenkov,Mon Feb 21 13:20:47 2022 +0300,Cache chunk data when performing chunk exclusion,"We cache the Chunk structs in RelOptInfo private data. They are later used to estimate the chunk sizes, check which data nodes they belong to, et cetera. Looking up the chunks is expensive, so this change speeds up the planning. " 0f7ab056c23f6a5b214858885123d40b4a42065e,Dmitry Simonenko,Mon Feb 21 11:11:28 2022 +0300,Move copy/move chunk tests to separate file,Fix flaky data_node test by moving tests associated with copy/move chunk functionality into a separate test file/groupand with increased background workers (using max_bgw_8.conf). Issue: #4047 eedaaecc46966c75983f34e269fb28f7c8ae7277,Aleksander Alekseev,Wed Feb 16 16:38:24 2022 +0300,Custom origin's support in CAGGs,"This patch allows using custom origin's in CAGGs, for instance: time_bucket_ng('7 days', day, '2000-01-03' :: date) AS bucket For weekly buckets this allows the user to choose what should be considered the beginning of the week - Sunday or Monday. Also by shifting the origin one second forward or backward user can tweak the inclusiveness of the buckets. This works for date's, timestamp's and timestamptz's. The bucket size is considered variable-sized in all these cases. CAGGs on top of distributed hypertables, compressed hypertables and compressed distributed hypertables are supported as well. Additionally, this patch does several small refactorings. Firstly, it makes sure that experimental features of CAGGs will be tested in both Debug and Release builds. This was previously overlooked. Secondly, it renames the tests so that a person who is working on experimental features in CAGGs will be able to easily execute all the related tests: `TESTS='exp_cagg_*' make installcheck` Last but not least the patch refactors is_valid_bucketing_function() and renames it to function_allowed_in_cagg_definition(). The reason to do it in this patch is that otherwise, the logic of the function gets rather confusing which complicates code review. fix " 45cf7f1fa1e2c00afde506bfada625af9d7d4eb3,Sven Klemm,Sat Feb 12 18:05:20 2022 +0100,Document requirements for statements in sql files,Since we now lock down search_path during update/downgrade there are some additional requirements for writing sql files. f29340281e47a4bade690e0b81c1ef6f18507834,Alexander Kuzmenkov,Thu Feb 17 19:32:58 2022 +0300,Post release 2.6.0, df49c2b4cbba831cbd6c8d5e4f88a7a7b781c3ac,Sven Klemm,Thu Feb 17 21:26:52 2022 +0100,Fix non-debug build with older cmake,"On non-debug builds we might end up with an empty list of tests when generating the schedule. On older cmake versions < 3.14 trying to sort an empty list will produce an error, so we check for empty list here. " 9e7fbf7f6982bad80cd0129d38dcf6273baad2a2,Alexander Kuzmenkov,Mon Feb 14 12:19:41 2022 +0300,Release 2.6.0,"This release is medium priority for upgrade. We recommend that you upgrade at the next available opportunity. This release adds major new features since the 2.5.2 release, including: Compression in continuous aggregates Experimental support for timezones in continuous aggregates Experimental support for monthly buckets in continuous aggregates It also includes several bug fixes. Telemetry reports are switched to a new format, and now include more detailed statistics on compression, distributed hypertables and indexes. **Features** * #3768 Allow ALTER TABLE ADD COLUMN with DEFAULT on compressed hypertable * #3769 Allow ALTER TABLE DROP COLUMN on compressed hypertable * #3943 Optimize first/last * #3945 Add support for ALTER SCHEMA on multi-node * #3949 Add support for DROP SCHEMA on multi-node **Bugfixes** * #3808 Properly handle max_retries option * #3863 Fix remote transaction heal logic * #3869 Fix ALTER SET/DROP NULL contstraint on distributed hypertable * #3944 Fix segfault in add_compression_policy * #3961 Fix crash in EXPLAIN VERBOSE on distributed hypertable * #4015 Eliminate float rounding instabilities in interpolate * #4019 Update ts_extension_oid in transitioning state * #4073 Fix buffer overflow in partition scheme **Improvements** Query planning performance is improved for hypertables with a large number of chunks. **Thanks** * @fvannee for reporting a first/last memory leak * @mmouterde for reporting an issue with floats and interpolate " fc5341c47bc7af4163aa2acc0fae5c10f2881238,Alexander Kuzmenkov,Wed Feb 16 20:37:33 2022 +0300,Increase psql verbosity to see error line numbers in remote_connection test, 2bc3efa44ed50e6d3cffd6ed3276a473650df33d,Erik Nordström,Wed Feb 16 11:17:07 2022 +0100,Add missing OS-information in telemetry,Some telemetry fields were removed by mistake as part of an earlier refactoring. Add back the missing fields that provide information about the operating system. a956faaa16db96b6feda50c5d454fbcc7adc2562,Sven Klemm,Tue Feb 15 22:27:24 2022 +0100,Set search_path again after COMMIT,SET LOCAL is only active until end of transaction so we set search_path again after COMMIT in functions that do transaction control. While we could use SET at the start of the function we do not want to bleed out search_path to caller. 264540610e02f7f46d3213c103b66b07c1ce12c5,gayyappan,Mon Feb 7 13:59:37 2022 -0500,Fix tablespace for compressed chunk's index,"When a hypertable uses a non default tablespace, based on attach_tablespace settings, the compressed chunk's index is still created in the default tablespace. This PR fixes this behavior and creates the compressed chunk and its indexes in the same tablespace. When move_chunk is executed on a compressed chunk, move the indexes to the specified destination tablespace. Fixes #4000 " 72d03e6f7d30cc4794c9263445f14199241e2eb5,Sven Klemm,Sat Feb 12 17:02:03 2022 +0100,Remove search_path modifications from reverse-dev,Resetting search_path in reverse-dev was necessary before the release of 2.5.2 as the previous timescaledb version scripts didn't handle locked down search_path. We can remove setting search_path too as the downgrade script includes pre-update.sql which locks down search_path. 65ffa44711d3bc0d58563ff8179706819095d27c,Sven Klemm,Sat Feb 12 02:55:54 2022 +0100,"Bump postgres versions used in CI to 14.2, 13.6, 12.10", 531f7ed8b16e4d1a99021d3d2b843bbc939798e3,Sven Klemm,Thu Feb 10 19:32:22 2022 +0100,Don't use pg_temp in update scripts,Scripts run under pgextwlist cannot use pg_temp so we replace pg_temp usage in update scripts with _timescaledb_internal. ee9b9ad318719b715caeebd20875692e80ff82dc,Aleksander Alekseev,Fri Feb 11 00:21:25 2022 +0300,time_bucket_ng() bugfix,"time_bucket_ng() may return wrong results for monthly buckets with overwritten origin. As an example: time_bucket_ng('1 month', '2001-01-01' :: date, origin => '2000-06-01') ... returns 2000-01-01 instead of 2001-01-01. In fact, our tests showed this but we overlooked the fact that the results are incorrect. The reason is that the month of the origin is not accounted for when calculating the resulting year. This patch fixes this. Also it updates linux-32bit-build-and-test.yaml workflow due to todays 14.2/13.6/12.10 releases. ""Memoize"" test doesn't pass on 14.2 thus the patch adds it to the list of skipped tests. The patch doesn't rename the checks because we list ""PG14.1 Debug linux-i386"" etc as required checks on GitHub. This will be addressed in the follow-up patches. " d3d0b8d94323af01f2a83316c3d7e67ee1181a98,Markos Fountoulakis,Thu Feb 10 07:33:18 2022 +0200,Fix memcpy in force_group_by_push_down, c56b33d2e778fbf5975cfadfae3748df218d6731,Sven Klemm,Thu Feb 10 00:16:52 2022 +0100,Fix openssl detection for PG14,PG14 introduced an alternative way to specify for postgres to be built with openssl support. --with-ssl=openssl may be used instead of --with-openssl. This patch adjusts our openssl check to detect both versions. https://github.com/postgres/postgres/commit/fe61df7f82 85e3307dd130c61b8dab20b6551ca66a1703dd6b,Sven Klemm,Thu Feb 10 10:59:22 2022 +0100,Adjust pg_md5_hash compat macro to upstream changes,The API for pg_md5_hash was changed for PG15 which was initially backported to the PG14 branch but that commit was later reverted to not introduce an ABI breakage for extensions. https://github.com/postgres/postgres/commit/ad5b6f24 https://github.com/postgres/postgres/commit/b69aba74 c07fe1f88374a68707e54121c983b27fd59e5a20,Sven Klemm,Wed Feb 9 23:52:34 2022 +0100,Post Release 2.5.2,Add 2.5.2 to update/downgrade scripts f0d163603c1797442e2acffc31ce135a0d38b820,Sven Klemm,Wed Feb 9 19:47:10 2022 +0100,Release 2.5.2,This release contains bug fixes since the 2.5.1 release. This release is high priority for upgrade. We strongly recommend that you upgrade as soon as possible. **Bugfixes** * #3900 Improve custom scan node registration * #3911 Fix role type deparsing for GRANT command * #3918 Fix DataNodeScan plans with one-time filter * #3921 Fix segfault on insert into internal compressed table * #3938 Fix subtract_integer_from_now on 32-bit platforms and improve error handling * #3939 Fix projection handling in time_bucket_gapfill * #3948 Avoid double PGclear() in data fetchers * #3979 Fix deparsing of index predicates * #4015 Eliminate float rounding instabilities in interpolate * #4020 Fix ALTER TABLE EventTrigger initialization * #4024 Fix premature cache release call * #4037 Fix status for dropped chunks that have catalog entries * #4069 Fix riinfo NULL handling in ANY construct * #4071 Fix extension installation privilege escalation * #4073 Fix buffer overflow in partition scheme **Thanks** * @carlocperez for reporting crash with NULL handling in ANY construct * @erikhh for reporting an issue with time_bucket_gapfill * @fvannee for reporting a first/last memory leak * @kancsuki for reporting drop column and partial index creation not working * @mmouterde for reporting an issue with floats and interpolate * Pedro Gallegos for reporting a possible privilege escalation during extension installation Security: CVE-2022-24128 8a33a79e0c489ddc15f252e710dab61aa96c4e82,Markos Fountoulakis,Wed Feb 9 18:35:19 2022 +0200,Fix buffer overflow in partition scheme,Reallocate the partitioning attributes arrays when forcing GROUP BY aggregates down. Fixes #4050 6dddfaa54e8f29e3ea41dab2fe7d9f3e37cd3aae,Sven Klemm,Tue Feb 8 20:02:55 2022 +0100,Lock down search_path in install scripts,"This patch locks down search_path in extension install and update scripts to only contain pg_catalog, this requires that any reference in those scripts is fully qualified. Additionally we add explicit create commands to all update scripts for objects added to the public schema. This change will make update scripts fail if a function with identical signature already exists when installing or upgrading instead reusing the existing object. " c8b8516e466c2bb7d2ae6a4b0b2e8e60b24b24a2,Sven Klemm,Tue Feb 8 19:55:28 2022 +0100,Fix extension installation privilege escalation,"TimescaleDB was vulnerable to a privilege escalation attack in the extension installation script. An attacker could precreate objects normally owned by the extension and get those objects used in the installation script since the script would only try to create them if they did not already exist. Thanks to Pedro Gallegos for reporting the problem. This patch changes the schema, table and function creation to fail and abort the installation when the object already exists instead of using the existing object. Security: CVE-2022-24128 " 7fb3feb60d72fef164940b5dd6d26ef1201e4988,Erik Nordström,Wed Feb 9 08:34:18 2022 +0100,Use proper types in telemetry JSON,Many fields in the generated telemetry JSON output values as strings instead of using integer or boolean types. Change those fields to use the type that match the original data. 5af9f45488d51027804cac16362811f71a89bb64,Erik Nordström,Tue Feb 8 09:57:23 2022 +0100,Add extra telemetry for continuous aggregates,Add the following telemetry fields for continuous aggregates: * The number of continuous aggregates created on distributed hypertables * The number of continuous aggregates using real-time aggregation e19fffc1482cf0a532ab252ac440bf52f9d8bf96,Nikhil Sontakke,Wed Feb 9 13:07:44 2022 +0530,Fix riinfo NULL handling in ANY construct,"If the ANY construct contains a singleton NULL then the logic in ""dimension_values_create_from_array"" barfs causing a crash. Fix it appropriately in the caller ""hypertable_restrict_info_add_expr"" function. " 454d32539ce47c1775c136e68a615d3453ee3ac1,Aleksander Alekseev,Tue Feb 8 12:33:30 2022 +0300,"Fix ""nm: unknown argument -defined-only"" error on MacOS","On MacOS Monterey with LLVM 13.0.1 when executing export_prefix_check.sh we get an error: /usr/local/opt/llvm/bin/nm: error: : unknown argument '-defined-only', did you mean '--defined-only'? This patch adds a better check of which flag does `nm` expect. " 418d71c2fef7978ec6350669fb01d2d076039d8f,Erik Nordström,Fri Feb 4 11:40:39 2022 +0100,Rename telemetry test,"All TSL-related telemetry testing was moved into the `telemetry_community` test, which has a name that is no longer accurate (given that it tests non-community features). Rename this test to `telemetry_stats`, which also doesn't conflict with the non-TSL `telemetry` test. " e56b95daec0f9fbf55d99b984d6236c1bbd4358c,Erik Nordström,Fri Jan 28 08:38:08 2022 +0100,Add telemetry stats based on type of relation,"Refactor the telemetry function and format to include stats broken down on common relation types. The types include: - Tables - Partitioned tables - Hypertables - Distributed hypertables - Continuous aggregates - Materialized views - Views and for each of these types report (when applicable): - Total number of relations - Total number of children/chunks - Total data volume (broken into heap, toast, and indexes). - Compression stats - PG stats, like reltuples The telemetry function has also been refactored to return `jsonb` instead of `text`. This makes it easier to query and manipulate the resulting JSON format, and also gives cleaner output. Closes #3932 " 7f05448d2af964264cd9bde9624a05522e786e31,Erik Nordström,Thu Jan 27 17:14:26 2022 +0100,Add rescan support to internal Scanner,"This change adds rescan support to the internal Scanner module. When scanning TimescaleDB catalog data, it is sometimes useful to be able to restart an index scan without having to close and reopen the scanned relation. " 7d413809916035d1f3e4dfb3210878dbd1b34744,Alexander Kuzmenkov,Thu Feb 3 18:35:51 2022 +0300,Fix double initialization of TsFdwRelInfo,"This doesn't cause any problems except degrading the performance. The TsFdwRelInfo is initialized twice for hypertable on access node: 1) when using it for chunk size estimation in `estimate_chunk_size`, 2) when expanding the distributed hypertable in `tsl_set_rel_pathlist` callback. The initialization has to happen inside the TSL module, this is why we can't easily move it up the call graph to, say, `import/ts_set_append_rel_size`, which is FDW-agnostic. As a fix, just accept that TimescaleDBPrivate structure can be allocated already, and don't discard the previous one. " c9d24703a86a63933b3d64ef5a9681ef6f714230,Aleksander Alekseev,Mon Jan 31 15:24:54 2022 +0300,Make add_continuous_aggregate_policy() work with variable-sized buckets,"Variable-sized buckets don't work with CAGG policies: SELECT add_continuous_aggregate_policy('conditions_summary_policy', start_offset => INTERVAL '65 days', end_offset => INTERVAL '1 day', schedule_interval => INTERVAL '1 hour'); ERROR: bucket width is not defined for a variable bucket This patch fixes it and adds corresponding tests. Reported by Miranda Auhl " b330dd94e13746069ec21d8cdcd2e43ade17f01d,Sven Klemm,Sun Feb 6 18:06:03 2022 +0100,Fix privilege check in extension_load_without_preload,The privilege check in extension_load_without_preload used is_member_of_role which does not respect the inherit flag of roles instead of has_privs_of_role. For additional context: https://www.postgresql.org/message-id/flat/CAGB 2BVh5RpzZroX-RKLXYWyTidpWX6,,,, 3DqcbMoVWERotAWh-_CeUQ,,,, 40mail.gmail.com#f08fc3998fad5d919c1660c6a1926158 ,,,, 9b1809152d7b6c1bc3a8977889328b2f1312b4ce,Dmitry Simonenko,Mon Feb 7 10:13:16 2022 +0300,Add a way to disable dist ddl on database objects,"This change adds `timescaledb_experimental.enable_distributed_ddl` session variable which allows to enable or disable distributed DDL operations on a database objects such as SCHEMA, DATABASE or execute GRANT command. Basically everything that does not involve particular distributed hypertable associated with the command can be disabled. Affected commands: CREATE/DROP SCHEMA ALTER SCHEMA ALTER DATABASE REASSING OWNER DROP OWNED ALTER DEFAULT PRIVILEGES GRANT ON DATABASE/TABLE GRANT IN SCHEMA This change also set this variable to false by default, which disables forwarding DDL commands on database objects to data nodes. Fix: #4001 " 5e50cb51dbf3543cd6ba2e44220b46ee2af137af,Dmitry Simonenko,Mon Feb 7 09:28:55 2022 +0300,Fix cmake file to run template tests,"Recent CMakeLists.txt related refactoring accidently missed one case with TEST_FILES variable renaming, which lead to the exclusion of template tests. " 485af7c19fc0a9f29858211d1703d2e81d8f6588,Sven Klemm,Sun Feb 6 08:33:12 2022 +0100,Fix centos 8 package test,Since CentOS 8 went EOL in december last year most of the repository URLs error out now. This patch switches the test to use Rockylinux which is a drop in replacement for CentOS. 9714e881e3422a38032bdb2e18384657fa18134d,Mats Kindahl,Fri Jan 28 13:37:21 2022 +0100,Increase default background workers,Increate the number of background workers since we have flaky tests as a result of groups running out of background workers. We also increate the total number of workers to 24 for the default configuration when running tests. Fixes #4023 05dd4787d1cd287d0636cd34d9295f29dda7c769,Mats Kindahl,Fri Jan 28 13:30:32 2022 +0100,Support test groups with different configurations,"To support tests with different configuration options, we split the tests into *test configurations*. Each test configuration NAME will have - A configuration template file `NAME.conf.in` that is used to run the suite of tests. - A variable `TEST_FILES_` listing the test files available for that test suite. - A variable `SOLO_TESTS_` that lists the tests that need to be run as solo tests. The code to generate test schedules is then factored out into a separate file and used for each configuration. " 9248de2f2a6628881aca9b73256fae1921b2ed26,Markos Fountoulakis,Tue Feb 1 13:07:50 2022 +0200,Fix shellcheck warnings, 481bf8b5a8b304c40ccba1d12fb232e8684807fd,Sven Klemm,Sun Jan 30 21:03:27 2022 +0100,Fix downgrade test artifact upload,The artifact upload for the downgrade test was using the filenames of the update test and so would never upload any files. e5db6a9eecdba2de76eeaee00b459fe35a556baa,gayyappan,Thu Jan 27 16:03:24 2022 -0500,Fix status for dropped chunks that have catalog entries,Chunks that are dropped but preserve the catalog entries have an incorrect status when they are marked as dropped. This happens if the chunk was previously compressed and then gets dropped - the status in the catalog tuple reflects the compression status. This should be reset since the data is now dropped. cf6aed71b94d473e5eb70536571d3ef0a6970222,Aleksander Alekseev,Mon Jan 31 12:57:24 2022 +0300,Refactoring: Use consistent naming for cagg_*.sql tests,"Rename cagg_variable_size_buckets.sql to cagg_monthly.sql. This makes the naming for the tests much more consistent. This file actually contains only tests for monthly buckets, not for all possible variable-sized buckets. CAGGs with timezones are covered in cagg_with_timezone.sql. " 22f9cf689dca8a831fee1249f141b4acff5aaf6b,Alexander Kuzmenkov,Mon Jan 31 14:46:14 2022 +0300,Don't leak Chunks in classify_relation,"This function is called often, at least 4 times per chunk, so these add up. Freeing these chunks allows us to save memory. Ideally, we should fix the function not to look up the chunk anew each time. Also make a couple other tweaks that reduce memory usage for planning. " ae02934c6a43524b6c17b562b283bac08a2bf4be,Aleksander Alekseev,Thu Jan 27 17:11:42 2022 +0300,Timezones support in CAGGs,"This patch allows using timezones in CAGGs, for instance: time_bucket_ng(bucket_size, ts, 'Europe/Moscow') Both months/years and days/hours/minutes can be used in bucket_size. The bucket size is considered variable-sized in all these cases because DST affects days/hours/minutes as well. CAGGs on top of distributed hypertables, compressed hypertables and compressed distributed hypertables are supported as well. " c0050a1315f9e79e593ba9f8906d6064606d5efc,gayyappan,Thu Jan 27 14:05:41 2022 -0500,Improve error message for compress_chunk,Fix the error message for compress chunks so that it specifies the cagg name instead of the materialized hypertable name. 9e16d9f4e44dff336e15c234632e4d280e68d64d,Sven Klemm,Thu Jan 27 00:05:05 2022 +0100,Add coccinelle scripts for detecting use-after-free,This patch adds coccinelle scripts for detecting use-after-free bugs in relation to ts_cache_release. This will find code using the following pattern: Use of the hypertable from cache after releasing the cache and use of the dimension of the hypertable from a cache after the cache was released. 7f5c5fd10d518916a435b8e52fb48e38b407a55e,gayyappan,Wed Jan 26 14:39:50 2022 -0500,Verify compression settings before adding policy,"When a compression policy is added for a continuous aggregate, verify that compression has been enabled, before adding a policy. " 44be03b5c6856a4d5a8ee43f606ddc5befeb263f,gayyappan,Tue Jan 25 14:23:12 2022 -0500,Fix premature cache release call,"The cache entry for a hypertable is created by calling ts_hypertable_from_tupleinfo. This sets up the ht->space structure, which in turn sets up the dimension information. These structures are allocated in the cache's memory context. The dimension information is accessed after the cache is released in ts_subtract_integer_from_now. This PR fixes this by releasing the cache before returning from the function. Fixes #4014 " 64ed2db8695e471b7a2d938762b84f26ccf32b6f,Sven Klemm,Mon Jan 24 21:05:39 2022 +0100,Fix ALTER TABLE EventTrigger initialization,"When ALTER TABLE SET (timescaledb.compress) is executed, the compression table is created as part of this command. But when run as part of an extension installation the EventTrigger for ALTER TABLE does not get initialized leading to a segfault. Fixes #4017 " b30d005657c5803c35ff609637644e484458f9ad,Dmitry Simonenko,Tue Jan 25 14:03:50 2022 +0300,Fix crash when execute add_data_node,"On disconnect libpq will create a result object without creating event data, which is usually done for a regular errors. This fix handles this case and forces the result object to always have have associated connection meta data. Fix: #3951 " 2e725dc0e2cf6d9f715cc91278a6713f6a847b06,Alexander Kuzmenkov,Fri Jan 14 14:54:55 2022 +0300,Use cached Chunks for data node assignment,"Before assigning chunks to data nodes, we have to look up the same Chunk structs to estimate the chunk relation size. Cache them in chunk RelOptInfos to avoid the costly lookup. " 60edef6ebaa35c4961398c8ba2cf7c1be911bdbc,Alexander Kuzmenkov,Tue Jan 11 17:37:58 2022 +0300,Don't guess historical/current chunk by ids,"For the chunks that don't have the ANALYZE stats, we have to estimate the number of pages and tuples in some other ways. One thing we factor in this estimation is the expected fill factor of the chunk. There are two ways we calculate the fill factor: 1. For time dimensions, by looking at whether the now() belongs to the chunks [begin, end) interval and calculating the fill factor accordingly. 2. For space dimensions, by looking at whether the chunk is one of the last chunks in the hypertable. To check (2), we used to compare the chunk ids. Turns out it didn't work correctly, because the chunk ids are global for all hypertables, and looking at them doesn't tell us anything if there are many hypertables. The code that did that was also very slow. This commit just removes this logic and considers all the chunks on space dimensions w/o the ANALYZE stats as recent chunks with fill factor of 0.5. " c2bfc5d17c01359621ed2ce00fb198653cda4ffa,Sven Klemm,Sat Jan 22 23:49:00 2022 +0100,Route delete on Hypertables through HypertableModify node,This patch changes DELETE handling for hypertables to have the postgres ModifyTable node be wrapped in a custom HypertableModify node. By itself this does not change DELETE handling for hypertables but instead enables subsequent patches to implement e.g. chunk exclusion for DELETE or DELETE on compressed chunks. Since PG 14 codepath for INSERT is different from previous versions this PR will only change the plan for PG14+. DELETE handling for distributed hypertables is not changed as part of this patch. 9f64df8567abdbe6af11a9388acefb8237629ff5,gayyappan,Thu Jan 20 13:23:12 2022 -0500,Add ts_catalog subdirectory,Move files that are related to timescaledb catalog access to this subdirectory 616d3b16b8e1dfca760cc4e4472f9af159bf44bd,Mats Kindahl,Mon Jan 24 15:14:23 2022 +0100,Update ts_extension_oid in transitioning state,"When executing the post-update scripts we get an assertion failure and subsequent segfault because `ts_extension_oid` is still set to `InvalidOid`. This happen because `ts_extension_oid` is not set up until after the post-update state has finished, but we need to set it up already in the extension transitioning state used during post-update execution. The reason for not setting up the `ts_extension_oid` is because we cannot issue queries to the catalog, but this is possible already in the post-update stage, so we add code to set up `ts_extension_oid` already in this state. Fixes #4009 " 89dceeabd4017f0843ddfc0a17bc505eb0eff895,Sven Klemm,Sun Jan 23 20:33:40 2022 +0100,Make pgtest input/output dir optional,PG15 removes the input and output directory from the pg regression test files and moves those files into sql and output directories. Currently cmake errors when those directories are not present in PG_SOURCE_DIR. This patch makes those directories optional so cmake does not fail when PG_SOURCE_DIR contains the checkout of a version without these directories. https://github.com/postgres/postgres/commit/d1029bb5 https://github.com/postgres/postgres/commit/dc9c3b0f 5c22ef3da2958a6e5eb876f984539be4d86f4def,Sven Klemm,Sun Jan 23 11:38:57 2022 +0100,Rename continuous aggregate tests,Change the prefix for continuous aggregate tests from continuous_aggs_ to cagg_. This is similar to commit 6a8c2b66 which did this adjustment for isolation tests because we were running into length limitations for the spec name. This patch adjusts the remaining tests to be consistent with the naming used in isolation tests. 29856fd0ace1ab0a1b27452e3b835f5971014848,Sven Klemm,Sun Jan 23 19:58:28 2022 +0100,Eliminate float rounding instabilities in interpolate,When interpolating float values the result of the calculation might be unstable for certain values when y0 and y1 are equal. This patch short circuits the formula and returns y0 immediately when y0 and y1 are identical. Fixes #1528 e603478e472b99c348881b22e401f31505b77985,gayyappan,Wed Jan 12 14:59:17 2022 -0500,Support cagg trigger for distributed compressed chunks,AFTER ROW triggers are not supported on inserts into compressed chunks. This change explicitly calls the cagg triggers so that continuous aggregates work as expected on inserts into compressed distributed hypertables. This fix is similar to PR 3724 that handles cagg triggers for non distrbuted hypertables. 4ebb5b72e95f788694855fafbb4fc5de732e72a6,gayyappan,Wed Jan 12 17:06:57 2022 -0500,Refactor ChunkInsertState struct,Move all compression related info into a separate struct. 07fd9a4298bde42297c28594e1c6f8b7120677a2,gayyappan,Wed Jan 12 11:32:17 2022 -0500,Support continuous agg trigger on copy into compressed chunks,AFTER ROW triggers are not supported on compressed chunks. Directly call the continuous aggregate trigger function for copies. This fix is similar to PR 3764 that handles cagg triggers for inserts into compressed chunks. afe3362778e0e950407e51291427e2b2524413f0,Sven Klemm,Tue Jan 18 11:03:31 2022 +0100,Don't REVOKE in runner cleanup function,This patch removes the REVOKEs added in commit e320679c. The REVOKEs were added to the cleanup function which gets executed for every individual test but the GRANTs only happens once during initial setup thereby creating an asymmetry. This lead to tests failing that relied on those GRANTs. This patch also makes those GRANTs only execute for PG >= 15. 8e6d3d44abe4163cba277a364155694b808531c0,Sven Klemm,Mon Jan 17 18:45:26 2022 +0100,Add support for wildcards to SKIPS,"This patch adds suport for wildcards to the SKIPS environment variable similar to wildcard support in TESTS. This enables the following test invocations: -- run all tsl tests expect background worker and multinode make regresscheck-t SKIPS=""*bgw* *dist*"" -- run cagg tests except background worker make regresscheck-t TESTS=""continuous_agg*"" SKIPS=""*bgw*"" " 9103d697fb3cdbae3450eaa596d66df4cf289b61,Aleksander Alekseev,Tue Jan 11 12:06:34 2022 +0300,Don't allow using buckets like '1 month 15 days' + some refactorings,"This is in fact a backport from the ""Buckets with timezones"" feature branch. While working on the feature a bug was discovered. We allow creating buckets like '1 month 15 days', i.e. fixed-sized + variable-sized, which is supposed to be forbidden. This patch fixes the bug and also simplifies code a little. timezone_in / timezone_out procedures are used instead of snprintf/scanf. Also, the CAggTimebucketInfo structure was changed slightly. These changes are going to be needed for timezones support anyway. " 2a2b394172109b693d2bc06c8c8df8f9ab0cd879,Nikhil Sontakke,Wed Jan 12 17:24:52 2022 +0530,Avoid distributed chunks scan plans,"The current approach to planning queries on distributed hypertables first plans a ""naive"" plan for scanning each remote chunk individually (the chunks are children of an append) and then that plan gets replaced by a datanode scan plan (each data node is a child in the Append). While we need to include each chunk in the planning for cost reasons (we need to know the stats for chunk rels) we need not actually create paths for scanning chunks. This unnecessary work is now avoided and we now plan datanode scans directly. Fixes #3685 " e320679c4cc5cd9fc0f35f59fd22284fc2ecfb89,Mats Kindahl,Fri Jan 7 14:07:09 2022 +0100,Remove grants on data node bootstrap,"Starting with PG15, default permissions on the public schema is restricted for any non-superuser non-owner. This causes test failures since tables can no longer be created without explicitly adding permissions, so we remove grant when bootstrapping the data nodes and instead grant permissions to the users in the regression tests. This keeps the default permissions on data nodes, but allow regression tests to run. Fixes #3957 Reference: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b073c3cc " 342f848d90b08effa60aaa2c33ad1ceb16fcb4ce,Fabrízio de Royes Mello,Fri Dec 10 10:40:10 2021 -0300,Refactor invalidation log inclusion,Commit 97c2578ffa6b08f733a75381defefc176c91826b overcomplicated the `invalidate_add_entry` API by adding parameters related to the remote function call for multi-node on materialization hypertables. Refactored it simplifying the function interface and adding a new function to deal with materialization hypertables on multi-node environment. Fixes #3833 c5796c0f1d6135729c772029d188afe8a2b69b1b,Sven Klemm,Mon Jan 17 10:21:37 2022 +0100,Ignore memoize test in pg14 snapshot,Upstream changed the explain output for the memoize node to include an additional `Cache Mode` line. While we could adjust our test runner to always ignore that line this would prevent us from testing the cache mode in future tests. https://github.com/postgres/postgres/commit/6c32c0977783fae217b5eaa1d22d26c96e5b0085 1f9bd81ee05e64f14db29f63b33f9d7d2ae712d7,Sven Klemm,Fri Jan 14 10:57:05 2022 +0100,Adjust CI ignore lists to isolation test renames,Commit 6a8c2b66 renamed cagg tests in isolationcheck-t to have a common prefix but did not adjust the CI ignorelists to reflect that change. 69b267071a9deab62ef204f4815c810da58b406e,Sven Klemm,Fri Jan 14 14:09:34 2022 +0100,Bump copyright year in license descriptions,Bump year in copyright information to 2022 and adjust same scripts to reference NOTICE that didn't have the reference yet. This patch also removes orphaned test/expected/utils.out. 28a3895235023d1271f460dbb73ee733714e3ab0,Rafia Sabih,Tue Jan 11 20:02:39 2022 +0100,Update syntactic attribute number,"When adjusting attribute numbers of chunk index, also adjust varattnosyn which is used at the time of reconstructing index definition. Fixes #3794 " e2d578cfac52c2f2a3fc5a1053954cdf9a6b0201,Sven Klemm,Fri Jan 14 02:20:24 2022 +0100,Fix cagg_multi_dist_ht isolation test,Adjust cagg_multi_dist_ht isolation test to no longer include chunk names. isolation tests that expose chunk names cannot be run by themselves or in a custom test list because chunk numbering is depending on the spec file list. 6a8c2b666e23eac77cfb383301a29a0fbb1efe94,Sven Klemm,Thu Jan 13 12:55:00 2022 +0100,Shorten isolation test spec file names,Isolation test identifiers have a length limit and when the isolationtester encounters names that are too long they get truncated. More recent versions will produce a warning when this truncation is done leading to flaky tests. Only continuous_aggs_concurrent_refresh_dist_ht.spec exceeded the truncation limit but since all the continuous_aggs isolation tests have quite long name this patch shortens the names from continuous_aggs_* to cagg_* to prevent this problem from happening to other isolation tests as well. 22fd4d44263342ce650c1e344c2a9424f103254e,Sven Klemm,Thu Jan 13 12:57:41 2022 +0100,Fix compression_ddl isolation test,This patch gets rid of all hardcoded chunk names from the compression_ddl isolation test and also gets rid of chunk names from the output files. Chunk names in isolation test files are problematic as it prevents changing the order of execution of isolation test runs as the database is shared between the individual tests. Output will also differ when only a subset of the tests is run leading to flaky tests. 10b48858e1b03f96d53f690de1501934a586148b,Fabrízio de Royes Mello,Wed Jan 12 15:25:58 2022 -0300,Add exception to invalidation SQL functions,Per Sven's sqlsmith run we got some assertion errors adding invalidation for hypertables and continuous aggregates. Fixed id adding the proper validation in the invalidation SQL functions to check properly the parameters `start_time` and `end_time`. 82e3f055371b39158fbe4d1f81f412d99763e304,Aleksander Alekseev,Tue Jan 11 12:06:34 2022 +0300,Fix the build against REL_14_STABLE,The number of arguments of pg_md5_hash() has changed in REL_14_STABLE. This patch adds a corresponding compat.h version of the procedure. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=3a0cced8 3b9beac5321390152843ad3df4bcc31cd3f34a2a,Sven Klemm,Mon Jan 10 21:21:31 2022 +0100,Adjust apt package test to match docs,Adjust the APT package test to closer match the instructions used in our documentation. 6d3f11cde0298a70c8f659c288e46d92ee273506,Erik Nordström,Tue Dec 21 18:56:56 2021 +0100,Avoid double PQclear() in data fetchers,"The macro `remote_result_elog` is used in the cursor and row-by-row fetchers and clears the result value using the function `PQclear()` before rethrowing its error. However, this error is later captured to clear the wrapping result value that holds a reference to the original result. To avoid this double-clearing, set the response to NULL before the error is thrown. " ab0f753e26a45e9a25cee3d243f431a5a15f4a91,Sven Klemm,Mon Jan 10 13:18:19 2022 +0100,Sort issue template options alphabetically,Change the bug report and enhancement issue template to have dropdown options sorted alphabetically. 4aeb1330f379ed1e8ebbab2f93a6e73aac24e94a,Nikhil Sontakke,Thu Jan 6 20:01:46 2022 +0530,Fix crash in EXPLAIN VERBOSE on dist table,"The crash is occurring in ""fdw_scan_explain"" function because the fsstate passed in is NULL and we try to access the ""fetcher_type"" field from it in VERBOSE mode. When we are doing EXPLAIN, we short circuit the init foreign scan calls and that's why fsstate is not filled up. Fixes #3960 " 4762908bc865e7c7fd27b2c52b7c82da04bd1698,Markos Fountoulakis,Mon Jan 3 14:30:39 2022 +0200,Clean up dead code, 0dd4329ab1e7451e2a50f466a05515e1336edc19,Aleksander Alekseev,Thu Dec 23 11:43:30 2021 +0300,Don't display an error message when creating an empty CAGG,"This is a follow-up to 95804069, ""Monthly buckets support in CAGGs"". In that patch we didn't check the creation of a CAGG on top of an empty hypertable. It works, but displays ""ERROR: origin must be before the given date"", which is a bit misleading. This happens because 95804069 didn't modify invalidation_threshold_compute() in the same way as tsl_process_continuous_agg_viewstmt(). As a result, compute_circumscribed_bucketed_refresh_window_for_months() calls time_bucket_ng() for ts_time_get_min() argument. It doesn't work because the minimum time is 4714-11-24 BC which is before any reasonable origin, and the user sees corresponding error message. This patch fixes the problem in the same way as 95804069 did and adds a corresponding test. " d7eaa55a476a0a1a2d4aa2d868db27c9e5a092d6,Aleksander Alekseev,Thu Dec 23 12:18:17 2021 +0300,Override the default ACL for public schema on PG15,"Since PG15, by default, non-superuser accounts are not allowed to create tables in public schema of databases they don't own. This default can be changed manually. This patch ensures that the permissions are going to be the same regardless of the used PostgreSQL version. Without this patch, none of our tests pass on PG15 because they fail with the ""access denied to schema public"" error. This is why runner.sh was modified. Then, some other tests keep failing because when we call create_distributed_hypertable() we create a new database on each of the data nodes, also not granting enough permissions to non-privileged users. This is what the fix of data_node.c addresses. This is not necessarily the best approach possible, but it preserves the same behavior on PostgreSQL >= 15 and PostgreSQL < 15. Maybe one day we will come up with something better (especially when there will be no need to support PG < 15) but until then the patch seems to be good enough. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b073c3cc " 13abd14c278418c4ccdc36630864c59486b5fdf7,Alexander Kuzmenkov,Tue Dec 14 19:56:20 2021 +0300,Use moving average to estimate chunk size,"When no size statistics is available for a chunk because it haven't been ANALYZEd, we have to resort to some estimates. A good way to estimate the size of the chunk is to look at the sizes of the nearby chunk. Before this commit, we would look up the ten nearby chunk for each chunk w/o statistics, which was slow. This commit adds the code to maintain exponential moving average of the recent chunk sizes in the parent RelOptInfo as we go through them, avoiding the additional lookups. " 4b3227663a8f0e189fe23a04ee44ecdfa94ab487,Dmitry Simonenko,Thu Dec 23 14:01:06 2021 +0300,Add support for ALTER SCHEMA command on multi-node,This change adds support for ALTER SCHEMA RENAME TO and ALTER SCHEMA OWNER TO commands to distributed database. Issue: #3909 24d9243f8b80db905cd864736466cd66574ccb40,Dmitry Simonenko,Thu Dec 23 13:33:24 2021 +0300,Add support for DROP SCHEMA on multi-node,"This PR changes logic and allows DROP SCHEMA command to be executed on each data node, even if the schema being dropped has no distributed hypertables in it. Fix: #3909 " a7dedf89c997099ec9598a964e87cf5096c74423,James Guthrie,Fri Dec 17 13:48:03 2021 +0100,Fix weirdly formatted comment,It appears as though this comment had a run-in with auto-formatting which left it somewhat mangled. ce73f25a87cdc54e911d7c84279be5a0dc48aa2f,Sven Klemm,Mon Dec 20 00:22:17 2021 +0100,Optimize first/last,This patch optimizes how first()/last() initialize the compare function. Previously the compare function would be looked up for every transition function call but since polymorphic types are resolved at parse time for a specific aggregate instance the compare function will not change during its lifetime. Additionally this patch also fixes a memory leak when using first()/last() on pointer types. These changes lead to a roughly 2x speed improvement for first()/ last() and make the memory usage near-constant. Fixes #3935 b735fdcf85e06856c015191d5e47783ecca5d1a0,Sven Klemm,Mon Dec 20 09:21:12 2021 +0100,Fix segfault in add_compression_policy,Postgres snprintf before 89ad14cd787 will segfault on NULL char pointers. This patch changes add_compression_policy to check for NULL get_rel_name before reporting the error. https://github.com/postgres/postgres/commit/89ad14cd787 7824b10a5ae7f4bcde5e874bc4be053e0ae357d3,Erik Nordström,Mon Dec 13 09:27:42 2021 +0100,Allow changing a data node via alter server,"Allow changing the name and configuration of a data node via `ALTER SERVER` since there is no `alter_data_node` command. The functions `add_data_node` and `delete_data_node` are wrappers around `CREATE SERVER` and `DROP SERVER`, respectively. They block the PostgreSQL commmands since they do additional things, like data node bootstrapping. However, there's currently no way to change a data node's configuration, and no additional functionality is required over standard `ALTER SERVER`, so we might as well allow it until a `alter_data_node` is implemented. Fixes #3915 " 0deffe2b643ea9ea51d65d73d656ddbe3e2aa2a2,Sven Klemm,Sat Dec 18 05:11:41 2021 +0100,Improve error handling in add_reorder_policy,When trying to add reorder policy to internal compressed hypertable add_reorder_policy would segfault. 1d40b2e4a621cf83d61e6849c1185f1dab43edce,Sven Klemm,Sat Dec 18 02:37:16 2021 +0100,Fix segfault when creating cagg,When trying to create a continuous aggregate on the internal compression hypertable a segfault would occur. This patch adds a check for this to the cagg validation logic and prevents creating cagg on the internal compressed hypertable. d989e61b56bc97ea5fc014a964a3ef4d1ccf9e77,Sven Klemm,Sat Dec 18 01:40:11 2021 +0100,Improve show_chunks and drop_chunks error handling,This patch fixes a segfault when calling show_chunks on internal compressed hypertable and a cache lookup failure for drop_chunks when calling on internal compressed hypertable. 39645d56da87521c3491efa97bafece00805d2b4,Sven Klemm,Thu Dec 16 23:16:34 2021 +0100,Fix subtract_integer_from_now on 32-bit platforms,"This patch fixes subtract_integer_from_now on 32-bit platforms, improves error handling and adds some basic tests. subtract_integer_from_now would trigger an assert when called on a hypertable without integer time dimension (found by sqlsmith). Additionally subtract_integer_from_now would segfault when called on a hypertable without partitioning dimensions. " a7608871452bd685ae8f87c3b554f929bdb9e575,Sven Klemm,Fri Dec 17 05:21:19 2021 +0100,Fix projection handling in gapfill,When getting the next tuple from the subplan gapfill would apply the projection to it which was incorrect since the subplan already did the projection and the projection for the gapfill tuple has to be done when the tuple is handed to the parent node. Fixes #3834 244568f23a7719283edc4f0a00c0e902b603c013,Fabrízio de Royes Mello,Fri Oct 22 09:47:56 2021 -0300,Add regression tests for caggs+compression,Closes timescale/timescaledb-private#962 d8d392914a7e37078c7a9589ad9b0b9e3859d755,gayyappan,Mon Sep 27 14:42:06 2021 -0400,Support for compression on continuous aggregates,Enable ALTER MATERIALIZED VIEW (timescaledb.compress) This enables compression on the underlying materialized hypertable. The segmentby and orderby columns for compression are based on the GROUP BY clause and time_bucket clause used while setting up the continuous aggregate. timescaledb_information.continuous_aggregate view defn change Add support for compression policy on continuous aggregates Move code from job.c to policy_utils.c Add support functions to check compression policy validity for continuous aggregates. 0d285d18a01f28a7b993efa310064ae45a21b7f5,gayyappan,Wed Sep 22 12:53:39 2021 -0400,Fix selectedCols list for materialized hypertable while creating view, 99521fef16d851a55f41b8e9d355774afe8beb8e,Aleksander Alekseev,Fri Dec 17 11:53:30 2021 +0300,Fix compilation against PG15,Instead of `#if PG14` added in 519e5de21 it should have been `#if PG14_GE`. da5ee6fc414e2a4c6a1dd323a1fb799a1aa4434f,Alexander Kuzmenkov,Thu Dec 16 20:29:21 2021 +0300,Add -Wundef to compiler flags,"Today was another time I used `#if PG14_LT` but forgot to include compat.h, spent a good deal of time debugging this. This flag will catch references to undefined macros. " e0f02c8c1a86eb4e4b913f39c34f53bc410dc4d7,Erik Nordström,Tue Dec 14 18:25:36 2021 +0100,Add option to drop database when deleting data node,"When deleting a data node, it is often convenient to be able to also drop the database on the data node so that the node can be added again using the same database name. However, dropping the database is optional since it should be possible to delete a data node even if it is no longer responding. With the new functionality, a data node's database can be dropped as follows: ```sql SELECT delete_data_node('dn1', drop_database=>true); ``` Note that the default behavior is still to not drop the database in order to be compatible with the old behavior. Enabling the option also makes the function non-transactional, since dropping a database is not transactional. Therefore, it is not possible to use this option in a transaction block. Closes #3876 " 91f3edf609ac77523b982634d1fe6bebd7117662,Aleksander Alekseev,Wed Dec 15 16:15:59 2021 +0300,Refactoring: get rid of max_bucket_width,"Our code occasionally mentions max_bucket_width. However, in practice, there is no such thing. For fixed-sized buckets, bucket_width and max_bucket_width are always the same, while for variable-sized buckets bucket_width is not used at all (except the fact that it equals -1 to indicate that the bucket size is variable). This patch removes any use of max_bucket_width, except for arguments of: - _timescaledb_internal.invalidation_process_hypertable_log() - _timescaledb_internal.invalidation_process_cagg_log() The signatures of these functions were not changed for backward compatibility between access and data nodes, which can run different versions of TimescaleDB. " beb8527defc766543fe0e22de0250a876e35fc42,Alexander Kuzmenkov,Fri Dec 10 17:32:30 2021 +0300,Don't excessively look up the extension oid,Use the one from the static variable. This improves performance by avoiding excessive catalog lookups. d437c70df8320a19ee43d9b2b809f624931d35c4,Sven Klemm,Sat Dec 11 12:05:00 2021 +0100,Enable downgrade test for PG14 packages,"When initially adding the package tests for PG14 Debian, Ubuntu and RPM packages we couldn't test downgrade with the packages because there was no previous version with PG14 support. Since we now have released a 2nd version with PG14 support we can enable the downgrade test for Debian, Ubuntu and RPM packages. " 1b6d2428bbb804bc72b4aeee7763b4d9cedab178,Alexander Kuzmenkov,Fri Dec 10 17:29:59 2021 +0300,Don't do unnecessary planning for the relations proven to be empty,This imporves performance. 7f494077eda069502723c4e79feea8b34af21bf5,Sven Klemm,Fri Dec 10 23:56:39 2021 +0100,Fix DataNodeScan plans with One-Time Filter,When a query has a filter that only needs to be evaluated once per query it will be represented as a Result node with the filter condition on the Result node and the actual query as child of the result node. find_data_node_scan_state_child did not consider Result node as valid node to contain a DataNodeScan node leading to a `unexpected child node of Append or MergeAppend: 62` for queries that had one-time filter with a subquery. 958040699c4986a98fab788bafd599d24281f1d8,Aleksander Alekseev,Mon Dec 6 18:19:33 2021 +0300,Monthly buckets support in CAGGs,"This patch allows using time_bucket_ng(""N month"", ...) in CAGGs. Users can also specify years, or months AND years. CAGGs on top of distributed hypertables are supported as well. " 1b4780df3127565991a7a6e7436f3743663c0a6d,Sven Klemm,Fri Dec 10 23:01:20 2021 +0100,Fix assertion failure in cursor_fetcher_rewind,The code in cursor_fetcher_rewind asserted that there always is an associated request which is not true if EOF was reached already. Found by sqlsmith. Fixes #3786 babcc8e9ee05219a638ff3a46b9779fac87d1778,Alexander Kuzmenkov,Fri Dec 10 17:26:09 2021 +0300,Use function oids instead of names when transforming the sort keys,"This is more explicit, and looking up the function name each time has a performance impact. " 519e5de21ce7ee5b88fde98e1646f36c45b8e1bb,Dmitry Simonenko,Mon Dec 13 14:29:24 2021 +0300,Fix role type deparsing for GRANT command,"This change fixes GRANT command deparsing by including handling of the special role types such as: PUBLIC, CURRENT_USER, SESSION_USER and CURRENT_ROLE (PG14). Fix #3910 " 765d7375cea124f9a39ab1497a0f062eefc493d5,Sven Klemm,Mon Dec 13 02:14:24 2021 +0100,Fix segfault on insert into internal compressed table,When trying to insert into the internal compressed hypertable timescaledb would segfault. This patch blocks direct inserts into the internal compressed hypertable through our tuple routing. Internally we don't use this code path for compression as we create chunks explicitly and insert directly into those chunks in compress_chunk. Fixes #3920 99746ed8baee7e3e6a874a584dd72c04f947dbf6,Mats Kindahl,Wed Dec 8 17:12:32 2021 +0100,Check custom scan node registration,"When loading several different versions of shared libraries after a restart, it might attempt to register the custom scan nodes several times. This is fixed by adding a function that checks if the custom scan method was already registered and skips registering it if it was. Fixes #3901 " fd5dc6002c1058b740e1d6695c2386341b03ede4,Aleksander Alekseev,Fri Dec 10 16:02:29 2021 +0300,Support UPPERREL_PARTIAL_DISTINCT stage in fdw_create_upper_paths(),UpperRelationKind enum was extended in the upstream and now can be UPPERREL_PARTIAL_DISTINCT as well. This patch rewrites the switch-case expression in the fdw_create_upper_paths() procedure accordingly. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=22c4e88e b208f5276f16272c8840a0a37b3e741ede453ca8,Mats Kindahl,Tue Dec 7 13:45:27 2021 +0100,Remove C language recompress_chunk,"Since we are re-implementing `recompress_chunk` as a PL/SQL function, there is no need to keep the C language version around any more, so we remove it from the code. " 0f81a60cbb7d0aa67a575782a6c1f66c86e5f9ec,Alexander Kuzmenkov,Fri Dec 10 14:22:10 2021 +0300,Use row-by-row fetcher to enable parallel plans on data nodes,"The row-by-row fetcher is more efficient, so we want to use it when we can -- that is, when the have to read only one table from the data node, without interleaving it with anything else. This patch adds an option of choosing the fetcher type automatically. It detects the simplest case of only one distributed table in the entire query, and enables row-by-row fetcher. For other cases, the cursor fetcher is used. " 7f2f7e922e3506c0de86b8507611cced7b22b722,Nikhil Sontakke,Fri Dec 10 15:10:58 2021 +0530,Fix use-after-free crash in remote heal code,Address sanitizer reported a crash due to a use-after-free code block. Fixed that and verified with a subsequent sanitizer local run. Fixes #3907 aae19319c01ceffb18d6a6eb374132e0f2678290,Mats Kindahl,Fri Dec 3 10:31:37 2021 +0100,Rewrite recompress_chunk as procedure,"When executing `recompress_chunk` and a query at the same time, a deadlock can be generated because the chunk relation and the chunk index and the compressed and uncompressd chunks are locked in different orders. In particular, when `recompress_chunk` is executing, it will first decompress the chunk and as part of that lock the uncompressed chunk index in AccessExclusive mode and when trying to compress the chunk again it will try to lock the uncompressed chunk in AccessExclusive as part of truncating it. Note that `decompress_chunk` and `compress_chunk` lock the relations in the same order and the issue arises because the procedures are combined inth a single transaction. To avoid the deadlock, this commit rewrites the `recompress_chunk` to be a procedure and adds a commit between the decompression and compression. Committing the transaction after the decompress will allow reads and inserts to proceed by working on the uncompressed chunk, and the compression part of the procedure will take the necessary locks in strict order, thereby avoiding a deadlock. In addition, the isolation test is rewritten so that instead of adding a waitpoint in the PL/SQL function, we implement the isolation test by taking a lock on the compressed table after the decompression. Fixes #3846 " 4cecdb50f9def569547b01bbb26118dbed377cd9,Nikhil Sontakke,Mon Nov 29 18:28:33 2021 +0530,Fix remote txn heal logic,* A few tweaks to the remote txn resolution logic * Add logic to delete a specific record in remote_txn table by GID * Allow heal logic to move on to other cleanup if one specific GID fails * Do not rely on ongoing txns while cleaning up entries from remote_txn table Includes test case changes to try out various failure scenarios in the healing function. Fixes #3219 ea342f139627d657b09801971dadaa77cf81b281,Aleksander Alekseev,Thu Dec 9 15:14:07 2021 +0300,Remove the rest of Value* type usage,"This is a follow-up to 8afac602. In this commit we refactored the code that used Value* type because the type was removed in the upstream. However, we missed a few places. This patch fixes it. " 19fa286200c1f7eeb866559d55c73aaa370dcb5e,Aleksander Alekseev,Wed Dec 8 16:10:00 2021 +0300,Add make_new_heap_compat() macro,"The number of arguments of make_new_heap() has changed in PG15. In order to be able to compile against PostgreSQL 15 use a _compat() macro instead of the direct function call. Also, it should have been #if PG15_GE, not #if PG15 - fix this too! https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b0483263 " eeb1ea96ef76cbcd1468d34ee6b475c97f170ad6,Aleksander Alekseev,Tue Dec 7 16:05:52 2021 +0300,Use compat.h version of FirstBootstrapObjectId,The macro FirstBootstrapObjectId was renamed in the upstream. This patch adds a corresponding FirstBootstrapObjectIdCompat macro to compat.h and uses it instead of FirstBootstrapObjectId. This will allow us to compile agains PG15. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=a49d0812 3a83e75789358258ff7fe92f99427d21a110b506,Aleksander Alekseev,Mon Dec 6 17:43:00 2021 +0300,Add shm_mq_send_compat() macro for compatibility with PG15,There was an additional argument added to shm_mq_send() in the upstream. This patch adds a corresponding _compat() macro to be able to compile against future versions of PostgreSQL. Better handle this now than solve a bunch of problems one year later. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=46846433 9d4c94f971da44afb446a7d6761dd049daadc6db,Dmitry Simonenko,Tue Dec 7 12:37:04 2021 +0300,Fix ALTER SET/DROP NULL contstraint on dist hypertable,This change adds support for ALTER TABLE SET/DROP NULL commands to the list of supported commands that can be run with distributed hypertable. Fix #3860 613e08a27d4234f516b6a4bb492a8d05192171a9,Sven Klemm,Mon Dec 6 20:00:36 2021 +0100,Remove dead code in dist_ddl_process,Commit 6424dd20 added a redundant early exit to dist_ddl_process which triggered a Coverity dead code warning. This patch refactors the code slightly to remove the redundant code and get rid of the coverity warning. ad8a5e68c91a1d7bb6f92cd226578400716b0b05,Dmitry Simonenko,Mon Dec 6 13:03:33 2021 +0300,Support CREATE SCHEMA on distributed database,This PR makes CREATE SCHEMA command shippable to all data nodes in the distributed database. Additional support for `GRANT ON SCHEMA` command added. Fix #3690 1bc1993c61c61fcf4d75284f44a46d330d1c4fb8,Duncan Moore,Thu Dec 2 23:21:54 2021 +0000,Post release 2.5.1, a82f97dfc2b38c1ef8bb87470ad1904d0ce91db5,Aleksander Alekseev,Fri Dec 3 14:54:34 2021 +0300,Use standard random() instead of pg_lrand48(),pg_*rand48() functions family was removed in the upstream and the code that uses it will not compile against PG15. This patch replaces pg_lrand48() with standard random() function from stdlib.h. Also the patch rewrites the bgw_db_scheduler test so that the output doesn't contain any pseudo-random data. 21d9b6a6c765a09b1f25ade8cd0f2e8cb254d3f5,Sven Klemm,Fri Dec 3 11:41:30 2021 +0100,Use generic URL in windows package test,Change the windows package test to use an URL with latest instead of explicit package version since this is what we use in docs. 6424dd201293b70b1c509652bae540d90ac5ffa4,Erik Nordström,Wed Nov 24 14:21:03 2021 +0100,Handle default privileges on distributed databases,Statements that alter default privileges are now by default forwarded to all data nodes. For instance: ```sql ALTER DEFAULT PRIVILEGES IN SCHEMA myschema GRANT INSERT ON TABLES TO myrole; ``` is applied across the entire cluster of data nodes. Fixes #3851 1ff6dfe6ab2cbc2d6c655e9a00b185099f972e4b,Mats Kindahl,Thu Dec 2 10:26:50 2021 +0100,Fix race condition in deadlock_recompress_chunk,"After the synchronizing lock is released and the transaction is committed, both sessions are free to execute independently. This means that the query can actually start running before the recompress step has completed, which means that the order for completion is non-deterministic. We fix this by adding a marker so that the query is not reported as completed until the recompress has finished execution. Since markers in isolation tests is a recent thing, we only run the test for PostgreSQL versions with markers added. Part-Of: #3846 " 2ad1eb9afea153a77260b8197489a2753de55d36,Sven Klemm,Mon Nov 29 20:55:10 2021 +0100,Add package test for Ubuntu 21.10,Add 21.10 to the list of tested distributions since we build packages for it. 94dc8373943e01561577df610fcba6dda08301db,Duncan Moore,Tue Nov 30 22:47:40 2021 +0000,Release 2.5.1, 069cb4ed19231c439b5bb2e086a1232ef68d75fe,Markos Fountoulakis,Thu Dec 2 17:36:50 2021 +0200,Fix create_distributed_restore_point memory issue,Sanitizer reports a heap buffer overflow in `create_distributed_restore_point` because we are not copying the restore node name. This commit fixes this by copying the string to a new name buffer. 320119d81ed23752dcdefc86a6ac16d8050df412,Erik Nordström,Thu Dec 2 11:40:15 2021 +0100,Fix SkipScan crash due to pruned unique path,"When SkipScan paths are added it can result in a crash if the new SkipScan dominates the PostgreSQL planned unique path. In that case, the previous unique path is pruned and freed, leading to a potential memory corruption since a pointer to the original unique path is used when looping over all subpaths to try to apply SkipScan. Thus, if the original unique path is pruned, and there are multiple subpaths, a crash will occur in the next iteration over the subpaths. The crash is fixed by making a shallow copy of the original unique path while iterating the subpaths. " 210886e508632d858d2be5b55593f2e49780ebf5,Mats Kindahl,Wed Dec 1 20:07:02 2021 +0100,Ignore deadlock_recompress_chunk in early version,"This isolation test will fail in early versions of the PG servers, just like the other isolation tests, so ignoring failures for it. " 112107546f0be2255e7d872824b41cbc658087c1,Mats Kindahl,Wed Nov 24 17:12:46 2021 +0100,Eliminate deadlock in recompress chunk policy,"When executing recompress chunk policy concurrently with queries query, a deadlock can be generated because the chunk relation and the chunk index or the uncompressed chunk or the compressed chunk are locked in different orders. In particular, when recompress chunk policy is executing, it will first decompress the chunk and as part of that lock the compressed chunk in `AccessExclusive` mode when dropping it and when trying to compress the chunk again it will try to lock the uncompressed chunk in `AccessExclusive` mode as part of truncating it. To avoid the deadlock, this commit updates the recompress policy to do the compression and the decompression steps in separate transactions, which will avoid the deadlock since each phase (decompress and compress chunk) locks indexes and compressed/uncompressed chunks in the same order. Note that this fixes the policy only, and not the `recompress_chunk` function, which still is prone to deadlocks. Partial-Bug: #3846 " 8afac6029b1719e42def0374b2d498a77c06bbb8,Aleksander Alekseev,Mon Nov 29 17:10:54 2021 +0300,Don't use Value type,Value node struct was removed in the upstream and the code that uses it will not compile against PG15. Better rewrite the code now than deal with a whole bunch of changes one year later. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=639a86e3 fed8f1ac88e3438c0f7b57fb296d100224a1a016,Nikhil Sontakke,Thu Nov 25 12:31:34 2021 +0530,Support [DROP|REASSIGN] OWNED BY in multinode,"We ship the ""DROP OWNED BY"" and ""REASSIGN OWNED BY"" queries as is to the datanodes. This assumes that the roles that are referred in these queries are present on the datanodes. " 58ad8bab663f24ec0780e4f91bb9949711b07bec,Markos Fountoulakis,Wed Nov 24 18:44:08 2021 +0200,Add deparser for CreateTrigStmt,Add deparser for CreateTrigStmt to use for distributed hypertables. Fixes #3825 da8ce2e140cb056b7a7bfbc5b84e5b19771a1180,Fabrízio de Royes Mello,Mon Nov 15 16:23:00 2021 -0300,Properly handle `max_retries` option,Surprisly we're not taking care of `max_retries` option leading us to failed jobs running forever. Fixed it by properly handle the `max_retries` option in our scheduler. Fixes #3035 1bdd825b80370dbf3b759e635b2d174034a3d8ab,Erik Nordström,Fri Nov 19 10:56:31 2021 +0100,Add configuration for the GitHub issue template chooser,"Configure the GitHub issue template chooser to include options with links to the page about support requests, the Slack community, and documentation changes. " 7719b7fcff3150cd68f6b0c46e3460ce3c306efc,Erik Nordström,Wed Nov 17 18:52:04 2021 +0100,Add a pull request template,Add a pull request template with Guidelines for a well-formed pull request and the checklist to follow in the PR process. a459b327b1164df9cadc20fd248f4e71701df1e9,Erik Nordström,Thu Nov 18 10:31:12 2021 +0100,Add a feature request issue template,The feature request issue template adds a form for people to fill out so that we can ensure we get the information we need. ea7ebe17de086d3596ecd7194e231602536358c0,Erik Nordström,Fri Nov 19 13:50:45 2021 +0100,Add enhancement issue template,"Add an issue template for ""Enhancement"" requests. An enhancement is an improvement to an existing feature or functionality. The user must pick from a set of features or subsystems to improve. " 87a319810b2329d2298a5e460baa29b615dd80e0,Erik Nordström,Thu Nov 18 16:44:30 2021 +0100,Update bug report issue template,"Convert the bug report issue template into a form that the user has to fill out. The form allows making some fields required. Also add the ""bug"" and ""triage"" labels to all bug reports. The ""triage"" label can be used to find bugs that haven't been triaged, assuming the label is removed as part of the triaging process. " 9e075fa63626df32fc217fb774b7928066d97d0c,Erik Nordström,Thu Nov 18 17:55:01 2021 +0100,Add an issue template for reporting flaky tests,"Tests are sometimes flaky, so add an issue template to report such tests. The template ensures the issue is labeled and has the required information we need to fix the flake. " 387d6842cf17b713d48e23ae15b2150c3537d653,Fabrízio de Royes Mello,Mon Nov 22 17:04:50 2021 -0300,Add missing gitignore entry,Pull request #3779 introduced a new template SQL test file but missed to add the properly gitgnore entry to ignore generated test files. ab61b90f5881217c000f8d691c513c8f6861949a,Dmitry Simonenko,Mon Nov 22 17:36:25 2021 +0300,Test enabling dist compression within a procedure,"Make sure ALTER TABLE SET (timescaledb.compress) command properly propagated to the data nodes when executed within a procedure. This includes only test a case, actual fix was done in #3663. Fix #3705 " bedc624c7301897b28a37d848c0f73750359a6bb,Sven Klemm,Mon Nov 22 13:23:25 2021 +0100,Fix homebrew test to work on macos 11,In CI macos-latest currently may resolve to MacOS 10.15 or 11. But in MacOS 11 postgres seems to not be started by default leading to the package check failing because postgres is not running. This patch changes the test to start postgres which works for both MacOS environments and fixes the homebrew test on MacOS 11. de24d81b47b8c1e8f543adb7b9e4899fb2b2512c,Sven Klemm,Sun Nov 21 17:10:58 2021 +0100,Fix snapshot pg regresscheck,Upstream added a new regression test for binary compatibility check. Since we define our own types the test will fail with timescaledb installed as the test does not expect to see those additional types. This patch changes our CI to ignore result of type_sanity test. Currently those are only in REL_XX_STABLE but with release of next minor version those will enter normal tests as well. https://github.com/postgres/postgres/commit/cf3d79aa31f 3a1ff28d3e202aaa63fd71b6104934db3b107fb6,Alexander Kuzmenkov,Wed Nov 17 15:32:06 2021 +0300,Evaluate more timestamp-related stable functions on data nodes,"This commit extends the stable function whitelist with some timestamp-related functions that were not there before. They depend on timezone settings which we synchronize on data nodes, so they are safe to evaluate there. " 48cdf71da9c1f015320a085dc11d47f3c45957ea,Nikhil Sontakke,Tue Nov 16 20:07:48 2021 +0530,Fix INSERT..SELECT involving dist hypertables,If the relation being inserted into and the relation in the SELECT are both distributed hypertables then do not use COPY. This is because we will need two connections to the same set of datanodes (one to ingest data into src table using COPY and one to SELECT from dst table) which is not supported currently in a single transaction. This restriction will be lifted in the future. Fixes #3447 0a82e7f0ab1b406ae91230621ad7ce75c8ff3468,Sven Klemm,Wed Nov 17 18:41:24 2021 +0100,Add ABI test for minimum required pg version,This patch adds a test to detect changes in minimum required pg versions for our packages. We build timescaledb against latest postgres version and then run the tests with that library loaded into previous postgres version. This patch also changes the ABI test for maximum compatible postgres version to use the same format and removes the old ABI test code. 0de4955ca56679470589742a8494ad78f5308b46,Aleksander Alekseev,Wed Nov 17 18:30:12 2021 +0300,Treat clang-tidy warnings as errors,"This change makes the build fail on macos-* if clang-tidy detects any bugs. This improves our process in two ways. Firstly, currently we execute clang-tidy on macos-* but ignore the warnings. As a result we waste CPU resources. Also, somebody who uses MacOS has to fix the warnings manually while this could be done by the person who introduced corresponding changes. Secondly, running clang-tidy on a laptop is relatively expensive. This change allows to disable clang-tidy locally and speed up the development process. " 9a4c4ca992fe1c7ad762ba098e962b4e544d29d3,Aleksander Alekseev,Wed Nov 17 17:24:31 2021 +0300,Fix reading garbage value from TSConnectionError,clang-tidy reports that after eed2810d under certain conditions remote_connection_error_elog() macro can read from the memory that was not initialized. Make sure this can't happen. a4a2c4e9d1c8e8ad585913feea885df1260d8424,Markos Fountoulakis,Thu Nov 18 14:53:24 2021 +0200,Remove pointers from CAGG lists for 64-bit archs,"Currently, the int64 elements are always passed by reference, which is not necessary on 64-bit architectures where it can be passed by value. Make it so they are only passed by reference for 32-bit architectures and by value for 64-bit architectures, since the element fits in a 64-bit architecture pointer. " 103c1b25b84e93c71ee4ec88f61db32aba559dc0,Dmitry Simonenko,Thu Nov 18 12:02:35 2021 +0300,Support GRANT/REVOKE on distributed database,"This PR adds support for ""[GRANT|REVOKE] ON DATABASE"" command when executed on distributed databse. If current database is distributed, the command will be rewritten to respect each data node database name and send to each data node accordingly. Fix #3594 " ca82a9b13569f8fb12ff85f4df052d32a682bee2,Mats Kindahl,Fri Nov 12 09:24:22 2021 +0100,Fix error printout on correct security label,"For all security labels from provider `timescaledb` an error was printed, which make `pg_restore` fail because the security labels are printed to the dump. Instead, we only print an error if the security label is not of the correct format. Fixes #3760 " 1df000e0dc970726a9736641590b79bb0b372823,Fabrízio de Royes Mello,Wed Nov 17 10:53:49 2021 -0300,Backport PG13 typalign macros for PG12,Postgres 13 introduced macros for typalign and typstorage constants in commit postgres/postgres@3ed2005ff59. For better code readability let's port `TYPALIGN_CHAR` macro for the compatibility layer. Closes #3817 2ccba5ecc97c7e6086eab8e0be60c630d8018d4c,Fabrízio de Royes Mello,Tue Nov 16 16:43:16 2021 -0300,Refactor isolation tests to prevent SQL injection,During the developing of Continuous Aggregates for Distributed Hypertables we left some work to be done later and refator the isolation tests to prevent SQL injection was on of them. Per discussion: https://github.com/timescale/timescaledb/pull/3693#discussion_r735098888 Epic issue #3721 eed2810d2b913cb4784430c1f7529bc85783252e,Dmitry Simonenko,Wed Nov 17 15:45:55 2021 +0300,Fix NULL pointer evaluation in fill_result_error(),Fix #3781 f1e103fab1fd59fb3a8ebe43ae9c084120f1ec68,Alexander Kuzmenkov,Tue Nov 16 17:48:30 2021 +0300,Fix DISTINCT ON queries for distributed hyperatbles,"Previously, we would push DISTINCT ON down to the data nodes even when the pathkeys of the resulting paths on the data nodes were not compatible with the given DISTINCT ON columns. This commit disables pushdown when the sorting is not compatible. Fixes #3784 " ecb59170d3c6c69b21f4cdad0e272a9c55dd0b00,Mats Kindahl,Wed Nov 17 10:16:21 2021 +0100,Add krb5-dev dependency to alpine build,"The header file `gssapi/gssapi.h` is present in Alpine package `krb5-dev` but for some reason it was not installed prior to building, causing upgrade and downgrade test failures. " b78b25d3178945d73977c2f8dc2c30b4248f500b,Erik Nordström,Fri Nov 12 14:33:18 2021 +0100,Fail size utility functions when data nodes do not respond,"Size utility functions, such as `hypertable_size()`, excluded non-responding data nodes from size calculations, which led to the functions succeeding but returning the wrong size information. To avoid reporting confusing numbers, it is better to fail. This change updates the SQL queries for the relevant functions to no longer exclude non-responding data nodes and also adds a TAP test to illustrate the error when data nodes are not responding. Fixes #3713 " 8a5cb543432ddeb737be1a3de01057b8f42ba906,Fabrízio de Royes Mello,Fri Oct 29 11:40:24 2021 -0300,Add Postgres 14 prerequisites for compilation, aee832c3cbca25fe65764624844e8fde3ed2cff1,Erik Nordström,Fri Nov 12 09:34:29 2021 +0100,Tweak chunk creation error message,"When a data node tries to create a chunk itself, and not via the access node, an error is raised. The hint in this error was not conforming to the error style guide, so fix it. " 4a17d4c40297d0295bb80a1575d296e6426d4e0f,Alexander Kuzmenkov,Mon Nov 15 14:10:14 2021 +0300,Add shellcheck to CI,"Writing a shell script correctly can be hard even for a skilled programmer. shellcheck is a static analysis tool that helps catch common errors in shell scripts. We now have 36 executable scripts in our repository, for which shellcheck reports 126 errors (calculated like find . -type f -executable -exec bash -c '[ ""$(file --brief --mime-type ""$1"")"" == ""text/x-shellscript"" ]' sh {} \; -exec shellcheck -f gcc --exclude=SC2086 {} \; | cut -d: -f1 | sort | uniq | wc -l). This commit fixes these warnings and adds a GitHub actions workflow that runs shellcheck on all the executable shell scripts in the repository. The warning SC2086: Double quote to prevent globbing and word splitting is disabled globally, because it has little practical consequences, sometimes leads to false positives, and is general is too widespread because people forget to quote. " c91faad221248c35304f5d38f8b4ebf8a6ad6d64,Sven Klemm,Wed Oct 27 18:28:01 2021 +0200,Add coccinelle workflow to CI,This patch adds a workflow to run coccinelle scripts against our codebase. This can be used to find errors in the code base. The initial patch adds a check to find unbalanced heap_form_tuple heap_freetuple occurances. https://coccinelle.gitlabpages.inria.fr/website/ e4ec15d8812455099293523b698d0163e6faab1c,Sven Klemm,Wed Nov 10 09:45:32 2021 +0100,"Bump postgres versions used in CI to 12.9, 13.5, 14.1", 509c2b99ab7e031f06ffd8815f11b40b8c981eed,Aleksander Alekseev,Thu Nov 11 15:36:03 2021 +0300,Silence 3 clang warnings for simple8b_rle.h/dictionary.c,"CLang 11 generates several warnings regarding possible null pointer dereferences and division by zero. The warnings seem to be false-positive. However, they may hide new valuable warnings. Thus this patch double-checks that there is nothing to be concerned about and also silences these warnings. " 10b8979db930600bf2e82dfddfee980d7781dbb0,Mats Kindahl,Wed Oct 27 13:18:11 2021 +0200,Add skeleton release workflow,"In order to be able to run workflows using `workflow_dispatch`, the event need to be defined on the default branch. This commit adds a basic workflow for the release pipeline containing only a workflow that can be triggered using `workflow_dispatch` with the correct parameters and will just list the commits in the branch since the last tag. With this workflow in the default branch, you can run the workflow either from the web interface, or using something like: gh workflow run release.yaml --ref 2.1.x -f version=2.1.1 " 7591f7f1f3a03b25d69f96159244e7f8ce84574b,Sven Klemm,Mon Nov 8 15:00:29 2021 +0100,Fix time_bucket comparison transformation,The time_bucket comparison transformation code assumed the value and the width of the time_bucket comparison expression were both Const. But this was not validated only asserted. This can lead to wrong query results. Found by sqlsmith. 0f5268ad49cab42b6876623b7b01b048d9ab0166,Aleksander Alekseev,Mon Nov 8 12:22:53 2021 +0300,Fix possible uninitialized value in test_utils.c,Supress the following CLang warning: ``` warning: 2nd function call argument is an uninitialized value [clang-analyzer-core.CallAndMessage] ``` 82656a6807cc66dd2e0ed9c7059a6222f53c7698,Mats Kindahl,Mon Oct 25 14:27:45 2021 +0200,Fix flaky pg_dump,"Calling `CREATE DATABASE` we cannot have any backends connected to the source database, except the current one. Since background workers connect to any database that has installed the extension, they can linger and cause a flaky error. The test changed to just check that the extension can be implicitly installed, but this require waiting for backends to be unregistered from `procArray` in `procarray.c`, which the function `pg_terminate_backend` does not handle properly and hence results in a flaky test. This happen because `pg_terminate_backend` only signals the process to terminate, but does not wait for it to actually terminate. Instead, we add an explicit function that checks the `procArray` and waits for it to not have any backends for a database and uses that to wait for termination. " 7e3e771d9f4c2feac6c6d75ba1909af4b5c2a682,Fabrízio de Royes Mello,Mon Oct 25 17:26:22 2021 -0300,Fix compression policy on tables using INTEGER,"Commit fffd6c2350f5b3237486f3d49d7167105e72a55b fixes problem related to PortalContext using PL/pgSQL procedure to execute the policy. Unfortunately this new implementation introduced a problem when we use INTEGER and not BIGINT for the time dimension. Fixed it by dealing correclty with the integer types: SMALLINT, INTEGER and BIGINT. Also refatored the policy compression procedure replacing the two procedures `policy_compression_{interval|integer}` by a simple `policy_compression_execute` casting dimension type dynamically. Fixes #3773 " 151637db89b57b43c1974fd469cd37f98b77e8b6,Sven Klemm,Wed Nov 3 12:20:38 2021 +0100,Allow ALTER TABLE DROP COLUMN on compressed hypertable,This patch allows dropping columns from hypertables with compression enabled when the dropped column is neither an orderby nor a segmentby column. Fixes #3683 5eadbbd0978fa327b57a9e91b3b321ca0b48c053,Aleksander Alekseev,Thu Nov 4 16:28:00 2021 +0300,const'ify CaggsInfo structure,CaggsInfo is an immutable structure and a good candidate to be const'ified. This simplifies the maintenance of the code and probably could save a CPU cycle or two here and there. 1aabbc83f8ead6fe86961d02d9f481c85d1c86d8,Nikhil,Mon May 24 14:48:58 2021 +0530,Enable 1PC and DN reads for read only AN txns,"In high availability setups, a streaming replica of the access node can be configured. Since it's a hot standby users should be able to run read-only queries from this replica and should be able to query the distributed hypertables from it. Additionally, the internal code which sets up a connection from the standby access node to the data nodes now marks the ongoing transaction explicitly as READ ONLY ones. This ensures that any functions or any activity which tries to make modifications on the data nodes errors out. Note that even if ""timescaledb.enable_2pc"" is enabled, we will still use 1PC for these READ ONLY transactions. While we are doing this, we also handle the case when a user explicitly marks a transaction as ""READ ONLY"" on the primary AN. Even in that case the connection to the DNs will be marked read only. Additionally, similar to the above standby case, even if ""timescaledb.enable_2pc"" is enabled, we will still use 1PC for these READ ONLY transactions on the primary AN. In normal Postgres standby, the ongoing transaction becomes READ WRITE immediately on promotion whereas in our case the ongoing transaction will remain READ ONLY till completion. New transactions on the same session will be READ WRITE as expected. " 449019f3c1d68359537b53196a0f3276c34acf19,Aleksander Alekseev,Thu Nov 4 14:12:24 2021 +0300,Remove dead code,"Remove two procedures declared in continuous_agg.c: - ts_create_arrayexprs_from_caggs_info - ts_continuous_agg_get_user_view_oid This code is not used for anything and, nor is test-covered. " 77d1267eb1ae01bb503c38b5f2ab8b21601e04a8,Sven Klemm,Sun Oct 31 23:47:07 2021 +0100,Remove unused function arg from metadata functions,Remove the key_type argument from metadata functions as all callers passed STRINGOID and there is no need to support other types for the key. 7d9ea6237ce581e4e74757fd5b3dccf0e7609d72,Sven Klemm,Sun Oct 31 14:42:06 2021 +0100,Remove namein calls from scankey initialization,A lot of our scankey initialization when scanning indexes for a name had a superfluous namein call. This patch remove those unnecessary calls. 93ffec7c10c70e1ba82a0aafb36614805ed26848,Sven Klemm,Mon Nov 1 08:22:52 2021 +0100,Allow ALTER TABLE ADD COLUMN with DEFAULT on compressed hypertable,When adding a new column with a default postgres doesnt materialize the default expression but just stores it in catalog with a flag signaling that it is missing from the physical row. Any expressions used as default that do not require materialization can be allowed on compressed hypertables as well. The following statements will work on compressed hypertables with this patch: ALTER TABLE t ADD COLUMN c1 int DEFAULT 42; ALTER TABLE t ADD COLUMN c2 text NOT NULL DEFAULT 'abc'; 6bce5da97b171c7cf8e25301eab57c7803967876,Sven Klemm,Sun Oct 31 20:30:50 2021 +0100,Fix segfault in ts_hist_sfunc,Since the number of buckets of the histogram function is an argument to the function call it is possible to initialize the histogram state with a lower number than actually needed in further calls leading to a segfault. This patch changes the memory access to use the number the state was initialized with instead of the number passed to the call. This also changes the function to error when the passed number differs from the initialized state. 20543bf66b4de035d7890cdb642eed4cef2fed53,Markos Fountoulakis,Fri Oct 29 17:56:19 2021 +0300,Refactor function in dist_commands.c,"Fix coverity issue for Out-of-bounds access (ARRAY_VS_SINGLETON): ""Passing &sql to function ts_dist_multi_cmds_params_invoke_on_data_nodes which uses it as an array"" " af930685665b79b4dcb29f39c30e043864e25336,Chris Bisnett,Sat Oct 30 09:18:23 2021 -0400,Fix typo in error message,This patch fixes a typo in the error message of CREATE INDEX when using timescaledb.transaction_per_chunk. Co-authored-by: Sven Klemm 51a007d7065cef212abf342a40487b528e76752b,Sven Klemm,Fri Oct 29 21:24:48 2021 +0200,Refactor ts_hypertable_insert_path_create,"Change ts_hypertable_insert_path_create and add hypertable as argument since we always check for that in caller allowing us to remove an error branch from the function, save an additional lookup for the hypertable and save a lookup for the range table entry. This patch also changes the name of ts_chunk_dispatch_is_state to ts_is_chunk_dispatch_state to be consistent with naming of similar functions. " 999f81bfd7339f702e151ec8b61adca88c324915,Sven Klemm,Fri Oct 29 19:46:42 2021 +0200,Move ChunkAppend and AsyncAppend code into nodes directory,Move the ChunkAppend and AsyncAppend code into nodes directory to make it consistent with other custom node code. e0e3bd1df52410890e7ca4866201fd5f10d12abb,Sven Klemm,Fri Oct 29 18:53:36 2021 +0200,Remove obsolete prototypes,This patch removes the prototypes for the following functions: policy_compression_proc tsl_hypertable_invalidation_log_delete tsl_materialization_invalidation_log_delete These were left in the code in previous refactorings even though they are no longer used. 3d119275677dafc804637bf86ef107a9aad4ae23,Dmitry Simonenko,Fri Oct 29 15:41:50 2021 +0300,Rework distributed DDL processing logic,"This patch does refactoring and rework of the logic beside dist_ddl_preprocess() function. The idea behind it is to simplify process by splitting each DDL command logic inside separate function and avoid relaying on the hypertable list count to make decisions. This change allows easier to process more complex commands (such as GRANT), which would require query rewrite or to be executed on a different data nodes. Additionally this would make it easier to follow and be more alike as main code path inside src/process_util.c. " 56d31d855fd8805e39ca9c1c0332e27b2907de00,Sven Klemm,Thu Oct 28 12:44:08 2021 +0200,Add PG14 to package tests,This patch adds PG14 to the package tests. Additionally the package downgrade test is disabled for PG14 since we do not have a released version to downgrade to yet. df7f058ad13802a34290ebe332962b95296568c6,Fabrízio de Royes Mello,Thu Oct 28 05:48:22 2021 -0300,Post release 2.5.0,Add 2.5.0 to update test scripts for PG12 and PG13 and create update test script for PG14. Fix missing CHANGELOG thanks for external contributors. 8925dd8e15c30ab5bdc1013abb063d4d05f64f06,Fabrízio de Royes Mello,Tue Oct 26 17:23:45 2021 -0300,Release 2.5.0,"This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode " e02f48c19de1f0b0cfe705ebf864801ac2b7d626,Erik Nordström,Wed Oct 27 09:36:41 2021 +0200,Add missing downgrade script files,"Some of the downgrade script files were only added to the release branch and no downgrade path existed for downgrading from the 2.4.2 release to the 2.4.1 release. Add the missing downgrade files for 2.4.x releases to the main development branch. Also, avoid generating downgrade scripts for old versions, so that only the downgrade script for the current version is generated by default. A new CMake option `GENERATE_OLD_DOWNGRADE_SCRIPTS` is added to control this behavior. Note that `GENERATE_DOWNGRADE_SCRIPT` needs to be `ON` for the new option to work. The reason we want to keep the ability to generate old downgrade scripts is to be able to fix bugs in old downgrade scripts. " d117d8772f3cf59d66c7d0f85ffa915958e29222,Fabrízio de Royes Mello,Wed Oct 27 09:48:25 2021 -0300,Add missing gitignore entry,Pull request #3717 introduced a new template SQL test file but missed to add the properly gitgnore entry to ignore generated test files. f8bf3b9767a66c97d49d6274f4c5c4eba2ad2c67,Mats Kindahl,Wed Oct 27 10:04:06 2021 +0200,Always reset expr context in DecompressChunk,"When decompressing very large compressed chunks as part of executing a `DecompressChunk`, the expression memory context was not reset after each qual evaluation if the qual did not match, meaning that if the qual did not match often, memory allocations would start to accumulate. This commit fixes this by resetting the expression memory context for each tuple when executing a `DecompressChunk` node, even if the qual did not match. Fixes #3620 " 6db012dcc9c20b4777e37e127609e620e1a3544c,Nikhil Sontakke,Wed Oct 27 11:38:01 2021 +0530,Fix ANALYZE crash with custom types,Extensions like PostGIS introduce custom datatypes for columns. They also have custom statistics for such custom types. We already state in our comments that stats for custom types are not supported but it was never tested. Fix that now. Fixes #3733 0fecefdafa44446595a0cf69b3986059da2b9711,Sven Klemm,Tue Oct 26 20:01:33 2021 +0200,Skip chunk_utils for PG12,Skip the chunk_utils test on PG12 instead of ignoring it because since the test segfaults it might actually cancel other tests even though it doesn't run in parallel. a9e00497a2a8fce8e4016b343587d3a11662df45,Sven Klemm,Mon Oct 25 12:48:00 2021 +0200,Improve memory tests,"This patch implements various improvements to the memory tests. Memory tests are split off into separate workflow to allow running them separately from prerelease tests by pushing to branch memory_test. The scripts are adjusted to no longer build docker images to run the tests but instead running the tests directly. The python script to monitor memory usage was improved and now flushes after printing to make output appear consistently in CI. This patch also changes the memory test to run on PG 12,13 and 14 instead of only 12. " b27c9cbd47870f17e96e86f36cf6c68a7ae5faf5,Sven Klemm,Sat Oct 23 00:27:41 2021 +0200,Add missing heap_freetuple calls,This patch adds missing heap_freetuple calls in 2 locations. The missing call in compression.c was a leak making the allocation live for much longer than needed. This was found by coccinelle. d740e19c5fb0e57be20e58de282e29f00f262888,Erik Nordström,Thu Oct 21 11:06:21 2021 +0200,Fix DirectFunctionCall crash in distributed_exec,"This change fixes a crash that occurred when calling `distributed_exec` via a direct function call. The crash was triggered by a dynamic lookup of the function name via the function Oid in the `FunctionCallInfo` struct in order to generate error messages for read-only and transaction block checks. However, this information is provided by the parsing stage, which is not executed when doing direct function calls, thus leading to a segmentation fault when trying to dereference a pointer in the `FunctionCallInfo` that wasn't set. Note that this problem is not limited to `distributed_exec`; it is present in all SQL-callable functions that use the same pattern and macros. To fix the problem, update the macros and patterns used for checking for read-only mode and transaction blocks to avoid doing the function name lookup when the pointer is not set. Instead fall back to the C function name in that case (via C macro `__func__`). A test case is added in C code to call `distributed_exec` via a direct function call within a transaction block in order to hit the previously crashing error message. The `distributed_exec` function has also been updated with better handling of input parameters, like empty arrays of data nodes, or arrays containing NULL elements. " 6d9715a95139f4d91fed99dd840b78e6533ca7d1,Markos Fountoulakis,Tue Oct 26 12:36:53 2021 +0300,Fix compiler warnings,Fix compiler warnings for MacOS 10.15 build. 221437e8efbfd2dfee084adb3c3b05ae657d90c5,Markos Fountoulakis,Mon Oct 25 17:41:10 2021 +0300,Continuous aggregates for distributed hypertables,"Add support for continuous aggregates for distributed hypertables by allowing a continuous aggregate to read from a distributed hypertable so that the continuous aggregate is on the access node while the hypertable data is on the data nodes. For distributed hypertables, both the hypertable and continuous aggregate invalidation log are kept on the data nodes and the refresh window is computed at refresh time on each data node. Since the continuous aggregate materialization hypertable is not present on the data nodes, the invalidation log was extended to allow using a non-local hypertable id on the data nodes. This means that you cannot create continuous aggregates on the data nodes since those could clash with continuous aggregates on the access node. Some utility statements added entries to the invalidation logs directly (truncating chunks and hypertables, as well as dropping individual chunks), so to handle this case, internal functions were added to allow logging invalidation on the data nodes from the access node. The commit also includes some fixes to memory context usage that caused crashes for invalidation triggers and also disable per data node queries during refresh since that would otherwise generate an exception. Fixes #3435 Co-authored-by: Mats Kindahl " bb1f046a37de782b85d00f6173e857e2f763bc83,Sven Klemm,Fri Oct 22 13:06:22 2021 +0200,Fix SkipScan with varchar column,This patch changes the operator lookup in SkipScan to fall back to the type of the operator class when it finds no direct match for column type but the type is binary compatible with the type of the operator class. This is required for SkipScan to work with character data column that are not of type text since the operator family will be text_ops which has no operators for varchar defined. Fixes #3720 4ff1bd3636fbb518dce8251f23a986892050a3c3,Sven Klemm,Fri Oct 22 15:03:26 2021 +0200,Ignore chunk_utils test results on PG 12.0,Postgres 12.0 has a use-after-free bug in the event trigger code leading to a failing chunk_utils test in our nightly CI runs. This patch ignores this test for runs on PG 12.0. https://github.com/postgres/postgres/commit/93f726c0 68697859df96ee7a8c69b5d75851d5537fc81a33,Nikhil Sontakke,Mon Oct 18 18:58:34 2021 +0530,Fix GRANT/REVOKE ALL IN SCHEMA handling,"Fix the ""GRANT/REVOKE ALL IN SCHEMA"" handling uniformly across single-node and multi-node. Even thought this is a SCHEMA specific activity, we decided to include the chunks even if they are part of another SCHEMA. So they will also end up getting/resetting the same privileges. Includes test case changes for both single-node and multi-node use cases. " b0886c1b6dacac381de1bd2ee0bac6a3db34d9ec,gayyappan,Wed Oct 20 14:30:17 2021 -0400,Support cagg invalidation trigger for inserts into compressed chunks,After row triggers do not work when we insert into a compressed chunk. This causes a problem for caggs as invalidations are not recorded. Explicitly call the function to record invalidations when we insert into a compressed chunk (if the hypertable has caggs defined on it) Fixes #3410. 2e6b05dfc02ed30678f5c20f5dd091b2274a719b,gayyappan,Wed Oct 20 13:16:20 2021 -0400,Move test for error reporting to compression_errors.sql,compression_insert.sql has tests related to inserts The test that is being moved verifies the error message. acc6abee92d2f215f0de37d269f1613ad505dfed,Sven Klemm,Mon Oct 18 23:54:15 2021 +0200,Support transparent decompression on individual chunks,This patch adds support for transparent decompression in queries on individual chunks. This is required for distributed hypertables with compression when enable_per_data_node_queries is set to false. Without this functionality queries on distributed hypertables with compression would not return data for compressed chunks as the generated FDW queries would target individual chunks. Fixes #3714 8862081e1c02a30108f593a9f55751ba77a3d728,Erik Nordström,Mon Oct 18 13:17:48 2021 +0200,Merge version tests for distributed partial agg,"The `dist_partial_agg` test no longer produces different output for the PostgreSQL versions we support, so the version-specific output files can be merged into one. " 3ee718c6d079b39931fb8a9f88380c37b8fce87e,Fabrízio de Royes Mello,Mon Oct 18 12:26:43 2021 -0300,Fix contributors documentation,We're not using Travis CI anymore and moved all our CI infrastructure to Github Actions. Replaced all old Travis CI mentioning to Github Actions on contributors documentation. Closes #3700 01bf6518d4991ee98f8f29efd0f3a47e14d6da25,Sven Klemm,Mon Oct 18 21:15:59 2021 +0200,Ignore pg_dump test result in macos PR tests,Since pg_dump fails very often on macos we ignore the result of this test for PRs on macos. 7f43a15ae8a899e6015d74b1bbb041acbb601aaa,Sven Klemm,Sun Oct 17 07:15:42 2021 +0200,Fix ordered append pathkey check,Even though ordered append checked the pathkey for a match of the attribute number with the initial ordering it did not double check the attribute number was actually from the hypertable. Under certain circumstances (e.g. with views) it was possible that the check passed even though the order by was not on the primary partitioning column. This patch changes the code to get the equivalence member for the hypertable relation instead of iterating over all equivalence members. Fixes #3409 3c3290976cf71712736a53d52e86ba6df392ebe8,Sven Klemm,Sun Oct 17 14:17:16 2021 +0200,Use postgres implemention of find_em_expr_for_rel,find_em_expr_for_rel used to be a function in postgres_fdw which we imported but got moved to postgres main in PG13. This patch changes our code to use the postgres implementation when it is available and switches to our own implementation only for PG12. e321d5212c4012bd84a54811bc4219516971fad4,Sven Klemm,Mon Oct 18 12:46:58 2021 +0200,Add more PG14 tests to CI,"This patch adds PG14 coverity, release, apache and snapshot tests to CI. " b243f82f61e488a54761b90ccba9ff65e6b7ef4c,Sven Klemm,Sat Oct 16 21:23:53 2021 +0200,Fix wrong assert in cagg_update_view_definition,When the view query has references to columns not selected in the resultset those columns will be at the end of the view query targetlist as junk columns. So the length of the targetlist of those queries might be different but they should always have the same amount of nonjunk columns. fb3613fe04d6ec5c3216a077ac1d7078640290a6,Alexander Kuzmenkov,Mon Oct 18 17:59:26 2021 +0300,Improve evaluation of stable functions such as now() on access node,"We have to evaluate the stable functions on the access node and not on data nodes, where the result might differ, to get consistent results of the query. Before this change, we used to substitute 'now()' with textual value of the current timestamp in the query text before sending it to the data node. This is not sufficient if now() is further wrapped in a whitelisted (for sending to data nodes) stable function such as '::timestamp'. Use the standard expression evaluation machinery to also handle such cases. Fixes #3598 " 02f273dfe3bfa8a2aa868b46e8fc3163235b5c8f,Sven Klemm,Fri Oct 15 18:16:40 2021 +0200,Fix chunk_get_relstats Datum handling,The code in chunk_get_single_stats_tuple was using Datum getter wrongly on FormData class which already holds primitive datatypes and did not correctly convert to Datum when setting the value for the tuple. Since most of the Datum macros are noops on amd64 this only lead to failing tests on 32bit. eec04640087ece3c17d93d17fec7fba4645be7d5,Sven Klemm,Sat Oct 16 23:55:48 2021 +0200,Fix crash in get_aggsplit,When looking for the Aggref to determine whether partial or full aggregation is used get_aggsplit only checked for top-level Aggrefs in the targetlist. So a targetlist where all Aggrefs where nested in other expressions would lead to a crash. This function also only looked for the Aggref in the targetlist but in a query with a HAVING clause the aggregate might not be in the targetlist if it is only referenced in the HAVING clause. Fixes #3664 Fixes #3672 f25e795ec8450c15b08aa0c7593f81ca1bc5874e,Fabrízio de Royes Mello,Thu Oct 14 17:55:54 2021 -0300,Add regression tests for Memoize Node,PostgreSQL 14 introduced new `Memoize Node` that serve as a cache of results from parameterized nodes. We should make sure it will work correctly together with ChunckAppend custom node over hypertables (compressed and uncompressed). Closes #3684 57744c84ceedaa44178d6b27b1074e13524713e3,Erik Nordström,Thu Oct 14 15:07:11 2021 +0200,Adjust cost estimates for distributed queries,"To improve remote query push down, do the following: * Import changes to remote cost estimates from PostgreSQL 14 `postgres_fdw`. The cost estimations for distributed (remote) queries are originally based on the `postgres_fdw` code in PG11. However, fixes and changes have been applied in never PostgreSQL versions, which improves, among other things, costing of sorts and having clauses. * Increase the cost of transferring tuples. This penalizes doing grouping/aggregation on the AN since it requires transferring more tuples, leading to the planner preferring the push-down plans. * As a result of the above, the improved costing also makes distributed queries behave similar across all currently supported PostgreSQL versions for our test cases. * Enable `dist_query` tests on PG14 (since it now passes). * Update the `dist_partial_agg` test to use additional ordering columns so that there is no diff in the test output due to ordering of input to the `first` and `last` functions. " 28a56503821d7de3ebc6c3c0f6a2c16fe0a0c095,Erik Nordström,Fri Oct 15 13:20:16 2021 +0200,Allow anyone to use size utilities on distributed hypertables,"This change removes a check for `USAGE` privileges on data nodes required to query the data node using utility commands, such as `hypertable_size`. Normally, PostgreSQL doesn't require `USAGE` on a foreign server to query its remote tables. Also, size utilities, like `pg_table_size` can be used by anyone---even roles without any privileges on a table. The behavior on distributed hypertables is now consistent with PostgreSQL. Fixes #3698 " cb13754c918c2709380925bd7f06846b57b29dff,Aleksander Alekseev,Wed Oct 6 14:49:02 2021 +0300,Clarify the comment for invalidation threshold,Explicitly state that the invalidation threshold is ignored by the transactions that use an isolation level above READ COMMITTED. 3729c5001741596567fac31c3b82371964585168,Aleksander Alekseev,Thu Oct 14 15:30:33 2021 +0300,"Fix two ""unused function"" errors","When building on MacOS with Clang 11 the build fails with: error: unused function 'get_chunk_rri' error: unused function 'get_hyper_rri' [-Werror,-Wunused-function] This PR fixes it. " 2f7bb43fe0f3ac806e51e262af17dd4be2cc8a7b,Sven Klemm,Wed Oct 13 11:44:58 2021 +0200,No longer require -DEXPERIMENTAL for PG14,"Change cmake to no longer require -DEXPERIMENTAL when building against PG14. Run more PG14 tests (32bit, windows, sanitizer). " f686b2af40ba3219e6c4d616aee184d02d9484d6,Sven Klemm,Thu Oct 14 00:34:13 2021 +0200,Fix various windows compilation problems with PG14,The windows compiler has problems with the macros in genbki.h complaining about redefinition of a variable with a different storage class. Since those specific macros are processed by a perl script and not relevant for the build process we turn them into noops for windows. a82826405e48b313438ae4f04c2765b74b28ac6a,Sven Klemm,Wed Oct 13 12:09:57 2021 +0200,Minor create_compress_chunk_result_relation_info refactor,Refactor create_compress_chunk_result_relation_info to get rid of compiler warning about unused variable when compiling on PG14 without assertions. 35bd83d08312ff284a47bd5f55e92a605aa8f8a2,Dmitry Simonenko,Wed Oct 13 16:14:32 2021 +0300,Allow owner change of a data node,Allow previously blocked `ALTER SERVER OWNER TO` operation on a data node foreign server. Fix #3617 fffd6c2350f5b3237486f3d49d7167105e72a55b,gayyappan,Thu Oct 7 17:08:27 2021 -0400,Use plpgsql procedure for executing compression policy,"This PR removes the C code that executes the compression policy. Instead we use a PL/pgSQL procedure to execute the policy. PG13.4 and PG12.8 introduced some changes that require PortalContexts while executing transactions. The compression policy procedure compresses chunks in multiple transactions. We have seen some issues with snapshots and portal management in the policy code (due to the PG13.4 code changes). SPI API has transaction-portal management code. However, the compression policy code does not use SPI interfaces. But it is fairly easy to just convert this into a PL/pgSQL procedure (which calls SPI) rather than replicating portal managment code in C to manage multiple txns in the compression policy. This PR also disallows decompress_chunk, compress_chunk and recompress_chunk in txn read only mode. Fixes #3656 " c55cbb9350c803b5c5ab62439317c151f41d1103,gayyappan,Mon Oct 11 11:20:13 2021 -0400,Expose subtract_integer_from_now as SQL function,Move subtract_integer_from_now to src directory and create a SQL function for it. bfe2b08350d4d75a276f12972221d3d8bac8b303,Sven Klemm,Tue Oct 12 17:29:18 2021 +0200,Make dist_partial_agg output pg version specific,The plan output of the dist_partial_agg test is different on PG14 so we need to make it PG version specific. On PG14 sorts are pushed down more often leading to better plans in some cases. This also updates the dist_hypertable-14 test output which differs to previous PG versions due to some renumbering of relation aliases. de2d446cb2704f498bf4bbeaa4d7bc3ff259dde9,Fabrízio de Royes Mello,Mon Oct 11 16:16:31 2021 -0300,Add handling of new ALTER TABLE options in PG14,PG14 introduced new `ALTER TABLE` sub-commands: * `.. ALTER COLUMN .. SET COMPRESSION`: handled it properly on `process_utility` hook code and added related regression tests * `.. DETACH PARTITION .. {CONCURRENTLY | FINALIZE}`: handled it properly on `process_utility` hook code but there's no need to add regression tests because we don't rely to native partitioning in hypertables. Closes #3643 f9d8667b1d6716a1dd8898979e4f11d4eab29264,Dmitry Simonenko,Tue Oct 12 22:47:48 2021 +0300,Fix distributed hypertable DROP within a procedure,When DROP being executed inside procedure ddl_command_start was not handled which lead to ignoring this operation on the data nodes. Fix #3663 8d12b28a0033cbf4bc896188c24a2c17792d15d7,Sven Klemm,Mon Oct 11 15:10:31 2021 +0200,Fix UPSERT for PG14,Adjust code to upstream UPDATE/DELETE changes. PG14 refactored the way UPDATEs and DELETEs are planned and executed. https://github.com/postgres/postgres/commit/86dc90056d bfe3603d5778eab741b65bfc6310a41f7c5d550e,Sven Klemm,Mon Oct 11 19:46:18 2021 +0200,Move pg version specific part of create_hypertable test with custom partition type,Since custom types are hashable in PG14 the partition test will be different on PG14. Since the only difference was testing whether creating hypertable with custom type paritition throws errors without partitioning function that specific test got moved to ddl tests which already is pg version specific. d959828155a41d805183bcd112ed9a2a935312d5,Sven Klemm,Fri Oct 8 13:44:25 2021 +0200,Don't use VACUUM VERBOSE in vacuum tests,VACUUM VERBOSE is the source for flaky tests and we don't gain much by including the verbose output in the test. Additionally removing the verbose option prevents us from having to make the vacuum tests pg-version specific as PG14 slightly changes the formatting of the VACUUM VERBOSE output. 4d425d94702fff6bd7621d890bd05593d37af4ef,Sven Klemm,Fri Oct 8 15:07:18 2021 +0200,Disable memoize node for append and transparent_decompression tests,With memoize enabled PG14 append tests produce a very different plan compared to previous PG versions. To make comparing plans between PG versions easier we disable memoize for PG14. PG14 also modified how EXTRACT is shown in EXPLAIN output so any query using EXTRACT will have different EXPLAIN output between PG14 and previous versions. 9486247798cc8faceedc2cf1d8d87901b3a35353,Sven Klemm,Fri Oct 8 23:05:23 2021 +0200,Adjust bgw_db_scheduler test to work across PG versions,PG14 changes bgw log message to include the job type so we filter it out in the log message view to make the output consistent across PG versions. c873731e25dfa050ba8745945c7fdabfebf8d49e,Sven Klemm,Fri Oct 8 22:49:00 2021 +0200,Remove unused variables from ExecModifyTable,Remove two unused variables from ExecModifyTable that were accidentally left in when removing dead code. 90d1a390c724bc69d454d852dffb5852b988f651,Sven Klemm,Wed Oct 6 15:08:11 2021 +0200,Require successful PG14 tests for PRs,The previous PR enabling tests on PG14 did not actually require tests to pass but with INSERT support merged and most of the tests passing it makes sense to require tests to pass to not introduce regression and explicitly not require the currently known failing tests. 26c5f276836df67d21d707f7368c5d15e30f5a47,Sven Klemm,Mon Oct 4 14:39:55 2021 +0200,Adjust hypertable insert for PG14,PG14 refactors the INSERT path and removes the result relation from executor state which means plan nodes don't have easy access to the current result relation and can no longer modify it. This patch changes the chunk tuple routing for PG14 to pull in code from ModifyTable which unfortunately is static and adjust it to allow chunk tuple routing. 9e53cbb6d32b59a90c2d263310c75d5fd153d9c7,Sven Klemm,Wed Oct 6 19:05:12 2021 +0200,Fix SkipScan path generation with constant DISTINCT column,When a DISTINCT query has a WHERE clause that constifies the DISTINCT column the query might use an index that does not have include the DISTINCT column even though it is referenced in the ORDER BY clause. The skipscan path generation would error on any path with such a configuration. This patch changes the path generation code to skip generating SkipScan path under these circumstances. Fixes #3629 74ca546565076676b4b18af3d369465dea4147e3,Fabrízio de Royes Mello,Tue Oct 5 11:34:08 2021 -0300,Fix memory context bug executing TRUNCATE,"Inside the `process_truncate` function is created a new relations list removing the distributed hypertables and this new list is assigned to the current statement relation list. This new list is allocated into the `PortalContext` that is destroyed at the end of the statement execution. The problem arise on the subsequent `TRUNCATE` call because the compiled plpgsql code is cached into another memory context and the elements of the relations inside this cache is pointing to an invalid memory area because the `PortalContext` is destroyed. Fixed it by allocating the new relations list to the same memory context of the original list. Fixes #3580, fixes #3622, fixes #3182 " 0f64e11db439b0a6f9dd2b66909e2e96696d43fe,Sven Klemm,Wed Oct 6 12:42:27 2021 +0200,Fix CI telemetry test to use correct build args,The telemetry without openssl test did not use the correct args supplied in the build matrix leading to a failure on PG14 as building against PG14 currently requires -DEXPERIMENTAL=ON ecac52f4942570e9bd62066130da267e6e7186bf,Daniel Orf,Tue Oct 5 17:10:32 2021 -0700,Fix a typo in README.md, a118f384c15b902265bdd7927330af4984181f0a,Sven Klemm,Wed Oct 6 03:26:05 2021 +0200,Fix index attnum mapping in reorder_chunk,When we clone an index from a chunk we must not do attnum mapping if the supplied index template is not on the hypertable. Ideally we would check for the template being on the chunk but we cannot do that since when we rebuild a chunk the new chunk has a different id. Fixes #3651 ef0ff450061eb47fb7bfa22ee4433dcb5bee0b80,Sven Klemm,Tue Oct 5 13:34:25 2021 +0200,Add PG14 regresscheck test output files,"This patch adds PG14 specific test output files for regresscheck, regresscheck-t and regresscheck-shared. " 6b69c781ba7b5b26fb68143daac176ff875779a3,Sven Klemm,Tue Oct 5 12:59:30 2021 +0200,Add PG14 to our CI, b3380e8174c6b7b999fd24cea63876412f15af5b,Fabrízio de Royes Mello,Thu Sep 23 17:44:13 2021 -0300,Fix execution of refresh_caggs from user actions,Segmentation fault was ocurring when calling the procedure `refresh_continous_aggregate` from an user defined action (job). Fixed it by adding the `SPI_connect_ext/SPI_finish` during the execution because there are underlying SPI calls that was leading us to an invalid SPI state (nonexistent `_SPI_current` global). Fixes #3145 27d547853d9de16d313f58f37f7a6e89e6687da3,Avthar Sewrathan,Wed Sep 22 17:34:45 2021 -0400,README Update for Project Prince,"Updated README for TimescaleDB GitHub page for Project Prince, with various references and links updated." a60b2e5f6a6c26e5a172538443467432dfd9cf77,Dmitry Simonenko,Mon Oct 4 17:18:03 2021 +0300,Allow alter column type on distributed hypertable,Unlock and support ALTER TABLE ALTER TYPE on distributed hypertable. Fix #3582 55840eb48d685a5eb107eac5acb214d105c2bbbd,Dmitry Simonenko,Mon Oct 4 16:45:58 2021 +0300,Make ssl_dir/passfile gucs accessible for read,timescaledb.ssl_dir/passfile GUC is currently only visible for superusers. This makes it impossible for non superusers to read GUC value which might be useful in some cases (eg. troubleshooting or building multinode automation). Since this GUC's does not contain any sensitive information it makes sense to make it public. Fix #3608 7570f27638557d79c0711dbd0b463ca3c3a91cb3,Nikhil,Thu Sep 30 15:00:19 2021 +0530,Schema qualify UDTs in multi-node,"""create_distributed_hypertable"" fails on the datanodes if the columns involved in the underlying non-default schema-qualified PG table are using user defined types (UDTs) from another non-standard schema. This happens because we explicitly set the namespace during the table creation on the datanode which doesn't allow us to lookup other schemas. We now unconditionally schema-qualify the UDT while sending the SQL from access node to the datanode to avoid this. Includes test case changes. Issue reported by and general fix provided by @phemmer Fixes #3543 " f0ba729acbac00a5310f9cd5e70b883090994621,Sven Klemm,Fri Oct 1 23:05:10 2021 +0200,Adjust code to PG14 es_result_rel_info removal,PG14 removes es_result_relation_info from executor state. https://github.com/postgres/postgres/commit/a04daa97 325f68a62d371c2b188218c9572193fc4efe3ac3,Sven Klemm,Fri Oct 1 12:16:30 2021 +0200,Adjust Code to PG14 ModifyTablePath changes,PG14 changes ModifyTablePath struct to have single child subpath instead of list of subpaths. Similarly ModifyTableState mt_nplans gets removed because ModifyTable will only have single child in PG14. The same patch also removes ri_junkFilter from ResultRelInfo. https://github.com/postgres/postgres/commit/86dc9005 d01141823948b6992b4f264d64e2d6ced7ea3fc9,Sven Klemm,Fri Oct 1 12:24:34 2021 +0200,Adjust ts_set_append_rel_pathlist to PG14 changes,PG14 removes partitioned_child_rels from RelOptInfo https://github.com/postgres/postgres/commit/a929e17e 4c4eb635ce86c797ccba474b91a2b5d52a6c0650,Sven Klemm,Mon Oct 4 12:41:25 2021 +0200,Switch appveyor to preview image,Since there is an issue with the current appveyor image we switch appveyor to the preview image which runs CI fine. 98f4cd6581209cd7d4b252ffb692611b2fdd1efe,Sven Klemm,Sat Oct 2 12:41:04 2021 +0200,Fix flaky drop_owned test,Using now() in regression tests will result in flaky tests as this can result in creating different number of chunks depending on alignment of now() relative to chunk boundaries. 609b5ea34af93bde6bfe55f21ef47ebf6f53d2be,Fabrízio de Royes Mello,Fri Sep 17 11:19:01 2021 -0300,Refactor SQL function approximate_row_count,Simplify the CTE to recursively inspect all partitions of a relation and calculate the sum of `pg_class.reltuples` taking in account the differences introduced by PG14. 0fd7ad4c0fa66de099ede9ede1a0051bcb86e0e1,Mats Kindahl,Fri Oct 1 10:50:15 2021 +0200,Install OpenSSL in MacOS workflow,MacOS 10.15 image version 20210927.1 does not have OpenSSL installed so we have to explicitly install OpenSSL in the workflow for PostgreSQL to build. 9783144becd3c8a26cb39a88b3fd0c0b21c0c103,Mats Kindahl,Tue Sep 28 21:14:45 2021 +0200,Add shared dependencies when creating chunk,"When a new chunk is created, the ACL is copied from the hypertable, but the shared dependencies are not set at all. Since there is no shared dependency, a `DROP OWNED BY` will not find the chunk and revoke the privileges for the user from the chunk. When the user is later dropped, the ACL for the chunk will contain a non-existent user. This commit fixes this by adding shared dependencies of the hypertable to the chunk when the chunk is created. Fixes #3614 " 5b16ac097ea81644bf66185867daea56a70f7896,Mats Kindahl,Wed Sep 22 09:45:28 2021 +0200,Improve error message when chunk not found,"When a chunk is not found, we print a generic error message that does not hint at what we are looking for, which means that it is very hard to locate the problem. This commit adds details to the error message printing out values used for the scan key when searching for the chunk. Related-To: #2344 Related-To: #3400 Related-To: #153 " 46a584e6b58a8fc9bcb0925160f1538f1ea2ed6b,Mats Kindahl,Wed Sep 22 11:10:04 2021 +0200,Show details on compressed chunk error,"If insertion is attempted into a chunk that is compressed, the error message is very brief. This commit adds a hint that the chunk should be decompressed before inserting into it and also list the triggers on the chunk so that it is easy to debug. " 6838fcf90647a3cedff0aa3493607c6d59789f32,Sven Klemm,Mon Sep 20 22:21:12 2021 +0200,Post 2.4.2 release,Update version numbers and add 2.4.2 to update tests. We have to put the DROP FUNCTION back in latest-dev because 2.4.2 did not include the commit which removed the function definitions. 0098829cf52751d214e8599cb7afbb83a6831bb8,Erik Nordström,Mon Sep 20 16:59:40 2021 +0200,Update connection cache on role changes,"The access node maintains a cache of connections to its data nodes. Each entry in the cache is a connection for a user and remote data node pair. Currently, a cache entry is invalidated when a foreign server object representing a data node is changed (e.g., the port could have been updated). The connection will remain in the cache for the duration of the current command, but will be remade with the updated parameters the next time it is fetched from the cache. This change invalidates a connection cache entry if the connection's role/user changes and drops an entry if the role is dropped. One reason for invalidating a connection is that a role rename invalidates the certificate the connection is using in case client certificate authentication is used. Thus, connections that have been authenticated with a certificate that is no longer valid will be remade. In some cases, this extra invalidation leads to purging connections when not strictly necessary. However, this is not a big problem in practice since role objects don't change often. " b449803878932875ec789f21e634e57321e30c38,Mats Kindahl,Wed Sep 15 14:10:24 2021 -0400,Make continuous aggregate errors more descriptive,"Users often have trouble creating continuous aggregates because many error messages are the same and quite terse offering no guidance for the user in how to fix. This makes some of them less terse. Co-authored-by: David Kohn Co-authored-by: Mats Kindahl "" " 3f944bee826c9fc0fab84d8e8d5888e43e031250,Sven Klemm,Mon Sep 20 17:02:05 2021 +0200,Release 2.4.2,This release contains bug fixes since the 2.4.1 release. We deem it high priority to upgrade. **Bugfixes** * #3437 Rename on all continuous aggregate objects * #3469 Use signal-safe functions in signal handler * #3520 Modify compression job processing logic * #3527 Fix time_bucket_ng behaviour with origin argument * #3532 Fix bootstrap with regresschecks disabled * #3574 Fix failure on job execution by background worker * #3590 Call cleanup functions on backend exit **Thanks** * @jankatins for reporting a crash with background workers * @LutzWeischerFujitsu for reporting an issue with bootstrap 592e0bd46e5fa1ae9df85f8e5f1857aadabd72fb,Mats Kindahl,Tue Jul 20 17:47:59 2021 +0200,Rename on all continuous aggregate objects,"When renaming a column on a continuous aggregate, only the user view column was renamed. This commit changes this by applying the rename on the materialized table, the user view, the direct view, and the partial view, as well as the column name in the dimension table. Since this also changes some of the table definitions, we need to perform the same rename in the update scripts as well, which is done by comparing the direct view and the user view to decide what columns that require a rename and then executing that rename on the direct view, partial view, and materialization table, as well as updating the column name in the dimension table. When renaming columns in tables with indexes, the column in the table is renamed but not the column in the index, which keeps the same name. When restoring from a dump, however, the column name of the table is used, which create a diff in the update test. For that reason, we change the update tests to not list index definitions as part of the comparison. The existance of the indexes is still tracked and compared since the indexes for a hypertable is listed as part of the output. If a downgrade does not revert some changes, this will cause a diff in the downgrade test. Since the rename is benign and not easy to revert, this will cause test failure. Instead, we add a file to do extra actions during a clean-rerun to prevent these diffs. In this case, applying the same renames as the update script. Fixes #3405 " e22aaad48c1fcdd2d1ca970593503652b30032a4,Fabrízio de Royes Mello,Tue Sep 14 16:58:02 2021 -0300,Fix failure on job execution by background worker,"If the registered procedure has COMMIT inside the code, the execution of the job by the background worker was failing. Fixed it checking if there are an ActivePortal, if NO then create a Portal from scratch and execute the existing job execution code. " 9ee9a1e1c0151721ce8c6d4c6b66cb038284fab8,Erik Nordström,Thu Sep 16 11:49:22 2021 +0200,Call cleanup functions on backend exit,"A number of cleanup functions for various data structures (e.g., caches) are designed to be called when the extension library is unloaded (via `_PG_fini`). But unloading is disabled in PostgreSQL and the library is also not unloaded when the backend exits. Therefore, cleanup functions are never called in practice. In most cases this is not a problem since memory is released in any case. However, in the case of the connection cache, connections need to be closed before memory is released or otherwise the DNs will emit ""Connection reset by peer"" log entries when the AN backend exits. Add proper cleanup of caches and other data structures by adding a callback via `on_proc_exit`, which gets called when the backend exits. The closing of AN to DN backend connections are also logged on the AN if `log_connections=true`. " 43bb5ba7d124dcdf04b02b75139e20cccca21c7e,Sven Klemm,Sat Sep 11 20:46:32 2021 +0200,Optimize approximate_row_count,Rewrite approximate_row_count to SQL instead of PLpgSQL and remove superfluous JOINs against pg_namespace. Adjust tuple calculation for PG14 since in PG14 reltuples for partitioned tables is the sum of it's children so we need to exclude those from calculation to not doublecount. cfbc9eae641dc059c2f4fa1641711a58001ac2e8,Sven Klemm,Thu Sep 16 12:39:24 2021 +0200,Fix find_ec_member_matching_expr backport for 13.2,find_ec_member_matching_expr is not available in PG13.2 so this patch adjusts the macro that enables the function for our code. https://github.com/postgres/postgres/commit/7bfba4f1 5b1292062a80b37041059bedfe81733f73a8f6b8,Sven Klemm,Tue Sep 7 05:14:16 2021 +0200,Update prepare_sort_from_pathkeys import,Update prepare_sort_from_pathkeys postgres import according to PG13 and PG14 changes. This also backports find_ec_member_matching_expr and find_computable_ec_member to PG12 where those are missing. https://github.com/postgres/postgres/commit/37539824 bdeccba11ba62993c1ee3559e579bb334a1b1487,Sven Klemm,Mon Sep 13 06:43:44 2021 +0200,Refactor ts_setup_append_rel_array,Remove superfluous repalloc from ts_setup_append_rel_array and rename that function to ts_add_append_rel_infos. The reallocation is already done by expand_planner_arrays. This patch also changes this function to only add new AppendRelInfos to the array that were created by parent instead of iterating over all AppendRelInfos. 77c969071c609f0b3992a35eec03f6e244fb7c7a,gayyappan,Tue Aug 17 15:33:26 2021 -0400,Modify compression job processing logic,"Instead of picking 1 chunk for processing, we find the list of chunks that have to be compressed by the compression job, and proceed to process each one in its own transaction. Without this, we could end up in a situation where the first chunk is continually picked for recompression (due to active inserts into the chunk) and we don't make any progress. We can limit the number of chunks processed by a single run of the job by setting the new config parameter: max_chunks_to_compress, for the compression job. Valid values are > 0, The job processes only maxchunks_to_compress number of chunks and defers any remaining items to the next scheduled run of the job. The default is to process all pending chunks. We have an additional job config parameter: verbose_log. This enables additional logging that logs the chunks that are processed by the job. " 04005a0ce4c05b527d23dae42c4ff26a2a117c3b,Sven Klemm,Thu Sep 9 03:45:06 2021 +0200,Use LookupFuncWithArgs in bgw job lookup,PG14 changes LookupFuncName to no longer match procedures but only functions. The only place where we actually expected to match procedures is the bgw. This patch changes bgw job lookup to use LookupFuncWithArgs which can return both functions and procedures depending on the requested object type. https://github.com/postgres/postgres/commit/e56bce5d 238da463525d479a47b4b87987797d3aa17aadc6,Sven Klemm,Thu Sep 9 11:48:38 2021 +0200,Add Ubuntu 21.04 to apt package test, 03ee0f4ba6998dde8225e92908228f7ffb9ec522,Sven Klemm,Thu Sep 9 09:02:30 2021 +0200,Adjust code to PG14 FDW API changes,PG14 changes the signature of AddForeignUpdateTargets and the method it must use for adding junk columns to the query. https://github.com/postgres/postgres/commit/86dc9005 265e18627b4675ccd729b06742f210d3498a2219,Sven Klemm,Tue Sep 7 19:12:06 2021 +0200,Adjust code to PG14 reindex_relation changes,PG14 changes the reindex_relation `params` argument from integer to a struct. https://github.com/postgres/postgres/commit/a3dc9260 06092fbd092a26822caf21418faca5e7e2ab420a,Sven Klemm,Tue Sep 7 00:23:00 2021 +0200,Adjust FuncnameGetCandidates calls to PG14 changes,PG14 adds an include_out_arguments parameter to FuncnameGetCandidates. https://github.com/postgres/postgres/commit/e56bce5d 29f3b517610e95d2997929a341984729d06d0ccc,Mats Kindahl,Tue Sep 7 13:28:23 2021 +0200,Use signal-safe functions in signal handler,"Reporting errors using `ereport` can call `malloc()`, which is not signal-safe. Using `ereport()` in a signal handler can therefore cause `malloc()` to run nested inside `malloc()` if the termination handler is called in the middle of a `malloc()` call, which will trigger an assertion in `malloc()` that will take down the server. This commit fixes this by using the signal-safe `write_stderr()` inside the signal handlers for the background workers. Fixes #3469 " ab71c4a1c1743897dbac8e283860681a85261044,Aleksander Alekseev,Mon Sep 6 16:55:21 2021 +0300,Mark time_bucket_ng() as IMMUTABLE,"This patch marks time_bucket_ng() as IMMUTABLE. Two exceptions are: - time_bucket_ng(interval, timestamptz) timestamptz - time_bucket_ng(interval, timestamptz, timestamptz) timestamptz ... due to their implementation, see the comments. These two overloaded versions were introduced only for backward compatibility with time_bucket() and are not needed for building continuous aggregates. " 8aee344264962d097ff9bcac8e0e1148411abf5d,Sven Klemm,Mon Sep 6 00:31:22 2021 +0200,Adjust code to processutility hook API change,PG14 adds an additional readonlyTree argument to the process utility hook. https://github.com/postgres/postgres/commit/7c337b6b de4a601a95c15e2430316d379ca824ce48948809,Sven Klemm,Tue Sep 7 18:03:45 2021 +0200,Revert appveyor to previous windows image,A recent update of the appveyor windows image updated the installed openssl version to 1.1.1l which does not match the openssl version postgres is built with (1.1.1k) leading to a segfault in one of our tests. dc67eb75d6a8794c61f2fca3e38f55ab87c68655,Aleksander Alekseev,Wed Sep 1 13:21:26 2021 +0300,time_bucket_ng() version with `origin` argument,This patch adds a version of time_bucket_ng() with 'origin' argument. It doesn't address any other known issues. E.g. volatility of the function will be changed in another patch. The error messages are going to be improved when the feature gets a little more stable. dfc63fe0634e6987d361019c6e2e90a8c2d7eb94,gayyappan,Wed Sep 1 12:14:02 2021 -0400,Remove legacy functions,Remove _timescaledb_internal.time_col_name_for_chunk and _timescaledb_internal.time_col_type_for_chunk functions. Fixes #3539 73cc9c502ef4fdef25847704b30af90358749f1e,Sven Klemm,Fri Jun 4 15:14:44 2021 +0200,Adjust code to PG14 aggregation changes,https://github.com/postgres/postgres/commit/0a2bc5d61e 1a05529c408e61d30143c57f16beb4aceacfa420,Sven Klemm,Sat Jun 5 13:12:57 2021 +0200,Adjust code to PG14 post_parse_analyze_hook changes,PG14 adds an argument to post_parse_analyze_hook https://github.com/postgres/postgres/commit/5fd9dfa5f5 044441e02a3a95b5630584991f6ff656f8f74f59,Sven Klemm,Fri Jun 4 17:13:55 2021 +0200,Adjust code to CLUSTER and REINDEX refactoring,PG14 refactors CLUSTER and REINDEX option handling. https://github.com/postgres/postgres/commit/b5913f6120 https://github.com/postgres/postgres/commit/a3dc926009 dae4a904eac114e57ed79a71d1a7d405a060247e,Sven Klemm,Fri Jun 4 17:33:25 2021 +0200,Adjust code to PG14 raw_parser changes,PG14 adds a parse mode argument to raw_parser. https://github.com/postgres/postgres/commit/844fe9f159 cb843e6a8dd6befe5ac4cf94f854bf6f8069a60b,Sven Klemm,Wed Sep 1 12:34:55 2021 +0200,Adjust various compatibility macros for PG14,"PG14 adds an update flag to ExecInsertIndexTuples, adds ResultRelInfo argument to ExecComputeStoredGenerated and adds PlannerInfo to make_restrictinfo, make_simple_restrictinfo and pull_varnos. https://github.com/postgres/postgres/commit/9dc718b https://github.com/postgres/postgres/commit/a04daa9 https://github.com/postgres/postgres/commit/55dc86eca7 " 47d9eaca7466d7d66052d6144bc2d1c0074b588e,Sven Klemm,Fri Jun 4 12:05:15 2021 +0200,Adjust code to PG14 copy code changes,PG14 splits Copy code into separate code for COPY FROM and COPY TO since we were only interested in the COPY FROM parts we macro CopyFromState to CopyState for versions < 14 https://github.com/postgres/postgres/commit/c532d15ddd 071b391e0561080b01c9f8d0b2fa47c69a938ea2,Sven Klemm,Thu Aug 26 15:15:47 2021 +0200,Adjust code to PG14 append changes,PG14 removes partitioned_rels from create_append_path and create_merge_append_path https://github.com/postgres/postgres/commit/f003a7522b 22e77a77ad7ba7755fca37390ba11bd68814217a,Aleksander Alekseev,Wed Aug 18 15:54:57 2021 +0300,Support timezones in time_bucket_ng(),This patch adds support of timezones in time_bucket_ng(). The 'origin' argument can't be used with timezones yet. This will be implemented in a separate pull request. 76ad38636b846f4fa2ddc4a63b46be588afab2c0,Sven Klemm,Tue Aug 31 22:56:23 2021 +0200,Disable RPM packages tests for PRs,Running the RPM package test workflow was accidently left on for PR from testing when adding the downgrade test for RPM packages. 05c41aa276d47395da5cac0d655fb199e84203df,Sven Klemm,Tue Aug 31 22:21:55 2021 +0200,Adjust list_make5 macros for PG14,The list_make5 macros got removed in PG13 but PG14 adds them back. https://github.com/postgres/postgres/commit/b663a4136331de6c7364226e3dbf7c88bfee7145 2a9868b3a983036284e21a00126ac486b34c5851,Sven Klemm,Sat Jun 5 07:38:18 2021 +0200,Adjust estimate_num_groups calls to PG14 changes,PG14 adds an additional argument to estimate_num_groups to allow passing back additional information about the estimation. https://github.com/postgres/postgres/commit/ed934d4fa3 a8a9941d2178de16dd86514b463d9bad52a9300f,Sven Klemm,Tue Aug 24 14:32:35 2021 +0200,Add downgrade test to rpm package test,Test extension downgrade to previous version in RPM package test. c2e9a829a7e99f8892418d5b0bdfb27fa8e4ef24,Sven Klemm,Tue Aug 24 09:52:19 2021 +0200,Add downgrade test to APT tests,This patch adds a test to downgrade to the previous version to the ubuntu and debian package tests. 59a0ad4dbd3648666ef368edfe503d5df89aa8f0,Sven Klemm,Mon Aug 30 12:53:40 2021 +0200,Remove postgres_dev CI workflow,Running against specific commits in the master postgres branch is not very valuable as there is quite often some back and forth regarding API changes. Once PG14 support is implemented this should be part of our normal CI. 2ee30f9d8109b04818ab0abfbe023d141a99c88a,Sven Klemm,Wed Aug 25 18:02:51 2021 +0200,Run CI against PG snapshots,Run CI against PG snapshots to discover potential issues with upcoming PG releases earlier. ebe9cb42e328d05b5ad43631e4675743bd896c60,Sven Klemm,Fri Aug 27 12:00:17 2021 +0200,Fix bootstrap with regresschecks disabled,"When running bootstrap with regression checks disabled but tap tests enabled, bootstrap would fail when trying to add dependencies to the installcheck target which was not defined when regression checks are off. " 29ae7a7f77287657d2403a3967884ad323ab975c,Sven Klemm,Fri Aug 27 04:09:30 2021 +0200,Add debian bullseye to apt package tests, d0426ff23488abb8a1e7f684f7a9e8c86017411a,Sven Klemm,Mon Aug 16 17:21:55 2021 +0200,Move all compatibility related files into compat directory, f7eceba1c3c479cf85a730b9e58ef601730d5a55,Aleksander Alekseev,Thu Aug 26 15:09:32 2021 +0300,Fix time_bucket_ng() behavior with `origin` argument,"This PR fixes a silly bug in the time_bucket_ng() implementation: SELECT time_bucket_ng( '1 day', timestamp '2021-07-12 12:34:56', origin => timestamp '2021-06-01 00:00:00' ); Before the patch: 2000-02-11 00:00:00 After the patch: 2021-07-12 00:00:00 The tests cath this, but we didn't notice the wrong results :( " 6537e75b1fcd7a8c771c787b829360828e489f59,Sven Klemm,Sat Jun 5 10:58:06 2021 +0200,Adjust code to PG14 role renames,PG14 renames predefined roles https://github.com/postgres/postgres/commit/c9c41c7a33 60f7283fbb1d59072dd2eb2483732037b2d518c0,Sven Klemm,Tue Aug 24 20:32:24 2021 +0200,Add test for windows packages,Add a test that downloads and installs our windows installer to CI. b08284ba52029c8db0c10db7c39dcf2bcd5276d4,Joshua Lockerman,Mon Aug 16 12:46:14 2021 -0400,Add toolkit extension,The Analytics extension was renamed to TimescaleDB Toolkit. Add the new name to the list of related extensions so we can see when people migrate. 86cd8f6532a07ef80131563cfc089974d3a0dde9,Mats Kindahl,Tue Aug 24 09:46:29 2021 +0200,Post 2.4.1 release,Updates `version.config` and adds new version to update tests. e3b9b2bcbe57485bd076be552b94c739db410803,Mats Kindahl,Mon Aug 23 10:35:19 2021 +0200,Use SPI transaction manipulation functions,"The procedures `copy_chunk` and `move_chunk` internally manipulate the transaction state using plumbing-level commands such as `StartTransactionCommand` and `CommitTransactionCommand`. Since these affect the internal state of the SPI execution context, it generates warnings when used inside a PL/SQL procedures. This commit fixes this by switching to using SPI-level commands and connecting and finishing the SPI properly. Fixes #3495 " 4e0954e87f96bec59c558fb65f59ccf342b4ee38,Mats Kindahl,Thu Aug 19 16:17:52 2021 +0200,Release 2.4.1,"This release contains bug fixes since the 2.4.0 release. We deem it high priority to upgrade. The release fixes continous aggregate refresh for postgres 12.8 and 13.4, a crash with ALTER TABLE commands and a crash with continuous aggregates with HAVING clause. **Bugfixes** * #3430 Fix havingqual processing for continuous aggregates * #3468 Disable tests by default if tools are not found * #3462 Fix crash while tracking alter table commands * #3489 Fix continuous agg bgw job failure for PG 12.8 and 13.4 * #3494 Improve error message when adding data nodes **Thanks** * @brianbenns for reporting a segfault with continuous aggregates * @brianbenns for reporting a segfault with continuous aggregates * @usego for reporting an issue with continuous aggregate refresh on PG 13.4 " 04058e1ba09b67cb2a2c79166df115f46f6e45ef,Sven Klemm,Thu Aug 19 00:41:12 2021 +0200,Test Apache-only homebrew, 06b73795783f2ebce5e8a2d0f948b5160ddad034,Sven Klemm,Thu Aug 19 00:21:31 2021 +0200,Test Apache-only RPM packages, f4f90b22ad0a2e2865a95c7eb633e88d459775f1,Sven Klemm,Wed Aug 18 23:38:22 2021 +0200,Test APT Apache-only packages in CI, c4eb9b69c6b706bde01644a633e2309e6311e851,Mats Kindahl,Tue Aug 17 19:22:53 2021 +0200,Improve error message when adding data nodes,"When data nodes are added but are missing USAGE privileges, a hint will be shown suggesting to add more data nodes. This is misleading since data nodes are added. Instead, if there are data nodes but they are not used, the hint will be to fix the privileges for the data nodes. Fixes #3479 " 36a82d08519cb796a8e2ee5dd4dc9e9d0f65ce99,Sven Klemm,Mon Aug 16 23:18:55 2021 +0200,Fix compiler warning about missing braces,Older gcc versions will throw a warning about missing braces when a nested struct is initialized with {0}. c5ecbbc25bc7fa309ab4c162bfc89603086757c6,Sven Klemm,Mon Aug 16 23:41:13 2021 +0200,Bump postgres versions used in CI to latest,Since the isolationtester got modified to improve the query result output which got backported in to PG12 and PG13 we have to adjust all isolationtester output. https://github.com/postgres/postgres/commit/4a05406 217ba461ac0756336a1f291cd4e1c3dfea98b50f,gayyappan,Tue Jul 20 16:51:54 2021 -0400,Fix havingqual processing for caggs,"If the targetlist for the cagg query has both subexprs and exprs from the having clause, the havingqual for the partial view is generated incorrectly. Fix this issue by checking havingqual against all the entries in the targetlist instead of first match. Fixes #2655 " c221c7845afc228468f249010376c25d27474fdc,Mats Kindahl,Wed Aug 11 14:53:57 2021 +0200,Remove CMake debug printouts,Removing some debug printouts and moving some other status messages to more appropriate locations. a77f9724fb74136d8ea4b13db5a2896103c85119,gayyappan,Mon Aug 16 17:01:17 2021 -0400,Fix continuous agg bgw job failure for PG 12.8 and 13.4,Procedures that have multiple transcations have to manage their snapshots if they do not use SPI internally for SQL commands and are invoked via C code. See PG commit 84f5c2908dad81e8622b0406beea580e40bb03ac for details. Fixes #3486 bff87a8e2b506894adfbcb0b2235df99e141dee8,Mats Kindahl,Tue Aug 10 09:31:51 2021 +0200,Disable tests by default if tools are not found,"If tools like `pg_regress` or `pg_isolation_regress` are not found, an error is generated telling you that you need to disable the tests explicitly using `REGRESS_CHECKS`. This is an inconvenience for the user since in most cases you just want to build the system with what you have. This commit changes that by automatically disabling regression tests, isolation tests, TAP tests, and formatting targets when tools cannot be found, which means that users do not normally have to disable tests explicitly. A notice is still printed that the targets are not added. In addition, the commit adds an option `REQUIRE_ALL_TESTS` for the rare case when you want to make sure that all tests are indeed executed (for example, in CI runs), which by default is off. " e1c5a9848814aaf81d5df4cf3a80033a4cb0a2b8,Nikhil,Wed Aug 4 14:01:04 2021 +0530,Fix crash while tracking alter table commands,"In this specific case, when we create a hypertable then we add a ""not-null"" constraint to the ""time"" column if it does not exist. That is done via an internal ALTER TABLE subcommand in dimension_add_not_null_on_column function. If the currentEventTriggerState structure is enabled then it's necessary to set up the command tracking appropriately, otherwise crash ensues. This has been fixed via this now. Includes test changes. " d3cb880ed166d86110144d29d79e830048dae92b,Sven Klemm,Mon Aug 2 22:24:04 2021 +0200,Adjust ubuntu package source in package test,Ubuntu packages are now released on packagecloud similar to Debian and RPM. ed71382e8f63462c352d447d79379ac38df89dbc,Sven Klemm,Wed Jul 28 12:27:31 2021 +0200,Add test for homebrew,This adds a workflow to install the timescaledb homebrew formula and checks that it matches the expected version. be44b1d87fa71453b2a339d0906de4e4f5326fc0,Sven Klemm,Fri Jul 30 09:58:08 2021 +0200,Skip perl file installation without TAP_CHECKS,Don't install the TAP-related perl files when TAP checks has been disabled. Installing this files unconditionally creates problem when building packages as these files should not be part of the package. 9ea77fb97f463b64ab250cb2b0badf279580cd57,gayyappan,Fri Jul 30 09:15:42 2021 -0400,Post release 2.4.0,Add tests to update from 2.4.0 Fix up version.config Fix view creation stmt in views_experimental.sql that causes extension update failures. 63f2bdfc9e6f701132adbc163a6679ae04bd8ec7,gayyappan,Mon Jul 26 10:05:57 2021 -0400,Release 2.4.0,"This release adds new experimental features since the 2.3.1 release. The experimental features in this release are: * APIs for chunk manipulation across data nodes in a distributed hypertable setup. This includes the ability to add a data node and move chunks to the new data node for cluster rebalancing. * The `time_bucket_ng` function, a newer version of `time_bucket`. This function supports years, months, days, hours, minutes, and seconds. We’re committed to developing these experiments, giving the community a chance to provide early feedback and influence the direction of TimescaleDB’s development. We’ll travel faster with your input! Please create your feedback as a GitHub issue (using the experimental-schema label), describe what you found, and tell us the steps or share the code snip to recreate it. This release also includes several bug fixes. PostgreSQL 11 deprecation announcement Timescale is working hard on our next exciting features. To make that possible, we require functionality that is available in Postgres 12 and above. Postgres 11 is not supported with TimescaleDB 2.4. **Experimental Features** * #3293 Add timescaledb_experimental schema * #3302 Add block_new_chunks and allow_new_chunks API to experimental schema. Add chunk based refresh_continuous_aggregate. * #3211 Introduce experimental time_bucket_ng function * #3366 Allow use of experimental time_bucket_ng function in continuous aggregates * #3408 Support for seconds, minutes and hours in time_bucket_ng * #3446 Implement cleanup for chunk copy/move. **Bugfixes** * #3401 Fix segfault for RelOptInfo without fdw_private * #3411 Verify compressed chunk validity for compressed path * #3416 Fix targetlist names for continuous aggregate views * #3434 Remove extension check from relcache invalidation callback * #3440 Fix remote_tx_heal_data_node to work with only current database **Thanks** * @fvannee for reporting an issue with hypertable expansion in functions * @amalek215 for reporting an issue with cache invalidation during pg_class vacuum full * @hardikm10 for reporting an issue with inserting into compressed chunks * @dberardo-com and @iancmcc for reporting an issue with extension updates after renaming columns of continuous aggregates. " c9adab63b37a14bf7cce44e61a5236b30d451c91,gayyappan,Thu Jul 29 13:01:56 2021 -0400,Revert cagg changes from reverse-dev.sql,Cagg changes that were removed from reverse-dev.sql by PR 3443 were inadvertently added back in by PR 3446. 2ffa1bf4362559abed42f3ac52938733f00554d6,Nikhil,Tue Jul 6 14:28:23 2021 +0530,Implement cleanup for chunk copy/move,"A chunk copy/move operation is carried out in stages and it can fail in any of them. We track the last completed stage in the ""chunk_copy_operation"" catalog table. In case of failure, a ""chunk_copy_cleanup"" function can be invoked to bring the chunk back to its original state on the source datanode and all transient objects like replication slot, publication, subscription, empty chunk, metadata updates, etc are cleaned up. Includes test case changes for each and every stage induced failure. To avoid confusion between chunk copy activity and chunk copy operation this patch also consistently uses ""operation"" everywhere now instead of ""activity"" " f071f89ade4109512e11000f9ae245573ddb9724,Erik Nordström,Tue Jul 6 10:48:36 2021 +0200,Fix issues in the dist_chunk test,"This change fixes issues in the shared `dist_chunk` test that caused flakiness. Since the test shares a database and state with other tests running in parallel, it should modify the database (e.g., creating new tables and chunks) while the test runs. Such modifications will cause non-deterministic behavior that varies depending on the order the tests are run in. To fix this issue, all the table creations have been moved into the shared setup script and the test itself has been made less dependent on hard-coded IDs and chunk names. One of the tables used in the has been changed to use space-partitioning to make chunk placement on nodes more predictible. " 352dc9baec5155a410a8f8b73ec4e75666620110,Erik Nordström,Tue Jul 6 10:30:40 2021 +0200,Remove copy_chunk_data from downgrade script,The internal function `copy_chunk_data` was removed as part of refactoring and is no longer necessary to remove in the downgrade script since the function was never part of a release. 691c3dc952fefd389a04b9d3f482cde4914babe8,Dmitry Simonenko,Thu Jul 8 11:05:52 2021 +0300,Add basic copy/move chunk tests,Add tests to ensure API constraints and checks are working as expected as long as the basic functionality. bea2613455bea01aea9530c74c605800ed99d624,Erik Nordström,Tue Jul 6 09:26:53 2021 +0200,Add constraints when copying chunks across data nodes,"Chunk constraints are now added after a chunk has been copied from one data node to another. The constraints are added when the chunk is made visible on the destination node, i.e., after data has been copied and the chunk's metadata is created. As an alternative, the constraints could be added when the chunk table is first created, but before the metadata for the chunk is added. This would have the benefit of validating each copied (inserted) row against the constraints during the data copy phase. However, this would also necessitate decoupling the step of creating the constraint metadata from the creation of the actual constraints since the other chunk metadata that is referenced does not yet exist. Such decoupling would require validating that the metadata actually matches the constraints of the table when the metadata is later created. One downside of adding the constraints after data copying is that it necessitates validating all the chunk's rows against the constraints after insertion as opposed to during insertion. If this turns out to be a performance issue, validation could be initially deferred. This is left as a future optimization. " b4710501dd12e7ff0189adf257337645ef8a6021,Erik Nordström,Thu Jun 24 10:56:22 2021 +0200,Add experimental chunk replication view,A new view in the experimental schema shows information related to chunk replication. The view can be used to learn the replication status of a chunk while also providing a way to easily find nodes to move or copy chunks between in order to ensure a fully replicated multi-node cluster. Tests have been added to illustrate the potential usage. 478404def52f4f1cd8fdd1487dbc8dd421cb58f1,Nikhil,Tue Jun 29 11:54:14 2021 +0530,Make move_chunk use AN txns on DN,We used to run transactions in autocommit mode on DN while running the chunk copy/move activity. This meant that any failures on the access node were de-coupled from the activity on the DN. This can make future cleanup messy since we wouldn't know what's failed/succeeded on the data nodes. We now drive the entire activity via transactions started on the access node. 38c178174869a96ac6240b5d5db0edcc6d98ea9b,Dmitry Simonenko,Fri Jun 18 15:58:24 2021 +0300,Copy/move chunk refactoring,"Remove copy_chunk_data() function and code needed to support it, such as the 'transactional' argument. Rework copy chunk logic using separate stages. Introduce copy_chunk() API function as an internal wrapper for the move_chunk(). " f6b025055707c6e4f4a67ee607b9c5d4b6f8e7c5,Nikhil,Thu May 27 13:05:16 2021 +0530,Implement wrapper API for copy/move chunk,"The building blocks required for implementing end-to-end copy/move chunk functionality have now been wrapped in a procedure. A procedure is required because multiple transactions are needed to carry out the activity across the access node and the involved two data nodes. The following steps are encapsulated in this procedure 1) Create an empty chunk table on the destination data node 2) Copy the data from the src data node chunk to this newly created destination node chunk. This is done via inbuilt PostgreSQL logical replication functionality 3) Attach this chunk to the hypertable on the dst data node 4) Remove this chunk from the src data node to complete the move if requested A new catalog table ""chunk_copy_activity"" has been added to track the progress of the above stages. A unique id gets assigned to each activity and it is updated with the completed stages as things progress. " 2c66c1fd6481898c0506a17e49a8b71f53e6e30a,Dmitry Simonenko,Thu May 27 17:56:34 2021 +0300,Introduce function to copy chunk data between data nodes,Add internal copy_chunk_data() function which implements a way to copy chunk data between data nodes using logical replication. This patch prepared together with @nikkhils. b8ff780c50ef033ef853276942dfa8cef99344ab,Erik Nordström,Thu Apr 29 21:03:51 2021 +0200,Add ability to create chunk from existing table,"The `create_chunk` API has been extended to allow creating a chunk from an existing relational table. The table is turned into a chunk by attaching it to the root hypertable via inheritance. The purpose of this functionality is to allow copying a chunk to another node. First, the chunk table and data is copied. After that, the `create_chunk` can be executed to make the new table part of the hypertable. Currently, the relational table used to create the chunk has to match the hypertable in terms of constraints, triggers, etc. PostgreSQL itself enforces the existence of same-named CHECK constraints, but no enforcement currently exists for other objects, including triggers UNIQUE, PRIMARY KEY, or FOREIGN KEY constraints. Such enforcement can be implemented in the future, if deemed necessary. Another option is to automatically add all the required objects (triggers, constraints) based on the hypertable equivalents. However, that might also lead to duplicate objects in case some of them exist on the table prior to creating the chunk. " 3651e6e102a349f5ad915a5723c620023637b5ee,Nikhil,Tue May 4 14:03:06 2021 +0530,Move related tests into dist_chunk,"The ""chunk_drop_replica"" test is part of the overall chunk copy/move functionality. All related tests will go into this dist_chunk test. Also, fix the earlier flakiness in dist_chunk test by moving the compress_chunk calls into the shared setup " 762053431e35ea12f023d9d9d173f85cfbd77825,Nikhil,Mon Apr 26 11:01:36 2021 +0530,Implement drop_chunk_replica API,"This function drops a chunk on a specified data node. It then removes the metadata about the datanode, chunk association on the access node. This function is meant for internal use as part of the ""move chunk"" functionality. If only one chunk replica remains then this function refuses to drop it to avoid data loss. " 404f1cdbad86feb591c0c91aa2dc8782fb4c991e,Ruslan Fomkin,Mon Apr 26 14:15:49 2021 +0200,Create chunk table from access node,Creates a table for chunk replica on the given data node. The table gets the same schema and name as the chunk. The created chunk replica table is not added into metadata on the access node or data node. The primary goal is to use it during copy/move chunk. fac03f8882c49273e9f12e0b1c227aa7c73070c9,Ruslan Fomkin,Thu Apr 29 17:28:28 2021 +0200,Drop inherit from chunk table to hypertable,"When an empty chunk table is created, it is not associated with its hypertable in metadata, however it was inheriting from the hypertable. This commit removes inheritance, so the chunk table is completely standalone and cannot affect queries on the hypertable. In future, this fix of dropping inherit can be replaced with the chunk table creation, which does not create inheritance in the first place. Since it is larger amount of work, it was considered now. " 34e99a1c23a406c07ddf6971fab8c49d416ecab2,Ruslan Fomkin,Thu Apr 22 10:11:14 2021 +0200,Return error for NULL input to create_chunk_table,Gives errors if any argument of create_chunk_table is NULL instead of being STRICT. Utilizes newly added macros for this. 28ccecbe7cdd204aa16da1625c81c76bc52cde29,Ruslan Fomkin,Wed Apr 7 08:54:21 2021 +0200,Create an empty chunk table,"Adds an internal API function to create an empty chunk table according the given hypertable for the given chunk table name and dimension slices. This functions creates a chunk table inheriting from the hypertable, so it guarantees the same schema. No TimescaleDB's metadata is updated. To be able to create the chunk table in a tablespace attached to the hyeprtable, this commit allows calculating the tablespace id without the dimension slice to exist in the catalog. If there is already a chunk, which collides on dimension slices, the function fails to create the chunk table. The function will be used internally in multi-node to be able to replicate a chunk from one data node to another. " ded8e82ebb6a05a7fea734186e4c08ac3f166179,Sven Klemm,Tue Jul 27 14:17:22 2021 +0200,Fix downgrade files for 2.3.1,The cagg rebuild was required for a fix introduced in 2.3.1 but was not removed from reverse-dev.sql when that version got released. bad2fc14e284657bf32f783e608263c06a4b74c7,Dmitry Simonenko,Sun Jul 25 17:50:49 2021 +0300,Fix remote_txn_heal_data_node() for multiple DB's,"Since pg_prepared_xacts is shared between databases, the healing function tried to resolve prepared transactions created by other distributed databases. This change makes the healing function to work only with current database. Fix #3433 " 3a8f396bc7d014a2722b8113236b34b0eda73eec,Markos Fountoulakis,Sat Jul 24 13:20:02 2021 +0300,Remove check from relcache invalidation callback,"The timescaledb extension checking code uses syscache lookups to determine whether the proxy table exists. Doing syscache lookups inside the invalidation callback context was corrupting the syscache. For this reason, we remove this callback. " 0277ed74617b915a3e741a966b05998aa9a4d789,gayyappan,Tue Jul 13 11:43:19 2021 -0400,Verify compressed chunk validity for insert path,"When a insert into a compressed chunk is blocked by a concurrent recompress_chunk, the latter process could move the storage for the compressed chunk. Verify validity of the compressed chunk before proceeding to acquire locks. Fixes #3400 " 637fe3275bb4d710607ed7d6991983e37aa7af81,Mats Kindahl,Fri Jul 16 13:57:31 2021 +0200,Fix targetlist for cagg views,If the names for entries in the targetlist for the direct and partial views of the continuous aggregate does not match the attribute names in the tuple descriptor for the result tuple of the user view an error will be generated. This commit fixes this by setting the targetlist resource names of the columns in the direct and partial view to the corresponding attribute name of the user view relation tuple descriptor. Fixes #3051 Fixes #3405 00e4a910822ff754e9b65383a97886c3858fc5f7,Aleksander Alekseev,Tue Jul 20 12:53:37 2021 +0300,Improve test coverage of time_bucket_ng(),Also make the code more explicit about handling 'infinity' values. 768ff54ae2e5edf366aff4e6533f30a0d56f18f8,Nikhil,Thu Jul 15 19:22:17 2021 +0530,Add perl module files to PG install,"We have added additional functionality in timescaledb extension to use in tap tests. Install these perl files in the PG installation directory so that external modules (the current ""forge_ext"" extension as an example) can use this functionality without having to have an additional dependency on the timescaledb extension source code. Note that these perl files should be installed only if PG installation has the relevant ""${PG_PKGLIBDIR}/pgxs/src/test/perl"" subdirectory. Also rejig the configuration directory variable that's used for the tap tests so that external modules can specify their own locations by using their own values for it (the current variable was tightly tied to timescaledb source code location). " 99f7a2122ff6f8d08cf5999929fb03243b7f8d2d,Aleksander Alekseev,Tue Jul 6 15:57:36 2021 +0300,"Support seconds, minutes, and hours in time_bucket_ng()","As a future replacement for time_bucket(), time_bucket_ng() should support seconds, minutes, and hours. This patch adds this support. The implementation is the same as for time_bucket(). Timezones are not yet supported. " 78a21f412b65aa36979a395ac74679debae13f09,Sven Klemm,Sat Feb 27 12:25:00 2021 +0100,Adjust copy code to PG14 changes,"PG14 changes initialization of ResultRelInfo, removes es_num_result_relations and changes es_result_relations into an array index by rangetable index. https://github.com/postgres/postgres/commit/1375422c78 " d607e3d714029bb63877fa3b02a9a75ad4999062,Sven Klemm,Tue Jun 1 21:04:51 2021 +0200,Adjust ReindexTable calls to PG14 changes,PG14 removes the concurrent variable from ReindexStmt and turns it into an option instead. PG14 adds an isTopLevel argument to ReindexTable. https://github.com/postgres/postgres/commit/844c05abc3 https://github.com/postgres/postgres/commit/a6642b3ae0 c49a0b3e8ec43994773d65b46a132aed1d880478,Sven Klemm,Tue Jun 1 20:58:57 2021 +0200,Adjust cluster_rel and vacuum_set_xid_limits calls to PG14 changes,PG14 adds an isTopLevel argument to cluster_rel and vacuum_set_xid_limits. https://github.com/postgres/postgres/commit/a7212be8b9 c270891a4ecd1cb3a108f2709ed8a827b8a93109,Sven Klemm,Thu Jul 8 10:06:56 2021 +0200,Fix segfault for RelOptInfo without fdw_private,In nested function invocations hypertable expansion would not work correctly and a hypertable would not be expanded by timescaledb code but by postgres table inheritance leading to fdw_private not being properly initialized. Fixes #3391 2f69e4b20777ea6037f5e5769a44e5e35c016084,Aleksander Alekseev,Mon Jul 12 11:59:38 2021 +0300,Ignore debug_notice test during sanitizer runs,"This test sometimes fails because it generates extra ""rehashing catalog cache"" debug messages, which is not very helpful. " b5ffc7107171d8284c2976ffd589871321c34ded,Mats Kindahl,Tue Jul 6 09:44:08 2021 +0200,Post-release steps for release 2.3.1,Add 2.3.1 to the update tests and update the downgrade target for the downgrade version. This commit also fixes two issues that were fixed in the release branch: 1. Drop `_timescaledb_internal.refresh_continuous_aggregate` 2. Update the changelog to match the release branch. 0e505dd553f76fca1909d37d70e0c25c625c061f,Mats Kindahl,Mon Jul 5 08:39:26 2021 +0200,Fix version modifier,The variable `VERSION_MOD` is used by `config.h`. 994640858b52cc04c7970ea7ca766bc18cfa1010,Dmitry Simonenko,Mon Jul 5 10:09:22 2021 +0300,Fix fall-through warning in debug_point_enable(), 072b9682c7c303ecc7555b59c4738e47508f9d69,Mats Kindahl,Fri Jul 2 11:17:03 2021 +0200,Fix regex that matches version,The regex expected a trailing version like `-rc1` or `-dev` but only the non-dash part was optional. This correct the regex to ensure that the entire trailing part is optional. 40d2bf17b647beb8297beb8980d5f2f67bfb05c0,Dmitry Simonenko,Fri Jul 2 16:27:33 2021 +0300,Add support for error injections,Rework debug waitpoint functionality to produce an error in case if the waitpoint is enabled. This update introduce a controlled way to simulate errors scenarios during testing. a1a789be8e8127a9b2c4ab1ff1bb51b02f6669b3,Aleksander Alekseev,Fri Jun 25 16:10:10 2021 +0300,Allow to use experimental functions in caggs,"Current implementation of caggs can't find a bucketing function if it's declared in the experimental schema. This patch fixes it. Also the patch adds `debug_notice` test to IGNORE list on AppVeyor. The corresponding test generates an extra ""DEBUG: rehashing catalog cache"" message which is not critical. It seems to be stable on Linux. " 06433f62280796d79b2f3731916913bba80d70b1,Mats Kindahl,Thu Jul 1 08:51:52 2021 +0200,Release 2.3.1,"**Bugfixes** * #3279 Add some more randomness to chunk assignment * #3288 Fix failed update with parallel workers * #3300 Improve trigger handling on distributed hypertables * #3304 Remove paths that reference parent relids for compressed chunks * #3305 Fix pull_varnos miscomputation of relids set * #3310 Generate downgrade script * #3314 Fix heap buffer overflow in hypertable expansion * #3317 Fix heap buffer overflow in remote connection cache. * #3327 Make aggregate in caggs fully qualified * #3327 Make aggregates in caggs fully qualified * #3336 Fix pg_init_privs objsubid handling * #3345 Fix SkipScan distinct column identification * #3355 Fix heap buffer overflow when renaming compressed hypertable columns. * #3367 Improve DecompressChunk qual pushdown * #3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan " a58ebdb3b41073ea8166f5ae66f18ed0a7e54a23,Mats Kindahl,Thu Jul 1 12:53:30 2021 +0200,Split update and downgrade version,"During an update, it is not possible to run the downgrade scripts until the release has been tagged, but the update scripts can be run. This means that we need to split the previous version into two different fields: one for running the update tests and one for running the downgrade tests. " cb7fffe0ce107fbcfcd466f7aa8ccd3177314198,Mats Kindahl,Wed Jun 30 13:10:01 2021 +0200,Fix bad use of repalloc,"In contrast to `realloc`, the PostgreSQL function `repalloc` does not accept a NULL pointer and fall back on `palloc`. For that reason it is necessary to check if the pointer is NULL and either use `palloc` or `repalloc`. " bfd92ab822c17e4e0f19cb0a6a72a03ac775424a,davidkohn88,Tue Jun 8 18:05:17 2021 -0400,Use CREATE OR REPLACE AGGREGATE,"From PG12 on, CREATE OR REPLACE is supported for aggregates, therefore, since we have dropped support for PG11, we can avoid going through the rigamarole of having our aggregates in a separate file from the functions we define to support them. Nor do we need to handle aggregates separately from other functions as their creation is now idempotent. " ff5d7e42bb3e7b07e904d4d9301c093b7b296a8a,Sven Klemm,Mon Jun 28 11:41:03 2021 +0200,Adjust code to PG14 reltuples changes,PG14 changes the initial value of pg_class.reltuples to -1 to allow differentiating between an empty relation and a relation where ANALYZE has not yet run. https://github.com/postgres/postgres/commit/3d351d916b e28b9d0d601a1cd0ec9d2268d2ec3584ee9ca033,Mats Kindahl,Mon Jun 28 10:14:33 2021 +0200,Export variables to docker-build,This commit export the `GENERATE_DOWNGRADE_SCRIPT` and `PG_VERSION` variables from `test_downgrade_to_tag.sh` to `docker-build.sh`. a3e8fd0497d7f54e476d02c0969f6de256784d44,Mats Kindahl,Mon Jun 28 10:58:02 2021 +0200,Always define 'format' target,"If `clang-format` is not found `CLANG_FORMAT` will be set to a false value, which will prevent the `format` target from being defined if `cmake-format` is not installed. This commit fixes that by always creating a `format` target since `clang-format` is always created even if `CLANG_FORMAT` is false. We do not redefine `CLANG_FORMAT` to a non-false value since it is expected to contain a path to an executable and if set to a true value that is not a path it could be used in a way that leads to strange errors. " 45a7abc4de77dc189792ac1976d2466b30cd8104,Sven Klemm,Fri Jun 25 16:42:59 2021 +0200,Improve DecompressChunk qual pushdown,Allow pushdown of RelabelType expressions into the scan below the DecompressChunk node. When using varchar columns as segmentby columns constraints on those columns would not be pushed down because postgres would inject RelabelType nodes that where not accepted as valid expression for pushdown leading to bad performance because the filter could only be applied after decompression. Fixes #3351 e57155fdc0d4c9a1a5bab89f199c1916bcdfb37f,Aleksander Alekseev,Mon Jun 21 15:06:34 2021 +0300,time_bucket_ng() may be IMMUTABLE depending on arguments.,"This function is IMMUTABLE when it doesn't accept timestamptz arguments, and STABLE otherwise. See the comments in sql/time_bucket_ng.sql for more details. " 20db06891a85aa088570a6286d465e7f57e25879,Markos Fountoulakis,Fri Jun 25 10:48:28 2021 +0300,Fix ABI breakage test in CI,Add the files that had been missing from the docker environment since the introduction of the downgrade scripts. 4aba9a348be76fa75a5861b0e1b027a26a76826d,Sven Klemm,Sat Feb 27 09:50:23 2021 +0100,Adjust test_utils.h includes to postgres changes,PG14 changes xact.h to no longer include fmgr.h which is needed for PG_FUNCTION_ARGS definition. This patch includes fmgr.h explicitly to no longer rely on the indirect include. https://github.com/postgres/postgres/commit/3941eb6341 f23233055dadf7e1aa8e9e7f3bb9ec957700256f,Sven Klemm,Sat Feb 27 06:00:19 2021 +0100,Adjust process_utility to PG14 field renames,PG14 renames field relkind for CreateTableAsStmt and AlterTableStmt to objtype. https://github.com/postgres/postgres/commit/cc35d8933a 51531e36d0f9afdaf8aeb9e3478c973fd484c626,Sven Klemm,Fri Jun 18 11:48:38 2021 +0200,Allow building against PG14 and add it to CI,This patch adds a new EXPERIMENTAL flag to cmake allowing skipping the check for a compatible postgres version. It also adds macros needed for PG14 support. 4a599baf3407489ea9a0df1abe4393403ac31074,Markos Fountoulakis,Fri Jun 18 22:37:11 2021 +0300,Fix memory overflow,Fix heap buffer overflow in hypertable expansion. 15b46818ea77f0dc92a588d4cf285df1b43130ce,Mats Kindahl,Tue Jun 8 13:32:41 2021 +0200,Generate downgrade script,"This commit add functions and code to generate a downgrade script from the current version to the previous version. This requires execution from a Git repository since it retrieves the prolog and epilog for the ""downgrade"" file from the version given by `update_from_version` in the `version.config` file. The commit adds several CMake functions that simplifies the composition of script files, but these are not used to generate the update scripts. A potential improvement is to use the scripts to also generate the update scripts. This commit supports generating a downgrade script from the current version to the previous version. Other versions are handled using a variable containing file names of reverse update scripts and the source and target version is extracted from the file names, which is assumed to be of the form `--.sql`. In addition to adding support for generating downgrade scripts, the commit adds a downgrade test file that tests a release in a similar way to the update script and adds it as a workflow. " 41261e4e58d1e95bf55176c4c6ae24154d78ce38,Sven Klemm,Wed Jun 23 17:20:52 2021 +0200,Stabilize timestamp_limits test output,Remove the chunk name completely from output as the name might have different length leading to different output as table headers are adjusted to the length of the field values. 77e8f019d1951df51b724966022537e3dd64a86b,Sven Klemm,Sun Jun 20 01:11:01 2021 +0200,Fix SkipScan distinct column identification,The SkipScan code assumed the first entry in PathKeys would match the distinct column. This is not always true as postgres will remove entries from PathKeys it considers constant leading to SkipScan operating on the wrong column under those circumstances. This did most likely not cause any wrong results as the other constraints for SkipScan to apply still had to be satisfied but resulted in very inefficient query execution for those affected queries. This patch refactors the SkipScan code to use the distinctClause from the Query instead. Fixes #3330 3bcd35640b9f796469c562ff8ee5ee2a92bc0d10,Sven Klemm,Sun Jun 20 00:56:08 2021 +0200,Add compatibility macro for convert_tuples_by_name,PG13 removes the msg parameter for convert_tuples_by_name so this patch provides a compatibility macro that abstracts the version differences away. 7b67f72f86bf4c76da3e261210838b0ff41f1ce3,Sven Klemm,Sun Jun 20 13:44:33 2021 +0200,Move timestamp_limits and with_clause_parser test,Move the timestamp_limits and with_clause_parser test to regresscheck-shared since those tests don't need a private database incurring less overhead to run those tests. Also add missing ORDER BY clauses to some of the queries in timestamp_limits to make the output more stable. a5caa5562f8c1d1f0411035ff3be77bb773f74bd,Markos Fountoulakis,Wed Jun 23 10:44:15 2021 +0300,Fix memory overflow,Fix heap buffer overflow when renaming compressed hypertable columns. c82b5c3262d68050dbf1a258a184ea714b3fbada,Markos Fountoulakis,Wed Jun 23 12:58:24 2021 +0300,Fix Regression Linux i386 workflow,"Change the postgresql ftp URL from snapshot/ to source/. This way we do not need to react to every new commit upstream, but only whenever the postgresql minor version changes. Co-authored-by: Mats Kindahl " 9eec2a544ca22c19e82a7a8bbd6337ab0a006f02,Aleksander Alekseev,Mon Jun 21 12:50:26 2021 +0300,Refactor initialize_func_info().,Replace unnecessary chain of ts_extension_schema_oid() -> ts_extension_schema_name() -> get_namespace_oid() calls with a single OID lookup. 59d868bdf1ebd15ffd7ac78d9704dbdad71240eb,Markos Fountoulakis,Tue Jun 22 11:24:37 2021 +0300,Run sanitizer tests on PG12,Change sanitizer test to run on PG12 and make it use the same infrastructure as the other linux regression tests. Co-authored-by: Sven Klemm 374decb5acd10c5ac885a87cda1f44a05c784914,Sven Klemm,Tue Jun 22 08:33:11 2021 +0200,Don't skip default path when looking for prove, 33dfdcf5eab9e1e8ccbbc341f897107b07347866,Aleksander Alekseev,Fri Jun 18 11:08:26 2021 +0300,Introduce experimental time_bucket_ng() function,"This patch adds time_bucket_ng() function to the experimental schema. The ""ng"" part stands for ""next generation"". Unlike current time_bucket() implementation the _ng version will support months, years and timezones. Current implementation doesn't claim to be complete. For instance, it doesn't support timezones yet. The reasons to commit it in it's current state are 1) to shorten the feedback loop 2) to start experimenting with monthly buckets are soon as possible, 3) to reduce the unnecessary work of rebasing and resolving conflicts 4) to make the work easier to the reviewers " 8aaef4ae14b6b4f0f818bfd2c4eb189fb5a763ca,Nikhil,Wed Jun 16 13:38:21 2021 +0530,Fix update tests to handle sequences,"The post-update script was handling preserving initprivs for newly added catalog tables and views. However, newly added catalog sequences need separate handling otherwise update tests start failing. We also now grant privileges for all future sequences in the update tests. In passing, default the PG_VERSION in the update tests to 12 since we don't work with PG11 anymore. " d800e2f46f087a99d262659d686700275237a934,Sven Klemm,Fri Jun 18 17:17:09 2021 +0200,Add missing ORDER BY clause to dist_hypertable test, 71e8f13871539ff782a571f346e6cc9bd9d2ff5d,Mats Kindahl,Mon Jun 14 15:23:21 2021 +0200,Add workflow and CMake support for formatting,Add a workflow to check that CMake files are correctly formatted as well as a custom target to format CMake files in the repository. This commit also runs the formatting on all CMake files in the repository. cf5626bf0c7f78915f68b9d406cb65b741b6aa14,Sven Klemm,Mon Jun 14 12:54:09 2021 +0200,Combine dist_hypertable_pg12 test with dist_hypertable,Combine dist_hypertable_pg12 test since all the tests in that file can run on all supported PG versions now. We also rename the views test to information_views to make it clearer what the test is about and rename the join test to pg_join since this is the postgres join test ported to hypertables. e0e2e4308ba508cd9aaaea2e6aff612d608b61bf,Sven Klemm,Tue Jun 15 23:34:03 2021 +0200,Combine rowsecurity-12 and rowsecurity-13,With the removal of PG11 support we can use the same template for the rowsecurity test. We still need to keep the output version specific since the plan output differs between PG12 and PG13. 1b08f11297f386746b3607225c2781d733a39322,Sven Klemm,Wed Jun 16 18:57:53 2021 +0200,Filter out chunk and hypertable ids from test output,Currently any tests in regresscheck-shared can only include EXPLAIN output if they only access the precreated hypertables as hypertables and chunks created in the test itself will have ids depending on execution order of the shared tests. This patch filters out those ids so shared tests can include EXPLAIN output for local hypertables. 38e6c0aee5a82286c225ec54f048da926d0d1891,Erik Nordström,Tue May 18 10:48:30 2021 +0200,Improve trigger handling on distributed hypertables,"Triggers on distributed hypertables can now be renamed due to having the rename statement forwarded to data nodes. This also applies to other objects on such tables, like constraints and indexes, since they share the same DDL ""rename"" machinery. Tests are added for these cases. For convenience, trigger functions on distributed hypertables will now also be created automatically on the data nodes. In other words, it is no longer necessary to pre-create the trigger function on all data nodes. This change also fixes an issue with statement-level triggers, which weren't properly propagated to data nodes during `CREATE TRIGGER`. Fixes #3238 " c4c0c3de4f354688a4a75c52d3e67df20544e707,Sven Klemm,Tue Jun 15 19:50:33 2021 +0200,Harden pg_init_privs query,Since we are only interested in entries with classoid pg_class our queries should reflect that. Without these restrictions objects that have entries for multiple classoids can cause the extension update to fail. 3f9adffaf0a74d8d33a3de9f2994f86ee1c9bdf5,Aleksander Alekseev,Tue Jun 15 15:16:24 2021 +0300,Don't access bucket_width directly,"Refactoring. Since bucket_width will not be fixed in the future this patch introduces two new procedures: - ts_continuous_agg_bucket_width(), for determining the exact width of given bucket; - ts_continuous_agg_max_bucket_width(), for estimating the maximum bucket width for given continuous aggregate; This will allow determining the bucket width on the fly, which is not possible when ContinuousAgg* -> data.bucket_width is accessed directly. All accesses to data.bucket_width were changed accordingly. " b0cd6495b30362bcf90aef656a6b011620166af6,Sven Klemm,Mon Jun 14 09:20:29 2021 +0200,Fix flaky pg_dump test,Call pg_terminate_backend on all active sessions on the database to be used as template database to prevent on error on CREATE DATABASE. 6e7679e222dc6515caf7f6ea98c3f3eb4b4bfc85,Markos Fountoulakis,Tue Jun 1 17:24:28 2021 +0300,Fix failed update with parallel workers,"When executing ""ALTER EXTENSION timescaledb UPDATE TO ..."" it will fail if parallel workers spawn for the update itself. Disable parallel execution during the update. " 6d6172b02710415a36344534f066d8d454bc521d,Sven Klemm,Tue Jun 15 08:14:23 2021 +0200,Fix pg_init_privs objsubid handling,pg_init_privs can have multiple entries per relation if the relation has per column privileges. An objsubid different from 0 means that the entry is for a column privilege. Since we do not currently restore column privileges we have to ignore those rows otherwise the update script will fail when tables with column privileges are present. 4dc4e09dc258c42903ab79b85bc2b694bc291dec,Aleksander Alekseev,Mon Jun 14 14:12:51 2021 +0300,Remove two_buckets_to_str() procedure,"The two_buckets_to_str() procedure relies on fixed bucket_width which is not going to be fixed in the future. Since the procedure is used only to generate a hint that accompanies an error message, the simplest solution is to remove this procedure. We can improve error messages later if that would be necessary. " 98110af75b7676cf9b26709dffab04d6416aa3d8,Erik Nordström,Wed May 5 14:27:59 2021 +0200,Constify parameters and return values of core APIs,"Harden core APIs by adding the `const` qualifier to pointer parameters and return values passed by reference. Adding `const` to APIs has several benefits and potentially reduces bugs. * Allows core APIs to be called using `const` objects. * Callers know that objects passed by reference are not modified as a side-effect of a function call. * Returning `const` pointers enforces ""read-only"" usage of pointers to internal objects, forcing users to copy objects when mutating them or using explicit APIs for mutations. * Allows compiler to apply optimizations and helps static analysis. Note that these changes are so far only applied to core API functions. Further work can be done to improve other parts of the code. " f1ae6468d8dc5809a3fb12a945ff8f71327ef3c8,Sven Klemm,Sat Jun 12 17:39:26 2021 +0200,Make aggregate in caggs fully qualified,When querying continuous aggregate views with a search_path not including public the query will fail cause the function reference in the finalize call is not fully qualified. This can surface when querying caggs through postgres_fdw which resets search_path to contain only pg_catalog. Fixes #1919 Fixes #3326 ed63a2adf445f87abebcf60a3e1af74be080a708,Markos Fountoulakis,Fri Jun 11 14:18:03 2021 +0300,Fix compile error,Error: 'nest_level' may be used uninitialized in this function. e5837a55e664680659d38c70c97d75db2e6df266,Sven Klemm,Mon Jun 7 09:15:10 2021 +0200,Fix pull_varnos miscomputation of relids set,"Upstream fixed a bug with miscomputation of relids for PlaceHolderVar. Those fixes changed the signature of pull_varnos, make_simple_restrictinfo and make_restrictinfo. The fixes got backported to the PG12 and PG13 branch but to not break compatibility with extensions the old functions were left in. This patch makes our code use the new functions when compiling against a postgres version that has them. https://github.com/postgres/postgres/commit/1cce024fd2 https://github.com/postgres/postgres/commit/73fc2e5bab " 318d2fd80857d4dc70ca6e308bdd289dec1cdd8d,gayyappan,Thu Jun 3 17:53:56 2021 -0400,Remove paths that reference parent relids for compressed chunks,We explicilty filter paths for compressed chunks that have spurious join clauses between the compressed chunk and the original chunk or hypertable. However there are other cases where a chunk could be a child rel (i.e. RELOPT_OTHER_MEMBER_REL) such as when the chunk is referenced as part of a UNION ALL query. We remove all paths that have spurious join clauses between the compressed chunk and any implied parent for the chunk. Fixes #2917 ac031c8aedccdf3c50dfbd5eb69ed4d0349f5950,Aleksander Alekseev,Thu Jun 10 14:40:07 2021 +0300,Remove unused code,While looking for code that relies on fixed bucket_width I discovered several files that don't seem to be used anymore. This patch removes these files. Even if there is a value in them we can't rely on fixed bucket_width in the future anyway. bb43f9c421bf2dfcc78a791236ed46f2aee94c32,Markos Fountoulakis,Wed Jun 9 11:10:55 2021 +0300,Fix memory overflow,Fix heap buffer overflow in remote connection cache. 3763b20ff464158a96eb71524bbbdd6845cf90c6,Aleksander Alekseev,Wed Jun 9 18:43:27 2021 +0300,Look for PostgresNode.pm in the right path,$(pg_config --libdir)/pgxs/src/test/perl is the right path for PostgresNode.pm on Linux. On MaxOS and PG 13.x another path seems to be used: $(pg_config --libdir)/postgresql/pgxs/src/test/perl 4ad2440dad46949b8f7ba0ae70ae1d59ae780441,Aleksander Alekseev,Tue Jun 8 13:49:57 2021 +0300,Remove unused bucket_width argument,continuous_agg_update_materialization() doesn't use this argument and we can't rely on a fixed bucket_width in the future anyway. 093d021228a93b46ca6f30cd3fdd5fcbc2b39159,Nikhil,Tue Jun 8 18:44:27 2021 +0530,Make provecheck run specific tap tests,"Introduce a shell wrapper around perl prove utility to control running of TAP tests. The following control variable is supported: PROVE_TESTS: only run TAP tests from this list e.g make provecheck PROVE_TESTS=""t/foo.pl t/bar.pl"" Note that you can also use regular expressions to run multiple taps tests matching the pattern: e.g make provecheck PROVE_TESTS=""t/*chunk*"" If the existing ""TESTS="" option is used along with PROVE_TESTS then the subset represented by PROVE_TESTS will also get run. Otherwise tap tests will be skipped if ""TESTS="" is specified. e.g make installcheck TESTS=dist_hyper* PROVE_TESTS=""t/001_*"" " 264b77eb20474d26cefd29708db27a596c9a908e,Erik Nordström,Fri Jun 4 12:17:53 2021 +0200,Move internal API functions to experimental schema,"Move the ""block new chunks"" functions and the chunk-based continuous aggregate refresh function to the new experimental schema. " b72dab16c0822993f3260efde08b080c3385673c,Matvey Arye,Tue May 25 12:13:06 2021 -0400,Add some more randomness to chunk assignment,Previously the assignment of data nodes to chunks had a bit of a thundering-herd problem for multiple hypertables without space partions: the data node assigned for the first chunk was always the same across hypertables. We fix this by adding the hypertable_id to the index into the datanode array. This de-synchronizes across hypertables but maintains consistency for any given hypertable. We could make this consistent for space partitioned tables as well but avoid doing so now to prevent partitions jumping nodes due to this change. This also effects tablespace selection in the same way. a1cf610d3c276447b9e692498a9a04413ceeb323,Erik Nordström,Mon Jun 7 10:35:04 2021 +0200,Run ABI tests with TAP support,"Install the Perl prerequisites when building the image for ABI tests, including the `prove` binary. Although the ABI tests currently don't run TAP tests, CMake still failed the configuration since it expects the prerequisites to be installed unless it is run with `-DTAP_CHECKS=off`. " 7e39797d4657c13933a57100b5316fc26a5913b7,Sven Klemm,Fri Jun 4 10:37:02 2021 +0200,Use ExecComputeStoredGeneratedCompat macro,When adding support for PG13 we introduced a macro hide the function signature differences of ExecComputeStoredGenerated but the callers of this function never got adjusted to use the macro. 8e1273cd130e25b5ff0d963ce42a9db750304f2d,Erik Nordström,Thu Jun 3 12:54:26 2021 +0200,Detect TAP test prerequisites with CMake,"CMake now detects if the necessary prerequisites for running TAP tests are installed on the local system. This includes perl installation and other dependencies, such as the IPC::Run module and prove binary. " 10e339f5915b8cf143a36abc1c13a817b620cd16,Mats Kindahl,Wed Jun 2 11:07:11 2021 +0200,Add experimental schema,This commit adds an experimental schema where experimental features can be added. 5719c50e516604627c073968c2e243e56a6fc782,Sven Klemm,Thu Jun 3 07:40:50 2021 +0200,Remove TTSOps pointer macros,"Remove TTSOpsVirtualP, TTSOpsHeapTupleP, TTSOpsMinimalTupleP and TTSOpsBufferHeapTupleP macros since they were only needed on PG11 to allow us to define compatibility macros for TupleTableSlot operations. " 3b48d734cef231c0ecd416ac77165b558d590bf1,Sven Klemm,Wed Jun 2 07:05:56 2021 +0200,Fix flaky pg_dump test,Stop background workers in test after calling timescaledb_post_restore since this function restarts background workers. dd3e422964f9a0d8a913d03ae2da2ad64799f3f8,Sven Klemm,Thu May 27 10:56:46 2021 +0200,Remove update files for PG11,Remove update files that were only used for PG11 versions since those versions do not support postgres past PG11 110d77a2fed3135d72135f8eca6aefcfcf4bfb81,Sven Klemm,Thu May 27 10:01:32 2021 +0200,Combine test files,Merge test files that after the removal of PG11 support need to be no longer version specific. fb863f12c7b754def0b04d041e0b2b7399d40f5e,Sven Klemm,Thu May 27 06:43:50 2021 +0200,Remove support for PG11,Remove support for compiling against PostgreSQL 11. This patch also removes PG11 specific compatibility macros. 22ceabcb83b10cb45c9c1172de4d8a2e55beb862,Sven Klemm,Thu May 27 01:58:38 2021 +0200,Remove PG11 specific test output files, e9066d55319b4b8a23f8334c336a2813a96566aa,Sven Klemm,Thu May 27 20:07:56 2021 +0200,Remove PG11 from CI,Remove all CI runs against postgres 11 9fe90ebcea2562503db2968578ba126a8f865463,Juanito Fatas,Sat May 22 16:56:19 2021 +0900,Fix broken links in TimescaleDB NOTICE,Link to Telemetry page explains more than disable it Co-authored-by: Mike Freedman ec06f9a7d0b6ee6322e5c7567cd10ec0c3600da5,Erik Nordström,Tue May 25 16:38:08 2021 +0200,Add 2.3.0 to update test scripts,"The update tests for PG11, PG12, and PG13 are updated to include release 2.3.0. " e039043b04a827baeb7792d1d6e62e1b101c9696,Mats Kindahl,Thu May 27 08:24:55 2021 +0200,Fix issues in update smoke test,Corrected a function name and replaced a variable with the explicit commands to use. 04da7684b80a77969b90c951da02b5499940acaf,Erik Nordström,Wed May 26 13:45:39 2021 +0200,Use correct heading level in changelog,The 2.3.0 release used the wrong heading level. Fix it for future releases. c861efa0e2c10879fb0cddde65d230aed95340d2,Sven Klemm,Tue May 25 01:23:02 2021 +0200,Remove unused job_type from policies,When refactoring the job code the job_type was removed from our catalog but a few places got overlooked that still had a variable holding the job type. 77a3d5a033582e754556bcf12b4f74bc352d1da3,Erik Nordström,Fri May 21 16:46:01 2021 +0200,Release 2.3.0,"This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * #3116 Add distributed hypertable compression policies * #3162 Use COPY when executing distributed INSERTs * #3199 Add GENERATED column support on distributed hypertables * #3210 Add trigger support on distributed hypertables * #3230 Support for inserts into compressed chunks **Bugfixes** * #3213 Propagate grants to compressed hypertables * #3229 Use correct lock mode when updating chunk * #3243 Fix assertion failure in decompress_chunk_plan_create * #3250 Fix constraint triggers on hypertables * #3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * #3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code * @fabriziomello for documenting the prerequisites when compiling against PostgreSQL 13 " fe872cb684c2b9c790e92bb0cfa51094c71ce137,Sven Klemm,Sat May 22 23:24:28 2021 +0200,Add policy_recompression procedure,This patch adds a recompress procedure that may be used as custom job when compression and recompression should run as separate background jobs. 426918c59f714f5c3f000d4a17be7f1ae0a330a0,gayyappan,Mon May 24 23:14:22 2021 +0200,Fix locking issue when updating chunk status,"Two insert transactions could potentially try to update the chunk status to unordered. This results in one of the transactions failing with a tuple concurrently update error. Before updating status, lock the tuple for update, thus forcing the other transaction to wait for the tuple lock, then check status column value and update it if needed. " ad25f787fc959b99e79bc02ea904386a8cb5ea3e,gayyappan,Mon May 17 17:48:38 2021 -0400,Test support for copy on distributed hypertables with compressed chunks,Add a test case for copy on distr. hypertables with compressed chunks. verifies that recompress_chunk and compression policy work as expected. Additional changes include: Clean up commented code Make use of BulkInsertState optional in row compressor Add test for insert into compressed chunk by a different role other than the owner 1e05843738fae0055a8c7f135e023edea60164eb,Sven Klemm,Fri May 14 17:39:38 2021 +0200,Add trigger-support for insert into compressed chunks, 4f865f78705db5cc3600b45cf1b718abf22b7ee0,gayyappan,Mon May 24 23:12:44 2021 +0200,Add recompress_chunk function,"After inserts go into a compressed chunk, the chunk is marked as unordered.This PR adds a new function recompress_chunk that compresses the data and sets the status back to compressed. Further optimizations for this function are planned but not part of this PR. This function can be invoked by calling SELECT recompress_chunk(). recompress_chunk function is automatically invoked by the compression policy job, when it sees that a chunk is in unordered state. " 2c05ccf1e7de686b94e998ec4dadb496e9d81251,Sven Klemm,Fri May 7 23:45:57 2021 +0200,Support COPY into compressed chunk, fc5f10d4546ccc7c47f6301b11adbd6f3d9702ab,Sven Klemm,Thu May 13 23:30:22 2021 +0200,Remove chunk_dml_blocker trigger,Remove the chunk_dml_blocker trigger which was used to prevent INSERTs into compressed chunks. 5f6e492474a5eadc2fef6af5e04d590acc7ac361,Sven Klemm,Fri Apr 30 16:13:08 2021 +0200,Adjust pathkeys generation for unordered compressed chunks,Compressed chunks with inserts after being compressed have batches that are not ordered according to compress_orderby for those chunks we cannot set pathkeys on the DecompressChunk node and we need an extra sort step if we require ordered output from those chunks. d9839b9b61a248c6e099b9524f0a882a2d1a9ded,gayyappan,Fri May 7 16:14:04 2021 +0200,"Support defaults, sequences, check constraints for compressed chunks","Support defaults, sequences and check constraints with inserts into compressed chunks " 93be235d33f1a7539be6562ad73ce02753dca581,gayyappan,Fri Apr 9 11:50:52 2021 -0400,Support for inserts into compressed hypertables,Add CompressRowSingleState . This has functions to compress a single row. 45462c775eb8ce479e1a8d80564ba1e252ad62e6,gayyappan,Sun May 23 14:41:54 2021 -0400,Fix hypertable_chunk_local_size view,"The view uses cached information from compression_chunk_size to report the size of compressed chunks. Since compressed chunks can be modified, we call pg_relation_size on the compressed chunk while reporting the size The view also incorrectly used the hypertable's reltoastrelid to calculate toast bytes. It has been changed to use the chunk's reltoastrelid. " 4eff5b17c3ee91dd6423e93cccfb2d3abacf4ede,Sven Klemm,Sun May 23 09:52:23 2021 +0200,Explicitly install clang-9 in coverity workflow,GitHub only has the last 3 clang versions installed by default in the virtual environment and no longer installed clang-9 when they added clang-12. Since we need clang-9 to build postgres with llvm support we need to explicitly install the required package. https://github.com/actions/virtual-environments/pull/3381 58b1eb83d4d57ca9142475b6d8d8f181b38dbbe4,Mats Kindahl,Wed May 19 22:46:11 2021 +0200,Add TimescaleNode subclasses for TAP Testing,"Use subclassing to inherit from `PostgresNode` and create a hierarchy containing `AccessNode` and `DataNode` to simplify creating tests with multiple nodes. Also, two new functions are added: `TimescaleNode::create`: Creates the new node by calling `get_new_node`, `init` and `start` in that order. `AccessNode::add_data_node`: Adds a new data node to the access node. Also rewrite the test to use the new hierarchy. " 551ac5693795b769152a52500092b4d26675afdd,Mats Kindahl,Fri May 21 11:36:41 2021 +0200,Add Perl code check GitHub action,Add a run of `perltidy` to make sure that all the Perl code is correctly formatted. Also fix the formatting of one file that did not follow the style in `scripts/perltidyrc`. 80b915d001be96c2793db5484b66e764cd65d17e,gayyappan,Wed May 19 23:31:52 2021 -0400,Fix segmentation fault due to incorrect call to chunk_scan_internal,"chunk_scan_internal is called with 2 callback functions, a filter function and a tuple_found function that are invoked with the same argument. ts_chunk_set_compressed and ts_chunk_clear_compressed use chunk_tuple_dropped_filter but do not pass in the required ChunkStubScanCtx* as argument. Instead an argument of type int* is passed in. This causes a segmentation fault. Add a new chunk filter function that is compatible with the tuple_found function used by ts_chunk_clear_compressed_chunk and ts_chunk_set_compressed_chunk Fixes #3158 " 99ffe8fd6c7b2538f2643a86e3f4fb352979ace6,Sven Klemm,Thu May 20 05:52:05 2021 +0200,Fix blocking triggers with transition tables,Block creation of triggers with transition tables on trigger creation instead of erroring out when a chunk is created. Fixes #3234 3e28f10600199e4ac996024fe835d832472d3cdf,Sven Klemm,Thu May 20 03:36:09 2021 +0200,Fix constraint trigger on hypertables,"When creating a constraint trigger on a hypertable the command succeeds and the constraint trigger works correctly for existing chunks but any chunk creation after that would fail with an error, because the constraint trigger was treated like a normal constraint. But since pg_get_constraintdef does not return a SQL command for constraint triggers the corresponding command that was created would throw an error. Fixes #3235 " 72ed3a81c57e0e60fbb78b570a6e53a124add01a,Sven Klemm,Thu May 20 20:23:36 2021 +0200,Explicitly install clang-9 in CI ubuntu images,GitHub removed default installation of clang-9 from the ubuntu images but those are required to build postgres with llvm so we have to explicitly install the required packages. https://github.com/actions/virtual-environments/pull/3381 eef71fdfb1e9c0e9342301bff933e007e50dfd97,Sven Klemm,Sat Feb 27 08:21:20 2021 +0100,Replace StrNCpy with strlcpy,PG14 removes StrNCpy and some Name helper functions. https://github.com/postgres/postgres/commit/1784f278a6 4f72ab0377ea669486549e7ac167da2bf518d1db,Sven Klemm,Wed May 19 03:40:17 2021 +0200,Fix assertion failure in decompress_chunk_plan_create,decompress_chunk_plan_create used get_actual_clauses to extract RestrictInfo clauses and adds them as quals. This function is only supposed to be used when none of the RestrictInfos are pseudoconstant leading to an assertion failure when the query has pseudoconstant quals. Fixes #3241 600d3248984631d18e772476408f8b54aedfd913,Erik Nordström,Mon May 17 14:41:30 2021 +0200,Use correct lock mode when updating chunk,Use RowExclusive lock instead of AccessShareLock when updating a chunk's metadata. 4efa51cd8da311ed8441fae7a293c180c069cdfa,Nikhil,Mon May 17 16:32:07 2021 +0530,Add tap tests infrastructure support,"The TAP testing PG framework meets our requirements for doing multinode/stand-alone-binaries testing for scenarios which need individual multiple PG instances (multi-node testing is a prime example). This commit adds the necessary wrappers to allow the use of the TAP testing framework in timescale code base. The README in ""test/perl"" directory gives a fair idea of how to write tap tests. A simple tap test has been added to provide a reference point for developers to write new ones. One can go to ""build/tsl/test"" and invoke ""make checkprove"" to see the tap test in action. Also includes changes for enabling github CI to run these taptests by installing the requisite dependencies. Includes changes to license checking scripts to handle new *.pl and *.pm files. Also added a new scripts/perltidyrc to aid in formatting of these files. " adde40f548d4bbde92520b5c59718d93e9f5c666,Markos Fountoulakis,Mon May 17 17:19:58 2021 +0300,Use chunk status for is_compressed field in view,"The timescaledb_information.chunks view used to return NULL in the is_compressed field for distributed chunks. This changes the view to always return true or false, depending on the chunk status flags in the Access Node. This is a hint and cannot be used as a source of truth for the compression state of the actual chunks in the Data Nodes. " 45384357bded3f6346e151440aa730cdc8b5076a,Sven Klemm,Sat May 15 04:38:09 2021 +0200,Bump postgres versions used in CI,"Change CI tests to run against pg 11.12, 12.7 and 13.3. The latest postgres versions adjusts the event_trigger test to output the list of currently defined event triggers making the test output different when timescaledb is installed since we define our own event triggers. Due to this change we have to ignore the result of this test in our CI. https://github.com/postgres/postgres/commit/f3c45378 " b788168e594cd53fcb9314f64c637ecba852edb5,Mats Kindahl,Mon May 10 14:32:18 2021 +0200,Propagate grants to compressed hypertable,"Grants and revokes where not propagated to compressed hypertables, if the hypertable had a compressed hypertable, meaning that `pg_dump` and `pg_restore` would not be able to dump nor restore the compressed part of a hypertable unless they were owners and/or was a superuser. This commit fixes this by propagating grants and revokes on a hypertable to the associated compressed hypertable, if one exists, which will then include the compressed hypertable and the associated chunks in the grant and revoke execution. It also adds code to fix the permissions of compressed hypertables and all associated chunks in an update and adds an update test to check that the permissions match. Fixes #3209 " f6967b349f2065f8a350097b65b795d3d29462a9,Erik Nordström,Fri Apr 23 20:45:38 2021 +0200,Use COPY when executing distributed INSERTs,"A new custom plan/executor node is added that implements distributed INSERT using COPY in the backend (between access node and data nodes). COPY is significantly faster than the existing method that sets up prepared INSERT statements on each data node. With COPY, tuples are streamed to data nodes instead of batching them in order to ""fill"" a configured prepared statement. A COPY also avoids the overhead of having to plan the statement on each data node. Using COPY doesn't work in all situations, however. Neither ON CONFLICT nor RETURNING clauses work since COPY lacks support for them. Still, RETURNING is possible if one knows that the tuples aren't going to be modified by, e.g., a trigger. When tuples aren't modified, one can return the original tuples on the access node. In order to implement the new custom node, some refactoring has been performed to the distributed COPY code. The basic COPY support functions have been moved to the connection module so that switching in and out of COPY_IN mode is part of the core connection handling. This allows other parts of the code to manage the connection mode, which is necessary when, e.g., creating a remote chunk. To create a chunk, the connection needs to be switched out of COPY_IN mode so that regular SQL statements can be executed again. Partial fix for #3025. " 39b9457540002f7da6771e6e03b40d6021f12c30,Erik Nordström,Fri May 7 16:41:57 2021 +0200,Add trigger support on distributed hypertables,"This change makes it possible to create and use triggers on distributed hypertables. The DDL for `CREATE TRIGGER` and `DROP TRIGGER` are now forwarded to data nodes. Previously, `CREATE TRIGGER` worked, but didn't forward the DDL command. Note that triggers exist only on the data nodes and on the data node's root hypertable. Triggers need to execute on the data nodes where data is stored; in particular, row-based ""before"" triggers might modify the tuple. On the access node, chunks are foreign tables and such tables do not support triggers. Also note that statement-level triggers execute once on each node involved, including the access node. So, if, e.g., two rows are inserted into two different data nodes in the same statement, a statement-level insert trigger will fire once on the access node and once on each data node. An issue with `RETURNING` handling for distributed insert is also fixed (in the `DataNodeDispatch` executor node). The fix ensures that the modified tuple is returned when a trigger modifies the tuple on a data node. Fixes #3201 " 5606bf888318d1dff0217383af9748cc7c3be903,Fabrízio de Royes Mello,Mon May 10 10:24:58 2021 -0300,Added Postgres 13 prerequisites for compilation,Issue #2779 introduced support for PostgreSQL 13 bc740a32fb49f38ceb5570dcfc40a75a59e3f2c4,Markos Fountoulakis,Fri May 7 14:13:53 2021 +0300,Add distributed hypertable compression policies,Add support for compression policies on Access Nodes. Extend the compress_chunk() function to maintain compression state per chunk on the Access Node. 422440a2de9f0d542f5cb38a46761c7a9ab030da,Erik Nordström,Thu May 6 22:38:09 2021 +0200,Add GENERATED column support on distributed hypertables,"Add support for deparsing `GENERATED` columns when creating distributed hypertables, including support for generating values ""on-the-fly"" when a `RETURNING` clause is used with an `INSERT` statement. Previously, using generated columns on a hypertable led to errors or had unpredictible results. " 490e9405874d284735e27e3f9f63a2f6dd31a08f,Michael J. Freedman,Thu May 6 23:28:50 2021 -0400,Fix accidental misused acronym for Data Manipulation Language,"In the definition of ""Timescale Data Manipulation Interfaces"", the description of Data Manipulation Language was accidentally and erroneously given the acronym ""DDL"" as a parenthetical, when it should have been (and was meant to be) given the proper acronym of ""DML"". No intent is changed with this correction. " 2e444849fbfbe348c9eae4b7ea87a23b87c60055,Erik Nordström,Wed May 5 13:22:40 2021 +0200,Clean up cross-module function definitions,Use the `PGFunction` type on all PostgreSQL function definitions when defining cross-module (TSL) functions. ec658668903d634cce1b896798b9776753f8a810,Mats Kindahl,Thu May 6 13:58:53 2021 +0200,Add 2.2.1 to update test scripts,"The update tests scripts for PG11, PG12, and PG13 are updated to include 2.2.1. " e7daf74d403643f18d7266928b92f765ab770a94,Mats Kindahl,Wed Apr 28 10:32:18 2021 +0200,Release 2.2.1,"This maintenance release contains bugfixes since the 2.2.0 release. We deem it high priority for upgrading. This release extends Skip Scan to multinode by enabling the pushdown of `DISTINCT` to data nodes. It also fixes a number of bugs in the implementation of Skip Scan, in distributed hypertables, in creation of indexes, in compression, and in policies. **Features** * #3113 Pushdown ""SELECT DISTINCT"" in multi-node to allow use of Skip Scan **Bugfixes** * #3101 Use commit date in `get_git_commit()` * #3102 Fix `REINDEX TABLE` for distributed hypertables * #3104 Fix use after free in `add_reorder_policy` * #3106 Fix use after free in `chunk_api_get_chunk_stats` * #3109 Copy recreated object permissions on update * #3111 Fix `CMAKE_BUILD_TYPE` check * #3112 Use `" u` to format Oid instead of `,,,, "d` * #3118 Fix use after free in cache * #3123 Fix crash while using `REINDEX TABLE CONCURRENTLY` * #3135 Fix SkipScan path generation in `DISTINCT` queries with expressions * #3146 Fix SkipScan for IndexPaths without pathkeys * #3147 Skip ChunkAppend if AppendPath has no children * #3148 Make `SELECT DISTINCT` handle non-var targetlists * #3151 Fix `fdw_relinfo_get` assertion failure on `DELETE` * #3155 Inherit `CFLAGS` from PostgreSQL * #3169 Fix incorrect type cast in compression policy * #3183 Fix segfault in calculate_chunk_interval * #3185 Fix wrong datatype in integer based retention policy **Thanks** * @Dead2, @dv8472 and @einsibjarni for reporting an issue with multinode queries and views * @hperez75 for reporting an issue with Skip Scan * @nathanloisel for reporting an issue with compression on hypertables with integer-based timestamps * @xin-hedera for fixing an issue with compression on hypertables with integer-based timestamps ",,,, 86bf122479b114f1e003ee947100231d95506bac,Sven Klemm,Sat May 1 01:05:57 2021 +0200,Fix segfault in calculate_chunk_interval,Block calling calculate_chunk_interval on distributed hypertables to prevent segfault when finding min and max as distributed chunks have no tableam since they are foreign tables. d17e08470564a0e01a90dea17a488e705719c42d,Sven Klemm,Mon May 3 23:33:30 2021 +0200,Fix wrong datatype in integer based retention policy,When reading the configuration for retention policy of integer based hypertables int32 was used instead of int64. Fixes #2877 04328b51c9ceac2bd4ccc8d64f80c8256b3e54de,James Winegar,Thu Apr 29 21:36:44 2021 -0500,Fix spelling error in time_bucket error message, 6b2b4d936a56095fed8f230c0db3d486b935ebc5,Nikhil,Wed Apr 28 13:34:17 2021 +0530,Add a few miscellaneous code fixes,"1) Quell a compiler warning about uninitialized use of ""varno"" variable. 2) Fix ts_chunk_get_by_name to return early if rogue schema/table args are passed in. " ff5f6056c3a492ef672b76ee3099e26e0d8f8284,Sven Klemm,Thu Apr 29 07:07:55 2021 +0200,Add changelog entry for #3169, 37370908c80b5e9b49df4d402ed0386b013c33a6,Xin Li,Wed Apr 28 10:20:58 2021 -0500,Fix incorrect type cast in compression policy,Fix incorrect type cast when reading int64 compress after from compression policy Fixes #3009 Signed-off-by: Xin Li 1499ed4f20fa60102a37737547d4240bfe11551c,Sven Klemm,Wed Apr 28 19:21:13 2021 +0200,Add function to set chunk status,"Add 2 helper functions to modify status of a chunk ts_chunk_add_status(Chunk *chunk, int status); ts_chunk_set_status(Chunk *chunk, int status); " 1f0c591c88920d47f886a476bf3d37b793cec68a,Nikhil,Wed Apr 28 12:39:39 2021 +0530,Use no-unused-command-line-argument flag,"We started inheriting CFLAGS from postgresql via ba51adc3. Although we remove ""-W"" flags, we still end up accepting other options like ""-L"" from postgresql. This can cause compilation to fail with compilers like Clang which enable reporting of errors on unused command line arguments. This patch conditionally adds -Wno-unused-command-line-argument in our cmake files to avoid unhelpful unused-command-line-argument warnings/errors. " 0a39e2ebffe5c919591b77c7fbe0a9db70f7d333,Sven Klemm,Fri Apr 23 21:25:55 2021 +0200,Add coredump support for 32bit ci runs,This patch adds coredump support for the 32bit CI runs. This patch switches 32bit run to the debian base image and install postgres from PGDG instead of using the alpine image so we don't have to compile postgres ourselves and can just install the debugsymbol package for postgres. e0bff859e36bf66b8322a0f374110e022b195940,gayyappan,Mon Apr 19 10:41:43 2021 -0400,Add chunk_status column to catalog chunk table,Add a new column chunk_status to _timescaledb_catalog.chunk. ba51adc3bdfbe44becd16d4567a26dcb7f183369,Sven Klemm,Sun Apr 25 00:12:01 2021 +0200,Inherit CFLAGS from postgresql,There are quite a few compiler flags that need to be identical between postgresql and timescaledb otherwise there will be seemingly unexplainable failures during execution. We inherit all CFLAGS from postgresql except those starting with -W as we have our own set of warning options. 0de39140c3a2b4211f7550c504ebc31bad641aec,Sven Klemm,Thu Apr 22 18:31:46 2021 +0200,Fix fdw_relinfo_get assertion failure on DELETE,The code in fdw_relinfo_get assumed rel->fdw_private and rel->fdw_private->fdw_relation_info were always allocated which is not true for DELETEs for instance. aeb107659bc0815ec5e3cb6b8044aed5dd444ac9,Mats Kindahl,Tue Apr 13 10:07:36 2021 +0200,Copy recreated object permissions on update,"Tables, indexes, and sequences that are recreated as part of an update does not propagate permissions to the recreated object. This commit fixes that by saving away the permissions in `pg_class` temporarily and then copying them back into the `pg_class` table. If internal objects are created or re-created, they get the wrong initial privileges, which result in privileges not being dumped when using `pg_dump`. Save away the privileges before starting the update and restore them afterwards to make sure that the privileges are maintained over the update. For new objects, we use the initial privileges of the `chunk` metadata table, which should always have correct initial privileges. Fixes #3078 " 255b5715ba2ebefa18d0812bc7f537c97b085e75,Nikhil,Fri Apr 23 12:45:26 2021 +0530,Fix flaky dist_hypertable test,The chunks of the datanodes seem to contain very few rows. That can cause PostgreSQL to choose a SeqScan plan in some cases. Add a few more rows to make it choose IndexScan all of the time. ec733de130e3ab3e902231e781a18b8e29de519f,Sven Klemm,Sat Apr 17 00:44:34 2021 +0200,Add GitHub action for posting daily CI summary,This patch adds a GitHub action that posts all failed CI jobs in the last 24 hours with link to the failed job. 85cb9aa722104c6141ea71d0fba517f7b830b0b4,Sven Klemm,Sat Apr 17 01:29:57 2021 +0200,Remove slack notification from individual workflows,The patch removes the Slack notification of individual workflows in favor of the Slack summary message e7ffedbec7c2032015a2685a7630d5efabc11244,Mats Kindahl,Mon Mar 1 20:06:04 2021 +0100,Add update smoke test script,"The update smoke test script can be used to test that an extension works correctly. It will run the normal update tests, but do it against a single server. It also makes some minor refactorings of the update tests to make it possible to run the tests against different versions of the server and also support the smoke tests where some features cannot be tested. " a3d8f9fecd4141e23901234c4000736ec3a5ee46,Nikhil,Thu Apr 22 10:38:46 2021 +0530,Make SELECT DISTINCT handle non-var targetlists,"The current SELECT DISTINCT pushdown code assumed that the targetlist will always contain references to column attributes of the target table. So, even a simple ""SELECT DISTINCT 1 from table;"" causes a segmentation fault because the ""varno"" field is not assigned. Fix this oversight. Issue reported by @svenklemm Fixes timescale/timescaledb-private#920 " fc3f8305e37621aad513549f0eb8ca0c3f92d67b,Sven Klemm,Thu Apr 22 00:08:52 2021 +0200,Skip ChunkAppend if AppendPath has no children,"Since postgres has optimizations for Append paths without children we do not replace those paths with ChunkAppend. While this will prevent ChunkAppendPaths without children during planning, in the executor it is still possible for ChunkAppend to have no children as run-time exclusion may remove all of them. " 8fd4ecbdd941f754dde575d4fa0069abe1db1aee,Sven Klemm,Wed Apr 21 19:17:46 2021 +0200,Fix SkipScan for IndexPaths without pathkeys,"The SkipScan code assumed IndexPaths on ordered indexes always have pathkeys which is not true when the pathkeys are not useful for the query, leading to a segfault for those queries. " 242919bc4938438488faae406b084dbf9d06db74,Sven Klemm,Wed Apr 21 14:02:34 2021 +0200,Fix flaky pg_dump test,This patch fixes the flaky pg_dump test by switching off the current database before using it as template. It also disables background job scheduling instead of stopping them before creating the new database as stopping bgw jobs might not happen immediately. Fixes #2847 4e86a12ceb21653aeabd6f5c9f3d05bee1c62c1a,Ruslan Fomkin,Wed Apr 21 10:48:13 2021 +0200,Run code style regression test on all PRs,"Checking code formatting and other simple code style test is good to perform on PRs to feature branches, not only the master branch. This commit removes the limitation to perform tests only on PR to the master. " 507b5e5c155548d3286592ea0ce3bf5b9ea89911,Ruslan Fomkin,Wed Apr 21 10:14:32 2021 +0200,Ignore generated SQL test files from templates,`index` test was moved into SQL test template in #3123 and `generated_columns` - in #2927. They are added to relevant gitignore. 425cdd16e41fe774cf7479733a32dd8455758b7a,Nikhil,Tue Apr 13 16:32:52 2021 +0530,"Add ""SELECT DISTINCT"" pushdown in multi-node","Construct ""SELECT DISTINCT target_list"" or ""SELECT DISTINCT ON (col1, col..) target_list"" statement to push down the DISTINCT clause to the remote side. We only allow references to basic ""Vars"" or constants in the DISTINCT exprs So, ""SELECT DISTINCT col1"" is fine but ""SELECT DISTINCT 2*col1"" is not. ""SELECT DISTINCT col1, 'const1', NULL, col2"" which is a mix of column references and constants is also supported. Everything else is not supported. This pushdown also needs to work when timescaledb.enable_per_data_node_queries is disabled. All existing test cases in which ""SELECT DISTINCT"" is now being pushed down have been modified. New test cases have been added to check that the remote side uses ""Skip Scans"" as is suitable in some cases. " 202cceee56d1f8fb5844b0d7be6da2f9d65120f8,Sven Klemm,Tue Apr 20 17:33:21 2021 +0200,Increase number of connections for tsl tests,Since we run 20 tests in parallel in the tsl tests multiple multinode tests might exhaust the number of connections the test instance allowed. This patch changes the number of connections to 200 for regresscheck-t. 28d89091ea589a20c85d248ce64155e7cb339c44,Sven Klemm,Tue Apr 20 10:47:40 2021 +0200,Fix SkipScan path generation with expressions,"In DISTINCT when deciding about adding a SkipScan path we try to match the PathKey to the targetlist which would fail if PathKey was an expression but the index not an expression index. The check for the DISTINCT being a Var would only happen after this check leading to a ""skip column not found in targetlist"" error under these circumstances. This only affects DISTINCT queries with targetlist with 1 non-Const entry that is an expression as others would get rejected earlier when checking whether to build SkipScan path. Fixes #3134 " dc6e8637f89974cafa486b51d8d03349053b5c41,Dmitry Simonenko,Mon Apr 19 15:47:35 2021 +0300,Fix REINDEX TABLE for distributed hypertables,In case of distributed hypertable do not propagate the REINDEX command to chunks. Allow command to be passed for remote execution on data nodes. Fixes #3018 20fd4214375be19aa15a5eb95f1a54a007dc8690,Dmitry Simonenko,Mon Apr 19 13:20:01 2021 +0300,Fix crash while using REINDEX TABLE CONCURRENTLY,CONCURRENTLY option of the REINDEX TABLE is not supported and leads to a crash when using with hypertable. Block it and print the error message. Fix #3122 16accae67d207e2fbe1dd1abc296ae5614e7f455,Dmitry Simonenko,Mon Apr 19 12:28:25 2021 +0300,Fix use after free in cache,When calling hash_search with HASH_REMOVE the returned pointer should not be dereferenced because it returns a dangling pointer c8535f91af284bef6049a639158ca8574d26bc94,Sven Klemm,Sat Apr 17 02:43:39 2021 +0200,Fix windows CI PG13 failure,Installing the postgres13 windows package pulled in an install of vcredist140 which requires a reboot making chocolatey return with a non-zero exit code to signal the required reboot. This patch changes chocolatey to no longer give individual packages control over chocolatey exit code. This patch also changes chocolatey to longer show package download progress information to make the CI log less spammy. d26c7441154b7b890cb8d694c20e96b07e10aa03,Sven Klemm,Tue Apr 13 11:41:10 2021 +0200,Use , u to format Oid instead of ,,,, d,Since Oid is unsigned int we have to use ,,, u to print it otherwise oids >= 2^31 will not work correctly. This also switches the places that print type oid to use format helper functions to resolve the oids. ,,,, 796bf3e9ae4f75c9f21347209c7168f51cd26256,gayyappan,Tue Feb 23 17:15:18 2021 -0500,Move plan nodes to nodes subdirectory,Reorg source code to create a nodes subdirectory. Move plan nodes to this drectory. 6e437c2d952f0ff58aa5f405f0d00a5700bf5cc8,Sven Klemm,Mon Apr 12 18:39:47 2021 +0200,Fix use after free in chunk_api_get_chunk_stats, 498ee7c6774edfc21cc639650ddda39925328274,Sven Klemm,Mon Apr 12 11:41:32 2021 +0200,Fix use after free in add_reorder_policy,The hypertable object in policy_reorder_add was still used after releasing the cache leading to occasional segfaults in CI. baaa9abdddbd671ebe4007665c807b37da23a0e0,Ruslan Fomkin,Tue Apr 13 10:37:32 2021 +0200,Add 2.2.0 to update test scripts, 3443e3ff290453fe862394298a58c5a30efd0eb7,Feike Steenbergen,Tue Apr 13 10:57:31 2021 +0200,Fix CMAKE_BUILD_TYPE check,"Possible values are Release, Debug, RelWithDebInfo and MinSizeRel https://llvm.org/docs/CMake.html#frequently-used-cmake-variables " f1e78b847c077d0e29770f643e37eebd82ea2d28,Sven Klemm,Sat Apr 10 18:37:26 2021 +0200,Use commit date in get_git_commit(),get_git_commit used the author date instead of the commit date when showing information about the commit. For the purpose of the function the author date is not that significant and when the commit to the branch actually happened is much more relevant. b1143de7955543154d3d32ec1fe2d0162b28daba,Ruslan Fomkin,Wed Mar 24 09:48:50 2021 +0100,Release 2.2.0,"This release adds major new features since the 2.1.1 release. We deem it moderate priority for upgrading. This release adds the Skip Scan optimization, which significantly improves the performance of queries with DISTINCT ON. This optimization is not yet available for queries on distributed hypertables. This release also adds a function to create a distributed restore point, which allows performing a consistent restore of a multi-node cluster from a backup. The bug fixes in this release address issues with size and stats functions, high memory usage in distributed inserts, slow distributed ORDER BY queries, indexes involving INCLUDE, and single chunk query planning. **PostgreSQL 11 deprecation announcement** Timescale is working hard on our next exciting features. To make that possible, we require functionality that is unfortunately absent on PostgreSQL 11. For this reason, we will continue supporting PostgreSQL 11 until mid-June 2021. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 11 support will not be included going forward. **Major Features** * #2843 Add distributed restore point functionality * #3000 SkipScan to speed up SELECT DISTINCT **Bugfixes** * #2989 Refactor and harden size and stats functions * #3058 Reduce memory usage for distributed inserts * #3067 Fix extremely slow multi-node order by queries * #3082 Fix chunk index column name mapping * #3083 Keep Append pathkeys in ChunkAppend **Thanks** * @BowenGG for reporting an issue with indexes with INCLUDE * @fvannee for reporting an issue with ChunkAppend pathkeys * @pedrokost and @RobAtticus for reporting an issue with size functions on empty hypertables * @phemmer and @ryanbooz for reporting issues with slow multi-node order by queries * @stephane-moreau for reporting an issue with high memory usage during single-transaction inserts on a distributed hypertable. " 43698717f6e0b509bf9073eda52d29099f7cdfb7,Sven Klemm,Tue Apr 13 00:24:43 2021 +0200,Fix Skip Scan cost calculation for queries with excluded chunks,If the row estimate for the scan below Skip Scan is 1 we assume that the estimate got clamped to 1 and no rows would be returned by this scan and this chunk will most likely be excluded by runtime exclusion. Otherwise the cost for this path would be highly inflated due to (ndistinct / rows) * total leading to SkipScan not being chosen for queries on hypertables with a lot of excluded chunks. 292574b2054389ba1064b09198e7221a34b37227,Sven Klemm,Fri Apr 9 23:21:46 2021 +0200,Fix command execution in pr-validation,The github action to check for multiple commits would check the pullrequest message for a command to disable the check. While processing the message any commands enclosed in backticks would be executed as shell commands. 31e1a9ce4606e94f64351506ef55b7da7e74f28b,Sven Klemm,Thu Apr 1 17:21:23 2021 +0200,Add SkipScan optimization for DISTINCT queries,"This patch implements SkipScan; an optimization for SELECT DISTINCT ON. Usually for SELECT DISTINCT ON postgres will plan either a UNIQUE over a sorted path, or some form of aggregate. In either case, it needs to scan the entire table, even in cases where there are only a few unique values. A skip scan optimizes this case when we have an ordered index. Instead of scanning the entire table and deduplicating after, the scan remembers the last value returned, and searches the index for the next value after that one. This means that for a table with k keys, with u distinct values, a skip scan runs in time u * log(k) as opposed to scanning then deduplicating, which takes time k. We can write the number of unique values u as of function of k by dividing by the number of repeats r i.e. u = k/r this means that a skip scan will be faster if each key is repeated more than a logarithmic number of times, i.e. if r > log(k) then u * log(k) < k/log(k) * log(k) < k. Co-authored-by: Joshua Lockerman " e51b0f7f5c45a39321548d210b2b1bc4bb2a9968,Sven Klemm,Fri Apr 9 17:53:06 2021 +0200,Add ts_is_chunk_append_path and ts_is_constraint_aware_append_path,Add helper functions to check whether a CustomPath is a ChunkAppendPath and ConstraintAwareAppendPath. db27b23e15f0e07551192214886a5f269b30b3bb,Dmitry Simonenko,Thu Apr 8 21:48:36 2021 +0300,Fix dist_restore_point test on PG11.0,"The test fails on PG11.0 since there is no notice during data node deletion. Reduce client notice level to warning, to make it compatible with other versions. Fixes #3096 " b6d9cda55df715ce20b78f608ff040202d0e8a11,Sven Klemm,Thu Apr 8 13:51:03 2021 +0200,Ignore advisory_lock of pgregresscheck,The advisory_lock test of the postgres test suite is flaky so this patch changes CI to ignore failures in that test. b95052e206cc8249c60365dee83203066b1bcb3b,Sven Klemm,Sat Apr 3 01:23:31 2021 +0200,Fix chunk index column name mapping,When mapping index column references from the hypertable index to the chunk index the name of the index column was used instead of the column name of the hypertables. While those names are identical most of time unless modified this is not true for indexes with INCLUDE on a column that is also a key column. Additionally this fixes a bug where expression indexes would never get their column references adjusted. Fixes #3056 639aef76a4cb33008096fe7743b30b72b637e009,Ruslan Fomkin,Wed Mar 24 09:48:50 2021 +0100,Refactor chunk creation for future extension,"Separates chunk preparation and metadata update. Separates preparation of constraints names, since there is no overlap between preparing names for dimension constraints and other constraints. Factors out creation of json string describing dimension slices of a chunk. This refactoring is preparation for implementing new functionalities. " cc38ba324125155036130940f0c9e82528fbee16,Sven Klemm,Sat Apr 3 13:13:23 2021 +0200,Keep Append pathkeys in ChunkAppend,This changes ChunkAppend to always keep the pathkeys from the original path because the original path was either a MergeAppendPath and this will become an ordered append or the original path is an AppendPath and since we do not reorder children the order will be kept intact. For the AppendPath case with pathkeys it was most likely an Append with only a single child. We could skip the ChunkAppend path creation if there is only a single child but we decided earlier that ChunkAppend would be beneficial for this query so we treat it the same as if it had multiple children. Fixes #3030 498eb03acbd7789fb6eaf533763ef5d6dd267647,Ruslan Fomkin,Tue Apr 6 09:01:49 2021 +0200,Do not notify Slack for MacOS test,Slack notification only works on Linux and produces error if it is called during MacOS test. This fix avoids notifying Slack if a regression test is run on MacOS. 349b298a05beee64f101af438654a4c7e56761dd,Sven Klemm,Sun Apr 4 22:28:03 2021 +0200,Install clang-8 for PG11.0 CI test,clang-8 is no longer installed by default in the ubuntu images so we have to explicitly install it for any actions that require it. https://github.com/actions/virtual-environments/issues/2950 d38b8eed9db72dbd1d63779bfe7d7269cacc0211,Sven Klemm,Mon Apr 5 16:20:21 2021 +0200,Combine vacuum and vacuum_multi tests,The reason vacuum and vacuum_multi was separate is because vacuuming multiple relations is only supported in PG11+. Since we no longer support versions < PG11 we can combine those 2 tests. 5151be5647d5c64fc3ceafac78e247ada80ce534,Sven Klemm,Sat Apr 3 18:37:20 2021 +0200,Make VACUUM tests more resilient,PG13 changes index vacuum behaviour in versions after 13.2 so index vacuum is no longer triggered when inserting only. This lead to test failures in the ABI test which was running PG13 snapshot but will also lead to test differences between 13.2 and 13.3 when that is released. This patch removes the indexes from the vacuum and vacuum multi tests and makes them smaller for vacuum_parallel test to make the size stable. https://github.com/postgres/postgres/commit/9663d12 eace5ea23b299681a45efa312cf6f51f587d561c,Sven Klemm,Fri Apr 2 11:47:16 2021 +0200,Allow use of wildcards in TESTS,"This patch enables the use of wildcards in TESTS allowing you to specify multiple tests with a single pattern. TESTS=""compression*"" will match all tests whose name starts with compression TESTS=""*compression*"" will match all tests that have compression anywhere in the name Wildcard matching also applies to version specific tests so compression-13 would also be matched by those patterns. " 42b4566aebb43ec54da806736d563a0e84df5797,Sven Klemm,Thu Apr 1 13:57:33 2021 +0200,Explicitly install clang-format-8 in CI,clang-format-8 was removed from the Ubuntu image so we have to install it explicitly for the code formatting check. https://github.com/actions/virtual-environments/issues/2950 4235344be5cd90830bba1a6d86551c0b6296510f,Sven Klemm,Fri Mar 26 16:06:32 2021 +0100,Remove Hypertable argument from ts_constraint_aware_append_path_create,The hypertable argument to ts_constraint_aware_append_path_create was never actually used in the function so this patch removes it. 3c178024855c7b89390047cdeb02a058547e6b13,Nikhil,Mon Mar 29 16:33:09 2021 +0530,Fix chunks_in queries to use ChunkAppend plans,"It was observed on DNs that ""ORDER BY time"" queries when the ""chunks_in"" exclusion function was specified were running slower than expected. Examination of the EXPLAIN output revealed that when ""chunks_in"" was specified then the ChunkAppend plan which provides the exclusion of unwanted chunks was not getting picked up for execution. Logic in get_explicit_chunk_oids() function has now been improved to populate the fdw_private structure appropriately if ordering info is present. Care has been taken to also handle the case where multiple dimensions are present. Fixes #2733 " 574ef874266ebf276700a0d1c5f474e5553e46be,Ruslan Fomkin,Tue Mar 30 11:13:23 2021 +0200,Add 2.1.1 to update test scripts, 17574807b098a389ec2ac31f74e530abb1988cd5,Mats Kindahl,Tue Mar 23 10:22:16 2021 +0100,Add GitHub action to ensure a single commit,"This commit adds a GitHub action that will check that there is only a single commit in the pull request and trigger an error if that is not the case. The check can be disabled by adding the following line as a trailer to the pull request message (case-insensitive check). Note that only the pull request body is checked. Trailers are assumed to follow RFC2822 conventions, even though this is not formally enforced right now. Disable-Check: commit-count " 5c525bbf310b1712c55f07ee0d9065534f602202,Erik Nordström,Thu Mar 25 16:27:12 2021 +0100,Reduce memory usage for distributed inserts,"The data node dispatcher used a per-tuple memory context that wasn't automatically reset as part of each iteration of the execution loop. This lead to linearly growing memory usage over a transaction that inserted a lot of data. This change switches to using the right per-tuple memory context and also introduces a new ""batch"" memory context that maintains the batch data until it is fully processed and the next batch is read. These changes mean that memory usage never grows higher than what is needed for one batch, which will avoid out-of-memory situations. Fixes #3049 " 2cba4b1d8113cc20e73ab3e2b4c235045d926690,Ruslan Fomkin,Mon Mar 29 13:08:56 2021 +0200,Release 2.1.1,"This maintenance release contains bugfixes since the 2.1.0 release. We deem it high priority for upgrading. The bug fixes in this release address issues with CREATE INDEX and UPSERT for hypertables, custom jobs, and gapfill queries. This release marks TimescaleDB as a trusted extension in PG13, so that superuser privileges are not required anymore to install the extension. **Minor features** * #2998 Mark timescaledb as trusted extension **Bugfixes** * #2948 Fix off by 4 error in histogram deserialize * #2974 Fix index creation for hypertables with dropped columns * #2990 Fix segfault in job_config_check for cagg * #2987 Fix crash due to txns in emit_log_hook_callback * #3042 Commit end transaction for CREATE INDEX * #3053 Fix gapfill/hashagg planner interaction * #3059 Fix UPSERT on hypertables with columns with defaults **Thanks** * @eloyekunle and @kitwestneat for reporting an issue with UPSERT * @jocrau for reporting an issue with index creation * @kev009 for fixing a compilation issue * @majozv and @pehlert for reporting an issue with time_bucket_gapfill " f54fc16fed8a1095cec885894e57c7fbcbe7caf3,Sven Klemm,Wed Mar 24 16:14:15 2021 +0100,Fix gapfill/hashagg planner interaction,The hashagg optimization adds a hashagg plan with modified costs to the list of paths. This happens after gapfill pathes have been created so those newly created pathes would miss the GapFill node. If those created pathes would turn out to be cheaper than other pathes GapFill would fail to work as no GapFill node would be executed. This patch changes hashagg path creation to skip adding pathes when there is a GapFill path. Fixes #3048 4d871cbd2634475deba3c22ce8e94fda4f73c155,Sven Klemm,Thu Mar 25 22:09:45 2021 +0100,Make readme names consistant,Most places in the source tree use README.md as filename for additional documentation. This patch changes the filenames of files using a different capitalization. a83ed44ce3efc85cbcb053b67c4149206de43b9b,Sven Klemm,Thu Mar 25 18:22:21 2021 +0100,Fix UPSERT on hypertables with columns with defaults,When a column with a DEFAULT was added to a hypertable after hypertable creation this column would have atthasmissing true for the hypertable itself but not for chunks created after. This lead to the attribute mapping from the hypertable to the chunk being non-NULL but the chunk -> hypertable map being NULL leading to a segfault because the code assumed both maps would either be NULL or non-NULL. Fixes #3036 Fixes #3037 48d0fb737d01d256062bdb4e31b295d6e981286f,Nikhil,Thu Mar 11 17:24:16 2021 +0530,Fix flaky remote_txn.sql test,"The remote_txn.sql test used to be consistently flaky in CI runs. The RegisterXactCallback function in the test does the following on the access node: 1) Send an async command to the remote node 2) Invoke the eventcallback, for example: eventcallback(DTXN_EVENT_WAIT_COMMIT) which kills the remote node backend 3) Wait for async responses for (1) from remote node and expect it to fail due to the killing of the remote backend in (2) above. The above works most of the times but it's possible that (1) completes successfully and returns successful async response via (3) before the remote backend gets killed via (2). To make things more predictable, we now use the DEBUG_WAITPOINT infrastructure. To ensure predictability, we want to kill the remote backend when it's in the midst of processing the transaction. To ensure that the access node sets the event handler and then takes an exclusive lock on the ""remote_conn_xact_end"" advisory lock via ""debug_waitpoint_enable"" function The ""RegisterXactCallback"" callback on the remote backend tries to take this same advisory lock in shared mode and waits. This allows the event handler enough time to kill this remote backend at the right time. The DEBUG_WAITPOINT infrastructure was modified to provide a ""blocking"" and ""retry loop"" based implementation to take the advisory lock. This is needed in cases where this function gets called from deep down inside a transaction where interrupts are not being served currently. The ""retry loop"" part also returns with a WARNING if the ""retry_count"" is exhausted. We could provide an implementation later which decides to retry indefinitely but the present implementation is good enough for the current requirements. " 931da9a6566441ad7994c3e818af953014a5047e,Erik Nordström,Thu Feb 25 17:07:19 2021 +0100,Refactor and harden size and stats functions,"Fix a number of issues with size and stats functions: * Return `0` size instead of `NULL` in several functions when hypertables have no chunks (e.g., `hypertable_size`, `hypertable_detailed_size`). * Return `NULL` when functions are called on non-hypertables instead of simply failing with generic error `query returned no rows`. * Include size of ""root"" hypertable, which can have non-zero size indexes and other objects even if the root table holds no data. * Make `hypertable_detailed_size` include one additional row for storage size of objects on the access node. While the access node stores no data, the empty hypertable may still take up some disk space. * Improve test coverage for all size utility functions. In particular, add tests on regular tables as well as empty and compressed hypertables. * Several size utility functions that were defined as `PL/pgSQL` functions have been converted to simple `SQL` functions since they ran only a single SQL query. The `dist_util` test is moved to the solo test group because, otherwise, it gives different size output when run in parallel vs. in isolation. Fixes #2871 " 8553323b5e4ff8130282a19192265f89af54c9c0,Sven Klemm,Mon Mar 22 01:03:48 2021 +0100,Replace hardcoded DecompressChunk node name,This patch fixes a typo in DecompressChunk comment and also changes the initialization function to use the node name from the CustomScanMethods struct instead of hardcoding it. 862fab76c3476b1d4b05a5845dea2003725c715d,Mats Kindahl,Mon Mar 22 13:02:06 2021 +0100,Updates based on review comments, 9aceb2e6c52dcaa9787b9ef3144ac98da94454fd,Mats Kindahl,Mon Mar 22 09:03:42 2021 +0100,Commit end transaction for CREATE INDEX,"When executing `CREATE INDEX` with `transaction_per_chunk` one transaction is created at the start to mark the index as invalid, a session lock is taken to prevent competing `DROP INDEX` from modifying the indexes, then each chunk is updated inside a separate transaction, one transaction is created to mark the index as valid, and lastly the session lock is released. However, the last transaction was not committed and left open, so a competing `DROP INDEX` would try to read index status after the session lock was released, resulting in a concurrent access of the index row from the `CREATE INDEX`. This commit fixes that by committing the transaction before releasing the session lock (and starting another transaction to match the end-of-statement commit that is done later). Fixes #3021 " 5bde51a3d9be356bd454d4426c5e2f82888c41f8,Mats Kindahl,Fri Mar 19 12:48:04 2021 +0100,Re-enable CREATE/DROP INDEX isolation test,The flake in `multi_transaction_indexing` is because of a race between `CREATE INDEX` and `DROP INDEX` and this commit add a debug waitpoint so that we can reliably reproduce the race condition in the isolation test. 102064aab38ec5020acc818d049be9457de39361,Erik Nordström,Mon Mar 22 14:45:47 2021 +0100,Make data_fetcher solo test to fix failing tests,"The `data_fetcher` test currently causes tests to fail regularly when run in parallel. Therefore, this change moves `data_fetcher` to solo tests. " c5c24616f8577ff435bd7a0365d793a4543c155e,Sven Klemm,Mon Mar 15 15:03:07 2021 +0100,Switch CI to Ubuntu 20.04,Switch CI tasks currently running on Ubuntu 18.04 to 20.04 and also switch some tasks that dont require specific version because they run in docker containers to use ubuntu-latest instead. All tasks switched to ubuntu-latest are expected to be ubuntu-version agnostic because they run their actions in docker containers. f18d9bf38be8a3ca48e7e439d7767d540047fb26,Sven Klemm,Sun Mar 21 16:50:04 2021 +0100,Add mangle_path to ignored symbols,"When building timescaledb on Ubuntu 20.04 mangle_path is present as exported symbol in binaries, making the export prefix check fail. This patch changes the export prefix check to ignore mangle_path. " dcad0e2be15defc67bc67b169b6cbb90ca7a1198,Ruslan Fomkin,Mon Mar 22 09:41:35 2021 +0100,Add error detail for failed refresh policy,"Adds an error detail, which clarifies that only one continuous aggregate policy can be created per a continuous aggregate. It also reports the job id of the continuous aggregate policy, which already exists. " c09f7e425244d2a9569e9edcde67ee4c533d5f43,Dmitry Simonenko,Fri Mar 19 15:14:12 2021 +0300,Add script to test recovery using restore points, 0bc3f0b55a721b6d2bfb14ac194c36c146933f84,Mats Kindahl,Mon Mar 1 20:06:04 2021 +0100,Factor out repair test from update test,"In order to implement repair tests, changes are made to the `constraint_check` table to simulate a broken dependency, which requires the constraints on that table to be dropped. This means that the repair runs without constraints, and a bug in the update test could potentially not get caught. This commit fixes this by factoring out the repair tests from the update tests and run them as a separate pass. This means that the contraints are not dropped in the update tests and bugs there will be caught. In addition, some bash functions are factored out into a separate file to avoid duplication. " 80f72ddbbb33004238fc7b75ce607b7585838252,Mats Kindahl,Wed Mar 17 08:58:11 2021 +0100,Add declaration before code to guidelines,Declarations before code has been informally supported and we have several examples in the code. This commit adds that guideline to the coding guidelines. ed62df721f7e6a991a622cfce5248adc9568dd3e,Sven Klemm,Mon Mar 15 14:50:10 2021 +0100,Run update test in GMT timezone,Due to a validation bug involving interval validation and the current daylight savings time switch in the 1.7 releases the update tests that includes 1.7 versions is currently failing. This patch changes CI to run in GMT instead. 72c6114c683ceda0757cbae5de1ce811d33ffc40,Sven Klemm,Mon Mar 15 01:26:36 2021 +0100,Add update test for PG13, 51b950552422ed6b608b88f944a3d9ad7a44c0cf,Mats Kindahl,Thu Mar 11 14:23:50 2021 +0100,Disable test in multi_transaction_indexing,There is a race condition between `CREATE INDEX` and `DROP INDEX` so disabling that test in `multi_transaction_indexing`. 1d6b4983469dffc074171b27306e15939ff8695a,Sven Klemm,Fri Mar 5 17:57:53 2021 +0100,Run regresscheck-shared on appveyor, ba4ce686b4bcbaa0584691287f6be91b7406013c,Sven Klemm,Mon Mar 8 19:02:41 2021 +0100,Drop tablespaces after regress run,This patch makes TEST_PGPORT available in the environment pg_regress is run on and changes pg_regress to drop the test tablespaces at the end of the run to get a clean state for installchecklocal runs. 5cfec69cb50c8c01f867d5708c4f38a0424f5a37,Sven Klemm,Tue Mar 9 09:33:23 2021 +0100,Adjust telemetry_distributed database names,Change the telemetry_distributed test to not use hardcoded database names but instead derive it from test name. This patch also changes the test to drop any databases created by the test itself. a36099a686debec1beeec7bfd5dbe809f5878711,Mats Kindahl,Mon Mar 8 15:13:19 2021 +0100,Check CMake build type,"If an unsupported CMake build type is used, it is accepted but generate strange builds for the compiler. This commit adds a check that only one of the standard build types are used. " b6510a063a37682a7e8b8ece908a84a9c72411f7,Sven Klemm,Tue Mar 9 05:06:10 2021 +0100,Refactor regresscheck-t database creation,This patch removes hardcoded database names from regresscheck-t and replaces it with database names derived from the test name. This allows more of the regresscheck-t tests to be run in parallel leading to a roughly 15 speedup of regresscheck-t. This patch is also prerequisite for allowing repeatable installchecklocal runs. ,,,, 2623821a16b6ea378e382d826e9eafa08bf59519,Sven Klemm,Tue Mar 2 17:03:02 2021 +0100,Mark timescaledb as trusted extension,"PG13 adds the trusted attribute to extensions. Extensions marked as trusted do not require superuser privileges, but can be installed by a non-superuser with CREATE privilege in the database. The objects within the extension will be owned by the bootstrap superuser, but the extension itself will be owned by the calling user. https://github.com/postgres/postgres/commit/50fc694e " 231549184c5dca7e4454283c017d6162f440bf39,Mats Kindahl,Mon Mar 1 13:58:09 2021 +0100,Break out superuser operations in update tests,"In order to support smoke-testing with a single server, the update tests are refactored to not require a `postgres` user with full privileges. To support both running smoke tests and update tests, the following changes where made: - Database creation code was factored out of tests and is only executed for update tests. - Since the default for `docker_pgscript` was to use the `postgres` database and the database creation code also switched database to `single` as part of the exection, the default of `docker_pgscript` is now changed to `single`. - Parts of tests that changes roles during execution was removed since it is more suitable for a regression test. Some `GRANT` statements were kept for the update tests since they test that non-superuser privileges are propagated correctly. - Operations that require escalated privileges are factored out into support functions that execute the original code for update tests and are no-ops for smoke tests. - A dedicated `test_update_smoke` script was added that can run a smoke test against a single server. " 10b3da03ec207a153f01b569879608df4c50698a,Mats Kindahl,Mon Mar 1 14:48:44 2021 +0100,Break out multinode update tests,"This commit is part of a longer sequence to refactor the update tests for use with smoke testing. The multinode parts of the tests are broken out into a new version to make the logic simpler. - All multinode tests are now added to a new version, `v7` which allow us to create multinode objects only for versions that support multinode. - Test script `test_updates_pg11` and `test_updates_pg12` are changed to use `v6` for updates from versions preceeding 2.0 and `v7` for versions 2.0 and later. - Setup file `post.update.sql` is not needed any more since it was used to ""fix"" a pre-2.0 updated version to add data nodes so that it matched the clean setup. This is not necessary any more since v6 does not add data nodes for some versions and not for others. " f0675be2e213ebd67b8b6413bb8a8b4e786c5afd,Joshua Lockerman,Wed Mar 3 11:27:34 2021 -0500,Add timescale_analytics to telemetry,"Much like the promscale before it, this will allow us to track usage of the timescale_analytics extension. " 152bebe027c71438474326fa427627e2eff19ea0,Nikhil,Fri Feb 26 14:39:36 2021 +0530,Fix crash due to txns in emit_log_hook_callback,"A weird crash was coming up while running the bgw_db_scheduler test while using log_min_messages with DEBUG3 setting. So what's happening here is that we register a RegisterXactCallback callback which gets called at post-commit time. This remote_connection_xact_end callback was issuing a debug3 level message. As part of its processing, the emit_log_hook_callback function was trying to write this message to a catalog table via a transaction. Obviously trying to start a transaction at this post-commit stage will error out and leads to weird behavior and crashes. In general it's not a good idea to have emit_log_hook_callback functions start transactions because messages can be emitted at any point of a transaction cycle, but we have to live with the current code for now. The fix is to reset the emit_log_hook in these xact_end callback functions. Additionally the emit_log_hook_callback function is only interested in elog/ereport messages with elevel LOG or above. So add that check in the emit_log_hook_callback as well. " 38493e9ab7d00beb110af5d35156151962235490,Mats Kindahl,Mon Mar 1 14:28:56 2021 +0100,Refactor update tests for smoke testing,"In order to support smoke-testing with a single server, the update tests are refactored. This is part of a longer sequence of changes to support both running smoke tests and update tests and contain the following changes: - All timestamp columns now have `not null` added to avoid spamming the output file and help finding the real issue. - Extension of dump files are changed from `sql` to `dump` since they use the custom format and are not SQL files. " 67d840f70ee7b150ac3ce987ed70aa4f7d0261c6,Ruslan Fomkin,Fri Feb 26 11:18:59 2021 +0100,Explain commit message requirement in contribution,"Since we require that commit messages follow seven rules of writing good commit messages and include GitHub issues number, this commit clarifies these requirements in the contribution. " 3723104c68f41772ccae7196cf9a66cb9fbe3c69,Sven Klemm,Mon Mar 1 08:01:16 2021 +0100,Don't use deprecated clientcert option for PG12+,The pg_hba option clientcert=1 is deprecated for PG12+ and removed completely in PG14. 6d6c55e6c6a336aad5be3c2967990a793d9abfc8,Kevin Bowling,Mon Jan 11 19:10:42 2021 -0700,Annotate fallthrough with TS_FALLTHROUGH, a947b7e0dfd1fd8ccf1fc8f833bad7e59b8281fd,Sven Klemm,Fri Feb 26 20:53:24 2021 +0100,Fix segfault in job_config_check for cagg,When mat_hypertable_id in the config JSON of job was altered to an invalid hypertable id the alter_job would segfault. Fixes #2980 6f4cfc8886f2d5adb8c2bf36860a045d5eba3aa7,Sven Klemm,Sat Feb 27 16:24:12 2021 +0100,Add missing ORDER BY to flaky tests,"Add missing ORDER BY clause to queries in histogram_test, compression_hypertable, and continuous_aggs_refresh test. " e31a0f116e3aab14e6762b79e1558b9f47b31827,Nikhil,Fri Feb 26 14:07:03 2021 +0530,Fix mismatched dsm_attach/dsm_detach calls,The above issue was encountered while testing bgw_db_scheduler.sql. A backend is allowed to only do one dsm_attach call to attach to any given dynamic shared memory segment (it needs to detach before calling attach again). The code in params_open_wrapper took care of calling it once. However the code in params_close_wrapper was doing an unconditional dsm_detach everytime which can cause the dsm refcnt to go down unnecessarily and was causing weird crashes The fix is to track when we should be calling params_close_wrapper instead of doing an unconditional detach always. 91569898539b3c79dbf087294e2d224f938737c0,Sven Klemm,Mon Feb 22 18:36:27 2021 +0100,Add PG13 to APT and RPM package tests, 6a1c81b63ef3aeb2235697443e268fde62849f9c,Dmitry Simonenko,Thu Feb 25 15:26:28 2021 +0300,Add distributed restore point functionality,This change adds create_distributed_restore_point() function which allows to create recovery restore point across data nodes. Fix #2846 2e6fc7425408d0c34229f785e6a78ef114294002,Sven Klemm,Tue Feb 23 00:40:35 2021 +0100,Fix index creation for hypertables with dropped columns,The index creation code would take the IndexInfo from the hypertable and adjust it for the chunk but wouldnt reset IndexInfo for each individual chunk leading to errors when the hypertable has dropped columns. Fixes #2504 d21b1c2ba79d5512b5ba091651f22c856d1b7af0,Sven Klemm,Tue Feb 16 13:21:12 2021 +0100,Fix off by 4 error in histogram deserialize,"In the histogram deserialization function the StringInfoData's pointer is set to VARDATA(...) which does not include the varlen header, while the length is set to VARSIZE(...) which does include the header's length. Fixes #2684 " 22870cfec6785493d7b4cbb67f50dfef77b85921,Sven Klemm,Mon Feb 22 18:42:09 2021 +0100,Add 2.1.0 to update test scripts, a38b564218c32efaa8bef330eda1a3f308e26649,Nikhil,Tue Feb 23 17:10:22 2021 +0530,Clearer error messages for ON CONFLICT DO [NOTHING|UPDATE] scenarios,"Our current support is limited to ""ON CONFLICT DO NOTHING"" without specifying any inference index columns. While we do error out in the other cases, the error messages do not convey things clearly. So rework them. " 7f9077dc8ca597ee091fcf83eb348ece792ca2f6,Ruslan Fomkin,Tue Feb 23 11:22:09 2021 +0100,Do not notify Slack if in PR,"Notifying a Slack channel on regression test failures fails in a fork, which is common for PRs. The failed Slack error is confusing, since it disturbs from checking actual regression test failure. Furthermore, notifying Slack is not needed in the case of PRs as developers rely on GitHub's GUI around PRs. This commit fixes GH Actions workflows to not notify Slack on PRs. " 06593a1c244a3105ef600e4b89b83d8455d3e61f,Sven Klemm,Thu Feb 18 17:34:32 2021 +0100,Release 2.1.0,"This release adds major new features since the 2.0.2 release. We deem it moderate priority for upgrading. This release adds the long-awaited support for PostgreSQL 13 to TimescaleDB. The minimum required PostgreSQL 13 version is 13.2 due to a security vulnerability affecting TimescaleDB functionality present in earlier versions of PostgreSQL 13. This release also relaxes some restrictions for compressed hypertables; namely, TimescaleDB now supports adding columns to compressed hypertables and renaming columns of compressed hypertables. **Major Features** * #2779 Add support for PostgreSQL 13 **Minor features** * #2736 Support adding columns to hypertables with compression enabled * #2909 Support renaming columns of hypertables with compression enabled " ab736fda904eb30b015aa10d530211f34156d0e3,Sven Klemm,Sun Feb 21 17:33:17 2021 +0100,Add 2.0.2 to update test scripts, 9ddf375fd5619477223b0f65aaa4f96645a2ab26,Erik Nordström,Thu Feb 18 10:41:10 2021 +0100,Release 2.0.2,"This maintenance release contains bugfixes since the 2.0.1 release. We deem it high priority for upgrading. The bug fixes in this release address issues with joins, the status of background jobs, and disabling compression. It also includes enhancements to continuous aggregates, including improved validation of policies and optimizations for faster refreshes when there are a lot of invalidations. **Minor features** * #2926 Optimize cagg refresh for small invalidations **Bugfixes** * #2850 Set status for backend in background jobs * #2883 Fix join qual propagation for nested joins * #2884 Add GUC to control join qual propagation * #2885 Fix compressed chunk check when disabling compression * #2908 Fix changing column type of clustered hypertables * #2942 Validate continuous aggregate policy **Thanks** * @zeeshanshabbir93 for reporting the issue with full outer joins * @Antiarchitect for reporting the issue with slow refreshes of * @diego-hermida for reporting the issue about being unable to disable compression * @mtin for reporting the issue about wrong job status " 5be6a3e4e9fdab44e1a95978c0297b389bcd6bfd,gayyappan,Thu Feb 4 17:30:34 2021 -0500,Support column rename for hypertables with compression enabled,ALTER TABLE RENAME TO is now supported for hypertables that have compression enabled. Note: Column renaming is not supported for distributed hypertables. So this will not work on distributed hypertables that have compression enabled. cc287f966d8ee3356b2267a560105aab2457957e,Erik Nordström,Wed Feb 10 16:59:35 2021 +0100,Limit number of materializations per cagg refresh,"When there are many small (e.g., single timestamp) invalidations that cannot be merged despite expanding invalidations to full buckets (e.g., invalidations are spread across every second bucket in the worst case), it might no longer be beneficial to materialize every invalidation separately. Instead, this change adds a threshold for the number of invalidations used by the refresh (currently 10 by default) above which invalidations are merged into one range based on the lowest and greatest invalidated time value. The limit can be controlled by an anonymous session variable for debugging and tweaking purposes. It might be considered for promotion to an official GUC in the future. Fixes #2867 " d4a93c7745dd4040f38111f07e3a1fb9320bcf69,Erik Nordström,Fri Feb 5 14:51:08 2021 +0100,Optimize cagg refresh for single row inserts,The refreshing of a continuous aggregate is slow when many small invalidations are generated by frequent single row insert backfills. This change adds an optimization that merges small invalidations by first expanding invalidations to full bucket boundaries. There is really no reason to maintain invalidations that aren't covering full buckets since refresh windows are already aligned to buckets anyway. Fixes #2867 7a711fc37092aebc714c39b41072fd08c7adbab1,gayyappan,Tue Feb 16 17:34:24 2021 -0500,Adjust eclass_indexes for compressed chunks for PG13,"In PG13, eclass_indexes are used to traverse the equivalence classes. This should be set correctly for compressed chunks. " 82a30ae420890f8361c6c8f57ce77d3974c80336,Erik Nordström,Thu Feb 11 13:16:29 2021 +0100,Validate continuous aggregate policy,"This change adds validation of the settings for a continuous aggregate policy when the policy is created. Previously it was possible to create policies that would either fail at runtime or never refresh anything due to bad configuration. In particular, the refresh window (start and end offsets for refreshing) must now be at least two buckets in size or an error is generated when the policy is created. The policy must cover at least two buckets to ensure that at least one bucket is refreshed when the policy runs, since it is unlikely that the policy runs at a time that is perfectly aligned with the beginning of a bucket. Note that it is still possible to create policies that might not refresh anything depending on the time when it runs. For instance, if the ""current"" time is close to the minimum allowed time value, the refresh window can lag enough to fall outside the valid time range (e.g., the end offset is big enough to push the window outside the valid time range). As time moves on, the window would eventually move into the valid time range, however. Fixes #2929 " ff7fb0aedf11242d3717385c0c12698dd6ae3ae3,Erik Nordström,Thu Feb 11 09:13:27 2021 +0100,Fail continuous aggregate refresh smaller than one bucket,"When refreshing a continuous aggregate, we only materialize the buckets that are fully enclosed by the refresh window. Therefore, we should generate an error if the window is smaller than one bucket. " 3b42300407a89b8de6d788b5072efa73d7df2760,Sven Klemm,Thu Feb 18 04:53:27 2021 +0100,Add string mapping for CONNECTION_CHECK_TARGET,PG13 adds a new connection status when checking for a proper target connection. https://github.com/postgres/postgres/commit/b438e7e7a1c58e0c20b5f46e73cbd713e8033c69 2fb7034c8fa8ca9747b9ec6416aa857dc566d950,Dmitry Simonenko,Thu Feb 18 14:56:35 2021 +0300,Add 1.7.5 to update test scripts, 39cb98bcc4eb76e64db981ded9c4d69962967b40,Sven Klemm,Thu Feb 18 01:46:08 2021 +0100,Fix sanitizer test,The sanitizer test is running on PG11.1 where the cluster test is expected to fail. This patch changes the sanitizer test to ignore the test result of the cluster test. 00b84d2e614cfb38fd92dd25754f6bed49fd8aa2,Sven Klemm,Wed Feb 17 16:11:43 2021 +0100,Disable strict-overflow check for gcc < 8,The strict-overflow check on gcc < 8 produces false positives leading to build failures when compiling with -Werror 240df3262be2af3c0460211a5ca90e854500c2e4,Sven Klemm,Sat Feb 13 00:16:06 2021 +0100,"Add PG13 to 32bit tests, coverity and cron tests", 74d3c184b65926b03de75411eb4c74004076540a,Sven Klemm,Tue Feb 16 13:56:32 2021 +0100,Adjust windows ci to use explicit postgres package version,Installing just postgresql13 with chocolatey would result in 13.1 being installed which would fail to build since we require 13.2 9b5b2dc67a97af96ebdfe800289ef787fd8ae00b,Sven Klemm,Sun Feb 14 03:55:21 2021 +0100,Set minimum required PG13 version to 13.2,The minimum required PG13 version is 13.2 because we require a bugfix only present in that version otherwise decompression might not work properly for certain queries. 02020d568ad1089236bb398802a22ac32221720e,Sven Klemm,Mon Feb 15 12:35:43 2021 +0100,Fix detection of stringop truncation compiler support,When checking for -Wno-stringop-truncation gcc reports success even though it doesn't support the flag. This patch changes the check to check for actual flag instead of the flag that turns it off which produces the correct result. 32bc42adfb0ae988fb09458a8941ba12d14f7455,Nikhil,Tue Feb 9 17:56:33 2021 +0530,Add test to check PARALLEL VACUUM functionality with PG13,"PG13 has introduced the parallel vacuum functionality in which if the table has two or more indexes (with sizes beyond 512KB and enough number of parallel worker processes), then the vacuum on those indexes can be done in parallel. " d43e54a03901e8738334a5b5f7d795a2a3c7e12a,Erik Nordström,Mon Feb 15 14:41:01 2021 +0100,Fix flaky distributed hypertable test on PG13,"The `dist_hypertable` test includes an `EXPLAIN` with the `BUFFERS` option, which is flaky on PG13 due to new planning-level output. This change turns off the `BUFFERS` option in order to mute the output that differs between test runs. " 5d6c8f988cd855b10bf361452b657f7c61333cd7,Mats Kindahl,Fri Feb 12 15:08:15 2021 +0100,Add row security tests for PG13,"Copying the test file for row security checks from PG12 to PG13 and creating a new output file. The test is identical to the PG12 test, but the PG11 test is different, so we are not using a generated test. Subtask of #2779 " d6cd54b5a774d9a34aa61c3799550c87ed3907ef,Sven Klemm,Thu Feb 11 14:11:23 2021 +0100,Bump postgres versions used in CI,"Run tests on PG 11.11, 12.6 and 13.2 " 9c7b02470525cb03449ec67c9250e125f75b3268,Sven Klemm,Wed Feb 10 13:06:33 2021 +0100,Run regresscheck-t on PG13,This patch enables running regresscheck-t on PG13 a5467fcdec16adfb074a20740b49fd9b52701e29,Sven Klemm,Fri Feb 12 21:16:25 2021 +0100,Refactor setting RTE permission fields for cagg creation,Postgres 13 changes markVarForSelectPriv making it segfault when called with NULL state. This patch changes the cagg creation code to set the permissions directly instead of calling markVarForSelectPriv. 9fbc19e6c90e5025b89ceb91e27c4ad60b2a2a93,Sven Klemm,Fri Feb 12 21:03:42 2021 +0100,Filter out incremental sort memory usage,EXPLAIN ANALYZE output of incremental sort nodes contains details about memory usage which is not stable and might lead to flaky tests. This patch filters out memory usage from test output. 8cbba11cc80c7fb0bda50c63a4275bf7b4a62ed0,Sven Klemm,Tue Feb 2 19:15:18 2021 +0100,Make remote_copy test pg version specific,PG13 changes constraint error messages to include the relation name making the test output different to previous pg versions. b354e03082f796acb6da6f71a42126d59a47e65c,Sven Klemm,Fri Feb 12 20:50:05 2021 +0100,Adjust ChunkAppend limit pushdown,"Postgres 13 added a new incremental sort node which can lead to subpath pathkeys not matching sort_pathkeys, leading to a plan where LIMIT pushdown is not safe. " 0d99592b203f4d712d481d131bea6835ec8ee0da,gayyappan,Wed Feb 10 12:30:40 2021 -0500,Add tests for ALTER COLUMN .. DROP EXPRESSION,This PR adds tests for dropping defaults for generated columns. This is done by executing ALTER TABLE .. ALTER COLUMN .. DROP EXPRESSION This feature is introduced in PG13. 6d679a14d05b46a46d0a2a0804d671e1b86569cc,Ruslan Fomkin,Thu Feb 11 13:48:59 2021 +0100,Release 1.7.5,"This maintenance release contains bugfixes since the 1.7.4 release. Most of these fixes were backported from the 2.0.0 and 2.0.1 releases. We deem it high priority for upgrading for users on TimescaleDB 1.7.4 or previous versions. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables, and when upgrading from previous versions. **Bugfixes** * #2502 Replace check function when updating * #2558 Repair dimension slice table on update * #2619 Fix segfault in decompress_chunk for chunks with dropped columns * #2664 Fix support for complex aggregate expression * #2800 Lock dimension slices when creating new chunk * #2860 Fix projection in ChunkAppend nodes * #2865 Apply volatile function quals at decompresschunk * #2851 Fix nested loop joins that involve compressed chunks * #2868 Fix corruption in gapfill plan * #2883 Fix join qual propagation for nested joins * #2885 Fix compressed chunk check when disabling compression * #2920 Fix repair in update scripts **Thanks** * @akamensky for reporting several issues including segfaults after version update * @alex88 for reporting an issue with joined hypertables * @dhodyn for reporting an issue when joining compressed chunks * @diego-hermida for reporting an issue with disabling compression * @Netskeh for reporting bug on time_bucket problem in continuous aggregates * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort * @zeeshanshabbir93 for reporting an issue with joins " b95e93a6510d04342f71eff9ffa43212d89dd901,Sven Klemm,Fri Jan 15 17:45:01 2021 +0100,Run regresscheck-shared on PG13,This patch enables the regresscheck-shared testsuite to run on PG13 2bf69fd4ca2da6a66a8d18642029df4e217a6e99,Nikhil,Wed Feb 10 14:09:19 2021 +0530,"Explicitly check for ""-Wno-stringop-truncation"" compiler flag","Older versions of GCC do not have this flag, so check for it explicitly before adding it. Fix by Mats Kindahl " 63a66524dd8638516d382ed0efc60c75fb17250e,Mats Kindahl,Fri Jan 29 13:44:26 2021 +0100,Change bootstrap and CMake defaults,"In contrast to the default for CMake files, `Release` is used as the default in `CMakeFiles.txt` which causes the `bootstrap` script to do a release build with development flags, in particular `-Werror`. Since warnings are triggered in a release build, this cause the build to fail while a debug build works fine. This commit fixes this by removing the `-Werror` flag (by setting `WARNINGS_AS_ERRORS` to `OFF`) on anything that is not a debug build and also disable the warnings that (currently) trigger the warnings in the release build. The commit also changes some of the GitHub workflows to run without `WARNINGS_AS_ERRORS` since it should always work without this option regardless of build type (on release build this should be disabled, on debug builds this should be enabled). But it is set to `ON` for the full release and debug builds to ensure that we do not generate any warnings, which will capture new surfacing warnings. Fixes #2770 " 1461244bdb906d76c9b15b8de87360aee3498902,Sven Klemm,Fri Jan 29 12:53:37 2021 +0100,Fix join propagation for nested joins,Join propagation would propagate join quals for nested joins even when it was not safe to do so leading to wrong query results. fa42b973f1974aa0eee91db78338cb0798a4612e,Sven Klemm,Mon Feb 8 09:09:03 2021 +0100,Fix typo in CI config,When adding the bugfix for clustered hypertables a typo was introduced making the PG12.0 test run with PG11 settings. d823987fe9b6186dcba6ea86de03ba341331bbbc,Sven Klemm,Fri Feb 5 19:15:04 2021 +0100,Fix changing column type of clustered hypertables,When changing the column type of a column that is part of an index that is being clustered on with either postgres CLUSTER or reorder the alter type operation would fail with a segfault because it couldn't lookup the index. 65df0fe71a482855bb3192af1635d19656d9d0b7,Sven Klemm,Fri Jan 29 18:23:14 2021 +0100,Add GUC to control join qual propagation,This patch adds a `enable_qual_propagation` GUC to control propagation of JOIN quals. Since there have been a few instances where JOIN qual propagation was too aggressive this GUC can be used to disable qual propagation. 954c775275123d22e981261ad1043a287b6b432c,Mats Kindahl,Thu Jan 21 15:30:21 2021 +0100,Set status for backend in background jobs,"The status of background workers is recorded in PostgreSQL using the function `pgstat_record_activity` which allow the status of the background worker to be inspected using the `status` field in `pg_stat_activity`. This is used by the TimescaleDB `job_stats` view to show if a job is running, scheduled, or paused. However, the activity was never recorded in the code and hence the field was always NULL. Since this was the default for the `job_status` field of `timescaledb_information.job_stats` it would always show as `Scheduled`. This commit fixes this by adding calls to `pgstat_record_activity` at suitable locations. Fixes #2831 " e3c6725ad1471995902328fb512b989599f7ec99,Sven Klemm,Sat Jan 30 01:04:39 2021 +0100,Fix compressed chunk check when disabling compression,The check for existence of compressed chunks when disabling compression would not ignore dropped chunks making it impossible to disable compression on hypertables with continuous aggregates that had dropped chunks. This patch ignores dropped chunks in this check and also sets compressed_chunk_id to NULL in the metadata for deleted chunks. 04d614e32ecf24339110da1eb8ea3c7512be3d54,Sven Klemm,Mon Jan 18 09:02:02 2021 +0100,Run regresscheck on PG13, b2849d8f347ced6115369e80b3c9671d1134a6b8,Sven Klemm,Tue Jan 26 03:59:39 2021 +0100,Make cluster test pg version specific,The order of relations of cluster is different on PG13 compared to previous versions. 8b744f2cc49bf6559cf3a2584acd79fcbff30a77,Sven Klemm,Mon Jan 18 12:11:22 2021 +0100,Make plan_hypertable_inline test output PG version specific,PG13 changes the relation name aliasing in EXPLAIN output making explain output different from previous PG versions. b5a9d33ef3ffeb34c270e3160c47799c2a807c49,Sven Klemm,Mon Jan 18 11:48:09 2021 +0100,Make ddl test output PG version specific,PG13 adds the relation to constraint violation error messages making the test output different from previous PG versions. 9d0f38b12fe9c6efe65ee37c4748b95fac7a083b,Sven Klemm,Mon Jan 18 11:08:40 2021 +0100,Make copy test output PG version specific,PG13 changes the error message on constraint validation to include the relation making the error message different from previous PG versions. 0d63c79049e5d756a57ca9f99632538ce428ade2,Sven Klemm,Mon Jan 18 10:52:15 2021 +0100,Make partitionwise test output PG version specific,PG13 changes relation alias naming in EXPLAIN output making the output different from previous versions. ed55018af859b89b5ccfd85edaa99f90949d4a32,Sven Klemm,Mon Jan 18 10:39:32 2021 +0100,Make delete test output PG version specific,PG13 changes relation alias naming in EXPLAIN output making the output different from previous versions. c1ee623e335760fc6518412dbe8fe737f4d605d1,Sven Klemm,Mon Jan 18 10:29:22 2021 +0100,Make update test version specific,PG13 changes relation alias naming in EXPLAIN output making the output different from previous versions. 8d226a533167cc125034ad0dc331a04bab775b6c,Sven Klemm,Mon Jan 18 09:42:35 2021 +0100,Make constraint test output version specific,PG13 adds the relation name to constraint errors making the output differ from previous PG versions. 99e450d2a39ae6c5662e88e98c1aac9a1007e0bb,Sven Klemm,Thu Jan 28 18:43:15 2021 +0100,Add 2.0.1 to update test scripts, 636c8bbdae809c8f2a7231fc4f0755a222cf0384,Sven Klemm,Wed Jan 27 16:57:16 2021 +0100,Release 2.0.1,"This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * #2772 Always validate existing database and extension * #2780 Fix config enum entries for remote data fetcher * #2806 Add check for dropped chunk on update * #2828 Improve cagg watermark caching * #2842 Do not mark job as started when setting next_start field * #2845 Fix continuous aggregate privileges during upgrade * #2851 Fix nested loop joins that involve compressed chunks * #2860 Fix projection in ChunkAppend nodes * #2861 Remove compression stat update from update script * #2865 Apply volatile function quals at decompresschunk node * #2866 Avoid partitionwise planning of partialize_agg * #2868 Fix corruption in gapfill plan * #2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort " c71632542c03b0e25d7127e1297f34bd6f663105,Mats Kindahl,Thu Jan 28 10:55:07 2021 +0100,Fix repair in update scripts,"The commit fixes two bugs in the repair scripts that could prevent an update in rare circumstances. For the 1.7.1--1.7.2 repair script: if there were several missing dimension slices in different hypertables with the same column name, the repair script would be confused on what constraint had what type and generate an error. For the 2.0.0-rc1--2.0.0-rc2 repair script: if a partition constraint was broken, it would generate an error rather than repairing the dimension slices because BIGINT_MIN would be cast to a double float and then an attempt would be made to cast it back to bigint, causing an overflow error. This commit also creates an update repair test that breaks a few tables for pre-2.0 versions to ensure that the repair script actually fixes them. The integrity check for the update tests already contain a check that dimension slices are valid, so there is no need to add a test for that. This commit adds an extra dimension in the workflow to test updates with repair and run that separately. It also changes the update test scripts to by default run without repair tests and add the additional option `-r` for running repair tests in addition to the normal tests. Fixes #2824 " 0e86bbe5e15df8147e3df80147591b7440427c9b,Erik Nordström,Thu Jan 28 09:45:50 2021 +0100,Fix partitionwise agg crash due to uninitialized memory,"When building the partition information for hypertables, the memory for partition bound information wasn't properly initialized. The uninitialized memory could make the planner believe it could plan an ordered append (PostgreSQL's built-in version) when the contents of the memory happened to have the ""right"" value (specifically the `strategy` char field of the partition bound info). This change fixes the crash by properly initializing the partition bound info so that PostgreSQL's ordered append plans are not triggered on hypertables. Due to the randomness of the uninitialized memory, it isn't possible to reproduce the crash reliably and create a test for it. But the crash was manually reproduced by initializing the `strategy` field of the partition bound info to a value that triggered the crash in the same place reported by the issue below. Fixes #2873 " f98337cd3c848cab09ee6deb60b635a71a1c2cf0,Ruslan Fomkin,Wed Jan 27 14:17:47 2021 +0100,Avoid partitionwise planning of partialize_agg,"partialize_agg is an internal function, which serializes partial aggregate results. It is used to prepare partials for materialization in continuous aggregates and partial results on data nodes in distributed query execution. paritalize_agg doesn't expect push down of aggregates, which happens when partitionwise aggregate is enabled, and produces a query plan, which either crashes on assert during execution or produces incorrect result. This fix avoids adding partition info if the function is present in the query. This can be seen as a work around and it is good to fix planning of partialize_agg in the case of pushed down aggregates. This commit also contains few minor fixes of readability of comments and code around the changes. Fixes #2849 and fixes #2858 " 36c1cd849a2ede6d1d3c08a18caaf3648d725bdb,Erik Nordström,Wed Jan 27 16:11:55 2021 +0100,Fix corruption in gapfill plan,"This change fixes a bug with gapfill that caused certain query plans to fail with the error ""could not find pathkey item to sort"". This was caused by a corruption in the query plan which happened as a result of removing the last two arguments to the `time_bucket_gapfill` function expression during query planning. Since the function expression was modified via a reference to the original query plan tree, it affected also the expression in the target list. When the planner couldn't match the target list with the corresponding equivalence member (which still included the two removed arguments), the error was generated. The original reason for removing the two arguments was to avoid passing them on to `time_bucket`, which is called internally by `time_bucket_gapfill`. However, the last to arguments aren't passed on anyway, so it isn't necessary to modify the original argument list. Fixes #2232 " 87a190afcd35e266bdce71eff3eab0ab5b7e8413,gayyappan,Thu Jan 21 14:41:43 2021 -0500,Fix nested loop joins for compressed chunk,"This change adds equivalence class members for non segment by attributes of compressed tables. Before the change introduced by this PR, we added equivalence members to the compressed chunk only for segment by columns. If we have a join condition like: hyper.segment_col = A.col1 and hyper.compressed_attr = A.col2 The compressed rel had EM only for compressed_chunk.segment_col = A.col1 When a parameterized path is created for the compressed chunk, it has only 1 clause (corresponding to the segment by EM ) in its list of ppi_clauses. The other clause is missing since it did not have an EM. A parameterized path is expected to carry all the clauses that come from the outer_rel for the join (table A in this case).Since the ppi_clauses did not have all the clauses that come from table A, we ended up missing a join condition (for table A) when we had a parameterized index path for the compressed chunk. Fixes #2673 " 5e73ff52b75f0733c363f1b9b07a892d1e92ed3f,gayyappan,Tue Jan 26 17:10:12 2021 -0500,Apply volatile function quals at decompresschunk,Volatile functions that are in chunk's baserestrictinfo list were not correctly handled when we inserted a DecompressChunk node. This PR adds these quals as a filter to the DecompressChunk node. Fixes #2864 86f8e30ae00d821848e337cb43fb1b480b866d5d,Erik Nordström,Wed Jan 20 11:26:11 2021 +0100,Fix continuous aggregate privileges during update,"Copy ACL privileges (grants) from the query view (user-facing object) to the internal objects (e.g., materialized hypertable, direct view, and partial view) when updating the extension to the new version. A previous change added such propagation of privileges when executing `GRANT` statements, but didn't apply it retrospectively to the internal objects of existing continuous aggregates. Having the right permissions on internal objects is also necessary for the watermark function used by real-time aggregation since it queries the materialized hypertable directly. The update script copies the ACL information from the user-facing view of every continuous aggregate to its internal objects (including the materialized hypertable and its chunks). This is done by direct insert into `pg_class` instead of executing a `GRANT` statement in the update script, since the latter will record the grant/ACL as an init privilege (i.e., the system believes the GRANT is for an extension object). The init privilege will prevent this ACL from being included in future dump files, since `pg_dump` only includes non-init privileges as it believes such privileges will be recreated with the extension. Fixes #2825 " 126f1c815f8a5ca18a587d499b46102dc614e37b,Mats Kindahl,Mon Jan 25 13:34:30 2021 +0100,Remove compression stat update from update script,"Updating the compression stats can potentially be a very big task locking many tables, so we remove it from the update script. We also remove the check from the update test. Fixes #2859 " dc3561dd5214d124d7468c76562a73f89a6ed66b,Sven Klemm,Mon Jan 25 19:19:23 2021 +0100,Improve CI coredump detection,Occasionally tests would segfault but the coredump would not be uploaded as artefact because the coredump was still in progress. This patch changes CI to wait for potential in progress coredumps to upload coredumps more reliably. 302de38b7ee1c27783d1f52058507796a7f7604a,Sven Klemm,Mon Jan 25 12:13:38 2021 +0100,Fix projection in ChunkAppend nodes,"When the targetlist of a ChunkAppend node was modified after plan creation the ChunkAppend node would not properly project because input and output targetlist where pointing to the same list, so any modification to the output targetlist would carry over to the input targetlist. This patch decouples input targetlist from output targetlist to prevent this from happening. " b1dc0305d3201c6f1e1cd41efb3f59e940665d30,Mats Kindahl,Fri Jan 22 13:14:59 2021 +0100,Fix path to dist_gapfill,The `TEST_OUTPUT_DIR` for shared tests `dist_gapfill` is incorrect inside the test file (when called from `pg_regress`) because `TEST_OUTPUT_DIR` is set to the parent directory rather than the subdirectory. This commit fixes the paths in `dist_gapfill`. 4cd95b970a001d04fa0fafd970d15925798941d1,Sven Klemm,Fri Jan 22 11:37:55 2021 +0100,Fix appveyor failures,The postgres:12.5-alpine image got changed to the latest alpine version which has postgres 13.1 as the packaged version so the postgres-dev package is no longer available for 12.5 in that image. This patch switches the docker image used to alpine 3.12 which has the required packages for 12.5. 41cfabd37c597e71dd682ed88aec764ef1909fb7,gayyappan,Mon Jan 18 15:50:24 2021 -0500,Do not mark job as started when setting next_start,Fix mark_start field when inserting job stat record. The mark_start field should be set to false when a new job stat record is created. Fixes #2793 472df0502431796292afb010f4e6a0df73036294,Erik Nordström,Thu Jan 14 12:30:12 2021 +0100,Improve cagg watermark caching,"The internal function `cagg_watermark` returns the last/max bucket in a continuous aggregate and is used to get the point where to union the raw and materialized data when using real-time aggregation. Since the function is marked `STABLE` the planner should be able to constify the function so that it need not be called repeatedly during execution. However, this optimization is not guaranteed and the function might be constified many times during planning if it occurs, e.g., as an index scan condition on many chunks. This leads to slow queries due to repeated calls to the function. Previously, the watermark was cached in the function's `fn_extra` state, but this state doesn't survive all repeated calls of the function and doesn't help if the function occurs many times in the same plan. To make the performance more reliable, the watermark is now cached in a global variable that is cleared at transaction end, when the input argument (materialized hypertable queried) changes, or a new command is executed. This should guarantee that the watermark only needs to be looked up once per query. Fixes #2826 " 19d3912ff6721b30cda7729aef9ddf3fc0b345e5,Sven Klemm,Fri Jan 15 13:39:59 2021 +0100,Enable isolation tests for PG13,Enable isolation tests for PG13 in CI. Subsequent PRs will reenable further tests until all tests run against PG13. 23cd4098ef7dbc5c063c717a02b2b56ed68de493,Sven Klemm,Fri Jan 15 14:13:08 2021 +0100,Remove unreferenced steps from remote_create_chunk test,This patch removes unreferenced steps from the remote_create_chunk isolation test because PG13 will print warnings about unreferenced steps leading to test failures. 3afdf7cc725d089d0e30b07459681f87c2068041,Sven Klemm,Fri Jan 15 09:36:16 2021 +0100,Add pg13 to CI build matrix,Since the actual tests have not been adjusted for PG13 yet this only checks compiling against PG13. A subsequent PR will enable tests for PG13. 6c36e7d49de9508732b7db26f420a0df010071c7,Sven Klemm,Thu Sep 24 01:43:55 2020 +0200,Don't use lcons when adding EquivalenceClass,PG13 added a bitmap of EquivalenceClass indexes to RelOptInfo so any prepend to the list of EquivalenceClasses would invalidate the BitmapSet. https://github.com/postgres/postgres/commit/3373c71553 8bc2568f7d53730f7c9813841bef723133789d54,Sven Klemm,Fri Jan 15 10:29:11 2021 +0100,Adjust compression code to PG13 AlterTable changes,PG13 changes the signature of AlterTable so we switch to using AlterTableInternal instead. 5d3aeb28b039317719f6e287b7a4f0675ff826cc,Sven Klemm,Thu Sep 24 04:42:45 2020 +0200,Adjust hypertable expansion to PG13 changes,PG13 merges setup_append_rel_array into setup_simple_rel_arrays which we use to build the append_rel_array. https://github.com/postgres/postgres/commit/1661a40505 172098a6d94572726f96ed09aa0f8cfe65914907,Sven Klemm,Fri Oct 2 00:58:57 2020 +0200,Adjust copy to PG13 HEAP_INSERT_SKIP_WAL change,PG13 removes HEAP_INSERT_SKIP_WAL https://github.com/postgres/postgres/commit/c6b92041 df3359bb2bc3d81ca8a67983573aa058087e6b43,Dmitry Simonenko,Fri Jan 15 11:46:13 2021 +0300,Rename process_add_hypertable(),"This change renames the function to a more obvious name, regarding what it does: add_hypertable_to_process_args. " 69778c159c0811af6a7b80126a8a1fe8b60c572d,Sven Klemm,Wed Sep 30 01:47:01 2020 +0200,Adjust code to PG13 command completion tag changes,PG13 changes the representation of command completion tags to structs. https://github.com/postgres/postgres/commit/2f9661311b aee2da1346d84abe079f38fe7c67b9685bcd5771,fidomax,Mon Jan 11 14:16:06 2021 +0300,Fix for_each_cell_compat macro for PG13,"In PG 13: define for_each_cell(cell, lst, initcell) \ for (ForEachState cell##__state = for_each_cell_setup(lst, initcell); \ (cell##__state.l != NIL && \ cell##__state.i < cell##__state.l->length) ? \ (cell = &cell##__state.l->elements[cell##__state.i], true) : \ (cell = NULL, false); \ cell##__state.i++) Extra brackets is wrong. Trying to concatenate parameters with brackets generate an preprocessor error. " f649736f2f978d31704efa58e1d9f6a9e47fead1,gayyappan,Fri Dec 4 13:33:37 2020 -0500,Support ADD COLUMN for compressed hypertables,Support ALTER TABLE .. ADD COLUMN for hypertables with compressed chunks. 2c4584c71acf8ba20c414b028fa0ff0e88f474e6,Michael J. Freedman,Fri Jan 1 10:40:19 2021 -0500,Update copyright year to 2021, 79f8a07f52d48d09a69a0cca0d16187fa2c9f294,Sven Klemm,Sat Sep 26 02:57:38 2020 +0200,Adjust copy code to PG13 changes,PG13 adds a CmdType argument to ExecComputeStoredGenerated. https://github.com/postgres/postgres/commit/c6679e4fca bb7fbff9b34e80e85f7fabf0e06187af3e127c16,Sven Klemm,Sat Oct 3 15:53:19 2020 +0200,Adjust code to deparse_context changes,PG13 changes EXPLAIN to use Plan instead of PlanState as context. https://github.com/postgres/postgres/commit/6ef77cf46e 51745b35537ada8f2a8224dc06ad6da78beec154,Mats Kindahl,Tue Jan 12 14:18:03 2021 +0100,Add check for dropped chunk on update,"If `drop_chunks` has been executed on a hypertable that has a continuous aggregate defined, the chunks will be removed and marked as dropped in `_timescaledb_catalog.chunk` but the lines will not be removed. This will cause problems for the update script since it is missing a check to only process chunks that are not dropped and will try to cast the chunk name into a `REGCLASS` for a table that does not exist. This commit fixes this by adding a check that the chunk is not dropped and also fixes the update test to not count dropped chunks when comparing with the chunk index since the chunk index does not count dropped chunks. Fixes #2791 " 2bef612c04435cdb5257d40373c4955a0d617c39,Sven Klemm,Sat Sep 26 02:19:56 2020 +0200,Adjust code to PG13 tuptoaster changes,PG13 split tuptoaster.c into three separate files. This patch also removes unnecesary tuptoaster.h includes. https://github.com/postgres/postgres/commit/8b94dab066 44edb25e8f907b5e42ccdd6553ff37fc91b80fe8,Sven Klemm,Fri Sep 25 14:31:55 2020 +0200,Handle AT_DropExpression in process_utility,PG13 adds a new ALTER TABLE subcommand for dropping the generated property from a column. https://github.com/postgres/postgres/commit/f595117e24 b2b3ad10df8ffcaf811147e65ed893a8c38546be,Sven Klemm,Fri Sep 25 14:23:01 2020 +0200,Adjust decompress code to Var field renames,PG13 renames the varnoold and varoattno field of Var to varnosyn and varattnosyn. https://github.com/postgres/postgres/commit/9ce77d75c5 7446cf7c548fe9cc21007664f3706a8a3d03d53b,Sven Klemm,Wed Sep 30 01:45:29 2020 +0200,Adjust planner code to PG13 planner_hook signature change,PG13 adds the query string as argument to the planner_hook. https://github.com/postgres/postgres/commit/6aba63ef3e 627b7f162b806196bbe467245aba305b1bee5459,Sven Klemm,Sat Sep 26 02:14:28 2020 +0200,Adjust jsonb_utils to PG13 changes,PG13 moved jsonapi.h from utils to common. https://github.com/postgres/postgres/commit/beb4699091 2cb7c41276c8f8112bd225a020fef709a8e776ff,Sven Klemm,Thu Sep 24 15:21:04 2020 +0200,Adjust code to PG13 tuple conversion changes,PG13 changes the way attribute mappings are done and uses AttrMap instead of AttrNumber[] in the new function signatures. This patch changes ChunkInsertState to use TupleConversionMap which abstracts this change. https://github.com/postgres/postgres/commit/e1551f96e6 968de59f8cd981ba0ca9cca1929d4ba17fbc256f,Sven Klemm,Thu Sep 24 05:33:38 2020 +0200,Adjust code to PG13 convert_tuples_by_name signature change,PG13 removed the msg parameter from convert_tuples_by_name. https://github.com/postgres/postgres/commit/fe66125974 10d1c2d698c1b7cbf5af86082287adda4f0c6e97,Sven Klemm,Thu Sep 24 16:18:43 2020 +0200,Adjust copy code to PG13 addRTEtoQuery changes,PG13 removes addRTEtoQuery and provides a similar function addNSItemToQuery which has a different signature. https://github.com/postgres/postgres/commit/5815696bc6 f61f5a03310795447ff5765ececccf24987f33eb,Ruslan Fomkin,Tue Dec 29 20:29:59 2020 +0100,Fix config enum entries for remote data fetcher,"config_enum entries, which are used for GUC values, need to contain the end mark. Missing the end mark in the data fetcher entries resulted in crashing TimescaleDB in Release on Windows. This commit fixes the end mark for data fetcher config_enum entries. Fixes #2775 " 13d8aac33b6fc5104c8ad1da816dc0d009fc13a7,Sven Klemm,Sat Sep 19 23:20:37 2020 +0200,Adjust code to PG13 list sort changes,PG13 changes the name of the list sorting function from list_qsort to list_sort. Additionally PG13 does in-place sort. https://github.com/postgres/postgres/commit/569ed7f483 b1a9c3b7b7d44ee78456931292655d52c252930d,Sven Klemm,Sat Sep 19 22:20:34 2020 +0200,Add support for PG13 List implementation,"PG13 changes the List implementation from a linked list to an array while most of the API functions did not change a few them have slightly different signature in PG13, additionally the list_make5 functions got removed. https://github.com/postgres/postgres/commit/1cff1b95ab " c224bc7994b0a40a3bffd306bb9af8425ec2bb97,Erik Nordström,Tue Dec 22 14:26:43 2020 +0100,Always validate existing database and extension,"This change ensures the database and extension is validated whenever these objects aren't created, instead of only doing validation when `bootstrap=>false` is passed when adding a data node. This fixes a corner case where a data node could be added and removed several times, even though the data node's database was already marked as having been part of a multi-node setup. A new test checks that a data node cannot be re-added after deleting it on the access node, irrespective of whether one bootstraps the data node or not when it is added. " 71633c3388eb27d10d0d48399ceb5a771e5b0104,Ruslan Fomkin,Mon Dec 21 20:01:48 2020 +0100,Add 2.0.0 to update test scripts, 5fc3f451852f6bf8ce4b43b8468a5228331a2ef9,Ruslan Fomkin,Thu Dec 17 10:32:21 2020 +0100,Release 2.0.0,"With this release, we are officially moving TimescaleDB 2.0 to GA, concluding several release candidates. TimescaleDB 2.0 adds the much-anticipated support for distributed hypertables (multi-node TimescaleDB), as well as new features and enhancements to core functionality to give users better clarity and more control and flexibility over their data. Multi-node architecture: In particular, with TimescaleDB 2.0, users can now create distributed hypertables across multiple instances of TimescaleDB, configured so that one instance serves as an access node and multiple others as data nodes. All queries for a distributed hypertable are issued to the access node, but inserted data and queries are pushed down across data nodes for greater scale and performance. Multi-node TimescaleDB can be self managed or, for easier operation, launched within Timescale's fully-managed cloud services. This release also adds: * Support for user-defined actions, allowing users to define, customize, and schedule automated tasks, which can be run by the built-in jobs scheduling framework now exposed to users. * Significant changes to continuous aggregates, which now separate the view creation from the policy. Users can now refresh individual regions of the continuous aggregate materialized view, or schedule automated refreshing via policy. * Redesigned informational views, including new (and more general) views for information about hypertable's dimensions and chunks, policies and user-defined actions, as well as support for multi-node TimescaleDB. * Moving all formerly enterprise features into our Community Edition, and updating Timescale License, which now provides additional (more permissive) rights to users and developers. Some of the changes above (e.g., continuous aggregates, updated informational views) do introduce breaking changes to APIs and are not backwards compatible. While the update scripts in TimescaleDB 2.0 will upgrade databases running TimescaleDB 1.x automatically, some of these API and feature changes may require changes to clients and/or upstream scripts that rely on the previous APIs. Before upgrading, we recommend reviewing upgrade documentation at docs.timescale.com for more details. **Major Features** TimescaleDB 2.0 moves the following major features to GA: * #1923 Add support for distributed hypertables * #2006 Add support for user-defined actions * #2125 #2221 Improve Continuous Aggregate API * #2084 #2089 #2098 #2417 Redesign informational views * #2435 Move enterprise features to community * #2437 Update Timescale License **Previous Release Candidates** * #2702 Release Candidate 4 (December 2, 2020) * #2637 Release Candidate 3 (November 12, 2020) * #2554 Release Candidate 2 (October 20, 2020) * #2478 Release Candidate 1 (October 1, 2020) **Minor Features** Since the last release candidate 4, there are several minor improvements: * #2746 Optimize locking for create chunk API * #2705 Block tableoid access on distributed hypertable * #2730 Do not allow unique index on compressed hypertables * #2764 Bootstrap data nodes with versioned extension **Bugfixes** Since the last release candidate 4, there are several bugfixes: * #2719 Support disabling compression on distributed hypertables * #2742 Fix compression status in chunks view for distributed chunks * #2751 Fix crash and cancel when adding data node * #2763 Fix check constraint on hypertable metadata table **Thanks** Thanks to all contributors for the TimescaleDB 2.0 release: * @airton-neto for reporting a bug in executing some queries with UNION * @nshah14285 for reporting an issue with propagating privileges * @kalman5 for reporting an issue with renaming constraints * @LbaNeXte for reporting a bug in decompression for queries with subqueries * @semtexzv for reporting an issue with continuous aggregates on int-based hypertables * @mr-ns for reporting an issue with privileges for creating chunks * @cloud-rocket for reporting an issue with setting an owner on continuous aggregate * @jocrau for reporting a bug during creating an index with transaction per chunk * @fvannee for reporting an issue with custom time types * @ArtificialPB for reporting a bug in executing queries with conditional ordering on compressed hypertable * @dutchgecko for reporting an issue with continuous aggregate datatype handling * @lambdaq for suggesting to improve error message in continuous aggregate creation * @francesco11112 for reporting memory issue on COPY * @Netskeh for reporting bug on time_bucket problem in continuous aggregates * @mr-ns for reporting the issue with CTEs on distributed hypertables * @akamensky for reporting an issue with recursive cache invalidation * @ryanbooz for reporting slow queries with real-time aggregation on continuous aggregates * @cevian for reporting an issue with disabling compression on distributed hypertables " ec82d16154c7427f0df779cfac1649a96875976d,Erik Nordström,Sat Dec 19 15:42:30 2020 +0100,Remove unused field for ignoring invalidations,Remove an unused field called `max_ignore_invalidation_older_than` that was left in the `Hypertable` struct after refactoring of continuous aggregates. 3bd29da988f5f331b2ee48f5621936f0a5283306,Erik Nordström,Sun Dec 20 11:14:54 2020 +0100,Bootstrap data nodes with versioned extension,"When the access node bootstraps a data node and creates the extension, it should use the extension version of the access node. This change adds the `VERSION` option to the `CREATE EXTENSION` statement sent to a data node so that the extension versions on the access node and data nodes will be the same. Without the version option, data nodes will be bootstrapped with the latest version installed, potentially leading to data nodes running different versions of the extension compared to the access node. " 973263a4f18fe31e2ceaa15e57de0fb0254a7bfb,Erik Nordström,Mon Dec 21 08:57:33 2020 +0100,Add multi-node update test,"The update scripts now test extension updates for a multi-node setup with one access node and one data node. Currently the following is tested: * A distributed hypertable is created in the database where the update process is normally tested. * A new database representing a data node is created. This database is updated and restored in the same way as the other database. However, no integrity or other validation checks are currently run on the datanode database; only the ability to update the extension is tested. " 40367d2dbf479fc822221cc80cff41c8ab2bd3ec,Erik Nordström,Sat Dec 19 16:18:03 2020 +0100,Fix check constraint on hypertable metadata table,"The `replication_factor` is set to `-1` on hypertables that are created on data nodes as part of a larger distributed hypertable. However, the check constraint on the hypertable metadata table doesn't allow such values, causing update scripts to fail when this check constraint is recreated as part of updating to version `2.0.0-rc4`. The reason it is possible to insert violating rows is because check constraints aren't validated when inserting data using PostgreSQL's internal catalog functions (in C). Therefore, the violating row can exist until one tries to update a data node to `2.0.0-rc4`, at which point the update script tries to recreate the `hypertable` metadata table due to other changes that were made to the table. This change fixes the check constraint to account for `-1` as a valid value, and also changes the update scripts to account for the new check constraint so that updates to the latest version will no longer fail. " 2500172a806eb31997da306f8426e0193bb90f71,Ruslan Fomkin,Sun Dec 20 12:33:32 2020 +0100,"Revert ""Release 2.0.0""","This reverts the 2.0.0 release commit, since new commits need to be merged before the release. " 07175eb88861c3691e37095f9ba8748843b1bb4d,Ruslan Fomkin,Thu Dec 17 10:32:21 2020 +0100,Release 2.0.0,"With this release, we are officially moving TimescaleDB 2.0 to GA, concluding several release candidates. TimescaleDB 2.0 adds the much-anticipated support for distributed hypertables (multi-node TimescaleDB), as well as new features and enhancements to core functionality to give users better clarity and more control and flexibility over their data. Multi-node architecture: In particular, with TimescaleDB 2.0, users can now create distributed hypertables across multiple instances of TimescaleDB, configured so that one instance serves as an access node and multiple others as data nodes. All queries for a distributed hypertable are issued to the access node, but inserted data and queries are pushed down across data nodes for greater scale and performance. Multi-node TimescaleDB can be self managed or, for easier operation, launched within Timescale's fully-managed cloud services. This release also adds: * Support for user-defined actions, allowing users to define, customize, and schedule automated tasks, which can be run by the built-in jobs scheduling framework now exposed to users. * Significant changes to continuous aggregates, which now separate the view creation from the policy. Users can now refresh individual regions of the continuous aggregate materialized view, or schedule automated refreshing via policy. * Redesigned informational views, including new (and more general) views for information about hypertable's dimensions and chunks, policies and user-defined actions, as well as support for multi-node TimescaleDB. * Moving all formerly enterprise features into our Community Edition, and updating Timescale License, which now provides additional (more permissive) rights to users and developers. Some of the changes above (e.g., continuous aggregates, updated informational views) do introduce breaking changes to APIs and are not backwards compatible. While the update scripts in TimescaleDB 2.0 will upgrade databases running TimescaleDB 1.x automatically, some of these API and feature changes may require changes to clients and/or upstream scripts that rely on the previous APIs. Before upgrading, we recommend reviewing upgrade documentation at docs.timescale.com for more details. **Major Features** TimescaleDB 2.0 moves the following major features to GA: * #1923 Add support for distributed hypertables * #2006 Add support for user-defined actions * #2125 #2221 Improve Continuous Aggregate API * #2084 #2089 #2098 #2417 Redesign informational views * #2435 Move enterprise features to community * #2437 Update Timescale License **Previous Release Candidates** * #2702 Release Candidate 4 (December 2, 2020) * #2637 Release Candidate 3 (November 12, 2020) * #2554 Release Candidate 2 (October 20, 2020) * #2478 Release Candidate 1 (October 1, 2020) **Minor Features** Since the last release candidate 4, it includes several minor improvements: * #2746 Optimize locking for create chunk API * #2705 Block tableoid access on distributed hypertable * #2730 Do not allow unique index on compressed hypertables **Bugfixes** Since the last release candidate 4, it includes several bugfixes: * #2719 Support disabling compression on distributed hypertables * #2742 Fix compression status in chunks view for distributed chunks * #2751 Fix crash and cancel when adding data node **Thanks** Thanks to all contributors for the TimescaleDB 2.0 release: * @airton-neto for reporting a bug in executing some queries with UNION * @nshah14285 for reporting an issue with propagating privileges * @kalman5 for reporting an issue with renaming constraints * @LbaNeXte for reporting a bug in decompression for queries with subqueries * @semtexzv for reporting an issue with continuous aggregates on int-based hypertables * @mr-ns for reporting an issue with privileges for creating chunks * @cloud-rocket for reporting an issue with setting an owner on continuous aggregate * @jocrau for reporting a bug during creating an index with transaction per chunk * @fvannee for reporting an issue with custom time types * @ArtificialPB for reporting a bug in executing queries with conditional ordering on compressed hypertable * @dutchgecko for reporting an issue with continuous aggregate datatype handling * @lambdaq for suggesting to improve error message in continuous aggregate creation * @francesco11112 for reporting memory issue on COPY * @Netskeh for reporting bug on time_bucket problem in continuous aggregates * @mr-ns for reporting the issue with CTEs on distributed hypertables * @akamensky for reporting an issue with recursive cache invalidation * @ryanbooz for reporting slow queries with real-time aggregation on continuous aggregates * @cevian for reporting an issue with disabling compression on distributed hypertables " a2275c7479c232315239cae2b25c1d3d9d104d85,Mats Kindahl,Fri Dec 18 16:08:16 2020 +0100,Fix Coverity build to not use -Werror,"The coverity build should not build with warnings-as-errors, so disabling it for the build. " c7a12d18a16b55d1ad644f682a8eabdc703cfd20,Ruslan Fomkin,Fri Dec 18 11:22:23 2020 +0100,Fix version number verification to allow releases,The verification of version number in version.config required pre-release label always. This fixes the bug and allows release versions. b21d8631dceeea470ba4701df7e35676b36cd6ea,Erik Nordström,Fri Dec 18 11:03:01 2020 +0100,Revert to blocking connection establishment,Non-blocking connection establishment when connecting to data nodes caused flakiness for Windows/Appveyor tests. Revert to blocking for now until these issues can be investigated more thoroughly. eede24bcf2b2c18bf76f53ce71f154bf547ed854,Erik Nordström,Thu Dec 17 09:51:11 2020 +0100,Add error detail when adding a data node fails,"When `add_data_node` fails, it often gives an opaque error that it couldn't connect to the data node. This change adds the libpq connection error as a detailed message in the error. " 15e1edb76f53ceed643c5076ef272a0dd8575b82,Erik Nordström,Thu Dec 17 00:37:09 2020 +0100,Fix crash when user mapping has no user name,"When a user mapping is created without a user name in the options a crash will occur when querying a distributed hypertable. This change fixes the crash by using the ""current user"" when no user option exists (i.e., no user mapping is done). This makes it possible to create user mappings that only provide the password option. " 0f48f1b29d88a1cc6d437ffd0d9edf5f6ab9fdde,Erik Nordström,Thu Dec 17 21:51:25 2020 +0100,Fix polling when connecting to data nodes,"When waiting for connection establishment to a data node, one should wait on a write event on the socket in the first iteration of the polling loop. However, this wasn't done and sometimes led to connection failures on some platforms. " 7eb6a9d7522a482ae782fee840a5092225b359d7,Sven Klemm,Thu Dec 17 13:47:35 2020 +0100,Add 2.0.0-rc4 to update test scripts, 877f48230cbf8dbc16922a7ec1abeb3fcdbc90b7,Erik Nordström,Wed Dec 16 23:08:19 2020 +0100,Fix crash and cancel when adding data node,"This change fixes two issues with `add_data_node`: 1. In one case, a check for a valid connection pointer was not done, causing a segmentation fault when connection attempts failed. 2. Connections were made with a blocking API that hangs indefinitely when the receiving end is not responding. The user couldn't cancel the connection attempt with CTRL-C, since no wait latch or interrupt checking was used. The code is now updated to use a non-blocking connection API, where it is possible to wait on the socket and latch, respecting interrupts. " d13afc70c976852043a53677ebe0928f48d3fb3e,Ruslan Fomkin,Wed Dec 16 14:50:49 2020 +0100,Run chunk_utils test in solo to reduce flakiness,chunk_utils test experiences some inconsistency in permissions sometimes. To avoid this the test is run solo without other tests interfering. bbac35b094614e39ae83bb1e672995d9c2b4c7a5,gayyappan,Fri Dec 11 14:40:34 2020 -0500,Fix compression status in chunks view for distributed chunks,The compression status of individual chunks for distributed tables is unknown on the access node. Fix the view so that is_compressed is NULL instead of false for distributed chunks. 2a22b7e9e18bdd05837e2164e03bb0203fc7415a,Erik Nordström,Sat Dec 12 16:06:45 2020 +0100,Optimize locking for create chunk API,"The create-chunk API, which is used by the access node to create chunks on data nodes, always grabs a lock on the root hypertable. This change optimizes this locking so that the lock is only taken when the chunk doesn't already exist. A new isolation test is also added to test locking behavior with concurrent transactions creating a chunk. " c6d64e3e8d56e797c129ec261b1fd3068d4d3976,Mats Kindahl,Wed Dec 16 09:38:37 2020 +0100,Fix compile issues in prerelease tests,"There are some flags that cannot be used in preprelease tests, which this commit fixes. It uses the flag `-DWARNINGS_AS_ERRORS=OFF` to not turn warnings into errors for release builds. In addition, explicit checks for compiler flags are added instead of using a function, which for some reason causes problems. Note that the prerelease tests run on a regular schedule as well. " 3448bcf2afe958f07b49f54b66fc72b904c90f86,Ruslan Fomkin,Wed Dec 9 20:50:59 2020 +0100,Move gapfill tests into using shared database, 2e352664bac9543945563973e8671036dcb74f27,Ruslan Fomkin,Wed Dec 9 13:11:03 2020 +0100,Run same table gapfill test on multinode,"This commit moves all gapfill tests, which was against a table, into a separate include test file, which is then used to run the test against a distributed hypertable. It is also run in the original test and the test results were moved inside the expected output. Also all the tests with errors in the original test file were moved to single location in the file. This commit also tests gapfill queries on a hypertable distributed only to one data node. dist_gapfill.sql was reorganized according the database setup, not tests. As the result of this commit all planned tests of gapfill on a distributed hypertable are added. " 494fe908011388040143f26fb3444e2be213e512,Mats Kindahl,Tue Dec 15 09:39:31 2020 +0100,Make loader test a solo test,"The `loader` test is quite flaky, and it seems to be because of interactions with other tests, so making it a solo test to avoid the flakiness. " e7c2d76b4291a52a37761e76cfb07821d321bbb4,Mats Kindahl,Fri Dec 11 10:03:19 2020 +0100,Exclude PostgreSQL compiler flags when building,"From `pg_config` it is possible to get the flags that were used to build PostgreSQL and we include when building the extension. However, on some platforms and some compilers (CentOS 8 and gcc, for example) flags are included that cannot be used when building the extension and will result in a compiler error. If the compiler used to build the extension is different from the compiler used when building PostgreSQL, we can also get errors because some options are not supported (PostgreSQL built using GCC and extension built using CLang, for example). This commit solves this by not adding the C compile flags from `pg_config` and explicitly add the flags that we need for building the extension. " 9e142644b611fa8fed0d097f8974a98a13d62b1e,Mats Kindahl,Mon Dec 14 08:06:07 2020 +0100,Use #ifdef rather than #if,"Some compilers do not support `#if SYMBOL` for undefined symbols and will generate an error. This commit switches to use `#ifdef SYMBOL` for symbols for `DEBUG` and `TS_DEBUG`, which can be undefined. " be8dd086a089f7419824bbc3b77182b95d0bbba7,Sven Klemm,Sat Sep 26 13:58:17 2020 +0200,Add missing utils/acl.h includes,PG13 removed acl.h from objectaddress.h so the places that need it need to now include it explicitly if they got it indirectly this way previously. https://github.com/postgres/postgres/commit/3c173a53a8 002510cb01e1d09767a526560f89c1857c1738a2,Sven Klemm,Sat Sep 19 19:17:38 2020 +0200,Add compatibilty wrapper functions for base64 encoding/decoding,PG13 adds a destination length 4th argument to pg_b64_decode and pg_b64_encode functions so this patch adds a macro that translates to the 3 argument and 4 argument calls depending on postgres version. This patch also adds checking of return values for those functions. https://github.com/postgres/postgres/commit/cfc40d384a 21dc9b9c1a307e46eda5fa621488ebeb6ee9636c,Sven Klemm,Sat Sep 19 17:43:52 2020 +0200,Allow building against PG13 source,This patch adjusts the version checks to allow building against postgres 13. It also adjusts the cmake version check to allow building against RC and devel versions. 05d8ac2a59189fd9f154ec805b072590bef9852f,gayyappan,Wed Dec 9 12:15:28 2020 -0500,Do not allow unique index on compressed hypertables.,CREATE UNIQUE INDEX is not supported on a hypertable that has compression enabled. 1cb9bf7a97ebc35dca962320376364df3abb3bfc,Mats Kindahl,Thu Dec 10 09:59:26 2020 +0100,Change based on review comments, 772be681a2e9f9cc93e5afc351b7be1c8e9b3e2e,Mats Kindahl,Wed Dec 9 14:37:08 2020 +0100,Increase test coverage for rename,"When renaming views of a continuous aggregate, schema rename with the direct view was not covered. This commit move an explicit rename of a direct view so that we can test a schema rename including a direct view as well as a partial view. Closes #2138 " bca1e35a52392677f3957492c938faab784ee5f9,gayyappan,Mon Dec 7 11:41:10 2020 -0500,Support disabling compression on distributed hypertables,This change allows a user to execute ALTER TABLE hyper SET (timescaledb.compress = false) on distributed hypertables. Fixes #2716 e9e7a9acb8f5e01d2fbde5ea437999ccee013802,Dmitry Simonenko,Tue Dec 8 14:50:26 2020 +0300,Block tableoid access on distributed hypertable,This change adds a check which raises an error when system column (like tableoid) being requested in case if data node scan is enabled. Fixes: #2528 9b5f20dd740546b0d0237dbcf193fb22ebb74ce6,Mats Kindahl,Wed Dec 2 21:27:45 2020 +0100,Fix ABI check build dependencies and tests,"Some tests contain code that only work if the build is based on a Git clone, which caused these tests to fail when Git was not available. This commit splits out those tests and only enabling them if Git is found and Git information can be retrieved. " 3db3fd921d102fb95e2208c3ce51c58113f0ad4e,Ruslan Fomkin,Wed Dec 2 14:01:07 2020 +0100,Compare distributed gapfill test to single-node,"The regression tests of gapfill on distributed hypertable compares their results to executing the same gapfill queries against non-distributed hypertable. The distributed gapfill test executes queries with partitionwise_aggregate disabled and enabled and compares both results. The tests were moved from the previous location to new, more appropriate files. The output was also moved with fixing the table name. This commit doesn't add new tests and doesn't include explain. " 6729326c0a5f0d6ad0433aa6a036424ae0524dbe,Sven Klemm,Wed Dec 2 19:46:41 2020 +0100,Remove workaround for AT_CookedColumnDefault to build against snapshot,Since PG12.5 has been released the workaround for AT_CookedColumnDefault is no longer needed and a simple PG version check is sufficient. e945ac762c8bd2d6148de8837590b76d1cf4ccbf,Mats Kindahl,Wed Dec 2 13:51:37 2020 +0100,Release 2.0.0-rc4,"This release candidate contains bugfixes since the previous release candidate, as well as additional minor features. It improves validation of configuration changes for background jobs, adds support for gapfill on distributed tables, contains improvements to the memory handling for large COPY, and contains improvements to compression for distributed hypertables. **Minor Features** * #2689 Check configuration in alter_job and add_job * #2696 Support gapfill on distributed hypertable * #2468 Show more information in get_git_commit * #2678 Include user actions into job stats view * #2664 Fix support for complex aggregate expression * #2672 Add hypertable to continuous aggregates view * #2662 Save compression metadata settings on access node * #2707 Introduce additional db for data node bootstrapping **Bugfixes** * #2688 Fix crash for concurrent drop and compress chunk * #2666 Fix timeout handling in async library * #2683 Fix crash in add_job when given NULL interval * #2698 Improve memory handling for remote COPY * #2555 Set metadata for chunks compressed before 2.0 " d41aa2aff5ce721f9e63cc07476707d21f7073bd,gayyappan,Wed Dec 2 11:23:45 2020 -0500,Rename macro TS_HYPERTABLE_HAS_COMPRESSION,This PR cleans up some macro names. Rename macro TS_HYPERTABLE_HAS_COMPRESSION to TS_HYPERTABLE_HAS_COMPRESSION_TABLE. 7fe74d4f076b5c840875c1328f70c893195bf5c8,Sven Klemm,Thu Oct 15 22:48:02 2020 +0200,Add test for statistics on compressed chunks,Add a test for updating statistics on chunks compressed before statistics where kept. This patch also sets metadata for previously compressed chunks so running ANALYZE on them will set the correct tuplecount. 7f3feb8200046a38a921458ffda9cd81c850d772,niksa,Wed Dec 2 16:30:03 2020 +0100,Introduce additional db for data node bootstrapping,"We now try connecting to three databases before giving up: postgres, template1 and defaultdb. " 2ecb53e7bb3a38b94d0d4f8510b990c028dd7879,Erik Nordström,Tue Dec 1 14:28:34 2020 +0100,Improve memory handling for remote COPY,"This change improves memory usage in the `COPY` code used for distributed hypertables. The following issues have been addressed: * `PGresult` objects were not cleared, leading to memory leaks. * The caching of chunk connections didn't work since the lookup compared ephemeral chunk pointers instead of chunk IDs. The effect was that cached chunk connection state was reallocated every time instead of being reused. This likely also caused worse performance. To address these issues, the following changes are made: * All `PGresult` objects are now cleared with `PQclear`. * Lookup for chunk connections now compares chunk IDs instead of chunk pointers. * The per-tuple memory context is moved the to the outer processing loop to ensure that everything in the loop is allocated on the per-tuple memory context, which is also reset at every iteration of the loop. * The use of memory contexts is also simplified to have only one memory context for state that should survive across resets of the per-tuple memory context. Fixes #2677 " 7c76fd4d09fe86f741924015a7dbd1365c61ca98,gayyappan,Thu Nov 19 15:04:46 2020 -0500,Save compression settings on access node for distributed hypertables,"1. Add compression_state column for hypertable catalog by renaming compressed column for the hypertable catalog table. compression_state is a tri-state column. This column indicates if the hypertable has compression enabled (value = 1) or if it is an internal compression table (value = 2). 2. Save compression settings on access node when compression is turned on for a distributed hypertable For a distributed hypertable, that has compression enabled, compression_state is set. We don't create any internal tables on the access node. Fixes #2660 " d043ff1e041f22814f3d923860df85c84e99dbaf,Mats Kindahl,Tue Nov 17 13:49:48 2020 +0100,Check configuration in alter_job and add_job,"If a bad value is given to `alter_job` or `add_job` for a configuration parameter, no error will be given but the job will fail to execute. This commit adds checks of the configuration parameters to the functions so that an error is given immediately when calling it. The commit factors out the extraction of parameters from the configuration from the execution functions into a separate functions and calls them from `alter_job` and `add_job` as well as when executing the job. Only non-custom job checks are done. The commit also moves a few functions that were only used in TSL code from the `src/` directory to the `tsl/src/` directory and also removes a redundant permission check and does a minor refactoring of the `job_execute` function so that an active snapshot is always created regardless of whether a transaction is open or not. The corresponding code in the individual policy functions are removed since they are not needed. Closes #2607 " 3544e33fb4d08105f587f87069299229b3a597ba,Mats Kindahl,Tue Dec 1 18:25:47 2020 +0100,Fix clang-format wrapper,The clang-format wrapper is always using `clang-format` even if a different version of `clang-format` has been found and has the correct version. This commit fixes the issue by passing the name of the found version of `clang-format` to the wrapper script using the `CLANG_FORMAT` varible. It will default to use `clang-format` if `CLANG_FORMAT` is unset. cab2347942409947ac8aab2566fa2a17b256513b,Sven Klemm,Tue Dec 1 14:48:17 2020 +0100,Fix postgres versions in build matrix,Versions in the build matrix with significant leading/trailing zeroes need the version string quoted otherwise it will get treated as number and those zeroes will be removed. As currently those tests fixed by this patch run PG 11.1 instead of 11.10. 9ac67fe6668f3e9b1566b4a45aaa56ee2f5b0b69,Ruslan Fomkin,Tue Dec 1 14:30:39 2020 +0100,Support gapfill on distributed hypertable,"This commits enables support for executing queries with time_bucket_gapfill on distributed hypertables. Current implementation always plans to execute time_bucket_gapfill on the access node. In future, it can be optimized to be executed on data nodes in suitable cases. " c311b44a097ff86fca018c37e7769a352c53936e,Erik Nordström,Thu Nov 26 15:57:38 2020 +0100,Fix crash for concurrent drop and compress chunk,"This change fixes a segfault that occured when `drop_chunks` is concurrently executed with `compress_chunk` and the same chunk that gets dropped is also being compressed. The crash happened because the tuple lock status function for a dimension slice passed in a pointer to a dimension slice that was always NULL. An isolation test is also added to cover concurrent compression and drop of the same chunk. To make the test pass with identical errors for PG11 and PG12, additional changes are made to the scanner API to pass on the lock failure data so that it is possible to distinguish between an update and delete on PG11. " 1e7f9610552bacc81ac679b1a5be1e7e28611f14,Mats Kindahl,Wed Sep 30 10:00:47 2020 +0200,Show more information in get_git_commit,"The command `get_git_commit` shows quite a little information and especially if mistakes are made it is hard to verify what the binary is actually based on. This commit extends the function to provide some more information: specifically the SHA of the HEAD regardless of whether there is a tag on it or not and the time for the HEAD commit. ``` postgres=# select * from _timescaledb_internal.get_git_commit(); commit_tag | commit_hash | commit_time --------------------------+-------------+------------------------ 1.7.4-10-g09b0b778-dirty | 09b0b778 | 2020-09-13 17:50:38+02 (1 row) ``` If git is not installed, the function `get_git_commit` will return an error indicating that no git information is available. If some of the fields are available, they will be emitted and the remaining fields will be NULL. Fixes #2457 " 37cef82f67298c01ac7b27c695e54ee42ae0ca1e,Erik Nordström,Fri Nov 20 14:04:13 2020 +0100,Fix timeout handling in async library,"This change removes a hard-coded 1 minute connection timeout that would cause long-running queries and commands (e.g., compressing a chunk) to time out before completion. Queries can still be canceled by the user or the database when `statement_timeout` is reached. One exception where timeouts is still used is cleanup during transaction abort (e.g., sending a `ROLLBACK` statement to a data node in the abort handler). Queries sent at this stage should have a timeout since they occur in the transaction abort handler after the main query has already been canceled or reached `statement_timeout`. Fixes #2645 Closes #2646 " 196f6dc19f581d8ff3b2a70d1095f7a29b4be2a8,Erik Nordström,Wed Nov 25 20:26:06 2020 +0100,Fix crash in add_job when given NULL interval,"This change fixes a crash in `add_job` when given an NULL schedule interval as input. In addition, the following improvements have been implemented for all job API functions: * Validation of input parameters, specifically NULL checks * Improved error messages for clarity and consistency * Tests for bad input arguments " d971d4f472510959e3fbd328b707470c46c884fe,Sven Klemm,Wed Nov 25 17:54:49 2020 +0100,Fix appveyor build,Bump the version number of the postgresql-dev package to 12.5 to match docker image and prevent version conflict. 79b7f4b14d17f1b143e92fba617409716d1441c8,Ruslan Fomkin,Tue Nov 24 09:24:49 2020 +0100,Include user actions into job stats view,"timescale_information.job_stats view was missing information about jobs running user actions, since they don't have associated hypertables. This commit fixes that the view includes user jobs in addition to the predefined policies, which were already covered as all of them reference a hypertable. " c1abf3b5c1a394fc407d1258a1baae7ddcf33236,Erik Nordström,Mon Nov 23 12:05:12 2020 +0100,Add hypertable to continuous aggregates view,"Add the hypertable's schema and name to the continuous aggregates view in the information schema, since these fields where missing. The new fields use the same names and order in the view to be consistent with other information views that reference the hypertable. Fixes #2653 " 6a9a96540979d1ba0a4b4b53274791e6b5bb9288,Ruslan Fomkin,Fri Nov 20 11:48:54 2020 +0100,Fix support for complex aggregate expression,"Fixes support for continuous aggregates when the view query contains an expression with several aggregates, e.g., `max(val) - min(val)`. Usage of continuous aggregates with such expression was producing errors if the aggregate expression was not the last in the SELECT clause or not all GROUP BY expressions were present in the SELECT clause. An expression with several aggregates is materialized with partials per aggregate. For example, `max(val) - min(val)` will be materialized in two partial entry columns: one for `max` and one for `min`. Thus all columns in the materialized hypertable should account for the number of partials and cannot just use the position in the original query. This fix makes sure to account for such case. Fixes #2616 " 47da8790bc374ab075a79f2382efbcf4e449f752,Ruslan Fomkin,Fri Nov 13 14:52:34 2020 +0100,Test latest PG 11.10 and 12.5, acb87329224e719cbbd0ad7ea1b413bf07f11a70,Erik Nordström,Thu Nov 12 13:53:34 2020 +0100,Add 2.0.0-rc3 to update tests,Add the latest release candidate to the update tests. d8e3b1c68d12d7f0a914c41360b5bc0e6f2bdbd7,Ruslan Fomkin,Fri Nov 13 10:46:04 2020 +0100,Make compatible with old Windows compiler,"Fixes usage of const to be identical in declarations and definitions, since 15.9 VS compiler gets warnings otherwise. " 54f943b8f909b43df5b44a799f260607a17e23c5,Erik Nordström,Wed Nov 11 14:32:11 2020 +0100,Release 2.0.0-rc3,"This release candidate contains bugfixes since the previous release candidate, as well as additional minor features including support for ""user-mapping"" authentication between access/data nodes and an experimental API for refreshing continuous aggregates on individual chunks. **Minor Features** * #2627 Add optional user mappings support * #2635 Add API to refresh continuous aggregate on chunk **Bugfixes** * #2560 Fix SCHEMA DROP CASCADE with continuous aggregates * #2593 Set explicitly all lock parameters in alter_job * #2604 Fix chunk creation on hypertables with foreign key constraints * #2610 Support analyze of internal compression table * #2612 Optimize internal cagg_watermark function * #2613 Refresh correct partial during refresh on drop * #2617 Fix validation of available extensions on data node * #2619 Fix segfault in decompress_chunk for chunks with dropped columns * #2620 Fix DROP CASCADE for continuous aggregate * #2625 Fix subquery errors when using AsyncAppend * #2626 Fix incorrect total_table_pages setting for compressed scan * #2628 Stop recursion in cache invalidation " 5a654f5e9d89b3974e838426a13e1874c84a6a11,Erik Nordström,Thu Nov 12 09:54:10 2020 +0100,Fix flaky constraint test when run with sanitizers,The constraint test fails when running with sanitizers (ASAN and UBSAN) due to a tablespace conflict. Make sure tablespaces are cleaned up prior to recreating. 4085f7352b175b16765d40fe544e970577a54497,Erik Nordström,Thu Nov 12 09:32:17 2020 +0100,Fix minor issues found by Coverity,Fix an uninitialized field in a struct in the refresh code for continuous aggregates. Other fixes are mostly to silence false positives. 791b0a4db70bd6e7f8c02e4dbef0b5a403d0cbd9,Ruslan Fomkin,Tue Nov 10 13:05:30 2020 +0100,Add API to refresh continuous aggregate on chunk,"Function refresh_continuous_aggregate, which takes a continuous aggregate and a chunk, is added. It refreshes the continuous aggregate on the given chunk if there are invalidations. The function can be used in a transaction, e.g., together with following drop_chunks. This allows users to create a user defined action to refresh and drop chunks. Therefore, the refresh on drop is removed from drop_chunks. " e284b2dfc07c1639d85a330ab165f127d6b79e26,Erik Nordström,Wed Nov 11 22:46:24 2020 +0100,Fix uninitialized variable in data node validation,Initialize a boolean variable used to check for a compatible extension on a data node. Leaving it uninitialized might lead to a potential read of a garbage value and unpredictible behavior. 05319cd4244884d2d129db37bda27f30fbc94595,gayyappan,Thu Oct 29 15:28:30 2020 -0400,Support analyze of internal compression table,"This commit modifies analyze behavior as follows: 1. When an internal compression table is analyzed, statistics from the compressed chunk (such as page count and tuple count) is used to update the statistics of the corresponding chunk parent, if it is missing. 2. Analyze compressed chunk instead of raw chunks When the command ANALYZE is executed, a) analyze uncompressed chunks and b) skip the raw chunk, but analyze the compressed chunk. " 4e1346a227dfa596fe662ca368d66bd0a3427bcd,gayyappan,Fri Nov 6 09:37:45 2020 -0500,Fix incorrect total_table_pages setting for compressed scan,"The compressed scan path overwrites the total_table_pages setting on PlannerInfo. This results in an assertion failure in index_pages_fetched function at Assert(T <= total_pages) For PG12, when a hypertable has a mix of compressed and uncompressed chunks, the total_table_pages value is calculated in reenable_inheritance code path and create_compressed_scan_paths. create_compressed_scan_path should add to the value previously computed by reenable_inheritance instead of overwriting it. " 47d26b422e4c5d5b9e1137a2d93fc8532dacf794,Erik Nordström,Mon Nov 9 11:43:05 2020 +0100,Allow optional password when adding data node,"Add an optional password parameter to `add_data_node` so that users that don't have a password in a `passfile` on the access node can add data nodes using password authentication. Together with user mappings, this allows full multinode configuration without relying on passwords or certificates provided in external/on-disk files. While wasswords can be provided in the database via a user mapping object, such a mapping is created on a per-server basis and requires the foreign server to exist prior to creating the mapping. When adding a data node, however, bootstrapping and/or validation of the data node happens at the same time as the server object is created, which means no user mapping can be created prior to adding the data node. Therefore, the password must be provided as an argument to add data node instead of via a user mapping. Fortunately, using a function parameter might be preferred to a user mapping since the (plaintext) password won't be stored in the database. A user mapping for the user that created the data node can optionally be added after the data node has been added. But it might be desirable to only create user mappings for unprivileged users that will mostly interact only with specific distributed hypertables. " 45db3b9b64b5456b7de88278c5c6ee7d654f5363,Erik Nordström,Sat Nov 7 11:21:51 2020 +0100,Add optional user mappings support,"A user mapping is a standard PostgreSQL object that can be used to provide a remote username and password when connecting to a data node. The downside of a user mapping, however, is that one mapping needs to exist per server and user while, in comparison, only one password file needs to exist per instance of PostgreSQL. Still, since user mappings are database objects they provide an alternative to password files for environments that cannot easily modify files outside the database. Providing a user mapping is completely optional and doesn't change other methods of providing credentials for authentication. " 97254783d4b9bc9fedf45f2150d200cfb3f87e9d,Sven Klemm,Wed Nov 4 12:22:12 2020 +0100,Fix segfault in decompress_chunk for chunks with dropped columns,This patch fixes a segfault in decompress_chunk for chunks with dropped columns. Since dropped columns don't exists in the compressed chunk the values for those columns were undefined in the decompressed tuple leading to a segfault when trying to build the heap tuple. bbcb2b22fa5eb0486893feaf7c566572d3e593f2,Dmitry Simonenko,Tue Nov 10 11:29:34 2020 +0300,Fix SCHEMA DROP CASCADE with continuous aggregates,This change fixes the situation when schema object is dropped before the cagg which leads to an error when it tries to resolve it. Issue: #2350 fbf412fad356131eea0a2c063fc9cdc5a60c392e,Mats Kindahl,Mon Nov 9 14:54:37 2020 +0100,Stop recursion in cache invalidation,"Inside the `cache_invalidate_callback` the OID of the namespace is read using `get_namespace_oid`, which usually goes to the cache. However, if there is a cache miss inside that function, it will attempt to read the OID from the heap, which involves invalidating the cache, which further leads to `cache_invalidate_callback` being called recursively. This commit prevent an infinite recursion by not doing a recursive call and instead considering the cache invalidated. This will allow `get_namespace_oid` (and other functions) to proceed with reading the OID from the heap tables, fill in the cache, and return it to callers. Closes #2143 " 8b54c6dc082dde8a8b5defd15350de03acc14ec6,Erik Nordström,Tue Nov 10 01:09:33 2020 +0100,Ignore unexpected policy functions in internal schema,"Do not assert that there are no unexpected policy functions in our internal schema when gathering policy counts for telemetry. Although unlikely, this could otherwise trigger the assertion if someone creates a policy function in our internal schema. " 5be958d1e886b4a5f783437616b22b91dc5cdb9c,Sven Klemm,Wed Oct 28 23:07:25 2020 +0100,Add user-actions to telemetry,"This patch adjusts the telemetry code to the job refactorings. Additionally it adds telemetry entries for user-defined actions, continuous aggregate policies, compression policies and renames drop_chunks to retention to match the new policy names. " 65161cd40bebc238efd90340bb802810a976c862,Erik Nordström,Thu Nov 5 09:41:51 2020 +0100,Improve error when converting time arguments,"API functions that take ""any"" type to support any supported time argument type sometimes do implicit conversion to the ""right"" type when a type coercion path exists. This change improves the error message when such a path does not exist. Closes #2614 " 048d67af935687f7abff66dfc8089345eaf8d2fe,Erik Nordström,Fri Nov 6 13:10:48 2020 +0100,Fix subquery errors when using AsyncAppend,"This change fixes errors when using sub-queries together with AsyncAppend on distributed hypertables. Since all subqueries are sent to remote data nodes as separate queries on the same connection, it is important that one query does not block the connection once the other query starts to execute. In other words, a query must complete a request and retrieve the result before another (sub-)query can execute. During query execution, this means that the following invariant must hold: An executor node cannot leave a request (or result) pending on a connection after returning a tuple, since for the next tuple a different executor node can be called to execute another sub-query on the same connection. This would be the case if two sub-queries are, e.g., joined with a nested loop. AsyncAppend (and associated fetchers) failed the above invariant; the executor node could leave a request unfinished after returning a tuple, causing the connection to be in an unexpected state when another AsyncAppend was called as part of a join with another sub-query. It turns out that only cursor fetcher can be used in these cases without having to fetch and buffer the entire result set of a sub-query (with a `CURSOR` the request can be broken up in multiple separate `FETCH` requests that can be interleaved with other sub-queries). Unfortunately, when executing a query using a `CURSOR`, it doesn't support parallel execution (on the data node). Previously, this was solved by defaulting to another ""row-by-row"" method of fetching data. However, row-by-row suffers from the same issue of leaving the connection in an unfinished state unless the whole result set is read and buffered, which could potentially blow up memory. The following changes are made to address the situation: * The cursor fetcher is made the default fetcher so that all queries can be executed without errors. * Fixes are also applied to the cursor fetcher to make sure it does not have pending queries after AsyncAppend returns a tuple. * AsyncAppend is similarly tweaked to avoid leaving connections with pending queries. * The ability to set the `fetch_size` (number of tuples to fetch in each `FETCH` request) and other options at the foreign data wrapper level is added. This allows changing FDW options globally for all data nodes, as the previous method of setting them on each foreign server is currently blocked. Setting a smaller `fetch_size` is often necessary to trigger the erroneous behavior. Unfortunately, these changes might lead to a regression in query performance. With cursor fetcher, queries cannot execute in parallel mode on data nodes. Further, the previous code was more aggressive with sending new fetch requests after each batch (for performance reasons), but this is not possible when sub-query joins are present. A future optimization could, for instance, use the row-by-row fetcher by default and fall back to cursor fetcher if it detects joins between sub-queries. Fixes #2511 " 0e579927646655a0819564c1a5c96db22c5882ae,Dmitry Simonenko,Fri Nov 6 17:45:14 2020 +0300,Fix DROP CASCADE for continuous aggregate,"Currently we treat cagg materialized view as a view internally, but that was missed in case of the drop cascade. Issue: #2608 " 307ac6afae0a3893d93612f3f0d7ef3a4dd77539,Ruslan Fomkin,Fri Oct 30 14:39:16 2020 +0100,Refresh correct partial during refresh on drop,"When drop_chunks is called on a hypertable manually or through a retention policy, it initiates refresh in hypertable's continuous aggregates. The refresh includes all buckets of each dropped chunk. Some buckets are inside a dropped chunk only partially. In such case, this commit fixes to refresh only the correct partials of the buckets. The fix passes chunk id to refresh on drop of a chunk. The chunk id is used to identify and update correct partials. This is a workaround that the invalidation outside of the refreshed chunk doesn't update the other partial, which is outside the refreshed chunk. The commit also adds the test demonstrating correct refresh. Fixes #2592 " f8f53aaeedf6dc00e0b5e1836cc0f14ce953170d,niksa,Tue Nov 3 14:48:59 2020 +0100,Fix validation of available extensions on data node,We want to check all available extension versions and not just the installed one. This is because we might be setting up a cluster for a database that has different extension version then the `postgres` or `template1` database which we actually use to perform this validation. So instead of using `pg_available_extensions` view we use `pg_available_extension_versions` that should return the same list of extension versions no matter which database we connect to. This should also make it possible to add a data node that has run ALTER EXTENSION UPDATE. This gives no guarantees that installed version will be compatible because we currently we use default version (the one specified in control file) when installing an extension. f0756d340d232487ff4e18e8f8a012ac3b21b51e,Erik Nordström,Wed Oct 21 13:31:47 2020 +0200,Remove commented code blocks,Remove two commented code blocks in data node dispatch and the decompress chunk executor node. 1bccedf3f892c0ea8256e79bc66c410644627995,Tyler Fontaine,Thu Oct 29 16:43:04 2020 -0500,Limit Index Tablespace Lookup To p and u Constraints,"This resolves an issue when using both constraints with index tablespaces AND constraints that have no indexes on a single hypertable. chunk_constraint_add_table_constraint() was attempting to add `USING INDEX TABLESPACE` to all constraints when a given hypertable had any constraint configured to use a tablespace, resulting in a SYNTAX error and blocking the creation of new chunks. The solution here is to limit index tablespace lookups to only the constraint types which use indexes: primary key and unique so that only those constraits will have `USING INDEX TABLESPACE` prepended when necessary. " 9551de018de98f42b6828ee7040ff0b9b861c244,Erik Nordström,Mon Nov 2 21:54:36 2020 +0100,Optimize internal cagg_watermark function,"The internal function `cagg_watermark` is often used as part of an index scan condition and is therefore called on every row of such a scan. Ideally, the planner should be able to reuse the result of repeated calls of the same function since it is marked STABLE, but this doesn't seem to be the case for C functions, making scans very slow. To deal with this, we cache the watermark value in the `fn_extra` field of the function call info context. This avoids row-by-row max value calculations, which otherwise makes things very slow. The function could be optimized further by making the scan for the max value faster (e.g., by doing an internal scan for max instead of relying on SPI). However, by caching the max value, this is already faster than the corresponding watermark function in 1.7.x. Fixes #2602 " c5e0c64795a9e5697bb83a9e40cd7a13cb43a4f5,Ruslan Fomkin,Mon Oct 26 16:11:55 2020 +0100,Set explicitly all lock parameters in alter_job,Sets the flag parameter to pass for tuple lock during alter_job execution. Restores the lost comment explaining the value of the flag parameter. 0c27dfaa78f08839ec3102d211c62137ec9945a6,Erik Nordström,Tue Nov 3 10:31:10 2020 +0100,Ignore PostgreSQL's timetz test,PostgreSQL's `timetz` test fails outside of daylight savings. The test is ignored until fixed in PostgreSQL (likely to happen in 12.5). 8b8b318fe05d62540304479f49916bbb34da7e5f,gayyappan,Wed Oct 21 14:41:48 2020 -0400,Add test for continuous agg policies,Add mock timestamp support for subtract_from_now. Use mock timestamp to execute cont. agg policies for date and timestamp based tables. 2cf3af1eb69ee416fbd233710449e30abc30b633,Mats Kindahl,Wed Oct 14 08:40:37 2020 +0200,Fix dist_hypertable test for parallel execution,"Change database names to be unique over the test suite by adding the test database name in front of the created database names in the test. This will allow the test to be executed in parallel with other tests since it will not have conflicting databases in the same cluster. Previously, there were a few directories created for tablespaces, but this commit changes that to create one directory for each test where the tablespace can be put. This is done by using a directory prefix for each tablespace and each test should then create a subdirectory under that prefix for the tablespace. The commit keeps variables for the old tablespace paths around so that old tests work while transitioning to the new system. " e9cb14985ec6c2b1293e40ff20480c57e9f45dcc,Mats Kindahl,Wed Oct 21 10:16:42 2020 +0200,Read function name dynamically,"The function name is hard-coded in some cases in the C function, so this commit instead define and use a macro that will extract the function name from the `fcinfo` structure. This prevents mismatches between the hard-coded names and the actual function name. Closes #2579 " 7c7b4f995ce0e901e15f9aa1bbd247d08dc084f8,Sven Klemm,Tue Oct 20 22:27:34 2020 +0200,Fix distributed_exec create statement, 374c405db37d2a46faae3355707cf9b324a0fe79,Sven Klemm,Tue Oct 20 21:45:53 2020 +0200,Add 2.0.0-rc2 to update tests, 525e821055ed8ee349b2e3d9c92e5bb2a4a9daa4,Brian Rowe,Tue Oct 20 11:34:38 2020 -0700,Add missing increment for PG11 decompression,"There is a bug in some versions of PG11 where ANALYZE was not calling CommandCounterIncrement. This is causing us to fail to update pg_class statistics during compression for those versions. To work around this, this change adds an explicit CommandCounterIncrement call after ExecVacuum in PG11. Fixes #2581 " 709ed0d4d006e65a7f49175833d28e05f25f68a9,Sven Klemm,Thu Oct 15 19:03:41 2020 +0200,Release 2.0.0-rc2,This release candidate contains bugfixes since the previous release candidate. **Minor Features** * #2520 Support non-transactional distibuted_exec **Bugfixes** * #2307 Overflow handling for refresh policy with integer time * #2503 Remove error for correct bootstrap of data node * #2507 Fix validation logic when adding a new data node * #2510 Fix outer join qual propagation * #2514 Lock dimension slices when creating new chunk * #2515 Add if_attached argument to detach_data_node() * #2517 Fix member access within misaligned address in chunk_update_colstats * #2525 Fix index creation on hypertables with dropped columns * #2543 Pass correct status to lock_job * #2544 Assume custom time type range is same as bigint * #2563 Fix DecompressChunk path generation * #2564 Improve continuous aggregate datatype handling * #2568 Change use of ssl_dir GUC * #2571 Make errors and messages conform to style guide * #2577 Exclude compressed chunks from ANALYZE/VACUUM 8a11b022bcb1dcbc2dc8da1855bfe72ebcbeeda6,Brian Rowe,Mon Oct 19 21:18:59 2020 -0700,Exclude compressed chunks from ANALYZE/VACUUM,This change makes sure that ANALYZE and VACUUM commands run without any relations will not clear the stats on compressed chunks that were saved at compression time. It also will skip any distributed tables. Fixes #2576 3cf9c857c4a0eb5ac449d0fbb1fa7e3cf9a95def,Erik Nordström,Thu Oct 15 13:36:45 2020 +0200,Make errors and messages conform to style guide,"Errors and messages are overhauled to conform to the official PostgreSQL style guide. In particular, the following things from the guide has been given special attention: * Correct capitalization of first letter: capitalize only for hints, and detail messages. * Correct handling of periods at the end of messages (should be elided for primary message, but not detail and hint messages). * The primary message should be short, factual, and avoid reference to implementation details such as specific function names. Some messages have also been reworded for clarity and to better conform with the last bullet above (short primary message). In other cases, messages have been updated to fix references to, e.g., function parameters that used the wrong parameter name. Closes #2364 " dc913ef0d474f9d040b7f89465722e4028b462e5,Sven Klemm,Sun Oct 18 18:02:58 2020 +0200,Fix DecompressChunk path generation,The non-parallel pathes generated by DecompressChunk were incorrectly marked as parallel_safe even when the child scan was not parallel aware. Leading to incorrect query results when those pathes were used in a parallel plan. Additionaly DecompressChunk code didnt set total_table_pages on PlannerInfo leading to an assertion failure in BitmapHeapscan path creation. 6cd688c3666f77a0e055af93eb6fbfe4b0805020,Sven Klemm,Mon Oct 19 16:19:43 2020 +0200,Fix use-after-free in finalize,Fix use-after-free in fa_perquery_state_init 253ceeb880a3039f20856e5060a4cf411712a461,Mats Kindahl,Mon Oct 19 14:09:27 2020 +0200,Change use of ssl_dir GUC,"By default, user certificates are placed in the directory `timescaledb/certs` relative to the data directory. This location can, however, be changed by setting `timescaledb.ssl_dir` to another value. If `timescaledb.ssl_dir` is set, user certificates will be `timescaledb/certs` relative to this location. This commit changes the usage of `timescaledb.ssl_dir` to point to the actual directory for the user SSL certificates and keys rather than the subdirectory `timescaledb/certs` under that directory. The default location of the user certificates and keys are unchanged. Closes #2568 " 4050f97dcf87e66001699c6d773661ae1cdbb4b8,Matvey Arye,Mon Oct 19 10:47:55 2020 -0400,Add Promscale extension to telemetry,This tracks the extension usage. 5acf3343b513376a3d60e3d3c8f1085eceb0c4fd,Brian Rowe,Mon Sep 28 14:54:46 2020 -0700,Ensure reltuples are preserved during compression,"This change captures the reltuples and relpages (and relallvisible) statistics from the pg_class table for chunks immediately before truncating them during the compression code path. It then restores the values after truncating, as there is no way to keep postgresql from clearing these values during this operation. It also properly uses these values properly during planning, working around some postgresql code which substitutes in arbitrary sizing for tables which don't see to hold data. Fixes #2524 " 295817f18ea2e113c8f847f41cd95e67de057781,Sven Klemm,Mon Oct 19 01:44:19 2020 +0200,Improve cagg datatype handling,This patch improves datatype handling when the aggregate function argument type is a pseudotype. 03f2fbcf3248b55f9888634789bb1ee3617c546f,Mats Kindahl,Thu Oct 15 17:12:56 2020 +0200,Repair dimension slice table on update,In #2514 a a race condition between inserts and `drop_chunks` is fixed and this commit will repair the dimension slices table by re-constructing missing dimension slices from the corresponding constraint expressions. Closes #1986 9963ecd18188ec609c0a0794e59e96d36e51ec46,Sven Klemm,Sat Oct 17 21:13:19 2020 +0200,Use postgres builtin hton functions,Switch to postgres builtin functions for converting from host to network byte order since those are better optimized. https://github.com/postgres/postgres/commit/510b8cbff15fcece246f66f2273ccf830a6c7e98 8cfaeb820e7765db555f33683de092b7a9b80ff0,Sven Klemm,Sat Oct 17 19:02:30 2020 +0200,Use AttrNumberGetAttrOffset to convert AttrNumber,Change code to use AttrNumberGetAttrOffset to convert AttrNumber into AttrOffset instead of manual subtraction. 5642ccaa1dda26ca88d5e195534b344ac35c1304,Sven Klemm,Sun Oct 11 22:01:22 2020 +0200,Fix index creation on hypertables with dropped columns,Fix creating expression indexes on hypertables with dropped columns. 3f23cb64e8dd90fcc515d43842881e58e3a4a507,Brian Rowe,Thu Oct 15 09:18:01 2020 -0700,Suspend retention policies with caggs conflicts,"When upgrading from 1.7, it's possible to have retention policies which overlap with continuous aggregates. These make use of the cascade_to_materializations parameter to avoid invalidating the aggregate. In 2.0 there is no equivalent behavior to prevent the retention from disrupting the aggregate. So during the 2.0 upgrade, check for any running retention policies that are dropping chunks still used by a continuous aggregate and suspend them (scheduled=>false). This will also print a notice informing the user of what happened and how to resume the retention policy if that's what they truly want. Fixes #2530 " 94af62f8136394f5b056cfb39ec626468ba69a65,Sven Klemm,Thu Oct 15 05:43:23 2020 +0200,Adjust package test to 2.0 package name change, ff560a903c7acea3bd71739720688f26c4ef39e0,gayyappan,Wed Oct 7 12:31:32 2020 -0400,Fix outer join qual propagation,time_bucket_annotate_walker passes an incorrect status for outer join to the function that checks quals eligibility for propagation. Fixes #2500 2cc2df23bdd18d42dd91e5917baf04af261ac946,Erik Nordström,Thu Oct 8 12:51:05 2020 +0200,Lock dimension slices when creating new chunk,"This change makes two changes to address issues with processes doing concurrent inserts and `drop_chunks` calls: - When a new chunk is created, any dimension slices that existed prior to creating the new chunk are locked to prevent them from being dropped before the chunk-creating process commits. - When a chunk is being dropped, concurrent inserts into the chunk that is being dropped will try to lock the dimension slices of the chunk. In case the locking fails (due to the slices being concurrently deleted), the insert process will treat the chunk as not existing and will instead recreate it. Previously, the chunk slices (and thus chunk) would be found, but the insert would fail when committing since the chunk was concurrently deleted. A prior commit (PR #2150) partially solved a related problem, but didn't lock all the slices of a chunk. That commit also threw an error when a lock on a slice could not be taken due to the slice being deleted by another transaction. This is now changed to treat that case as a missing slice instead, causing it to be recreated. Fixes #1986 " 5564ad8c6cfed9e35775c29fec40a8efe57f140d,Erik Nordström,Wed Oct 14 17:46:46 2020 +0200,Handle negative values in saturating sub and add,The functions for saturating addition and subtraction of time values assumed positive intervals as input. This change generalizes the code to also handle negative intervals being added or subtracted. ce6387aa90c26a4ec4aa562f04f30fb50529293c,Erik Nordström,Wed Oct 14 14:09:49 2020 +0200,Allow only integer intervals for custom time types,"Fix a check for a compatible chunk time interval type when creating a hypertable with a custom time type. Previously, the check allowed `Interval` type intervals for any dimension type that is not an integer type, including custom time types. The check is now changed so that it only accepts an `Interval` for timestamp and date type dimensions. A number of related error messages are also cleaned up so that they are more consistent and conform to the error style guide. " c4a91e5ae8f534143c67fc29aca2d800bf1c1a6c,Erik Nordström,Wed Oct 14 10:57:08 2020 +0200,Assume custom time type range is same as bigint,"The database must know the valid time range of a custom time type, similar to how it knows the time ranges of officially supported time types. However, the only way to ""know"" the valid time range of a custom time type is to assume it is the same as the one of a supported time type. A previous commit tried to make such assumptions by finding an appropriate cast from the custom time type to a supported time type. However, this fails in case there are multiple casts available that each could return a different type and range. This change restricts the choice of valid time ranges to only that of the bigint time type. Fixes #2523 " e33a72a607854ec9be298093716ec58b5e99d3c0,Sven Klemm,Sun Oct 11 22:57:15 2020 +0200,Fix flaky remote_txn test,The error message about closed connections varies between platforms/environments leading to flaky tests. This patch changes expected test output to sqlstate only in tests affected by this. b1c28c9c7cc0a579472568c6783230e38c825be5,Sven Klemm,Sun Oct 4 16:01:07 2020 +0200,Remove unreferenced steps from isolation tests,Some isolation tests had steps that were not referenced in any of the permutations so this patch removes those. 0e507affc168f2b2aec316af3a7e4188e5eb4dd8,Mats Kindahl,Tue Oct 13 10:12:31 2020 +0200,Remove modification time from invalidation log,"The `modification_time` column is hard to maintain with any level of consistency over merges and splits of invalidation ranges so this commit removes it from the invalidation log entries for both hypertables and continuous aggregates. If the modification time is needed in the future, we need to re-introduce it in a manner that can maintain it over both merges and splits. THe function `ts_get_now_internal` is also removed since it is not used any more. Part of #2521 " cf34f7d2ce78bd85ff879a9c2969a2a5323ad95c,gayyappan,Mon Oct 12 16:25:03 2020 -0400,Pass correct status to lock_job,lock_job is passed incorrect Status as argument. The lock is incorrectly interpreted as a session lock when a transaction lock is desired. e79c0648cfe5a94cc7c1c8f713d0c8097da8f7c8,Sven Klemm,Fri Oct 9 01:08:42 2020 +0200,Fix member access within misaligned address in chunk_update_colstats,"The array argument passed to array_length is treated as AnyArrayType which is a union of ArrayType and ExpandedArrayHeader, which lead to member access within misaligned address when used on the argument passed to array_length by chunk_update_colstats which is ArrayType. " 4b4db04c1e52b1a26ff32359ec1f11dd308a978c,Sven Klemm,Tue Oct 13 15:37:09 2020 +0200,Clean up update tests,Refactor update test files and remove obsolete test files. 336d8f9c47ea6c0c3b255acb993fc828d6809b84,Sven Klemm,Mon Oct 12 16:30:03 2020 +0200,Check function linkage in update test,This patch adds a check that all c functions link to the correct library after an update. 7b8203fc5350727e16c01752811aac4935b65a0e,Mats Kindahl,Mon Oct 12 17:39:33 2020 +0200,Update test dist_ddl to execute in parallel,Change database names to be unique over the test suite by adding the test database name in front of the created database names in the test. This will allow the test to be executed in parallel with other tests since it will not have conflicting databases in the same cluster. ccfca446f2f66f72ee6631a384ec0dd83682788e,Sven Klemm,Mon Oct 12 16:23:46 2020 +0200,Fix timescaledb_fdw function handling in update script,This patch splits the timescaledb_fdw sql file into two parts to separate the idempotent parts from the non-idempotent ones so the function definitions can be included in the regular update script. 87f78b4844e6c93dbddb562ac114c5537b6edda3,Sven Klemm,Mon Oct 12 03:34:51 2020 +0200,Move distributed insert tests to shared test,Change the distributed insert test to shared test so it can run in parallel and doesn't require dedicated distributed setup. d22799e21c6022a02c48dded360bec46e93d6a09,Sven Klemm,Tue Oct 13 13:41:31 2020 +0200,Use unique datanode name in read_only test, d8048f2a33bfee8318e08905b1ef4601a3103239,Sven Klemm,Tue Oct 13 00:39:34 2020 +0200,Use environment files for github actions,GitHub deprecates use of set-env in workflows so switch to using environment files. https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/ 85095b6eef840397e1ddfdf5c076e92aaf3d3a78,Ruslan Fomkin,Mon Oct 12 14:14:30 2020 +0200,Cleanup public API,Removes unlrelated column schedule_interval from timescaledb_information.continuous_aggregates view and simplifies it. Renames argument cagg in refresh_continuous_aggregate into continuous_aggregate as in add_continuous_aggregate_policy. Part of #2521 85428bc383219db6744d1782e6d1044f2dab47ab,Mats Kindahl,Tue Jan 28 11:22:33 2020 +0100,Support non-transactional distibuted_exec,"Before this commit, executions of `distributed_exec` was always transactional and this could not be disabled. This cause problems when executing statements that cannot be executed inside a transaction, such as `CREATE DATABASE`. With this commit, we introduce a parameter `transactional` to `distributed_exec` that allow non-transactional executions on data nodes. The parameter defaults to `TRUE`. Generates an error if `distributed_exec` is non-transactional and inside a transaction. It also changes `distributed_exec` to be a procedure, which better matches it's usage (it returns `VOID`). Closes #1660 " 967a10afcb95e28ee5e326bde376b63dc941c1f3,Sven Klemm,Thu Oct 8 21:19:02 2020 +0200,Fix flaky update test,Disable background workers during update tests to prevent deadlocks in continuous aggregates on 1.7.x 03deb65b0f1528e412a2c2ee1a7b6e52c780a50a,Sven Klemm,Mon Oct 12 15:12:19 2020 +0200,Fix update test diff display,The latest update test refactoring renamed some of the output files to better state their origin but not all places in the github workflow were adjusted to handle the new names. ebc4fd9b9e22403b66916d1cc62fb33a0098fe33,Dmitry Simonenko,Thu Oct 8 15:26:49 2020 +0300,Add if_attached argument to detach_data_node(),This change makes detach_data_node() function consistent with other data node management functions by adding missing if_attach argument. The function will not show an error in case if data node is not attached and if_attached is set to true. Issue: #2506 8ddaef66ea9c403160f0d502f5eb4537330c85c6,Mats Kindahl,Wed Oct 7 09:06:45 2020 +0200,Remove error for correct bootstrap of data node,"If the database exists on the data node when executing `add_data_node` it will generate an error in the data node log, which can cause problems since there is an error indication in the log but there are no failing operations. This commit fixes this by first validating the database and only if it does not exist, create the database. Closes #2503 " 65f31122eecc623f66404c1ae6bd0ad37167db2d,niksa,Wed Oct 7 13:35:11 2020 +0200,Fix validation logic when adding a new data node,We stop enforcing an extension owner to be the same as a user adding a data node since that's not strictly necessary. In multi-node setups it is common that a data node is pre bootstrapped and an extension owner is already set. This will prevent getting an error when a non extension owner tries to add a data node. 2347ebd123b90f5be8520ea10ee6f10a1c14c7ae,gayyappan,Wed Sep 2 14:38:33 2020 -0400,Overflow handling for refresh policy with integer time,"When we compute the start and end intervals for the refresh job execution, there is a potential to run into overflow with integer computations. In such cases, clamp the start/end boundaries to the MIN and MAX for that datatype ( i.e. PG_INT64_MIN or PG_INT64_MAX for int64, PG_INT16_MIN or PG_INT16_MAX for int16 and so on). " 3f2297b1735ac71579d4ab0dff6a1a237e9f0d47,Dmitry Simonenko,Tue Oct 6 15:07:25 2020 +0300,Block reorder policy for distributed hypertables, 641eb4e86bbcfbda250277970128428ff2fbb7a6,Sven Klemm,Tue Oct 6 01:27:46 2020 +0200,Ignore details of telemetry job in update test,Since handling of telemetry differs between timescaledb versions when telemetry is disabled via environment variable we ignore the scheduled flag in the post update diff. c3b9ca9e49e870d1074a3f601bb2d5441774778b,Sven Klemm,Tue Oct 6 01:12:47 2020 +0200,Fix compression_settings view definition, 5b97d8b3a95a558816f52f92d2c2720bb96ea9ac,Sven Klemm,Tue Oct 6 00:57:05 2020 +0200,Fix update script generation for versions with suffix, cc94d56d0e4b8cf9fbe87e9e770eb11f7403ced4,Sven Klemm,Mon Oct 5 23:37:17 2020 +0200,Add 2.0.0-rc1 to update test scripts, 46f7914e190633d2aaf103d89eb45f2143811a3b,Sven Klemm,Thu Oct 1 16:41:40 2020 +0200,Release 2.0.0-rc1,"This release adds major new features and bugfixes since the 1.7.4 release. We deem it moderate priority for upgrading. This release adds the long-awaited support for distributed hypertables to TimescaleDB. With 2.0, users can create distributed hypertables across multiple instances of TimescaleDB, configured so that one instance serves as an access node and multiple others as data nodes. All queries for a distributed hypertable are issued to the access node, but inserted data and queries are pushed down across data nodes for greater scale and performance. This release also adds support for user-defined actions allowing users to define actions that are run by the TimescaleDB automation framework. In addition to these major new features, the 2.0 branch introduces _breaking_ changes to APIs and existing features, such as continuous aggregates. These changes are not backwards compatible and might require changes to clients and/or scripts that rely on the previous APIs. Please review our updated documentation and do proper testing to ensure compatibility with your existing applications. The noticeable breaking changes in APIs are: - Redefined functions for policies - A continuous aggregate is now created with `CREATE MATERIALIZED VIEW` instead of `CREATE VIEW` and automated refreshing requires adding a policy via `add_continuous_aggregate_policy` - Redesign of informational views, including new (and more general) views for information about policies and user-defined actions This release candidate is upgradable, so if you are on a previous release (e.g., 1.7.4) you can upgrade to the release candidate and later expect to be able to upgrade to the final 2.0 release. However, please carefully consider your compatibility requirements _before_ upgrading. **Major Features** * #1923 Add support for distributed hypertables * #2006 Add support for user-defined actions * #2435 Move enterprise features to community * #2437 Update Timescale License **Minor Features** * #2011 Constify TIMESTAMPTZ OP INTERVAL in constraints * #2105 Support moving compressed chunks **Bugfixes** * #1843 Improve handling of ""dropped"" chunks * #1886 Change ChunkAppend leader to use worker subplan * #2116 Propagate privileges from hypertables to chunks * #2263 Fix timestamp overflow in time_bucket optimization * #2270 Fix handling of non-reference counted TupleDescs in gapfill * #2325 Fix rename constraint/rename index * #2370 Fix detection of hypertables in subqueries * #2376 Fix caggs width expression handling on int based hypertables * #2416 Check insert privileges to create chunk * #2428 Allow owner change of continuous aggregate * #2436 Propagate grants in continuous aggregates " 3f5872ec61650519e2c5e6fe1dfb60d07696cac7,Sven Klemm,Mon Oct 5 18:33:49 2020 +0200,Run pg_format on SQL files, dbd707b17ee1087ea9a651179fb62f759c66707d,Sven Klemm,Mon Oct 5 17:39:47 2020 +0200,Fix datanode sql function script,This patch merge data_node and remote_txn sql scripts and fixes the CREATE FUNCTION statement to use CREATE OR REPLACE FUNCTION. 2dc15ee345c4402684c02244544678ceead5b187,Brian Rowe,Sun Oct 4 15:15:31 2020 -0700,Show unscheduled jobs as paused in job_stats,"This change updates the timescaledb_information.job_stats view to check whether a job is currently scheduled in the bgw_config table. If it is not, the `job_status` field will show `Paused` and the `next_start` field will be NULL. Fixes #2488 " a1cf324063031f165321f5ad222f9ac4ec4b307c,Sven Klemm,Mon Oct 5 17:18:33 2020 +0200,Fix timescaledb_fdw sql script,Since CREATE FOREIGN DATA WRAPPER is not idempotent it must not be grouped with the normal sql scripts but has to be in the pre_install group. ec41593978fdb5c6799a153929fd60497ce6f533,Sven Klemm,Mon Oct 5 15:08:25 2020 +0200,Ignore memory leaks in openssl code,Ignore memory leaks in openssl code when running sanitizer tests in CI. 4623db14ad73ef5ea624d071ad678e65dcb02910,Erik Nordström,Mon Oct 5 12:42:32 2020 +0200,Use consistent column names in views,Make all views that reference hypertables use `hypertable_schema` and `hypertable_name`. c029a7be0258d07699fb3c5c04a502cbee2d7f46,Mats Kindahl,Mon Oct 5 11:38:10 2020 +0200,Remove use of REGCLASS in views,Removing usage of `REGCLASS` from view `continuous_aggregates` and splitting it into schema and name. 7a406fa690795501665ea8b57e22c54cc9bda5f0,Erik Nordström,Mon Oct 5 11:09:15 2020 +0200,Make data nodes view consistent with other views,Rename the data_node view to data_nodes to be consistent with other views that all use the plural form. 5a5cbe1a93b332894b090fa92c2d6b538d4e8807,gayyappan,Fri Oct 2 10:47:54 2020 -0400,Rename column in job_stats view,Rename next_scheduled_run to next_start so that jobs and job_stats views have the same column names b0a1f639697afcdc2451808fe662341eb08a0eb8,Sven Klemm,Fri Oct 2 17:51:18 2020 +0200,Fix application_name for user-defined actions, f7e26ebb62b0df3f8d3b0fa55bfd05548ada8a0d,Erik Nordström,Fri Oct 2 14:45:26 2020 +0200,Fix parameter names in policy API code,"A previous change updated the public function definitions, but didn't update the code and error messages to match these changes. " 544d3af52219591eb1a9569e4ecb4bc65fb98528,Sven Klemm,Fri Oct 2 15:05:41 2020 +0200,Improve check for fallthrough support,Change the check for fallthrough support to check for the attribute instead of specific compiler versions. da97ce6e8ba7d6c7fe1a162089cb4e4d403192a8,Mats Kindahl,Wed Sep 30 16:35:40 2020 +0200,Make function parameter names consistent,Renaming the parameter `hypertable_or_cagg` in functions `drop_chunks` and `show_chunks` to `relation` and changing parameter name from `main_table` to `hypertable` or `relation` depending on context. b341506cb2f3dac94b965c22af68455ec5515601,gayyappan,Thu Oct 1 12:30:40 2020 -0400,Enable continuous_aggs_policy test,Disable NOTICE messages for time based tables in continuous_aggs_policy test. The refresh sometimes does not have anything to update as the intervals are computed based off now(). This causes intermittent failures due to NOTICEs that the cagg is up to date. Remove continuous_aggs_policy from ignores list in CI scripts. 05355ab0be6704c3c0348ccf4e1f4ef7373e7c7a,gayyappan,Fri Sep 11 15:38:45 2020 -0400,Check start > end for refresh policy,Add check to verify that start_interval > end_interval specified by the refresh continuous aggregate policy 0703822a83a3b0ac7f1f53d1841da4b31c4b0f58,Brian Rowe,Mon Sep 28 19:39:49 2020 -0700,Create low end invalidation when updating caggs,This change will add an invalidation to the materialization_invalidation_log for any region earlier than the ignore_invalidation_older_than parameter when updating a continuous aggregate to 2.0. This is needed as we do not record invalidations in this region prior to 2.0 and there is no way to ensure the aggregate is up to date within this range. Fixes #2450 ef7f21df6d14c3b5ade3d55582972a173bc1b7cd,gayyappan,Wed Sep 30 15:45:08 2020 -0400,Modify job_stats and continuous_aggregates view,Use hypertable_schema and hypertable_name instead of regclass hypertable in job_stats and continuous_aggregates views. 7f98d7fa0ceacf8e694ba7a0c47b297ba3d35f0b,Ruslan Fomkin,Thu Oct 1 09:25:09 2020 +0200,Test Apache license on Windows,Fixes the regression test of Apache license on Windows. cdb4c8fa4fa6b2725e051e0fd336d4751efda8a8,Ruslan Fomkin,Thu Oct 1 11:57:11 2020 +0200,Ignore flaky test for now,continuous_aggs_policy is flaky and prevents to proceed with current PRs. This commit ignores it until it is fixed. 7fa8b1c06fb3be49b0182294073fb045b46f989c,Sven Klemm,Tue Sep 29 21:31:14 2020 +0200,Use postgres implementation of estimate_hashagg_tablesize,Only use our own implementation of estimate_hashagg_tablesize on versions where the postgres implementation is static. a51aa6d04b1ca666bcf574560b0c6e4ae575e6dd,Dmitry Simonenko,Wed Sep 30 14:43:26 2020 +0300,Move enterprise features to community,This patch removes enterprise license support and moves move_chunk() function under community license (TSL). Licensing validation code been reworked and simplified. Previously used timescaledb.license_key guc been renamed to timescaledb.license. This change also makes testing code more strict against used license. Apache test suite now can test only apache-licensed functions. Fixes #2359 66c63476e5905a5f78d6aca263ccebae1e911105,Ruslan Fomkin,Thu Sep 24 14:19:43 2020 +0200,Change cagg refresh to cover buckets,"Refresh of a continuous aggregate was expanding refresh window to include buckets, which contain the start and end of the window. This was leading to refreshing dropped data into the first bucket in the corner case, when drop_before of a retention policy is the same as start_offset of a continuous aggregate policy and the last dropped chunk happens to intersect with the first bucket. See #2198 for detailed discussion. A behavior of a refresh, which is called when chunks are dropped, is not changed, i.e., buckets, which fully cover chunks ot be dropped, are refreshed if needed (i.e., there were changes in the chunks, which were not refreshed yet). It can be done separately if needed. Fixes #2198 " 423e8fa527e60c77d0530ab4d4c786c7117e041d,Sven Klemm,Fri Sep 25 16:46:44 2020 +0200,Remove ORDER BY clause from timescaledb_information.jobs view,Remove the ORDER BY clause from timescaledb_information.jobs to make it consistent with our other information views. db0e210b8fb09dd8f61f96d90d992b062ac94b9e,Sven Klemm,Mon Sep 28 11:05:00 2020 +0200,Block REFRESH MATERIALIZED VIEW on caggs, 98e1aa25efcb9df7e89eebc619e6ab984c496fa2,gayyappan,Mon Sep 28 11:05:54 2020 -0400,Rename policy_stats view,Rename policy_stats view to job_stats 02ad8b4e7ea86072ef76354b662d36e765515ef1,Mats Kindahl,Thu Sep 24 15:37:00 2020 +0200,Turn debug messages into DEBUG1,"We have some debug messages that are printed as notices, but are more suitable to have at `DEBUG1` level. This commit removes a notice about indexes being added and turns it into a `DEBUG1` notice. " d11256bad61cae917d9b7da4ec2f126df3ee76d8,Erik Nordström,Mon Sep 28 13:33:49 2020 +0200,Rename policy parameters,Policy parameters are renamed to be more consistent and intuitive across different policies. Fixes #2442 882a2472446695cc31561973b80a858cf7d18cbf,Erik Nordström,Mon Sep 28 12:06:40 2020 +0200,Support compiler-agnostic annotations,"This change adds a compiler-agnostic annotation for fall-throughs in switch statements. The definition is put in `annotations.h`, which can be expanded to hold definitions for similar functionality in the future. The `clang` compiler (as of version 12) seems to have have dropped support for the previous comment-based annotations to allow fall-throughs in favor of native annotations or GCC-style attributes. " 6cc9871be81c46ee9d972ef0c614b3f701deb5f0,Sven Klemm,Thu Sep 24 23:22:11 2020 +0200,Make update test consistent,This patch changes the update test to use the same checks between clean / updated install and dumped/restored install. Previously only a small subset of the checks would be run against the updated instance and most of the tests would only run against the dumped and restored container. 9a5a4c0ede5538affc419a737668189c6a6aec7b,Sven Klemm,Thu Sep 24 06:26:35 2020 +0200,Replace heap_open by table_open,Starting with PG11 heap_open/_close is a macro that redirects to table_open/_close. In PG13 those compatibility macros are being removed so we have to use the actual function. 44a39dca85225feed19fd84398c37af74610dc3e,Sven Klemm,Sat Sep 26 19:53:43 2020 +0200,Remove unused allocation in copy path,This patch removes allocation of 2 arrays in the copyfrom function that were never used. e1be29e1611bef91f0079ec85da6f1d9edf2cc7e,Sven Klemm,Wed Aug 26 02:59:29 2020 +0200,Remove load-language pg_regress option,"PL/pgSQL is installed by default starting with PostgreSQL 9.0 so the parameter is not needed, additionally the parameter will throw an error on PG13 pg_regress. " dbb9988eee9af0ebcd0ac1f2b9444b76d17da51f,Sven Klemm,Fri Sep 25 16:51:44 2020 +0200,Fix result ordering in tests,This patch fixes the result sorting in tests that had no ORDER BY clause or where ORDER BY clause did not result in fixed ordering. f874381e49bab548ee73b107b1f643e408fed944,Sven Klemm,Fri Sep 25 17:44:21 2020 +0200,Set autovacuum_enabled to false for compressed chunks,Commit 8e1e6036 changed chunk compression to disable autovacuum on compressed chunks but did not apply the setting to chunks compressed before that change. So this patch changes chunks compressed with previous version to disable autovacuum as well. 559007097fbecb8808579790aa663441e7536a3f,gayyappan,Fri Sep 25 16:41:34 2020 -0400,Rename continuous_aggregate_policy parameters,Continuous aggregate policy parameters start_interval/end_interval are renamed to start_offset/end_offset e79308218a3be1dfdb197e9ba0b712276f692757,Brian Rowe,Tue Sep 15 16:50:33 2020 -0700,Add invalidations for incomplete aggregates,"As part of the 2.0 continous aggregate changes, we are removing the continuous_aggs_completed_threshold table. However, this may result in currently running aggregates being considered complete even if their completed threshold hadn't reached the invalidation threshold. This change fixes this by adding an entry to the invalidation log for any such aggregates. Fixes #2314 " 2ecfefc2fd4762c8dce9e67e90e3a95ebd33565e,Erik Nordström,Thu Sep 24 12:50:08 2020 +0200,Cleanup invalidations tests,This change adds views for invalidation tables to simplify queries in the test. 608d52dffc4b47d1c694598aabc70e28404e05af,Erik Nordström,Thu Sep 24 13:55:59 2020 +0200,Merge adjacent invalidations during refresh,"Since invalidations are inclusive in both ends, adjacent invalidations can be merged. However, adjacency wasn't accounted for when merging invalidations, which meant that a refresh could leave more invalidations in the log than strictly necessary. Note that this didn't otherwise affect the correctness of a refresh. " 5f31fa53b34d5cc0aaf1f30888f836820e57e76f,Erik Nordström,Thu Sep 24 14:01:58 2020 +0200,Fix index attribute in invalidation scan,"When setting up an index scan for invalidations, a table attribute number was used instead of the corresponding index attribute number. While the attribute numbers happened to be the same, it isn't future proof to use the wrong attribute reference. " fb4fec1834ac40e658e6b2cc99a9d7966ccc8b42,Mats Kindahl,Tue Sep 22 16:07:26 2020 +0200,Propagate grants in continuous aggregates,"Before this commit, grants on continuous aggregates were not propagated to the materialized hypertable, direct view, and partial view. With this commit, grants on a continuous aggregate is propagated. Fixes #2413 " c2fb4ab75378191ea431a99c373857a2e0857c78,Michael J. Freedman,Wed Sep 23 02:05:09 2020 -0400,Update Timescale License,* Add right-to-repair * Add right-to-improve * Remove Enterprise tier * Remove Usage Limits from Community * Other rights and simplifications 9fedb63f8089aa532e685624ecb3354a741f46af,Ruslan Fomkin,Thu Sep 24 14:35:02 2020 +0200,Remove unused refresh arguments from invalidation,After recent refactoring the refresh window arguments are not used in many invalidation functions. This fixes the code and removes unused arguments to avoid future confusions. eb30e54d92ba63f7fff372c65b44a6590546678a,Sven Klemm,Wed Sep 23 22:38:24 2020 +0200,Fix format strings used in tests,This patch fixes the format strings used to construct object names in tests. The format strings used in those tests would break when object names are involved that require quoting. 6aae14c11ff7967c67d7fe69cfe27e348e3b0ce4,Mats Kindahl,Tue Sep 22 11:24:56 2020 +0200,Allow owner change of continuous aggregate,"Implements support for changing the owner of the continuous aggregate. This will change the owner of the materialized hypertable, the user view, the partial view, and the direct view. Fixes #2414 Fixes #1277 " 6c59f57ccf68344b417f370a9448de796e6b61c4,Sven Klemm,Fri Sep 18 15:46:28 2020 +0200,Add timescaledb_information.jobs view,This patch adds a convenience view to list all configured jobs. 519863f46004b7d43ebd1ea7338315184a217a35,Erik Nordström,Tue Sep 22 11:39:16 2020 +0200,Remove catalog options for continuous aggregates,"This change removes the catalog options `refresh_lag`, `max_interval_per_job` and `ignore_invalidation_older_than`, which are no longer used. Closes #2396 " ecffe5af7d4652ab953e2c085b23d644849883bb,gayyappan,Fri Sep 18 14:19:59 2020 -0400,Add test for dimensions view,Create more than 2 dimensions and verify that dimensions view displays information correctly. c321fe0ca0e3d3938880f00cb827c8b42fc53d49,Mats Kindahl,Fri Sep 18 13:34:04 2020 +0200,Check insert privileges to create chunk,"To create a chunk in a hypertable, it is currently necessary to be the owner of the hypertable of the chunk. If a user has insert privileges only, it will fail with an error message, which causes problems when inserting data into distributed hypertables since the user cannot create new chunks. This commit changes this and only requires that the user has insert privileges on the hypertable of the chunk for allowing creation of a new chunk. Closes #2393 " c15d8be7f7307319cd71ffc31c304ecdccd85c1e,Dmitry Simonenko,Mon Sep 21 14:29:43 2020 +0300,Avoid having to cast time arg for cagg policy,This patch does a minor refactoring and adds a way to guess interval argument type based on used cagg Issue: #2286 e1a00eb517a46b396a29909b9d7949c2e140bd48,Sven Klemm,Sun Sep 20 23:36:20 2020 +0200,Remove obsolete function definitions and macros,During refactoring continuous aggregates some function defintions for functions that got removed were not removed from the header file. Additionally a few other header files had function definitions that no longer had an implementation or macros that were not used. c85a0c4c331d095ced0633b333b1d994085ec556,Sven Klemm,Mon Sep 21 00:05:06 2020 +0200,Remove ts_dimension_info_create_open_interval_usec,Remove ts_dimension_info_create_open_interval_usec function because it was no longer used. This patch also moves the add_dimension record attribute numbers into the c file because it does not need to be exported. ddd6ce21e4c942761ac39aef720da2de76894c08,Sven Klemm,Sun Sep 20 20:55:49 2020 +0200,Remove duplicate find_em_expr_for_rel function,The functions find_em_expr_for_rel and ts_find_em_expr_for_rel are identical. This patch removes find_em_expr_for_rel and changes all call-sites to use ts_find_em_expr_for_rel. 6d7d99a588a8883cd06b70c65aea22cc9cd4b13b,Sven Klemm,Wed Sep 16 03:16:35 2020 +0200,Fix alter_job not updating new job fields,Fix bgw_job_tuple_update_by_id to also update scheduled and config field of bgw_job. 17cc6f6bd7ed6b057e4d9436f20d7627211e5630,Sven Klemm,Fri Sep 18 01:32:41 2020 +0200,Fix ApacheOnly regression test,The recently added test for hypertable detection used compression which is not available in ApacheOnly tests so we move that test to regresscheck-t. Additionally we move the other test in plan_hypertable_cache to plan_expand_hypertable to reduce the number of tests. 1c2142fd6c84938d12fcd65d04e2993edf980f72,Erik Nordström,Fri Sep 18 11:51:56 2020 +0200,Check owner when refreshing a continuous aggregate,This change checks for ownership of the user view when refreshing a continuous aggregate (it's assumed the other objects have the same owner). This mimics the checks done when refreshing a regular materialized view. Fixes #2408 e2cefd9a803c846e7f92efa84ae1b0b2dd92b909,Erik Nordström,Tue Sep 15 15:03:36 2020 +0200,Update README for continuous aggregates,The README that gives an overview of the implementation of continuous aggregates is updated to reflect changes to the feature. Fixes #2147 7abe65d87e8dd188aae3bcdb26dd684b78321033,Mats Kindahl,Fri Sep 18 08:46:34 2020 +0200,Fix field name in continuous_aggregate view,Rename the `refresh_interval` field in `timescaledb_information.continuous_aggregate` view to match the parameter name in `add_continuous_aggregate_policy`. 149d45289dd53d82f5838c78b52f8d44bccb1dd2,gayyappan,Thu Sep 17 15:38:03 2020 -0400,Fix typo in chunks view,The is_compressed column for timescaledb_information.chunks view is defined as TEXT instead of BOOLEAN as true and false were specified using string literals. Fixes #2409 c21839ddb92059dc2b02505b470fc0c081c77e93,gayyappan,Wed Sep 16 16:24:05 2020 -0400,Add test for tablespaces with views,Add test for chunks and hypertables view that shows tablespaces. c1bd79ebc73ed7efcdb3e1c32044163015b59954,Sven Klemm,Sun Sep 13 11:43:06 2020 +0200,Fix caggs width expression handling on int based hypertables,This patch changes the time_bucket validation to constify the width argument so any immutable expression will be allowed. This change allows continuous aggregates on hypertables with int8 time dimension without requiring a workaround from a user to get the time_bucket call accepted. e79158e7175b5b4712abf8a762f80e6d7fe5501c,Sven Klemm,Sun Sep 13 22:43:48 2020 +0200,Run postgres regression test suite as part of our CI,During migration of CI from travis to github actions running postgres regression test suite was missed. This patch changes our regression workflow to run postgres regression test suite as well. 517944761350b0db309d49c46addc681d53d961e,Erik Nordström,Thu Sep 10 20:42:58 2020 +0200,Remove completed threshold,"The completed threshold in the TimescaleDB catalog is no longer used by the refactored continuous aggregates, so it is removed. Fixes #2178 " c884fe43f08c2976016a19e5c7eaaacad0b00511,Erik Nordström,Thu Sep 10 20:29:42 2020 +0200,Remove unused materializer code,"The new refresh functionality for continuous aggregates replaces the old materializer, which means some code is no longer used and should be removed. Closes #2395 " 27e44f20acd8c566bce0c0c00d4aa2a2add876f3,Erik Nordström,Thu Sep 10 12:34:05 2020 +0200,Cleanup functions to find continuous aggregates,This change cleans up and removes duplicate code for internal lookups of continuous aggregates. A number of related error messages have also been cleaned up and made conformant with the error style guide. 01fc0de040ea2e8075399f2432beaff6fd0e378b,Ruslan Fomkin,Tue Sep 15 15:05:18 2020 +0200,Fix failing isolation test on PG11.0,Early version of PG11 produces slightly different output for isolation tests. Thus the affected tests are ignored. continuous_aggs_concurrent_refresh is one of them and this commit adds it to the ignored test list. 555e6458f8666ec6b7b08717200f4c62ce66bd78,Ruslan Fomkin,Mon Sep 14 08:28:30 2020 +0200,Release 2.0.0-beta6, 602d9a9bf5269507e9d59b10c4058a50cb6e6163,gayyappan,Fri Sep 11 18:07:07 2020 -0400,Remove continuous_aggregate_stats view,Stats for policies are exposed via the policy_stats view. Remove continuous aggregate stats view - the thresholds exposed via this view are not relevant with the new API. 94dd9a33bf0665c671b4fd0e482ae6c177ec679a,Erik Nordström,Mon Sep 14 16:31:48 2020 +0200,Allow REFRESH MATERIALIZED VIEW to refresh caggs,"It is now possible to use `REFRESH MATERIALIZED VIEW` as an alternative to `refresh_continuous_aggregate` when doing ""infinite"" refreshes. " 6f1a0bd24a98e076171d2484c26ee3d210d69a79,Ruslan Fomkin,Mon Sep 14 16:52:28 2020 +0200,Remove options from continuous aggregate,"Removes options refresh_lag, max_interval_per_job and ignore_invalidation_older_than from continuous aggregate creation with CREATE MATERIALIZED VIEW, since they are not related this statement any more. They are already replaced with the corresponding options in add_continuous_aggregate_policy. This commit removes only options, while the options are still stored in the catalog and need to be removed from there in a separate PR. " d5446dfcfd2dd43763d0bfef16ebe59ee6bce353,gayyappan,Thu Sep 10 11:40:32 2020 -0400,Update policy_stats view with additional fields,Update policy_stats view to make it more end user friendly. These changes mirror the fields in continuous_aggregate_stats. 61b5d91c0e5e5c9ad6ff0a13f4749de122fb9d9e,Mats Kindahl,Mon Sep 14 16:10:01 2020 +0200,Error on create continuous aggregate with data,"If a continuous aggregate is created using `CREATE MATERIALIZED VIEW` using the `WITH DATA` option, it will fail with a segmentation fault if executed inside a transaction or block. This is because starting a new transaction in the middle of the statement will reset the SPI stack. This commit fixes this by raising an error if `CREATE MATERIALIZED VIEW` is not executed on top-level and `WITH DATA` is used either directly or indirectly. Closes #2371 " 4f7426299177b056c07288ab20aabcbebb27c620,Erik Nordström,Mon Sep 14 10:13:29 2020 +0200,Filter materialized hypertables in view,"This change filters materialized hypertables from the hypertables view, similar to how internal compression hypertables are filtered. Materialized hypertables are internal objects created as a side effect of creating a continuous aggregate, and these internal hypertables are still listed in the continuous_aggregates view. Fixes #2383 " 1068188128622d7facf89e9cefe765edaeb7bc45,Erik Nordström,Mon Sep 14 08:34:02 2020 +0200,Make block chunks API internal,This change moves the block chunks functionality to the internal namespace since it won't be part of the public API for the 2.0 release. Fixes #2236 c9539b95eaba1fefdb8b01191712ee4765b26323,Sven Klemm,Sun Sep 13 19:08:04 2020 +0200,Fix segfault in ALTER VIEW SET,Fix a segfault in ALTER VIEW when trying to SET view options on normal postgres views. 0d8fc618866dc7aed1d1003d60f190e1cbf71039,Sven Klemm,Mon Sep 14 09:22:56 2020 +0200,Make regression test names consistent, a01e729375b5d2af9b3db3927667db7fad3a579e,Sven Klemm,Sun Sep 13 01:48:05 2020 +0200,Remove duplicate lines from apt package test, e28ce96141cb3dcd8f9de30ab1ac852d887fb86c,Sven Klemm,Fri Sep 11 18:09:34 2020 +0200,Check sequence values don't get reset in update test, aed711760a732fb71bca695d202c753f41a41f5b,Sven Klemm,Fri Sep 11 09:52:58 2020 +0200,Remember job_id sequence value in update script,When rebuilding the bgw_job table the update script wouldnt remember the state of the sequence and reset it back to the default leading to failed job inserts until the sequence catches up. b245360502e9320f3e1efdfc407171ae0fdb2416,Sven Klemm,Fri Sep 11 14:33:03 2020 +0200,Fix detection of hypertables in subqueries,When a hypertable was referenced in a subquery that was not already in our hypertable cache we would fail to detect it as hypertable leading to transparent decompression not working for that hypertable. 802524ec200c5f59c7f8f065de12a89237399d71,gayyappan,Wed Sep 2 12:48:39 2020 -0400,Migrate ignore_invalidation_older_than for continuous aggregates,"When the extension is updated to 2.0, we need to migrate existing ignore_invalidation_older_than settings to the new continuous aggregate policy framework. ignore_invalidation_older_than setting is mapped to start_interval of the refresh policy.If the default value is used, it is mapped to NULL start_interval, otherwise it is converted to an interval value. " 6d7edb99bad168cd633534a93e469fbace75fc20,Sven Klemm,Sat Sep 5 17:34:08 2020 +0200,Fix rename constraint/rename index,When a constraint is backed by an index like a unique constraint or a primary key constraint the constraint can be renamed by either ALTER TABLE RENAME CONSTRAINT or by ALTER INDEX RENAME. Depending on the command used to rename different internal metadata tables would be adjusted leading to corrupt metadata. This patch makes ALTER TABLE RENAME CONSTRAINT and ALTER INDEX RENAME adjust the same metadata tables. 202692f1ef5598a19e743eed4a5f275d51f2d0f6,Erik Nordström,Tue Sep 8 19:57:29 2020 +0200,Make tests use the new continuous aggregate API,"Tests are updated to no longer use continuous aggregate options that will be removed, such as `refresh_lag`, `max_interval_per_job` and `ignore_invalidation_older_than`. `REFRESH MATERIALIZED VIEW` has also been replaced with `CALL refresh_continuous_aggregate()` using ranges that try to replicate the previous refresh behavior. The materializer test (`continuous_aggregate_materialize`) has been removed, since this tested the ""old"" materializer code, which is no longer used without `REFRESH MATERIALIZED VIEW`. The new API using `refresh_continuous_aggregate` already allows manual materialization and there are two previously added tests (`continuous_aggs_refresh` and `continuous_aggs_invalidate`) that cover the new refresh path in similar ways. When updated to use the new refresh API, some of the concurrency tests, like `continuous_aggs_insert` and `continuous_aggs_multi`, have slightly different concurrency behavior. This is explained by different and sometimes more conservative locking. For instance, the first transaction of a refresh serializes around an exclusive lock on the invalidation threshold table, even if no new threshold is written. The previous code, only took the heavier lock once, and if, a new threshold was written. This new, and stricter locking, means that insert processes that read the invalidation threshold will block for a short time when there are concurrent refreshes. However, since this blocking only occurs during the first transaction of the refresh (which is quite short), it probably doesn't matter too much in practice. The relaxing of locks to improve concurrency and performance can be implemented in the future. " 1bf458052035b2a0b24c12346ac1f9032df4d028,Erik Nordström,Fri Sep 11 09:19:25 2020 +0200,Restructure SQL API for policies and jobs,"This moves the SQL definitions for policy and job APIs to their separate files to improve code structure. Previously, all of these user-visible API functions were located in the `bgw_scheduler.sql` file, mixing internal and public functions and APIs. To improved the structure, all API-related functions are now located in their own distinct SQL files that have the `_api.sql` file ending. Internal policy functions have been moved to `policy_internal.sql`. " 07ebd5c9b2e821a0ac48899b610d100b4f1b9f9b,Erik Nordström,Fri Sep 11 09:06:11 2020 +0200,Rename continuous aggregate policy API,This change simplifies the name of the functions for adding and removing a continuous aggregate policy. The functions are renamed from: - `add_refresh_continuous_aggregate_policy` - `remove_refresh_continuous_aggregate_policy` to - `add_continuous_aggregate_policy` - `remove_continuous_aggregate_policy` Fixes #2320 9565cbd0f71fd79152cd58f2a299313a375f207d,Mats Kindahl,Thu Sep 10 07:02:10 2020 +0200,Continuous aggregates support WITH NO DATA,This commit will add support for `WITH NO DATA` when creating a continuous aggregate and will refresh the continuous aggregate when creating it unless `WITH NO DATA` is provided. All test cases are also updated to use `WITH NO DATA` and an additional test case for verifying that both `WITH DATA` and `WITH NO DATA` works as expected. Closes #2341 86f8b9d481497851d9c2fc2f3f69b64800ebbedc,Sven Klemm,Thu Sep 10 12:07:21 2020 +0200,Run 32bit regression tests on PR,Enable 32-bit regression tests on pull requests to catch improper use of datatype conversion functions earlier. a2185329411aef24cad35b4892a4f21197f9b24d,Sven Klemm,Thu Sep 10 17:00:33 2020 +0200,Fix cross-platform explain output differences,"Enforce index scan for queries that would produce different output between 32bit and 64bit platform to make explain output for constraint_exclusion_prepared, ordered_append and ordered_append_join test output consistent across platforms. " f3242fe9323ac33bf7c509c2857142e40d171a59,Mats Kindahl,Wed Sep 9 14:28:09 2020 +0200,Support continuous aggregates table access method,If a table access method is provided when creating a continuous aggregate using `CREATE MATERIALIZED VIEW` it will be used to set the table access method for the materialized hypertable. Closes #2123 97af7d855db0565d031f563f252f498f764613ef,gayyappan,Thu Sep 3 11:58:26 2020 -0400,Fix invalidation log initialization,The invalidation log initialization did not adjust the timestamp to internal format. Use ts_get_now_internal in all places that write entries to the invalidation log. dd90ffd0099ad785e0bf56dd12e5babb55b59fb3,gayyappan,Wed Sep 9 16:18:20 2020 -0400,Report docker build failures for update test,The update tests use the scripts for building the timescaledb image. Build errors should be reported and return a failure code. This will force the failure of the rest of the pipeline. e6a61c065ce1aa38f3c1699f9defdc85f5fd9464,Erik Nordström,Tue Sep 8 08:48:56 2020 +0200,Fix hint when dropping chunks with dependencies,"When dropping chunks that have dependent objects, like a continuous aggregate view, a dependent object error is raised. The hint message is overridden to produce a more useful hint for the drop chunks use case. However, the hint is overriden without checking the error code, which means the hint is replaced for any error raised. This can produce confusing log output. The issue is fixed by first examining the error code before overriding the hint. " caf64357f4a7ba5c01d83a6409026e51e8507696,Erik Nordström,Tue Sep 1 10:43:22 2020 +0200,Handle dropping chunks with continuous aggregates,"This change makes the behavior of dropping chunks on a hypertable that has associated continuous aggregates consistent with other mutations. In other words, any way of deleting data, irrespective of whether this is done through a `DELETE`, `DROP TABLE ` or `drop_chunks` command, will invalidate the region of deleted data so that a subsequent refresh of a continuous aggregate will know that the region is out-of-date and needs to be materialized. Previously, only a `DELETE` would invalidate continuous aggregates, while `DROP TABLE ` and `drop_chunks` did not. In fact, each way to delete data had different behavior: 1. A `DELETE` would generate invalidations and the materializer would update any aggregates to reflect the changes. 2. A `DROP TABLE ` would not generate invalidations and the changes would therefore not be reflected in aggregates. 3. A `drop_chunks` command would not work unless `ignore_invalidation_older_than` was set. When enabled, the `drop_chunks` would first materialize the data to be dropped and then never materialize that region again, unless `ignore_invalidation_older_than` was reset. But then the continuous aggregates would be in an undefined state since invalidations had been ignored. Due to the different behavior of these mutations, a continuous aggregate could get ""out-of-sync"" with the underlying hypertable. This has now been fixed. For the time being, the previous behavior of ""refresh-on-drop"" (i.e., materializing the data on continuous aggregates before dropping it) is retained for `drop_chunks`. However, such ""refresh-on-drop"" behavior should probably be revisited in the future since it happens silently by default without an opt out. There are situations when such silent refreshing might be undesirable; for instance, let's say the dropped data had seen erroneous backfill that a user wants to ignore. Another issue with ""refresh-on-drop"" is that it only happens for `drop_chunks` and not other ways of deleting data. Fixes #2242 " 3f7c5d22c768dcc55b04c50b3c4a1196832d9f2b,gayyappan,Wed Sep 9 11:07:52 2020 -0400,Continuous aggregate view changes,"With the new continuous aggregate API, some of the parameters used to create a continuous agg are now obsolete. Remove refresh_lag, max_interval_per_job and ignore_invalidation_older_than information from timescaledb_information.continuous_aggregates. " f49492b83dd2b206cff20d5537cb68ab8d0596a5,Erik Nordström,Mon Sep 7 17:07:43 2020 +0200,Cap invalidation threshold at last data bucket,"When refreshing with an ""infinite"" refresh window going forward in time, the invalidation threshold is also moved forward to the end of the valid time range. This effectively renders the invalidation threshold useless, leading to unnecessary write amplification. To handle infinite refreshes better, this change caps the refresh window at the end of the last bucket of data in the underlying hypertable, as to not move the invalidation threshold further than necessary. For instance, if the max time value in the hypertable is 11, a refresh command such as: ``` CALL refresh_continuous_aggregate(NULL, NULL); ``` would be turned into ``` CALL refresh_continuous_aggregate(NULL, 20); ``` assuming that a bucket starts at 10 and ends at 20 (exclusive). Thus the invalidation threshold would at most move to 20, allowing the threshold to still do its work once time again moves forward and beyond it. Note that one must never process invalidations beyond the invalidation threshold without also moving it, as that would clear that area from invalidations and thus prohibit refreshing that region once the invalidation threshold is moved forward. Therefore, if we do not move the threshold further than a certain point, we cannot refresh beyond it either. An alternative, and perhaps safer, approach would be to always invalidate the region over which the invalidation threshold is moved (i.e., new_threshold - old_threshold). However, that is left for a future change. It would be possible to also cap non-infinite refreshes, e.g., refreshes that end at a higher time value than the max time value in the hypertable. However, when an explicit end is specified, it might be on purpose so optimizing this case is also left for the future. Closes #2333 " 4f324393629ad9aff51fefac5d0296bb126a5db4,Mats Kindahl,Tue Sep 1 16:56:35 2020 +0200,Update tablespace of table on attach and detach,"If a tablespace is attached to a hypertable the tablespace of the hypertable is not set, but if the tablespace is set it is also attached. A similar situation occurs if tablespaces are detached. This means that if a hypertable is created with a tablespace and then all tablespaces are detached, the chunks will still be put in the tablespace of the hypertable. With this commit, attaching a tablespace to a hypertable will set the tablespace of the hypertable if it does not already have one. Detaching a tablespace from a hypertable will set the tablespace to the default tablespace if the tablespace being detached is the tablespace for the hypertable. If `detach_tablespace` is called with only a tablespace name, it will be detached from all tables it is attached to. This commit ensures that the tablespace for the hypertable is set to the default tablespace if it was set to the tablespace being detached. Fixes #2299 " 74b01fbd1bee8b886d1c3595a0f0cee9c8a6e575,Sven Klemm,Tue Sep 8 19:57:40 2020 +0200,Add 1.7.4 to update test scripts, 8797493cb2f5007f9aac55758fc40848650ca9e3,Sven Klemm,Mon Sep 7 21:09:12 2020 +0200,Release 1.7.4,This maintenance release contains bugfixes since the 1.7.3 release. We deem it high priority for upgrading if TimescaleDB is deployed with replicas (synchronous or asynchronous). In particular the fixes contained in this maintenance release address an issue with running queries on compressed hypertables on standby nodes. **Bugfixes** * #2340 Remove tuple lock on select path 23cbb0998300ecf85e6d16bf4d29c3fe8f6bdd8b,Mats Kindahl,Mon Sep 7 19:54:32 2020 +0200,Remove tuple lock on select path,"When executing a SELECT on a hypertable, it is not possible to acquire tuple locks on hot standbys since they require a transaction id and transaction ids cannot be created when running as a standby running in ephemeral recovery mode. This commit removes the tuple lock from the SELECT code path if running in recovery mode. " f30d15a99779db4ee72fa4260e3e7d14a43b7e5f,Dmitry Simonenko,Tue Sep 8 14:08:14 2020 +0300,Block compression policy with dist hypertable,Issue: #2193 0641f9dba2af75ad3ea1048c2043ab48f034a696,Sven Klemm,Mon Sep 7 17:54:22 2020 +0200,Fix segfaults in tests in 32bit environment,While DatumGetInt64 is a noop on 64-bit systems calling it on the wrong underlying type will lead to segfaults on 32-bit systems. eb5420e485a6a47e238ea569ede300020081ba1c,Sven Klemm,Sat Sep 5 21:58:26 2020 +0200,Fix telemetry handling in background worker scheduler,This patch changes the scheduler to ignore telemetry jobs when telemetry is disabled. With this change telemetry jobs will no longer use background worker resources when telemetry is disabled. d8acf29a0f19dfcd004de426e1c96e1d609de06d,Mats Kindahl,Fri Sep 4 08:52:45 2020 +0200,Add common patterns to .gitignore,Patterns `#*#` and `.#*` are for auto-genereated files from Emacs and can end up in source directories. Pattern `.clangd` is the working directory for `clangd` and handles source code indexing. 5a5255e6b2a23a8354aec4a1fef836e83051a1dd,Mats Kindahl,Mon Sep 7 11:41:19 2020 +0200,Change start and end interval for policy in test,"If a hypertable has a time type that is a date, any interval of less than a day will be truncated to the same day. This created a time-triggered test failure in `continuous_aggs_policy` and this commit changes it to use full days for the start and end interval. " 9e6609cdad2f0b30c228a53c05cc6a1c7e0b5ec7,Mats Kindahl,Mon Sep 7 13:40:58 2020 +0200,Extend timeout for isolation test,"The test `deadlock_dropchunks_select` get a timeout on MacOS, but not on other platforms, so this commit extend the lock timeout to make the test pass. " 9d4322486153d8bfa6c9aa1fac6ba7ee520f9401,Sven Klemm,Mon Sep 7 14:06:04 2020 +0200,Bump appveyor docker image to postgres 12.4,Due to a version conflict between the 12.2 and the 12.4 postgres packages in alpine the 12.2 package was not installable on recent alpine images. This patch bumps the involved packages to the latest postgres version and also uses the most recent postgres image for running appveyor tests. bc9726607e57bea7bbc294a0f3cc7eea64bfa5cb,Erik Nordström,Fri Aug 28 11:14:38 2020 +0200,Use end of last bucket as cagg watermark,"The function `cagg_watermark` returns the time threshold at which materialized data ends and raw query data begins in a real-time aggregation query (union view). The watermark is simply the completed threshold of the continuous aggregate materializer. However, since the completed threshold will no longer exist with the new continuous aggregates, the watermark function has been changed to return the end of the last bucket in the materialized hypertable. In most cases, the completed threshold is the same as the end of the last materialized bucket. However, there are situations when it is not; for example, when there is a filter in the view query some buckets might not be materialized because no data matched the filter. The completed threshold would move ahead regardless. For instance, if there is only data from ""device_2"" in the raw hypertable and the aggregate has a filter `device=1`, there will be no buckets materialized although the completed threshold moves forward. Therefore the new watermark function might sometimes return a lower watermark than the old function. A similar situation explains the different output in one of the union view tests. " 417b66e9741ee3b80507232d3c275a2b18fa982b,Erik Nordström,Thu Sep 3 11:33:47 2020 +0200,Fix boundary handling in time types and constraints,"Time types, like date and timestamps, have limits that aren't the same as the underlying storage type. For instance, while a timestamp is stored as an `int64` internally, its max supported time value is not `INT64_MAX`. Instead, `INT64_MAX` represents `+Infinity` and the actual largest possible timestamp is close to `INT64_MAX` (but not `INT64_MAX-1` either). The same applies to min values. Unfortunately, time handling code does not check for these boundaries; in most cases, overflow handling when, e.g., bucketing, are checked against the max integer values instead of type-specific boundaries. In other cases, overflows simply throw errors instead of clamping to the boundary values, which makes more sense in many situations. Using integer time suffers from similar issues. To take one example, simply inserting a valid `smallint` value close to the max into a table with a `smallint` time column fails: ``` INSERT INTO smallint_table VALUES ('32765', 1, 2.0); ERROR: value ""32770"" is out of range for type smallint ``` This happens because the code that adds dimensional constraints always checks for overflow against `INT64_MAX` instead of the type-specific max value. Therefore, it tries to create a chunk constraint that ends at `32770`, which is outside the allowed range of `smallint`. The resolve these issues, several time-related utility functions have been implemented that, e.g., return type-specific range boundaries, and perform saturated addition and subtraction while clamping to supported boundaries. Fixes #2292 " 1fa072acb47f354a66058af7cf17bdb8010020a3,Ruslan Fomkin,Thu Sep 3 10:44:58 2020 +0200,Disable gapfill on distributed hypertable,There are number of issues when time_bucket_gapfill is run on distributed hypertable. Thus a non-supported error is returned in this case until the issues are fixed. d5a6a5b1934b81a966eb54be902b7268734037aa,Mats Kindahl,Fri Aug 28 15:51:25 2020 +0200,Support tablespaces for continuous aggregates,Adding support for tablespaces when creating a continuous aggregate using `CREATE MATERIALIZED VIEW` and when altering a continuous aggregate using `ALTER MATERIALIZED VIEW`. Fixes #2122 4179f4f324a529e254f846b31f1fb977a9351698,Ruslan Fomkin,Mon Aug 24 12:34:59 2020 +0200,Ignore build and IDE folders,Updating gitignore to exclude default build folders and CLion folder. 78c2b9a16298a8dc049c0eceafa5a07f302bf07d,Sven Klemm,Fri Sep 4 11:13:04 2020 +0200,Fix copy/paste error in AT_CookedColumnDefault handling, 74dabc4c77063071b8c4b7745ec4b8e4f985ea14,Mats Kindahl,Thu Sep 3 12:41:12 2020 +0200,Make tests with tablespaces solo tests,"Tablespaces are created cluster-wide, which means that tests that create tablespaces cannot run together with other tests that create the same tablespaces. This commit make those tests into solo tests to avoid collisions with other tablespace-creating tests and also fix a test. " e10b4377122ba213df9e877f85f9bbfc15b94141,Dmitry Simonenko,Wed Sep 2 12:04:58 2020 +0300,Make hypertable_approximate_row_count return row count only,This change renames function to approximate_row_count() and adds support for regular tables. Return a row count estimate for a table instead of a table list. 3d880ecd2b28e56cfd7b73193bc292ae3e4a8e18,Sven Klemm,Tue Sep 1 23:16:37 2020 +0200,Fix segfault in caggs on 32-bit systems,While DatumGetInt64 and Int64GetDatum are noops on 64-bit systems calling them on the wrong underlying type can lead to segfaults on 32-bit systems. b355a86b4e6c55ce2004f85fcb3261aa69f96684,gayyappan,Tue Sep 1 17:36:26 2020 -0400,Fix test failures on appveyor,Some tests fail on appveyor due to background worker timing issues or difference in timestamp outputs on different platforms. Fix affected tests bgw_reorder_drop_chunks and continuous_aggs_bgw. 97b4d1cae229a0d6691b86032fe03022fc531d41,gayyappan,Fri Aug 14 16:48:21 2020 -0400,Support refresh continuous aggregate policy,Support add and remove continuous agg policy functions Integrate policy execution with refresh api for continuous aggregates The old api for continuous aggregates adds a job automatically for a continuous aggregate. This is an explicit step with the new API. So remove this functionality. Refactor some of the utility functions so that the code can be shared by multiple policies. 4bc88cb6948f660bc8ec2f038fa35e38f24f6519,Sven Klemm,Tue Sep 1 16:51:27 2020 +0200,Merge index and reindex test,Combine index and reindex test to reduce number of test cases. 7f0ec49fa2062e280fb6ad813583d4cd4c56ce3f,Sven Klemm,Mon Aug 31 21:42:58 2020 +0200,Merge ddl and ddl_alter_column test, 3859b5a6d2a0e728a7c314980e151654b935c397,Sven Klemm,Mon Aug 31 21:22:36 2020 +0200,Change ddl test to not depend on PG version, d4240becda5216a8b396de76074a14a2641bce35,Sven Klemm,Mon Aug 31 13:53:14 2020 +0200,Remove ddl_single test,The ddl_single test was almost exactly the same as the ddl test except for 5 statements not part of the ddl_single test. So the ddl_single test can safely be removed. 4397e5749760483d6227c54693dc01f7fbe55a9e,Sven Klemm,Thu Aug 20 12:39:38 2020 +0200,Remove job_type from bgw_job table,Due to recent refactoring all policies now use the columns added with the generic job support so the job_type column is no longer needed. 08f6e6f2adef262e3a0ebad7ac3a07a814cce244,Sven Klemm,Fri Aug 28 16:13:39 2020 +0200,Remove Ubuntu 19.10 from APT package test,This patch removes Ubuntu 19.10 from APT package test because 19.10 is no longer supported by Ubuntu and we don't build packages for it anymore. 8bd7fcc123a3ae0a6031ecf495abe2e64dad7903,Erik Nordström,Sat Aug 29 14:48:18 2020 +0200,Add TRUNCATE support for continuous aggregates,"This change adds the ability to truncate a continuous aggregate and its source hypertable. Like other mutations (DELETE, UPDATE) on a continuous aggregate or the underlying hypertable, an invalidation needs to be added to ensure the aggregate can be refreshed again. When a hypertable is truncated, an invalidation is added to the hypertable invalidation log so that all its continuous aggregates will be invalidated. When a specific continuous aggregate is truncated, the invaliation is instead added to the continuous aggregate invalidation log, so that only the one aggregate is invalidated. " 829905769ad7c11e7b1a3a2ed560eed940056205,Erik Nordström,Mon Aug 31 14:34:44 2020 +0200,Fix issues related to non-inclusive refresh window,"This change fixes some corner-case issues that could lead to a refresh not actually refreshing when it should. The issues arise because invalidations are inclusive in both ends while the refresh window is exclusive in the end. In some case, this wasn't correctly accounted for. To fix the issues, the remainder after cutting invalidations has been adjusted and we always add 1 to the end of the refresh window when it is set from an invalidation. In addition, we add an extra check for this case when computing the bucketed refresh window and the end of the refresh window is at the start of the bucket (i.e., the window size is 1). The test suite has also been expanded to test for some of these corner cases. " 509e48ea9ad3dca3f43b638b50fd3cceb05dc6a6,Sven Klemm,Thu Aug 27 20:08:02 2020 +0200,Handle AT_CookedColumnDefault in process_utility,Postgres added a new alter table subcommand in a bugfix for CREATE TABLE LIKE with inheritance which got backported to previous versions. We check for presence of the subcommand with cmake so we can support building on snapshot versions which also fixes the ABI breakage test. https://github.com/postgres/postgres/commit/50289819230d8ddad510879ee4793b04a05cf13b d19f93e1915cb03285c49b27d09b7723b597cf5c,Sven Klemm,Mon Aug 31 15:11:46 2020 +0200,Merge telemetry_community and telemetry_compression, 82f82fbdafcbf582c2133c78c305435d880ce275,Sven Klemm,Sun Aug 30 00:57:38 2020 +0200,Fix ts_jsonb_add_int64 for 32-bit systems,ts_jsonb_add_int64 used int4_numeric instead of int8_numeric to do numeric conversion which lead to test failures on 32 bit systems. cb2da81bf78b79db5be2a9ac7628385ecf9431c5,Dmitry Simonenko,Mon Aug 31 14:19:18 2020 +0300,Fix ts_get_now_internal to use transaction time,Issue: #2167 4538fc6c40fc8d0e024f2b41f1fe6f66719b0026,Erik Nordström,Sat Aug 8 10:15:49 2020 +0200,Optimize continuous aggregate refresh,"This change ensures a refresh of a continuous aggregate only re-materializes the part of the aggregate that has been invalidated. This makes refreshing much more efficient, and sometimes eliminates the need to materialize data entirely (i.e., in case there are no invalidations in the refresh window). The ranges to refresh are the remainders of invalidations after they are cut by the refresh window (i.e., all invalidations, or parts of invalidations, that fall within the refresh window). The invalidations used for a refresh are collected in a tuple store (which spills to disk) as to not allocate too much memory in case of many invalidations. Invalidations are, however, merged and deduplicated before being added to the tuplestore, similar to how invalidations are processed in the invalidation logs. Currently, the refreshing proceeds with just materializing all invalidated ranges in the order they appear in the tuple store, and the ordering does not matter since all invalidated regions are refreshed in the same transaction. " 5b8ff384dddb39fc1a9f0128058ca34967af5472,Erik Nordström,Fri Aug 7 16:20:35 2020 +0200,Add infinite invalidations to cagg log,"In its initial state, a continuous aggregate should be completely invalidated. Therefore, this change adds an infinite invalidation `[-Infinity, +Infinity]` when a continuous aggregate is created. " c0e0f673b87f123fa8f85aa723612dd6e5620bf6,Erik Nordström,Thu Nov 28 16:51:47 2019 +0100,Minor refactor of distributed xact handling,Cleanup some inconsistencies and address nits in abort handlers for distributed transactions. 506b1189b1065f543873b22042e93ea0d8469d03,Erik Nordström,Tue Nov 26 15:12:06 2019 +0100,Refactor node killer test functionality,"This change refactors and cleans up some of the test infrastructure around distributed transactions. In particular, the node killer now waits for the killed process to exit in an attempt to make tests more predictible. " 7f93faad0205de703aeb8fd9dce8d2d4bd8e895b,Sven Klemm,Sat Aug 29 22:03:06 2020 +0200,Fix dist_hypertable test to use unique data node names,Change dist_hypertable test to use unique data node names. 0fa778d1defda63a9d84fd841bced19ea81bb17a,Sven Klemm,Sat Aug 29 21:11:43 2020 +0200,Fix dist_compression test to enable parallel execution,Change dist_compression test to use unique data node name so it can be run in parallel. 90a5995dfb88fad862bc358249d510537e00f81c,Sven Klemm,Sat Aug 29 20:28:52 2020 +0200,Merge deparse and deparse_fail test, 6ad98a45bb97bf7014f1abf0fd5982f7c7896947,Sven Klemm,Sat Aug 29 19:49:40 2020 +0200,Move license change test to regresscheck-shared, 9ae409259a328acb797601472af8db305ef13166,Sven Klemm,Sat Aug 29 17:41:02 2020 +0200,Merge gapfill tests into single test, 66f80d159b50323762e5d231509e9825e73a8143,Sven Klemm,Sat Aug 29 17:29:52 2020 +0200,Fix handling of non-reference counted TupleDescs in gapfill,Use ReleaseTupleDesc instead of DecrTupleDescRefCount in gapfill interpolate because the latter does not account for non-reference counted TupleDescs. 1eceee3151d1f8530e46286e5b262d6b71887749,Sven Klemm,Fri Aug 28 11:09:17 2020 +0200,Fix sanitizer test script,This patch removes the -t and -i options from docker exec since this script is not interactive and not run in terminal on CI. 4608529aba07517b6a54f5b346ecb90a2c527b06,Sven Klemm,Thu Aug 27 21:00:47 2020 +0200,Fix format string in debug_wait.c, 470ad5842f8a35cd873d52490b9b83e6b1620df9,Sven Klemm,Thu Aug 27 20:37:20 2020 +0200,Fix test_time_to_internal for 32-bit systems,Some tests in test_time_to_internal were missing GetDatum calls which made the tests fail on 32-bit systems. c5a202476e0e795448157290cec6276c2fcfb083,Erik Nordström,Wed Aug 26 13:34:22 2020 +0200,Fix timestamp overflow in time_bucket optimization,"An optimization for `time_bucket` transforms expressions of the form `time_bucket(10, time) < 100` to `time < 100 + 10` in order to do chunk exclusion and make better use of indexes on the time column. However, since one bucket is added to the timestamp when doing this transformation, the timestamp can overflow. While a check for such overflows already exists, it uses `+Infinity` (INT64_MAX/DT_NOEND) as the upper bound instead of the actual end of the valid timestamp range. A further complication arises because TimescaleDB internally converts timestamps to UNIX epoch time, thus losing a little bit of the valid timestamp range in the process. Dates are further restricted by the fact that they are internally first converted to timestamps (thus limited by the timestamp range) and then converted to UNIX epoch. This change fixes the overflow issue by only applying the transformation if the resulting timestamps or dates stay within the valid (TimescaleDB-specific) ranges. A test has also been added to show the valid timestamp and date ranges, both PostgreSQL and TimescaleDB-specific ones. " c054b381c657bb3ecc81edd16b2e58910ded506b,Mats Kindahl,Thu Aug 20 09:38:23 2020 +0200,Change syntax for continuous aggregates,"We change the syntax for defining continuous aggregates to use `CREATE MATERIALIZED VIEW` rather than `CREATE VIEW`. The command still creates a view, while `CREATE MATERIALIZED VIEW` creates a table. Raise an error if `CREATE VIEW` is used to create a continuous aggregate and redirect to `CREATE MATERIALIZED VIEW`. In a similar vein, `DROP MATERIALIZED VIEW` is used for continuous aggregates and continuous aggregates cannot be dropped with `DROP VIEW`. Continuous aggregates are altered using `ALTER MATERIALIZED VIEW` rather than `ALTER VIEW`, so we ensure that it works for `ALTER MATERIALIZED VIEW` and gives an error if you try to use `ALTER VIEW` to change a continuous aggregate. Note that we allow `ALTER VIEW ... SET SCHEMA` to be used with the partial view as well as with the direct view, so this is handled as a special case. Fixes #2233 Co-authored-by: =?UTF-8?q?Erik=20Nordstr=C3=B6m?= Co-authored-by: Mats Kindahl " ba93cbf50c271b4b5ade1633e3879d85402817a5,Sven Klemm,Thu Aug 27 13:22:03 2020 +0200,Add 1.7.3 to update test scripts, 843790a847010a251cef52a3341baa00c29a5aa5,Sven Klemm,Mon Aug 24 17:40:07 2020 +0200,Release 1.7.3,"This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in compression, drop_chunks and the background worker scheduler. **Bugfixes** * #2059 Improve infering start and stop arguments from gapfill query * #2067 Support moving compressed chunks * #2068 Apply SET TABLESPACE for compressed chunks * #2090 Fix index creation with IF NOT EXISTS for existing indexes * #2092 Fix delete on tables involving hypertables with compression * #2164 Fix telemetry installed_time format * #2184 Fix background worker scheduler memory consumption * #2222 Fix `negative bitmapset member not allowed` in decompression * #2255 Propagate privileges from hypertables to chunks * #2256 Fix segfault in chunk_append with space partitioning * #2259 Fix recursion in cache processing * #2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @prathamesh-sonpatki for contributing a typo fix * @sezaru for reporting an issue with background worker scheduler memory consumption " e33fc8c199a7bd222aad6d8d783f6803905be213,Matvey Arye,Tue Aug 25 12:03:12 2020 -0400,Fix recursion in cache processing,"Previously, cache invalidation could cause recursion in cache processing. PR #1493 fixed this for the cache_invalidate_callback() -> ts_extension_invalidate() call path. But the call path cache_invalidate_callback() -> ts_extension_is_loaded() could still go into recursion. So, this PR moves the recursion-prevention logic into extension_update_state(), which is common to both call paths. Fixes #2200. " 5300b682088c3875299e4e80e087f8aa418451ca,Dmitry Simonenko,Wed Aug 26 11:52:46 2020 +0300,Add test for hypertable_approximate_row_count() on dist hypertable,Issues: #1902 769bc31dc20d30468eb9c36b9171e109bedde4b3,Mats Kindahl,Wed Jul 29 12:29:07 2020 +0200,Lock dimension slice tuple when scanning,"In the function `ts_hypercube_from_constraints` a hypercube is build from constraints which reference dimension slices in `dimension_slice`. As part of a run of `drop_chunks` or when a chunk is explicitly dropped as part of other operations, dimension slices can be removed from this table causing the dimension slices to be removed, which makes the hypercube reference non-existent dimension slices which subsequently causes a crash. This commit fixes this by adding a tuple lock on the dimension slices that are used to build the hypercube. If two `drop_chunks` are running concurrently, there can be a race if dimension slices are removed as a result removing a chunk. We treat this case in the same way as if the dimension slice was updated: report an error that another session locked the tuple. Fixes #1986 " f263087a689594c4a59e2c3bb4e1abf24d3e198a,Mats Kindahl,Mon Aug 17 19:20:38 2020 +0200,Do not crash on broken tables,"If a hypertable accidentally broke because a dimension slice is missing, a segmentation fault will result when an attempt is made to remove a chunk that references the dimension slice. This happens because no check if the dimension slice was found is made and the assumption is that it should be there (by design). Instead of crashing the server, this commit adds code that prints a warning that the dimension slice did not exist and proceed with removing the chunk. This is safe since the chunk should be removed anyway and the missing dimension slice does not change this. " 8b2c30cee1691ecbb2ccdc1cba4a79a2a19710dd,Mats Kindahl,Mon Aug 17 19:11:49 2020 +0200,Add debug waitpoint implementation,"Add implementation for debug waitpoints. Debug waitpoints can be added to code and can be enabled using the `ts_debug_waitpoint_enable` function. Once execution reaches this point, the session will block waiting for a call to `ts_debug_waitpoint_release`. Waitpoints are added to code by using the macro `DEBUG_WAITPOINT` with a string, for example: DEBUG_WAITPOINT(""references_fetched""); The string is hashed to compute a 32-bit number that is then used as an shared advisory lock. The waitpoint can be enabled with the function `ts_debug_waitpoint_enable`. This function takes a string and computes a hash and will use the hash to take an exclusive advisory lock. This will cause all sessions reaching the waitpoint to block until the lock is released with `ts_debug_waitpoint_release`. " 3d9e77e316c72b98459f3f0bf951c3532a6e8285,Sven Klemm,Tue Aug 25 10:21:05 2020 +0200,Fix segfault in chunk_append with space partitioning,When postgres prunes children before we create the ChunkAppend path there might be a mismatch between the children of the path and the ordered list of children in a space partitioned hypertable. Fixes #1841 f8727756a6561d72cc462626e1ae04813308a690,Erik Nordström,Sat Aug 22 00:59:21 2020 +0200,Cleanup drop and show chunks,"This change removes, simplifies, and unifies code related to `drop_chunks` and `show_chunks`. As a result of prior changes to `drop_chunks`, e.g., making table relid mandatory and removing cascading options, there's an opportunity to clean up and simplify the rather complex code for dropping and showing chunks. In particular, `show_chunks` is now consistent with `drop_chunks`; the relid argument is mandatory, a continuous aggregate can be used in place of a hypertable, and the input time ranges are checked and handled in the same way. Unused code is also removed, for instance, code that cascaded drop chunks to continuous aggregates remained in the code base while the option no longer exists. " 81ed667392f4b7f4b2334dd4a6a0e9e90e13a9a3,Mats Kindahl,Mon Aug 24 17:34:01 2020 +0200,Use saved snapshot for scanner,"When scan is started using `ts_scanner_start_scan` it create a snapshot to use when scanning the data, but this is not subsequently used in `ts_scanner_next`. This commit fixes this by using the already created snapshot instead of calling `GetLastSnapshot()`. " e058236311f5cf7a19f3d9a2f8f03920f7fff126,Sven Klemm,Mon Aug 24 11:55:34 2020 +0200,Fix segfault when executing jobs with NULL config,This patch adds check for NULL config as makeConst will segfault when trying to detoast datum when passed a NULL config. 1c92a29ba3dd686efb248b7801047a22e7195798,Sven Klemm,Mon Aug 24 13:18:19 2020 +0200,Fix macos ci builds,"Since GitHub updated the macos image used for CI it updated llvm to a version that is buggy on macos. So this patch disables building postgres with llvm support on macos. It also fixes the cache suffix code for macos because it changed with the latest runner version, and uses the image version now as cache suffix. https://bugs.llvm.org/show_bug.cgi?id=47226 " 58b462234632e9ca7cfedccb395ae4da348c334a,Sven Klemm,Mon Aug 24 10:47:17 2020 +0200,Fix assertion failure in job_stat error handling,Since CopyErrorData asserts that the current MemoryContext is not ErrorContext we need to make sure to switch context before copying ErrorData. 8e1e6036af01124015451bbcba26f3580d7a41bf,Brian Rowe,Wed Jun 17 16:44:47 2020 -0700,Preserve pg_stats on chunks before compression,"This change will ensure that the pg_statistics on a chunk are updated immediately prior to compression. It also ensures that these stats are not overwritten as part of a global or hypertable targetted ANALYZE. This addresses the issue that chunk will no longer generate valid statistics durings an ANALYZE once the data's been moved to the compressed table. Unfortunately any compressed rows will not be captured in the parent hypertable's pg_statistics as there is no way to change how PostGresQL samples child tables in PG11. This approach assumes that the compressed table remains static, which is mostly correct in the current implementation (though it is possible to remove compressed segments). Once we start allowing more operations on compressed chunks this solution will need to be revisited. Note that in PG12 an approach leveraging table access methods will not have a problem analyzing compressed tables. " 33d5d11821a153b20c2b5e9084c68db10e39b2bb,Dmitry Simonenko,Fri Aug 21 12:17:48 2020 +0300,Check CREATE INDEX with transaction per chunk using dist hypertable,Issue: #836 c281dcdb260745a6f813e5506e7ff74e3fcf90af,Sven Klemm,Fri Aug 21 08:28:17 2020 +0200,Fix segfault in alter_job,When trying to alter a job with NULL config alter_job did not set the isnull field for config and would segfault when trying to build the resultset tuple. aec7c59538b6aade1e588d329fbd33c14044edfe,Mats Kindahl,Wed Aug 19 16:25:53 2020 +0200,Block data migration for distributed hypertables,"Option `migrate_data` does not currently work for distributed hypertables, so we block it for the time being and generate an error if an attempt is made to migrate data when creating a distributed hypertable. Fixes #2230 " 043c29ba487cd5fac7e238a69b9549387e9aa157,Sven Klemm,Thu Aug 20 10:09:07 2020 +0200,Block policy API commands in read_only transaction, a9c087eb1e7c86f337375d670287f3b0c092a0d6,Sven Klemm,Thu Jul 23 10:48:25 2020 +0200,Allow scheduling custom functions as bgw jobs,"This patch adds functionality to schedule arbitrary functions or procedures as background jobs. New functions: add_job( proc REGPROC, schedule_interval INTERVAL, config JSONB DEFAULT NULL, initial_start TIMESTAMPTZ DEFAULT NULL, scheduled BOOL DEFAULT true ) Add a job that runs proc every schedule_interval. Proc can be either a function or a procedure implemented in any language. delete_job(job_id INTEGER) Deletes the job. run_job(job_id INTEGER) Execute a job in the current session. " a48a4646b037c30d63886b3800c4c5ea6efa4241,Erik Nordström,Tue Aug 18 13:08:54 2020 +0200,Improve the concurrent refresh test,"This change moves the invalidation threshold in the setup phase of the concurrent refresh test for continuous aggregates in order to generate invalidations. Without any invalidations, the invalidation logs are never really processed and thus not subjected to concurrency. " e9cf0add4930d2ef5424c65c779f77011f3be27a,Erik Nordström,Fri Aug 14 02:07:36 2020 +0200,Always use bucketed refresh window internally,"This change makes sure we always use the bucketed refresh window internally for processing invalidations, moving thresholds and doing the actual materialization. " c4ca66b76cabb6ae29f205a2babdbb757164dee2,Erik Nordström,Fri Aug 7 16:39:15 2020 +0200,Improve locking when processing invalidations,"Invalidation processing during refreshing of a continuous aggregate is now better protected against concurrent refreshes by taking the lock on the materialized hypertable before invalidation processing. Since invalidation processing is now split across two transactions, the first one processing the hypertable invalidation log and the second one processing the continuous aggregate invalidation lock, they are now separately protected by serializing around the invalidation threshold lock and the materialized hypertable lock, respectively. " 21d033f76a4f7aee4b82b56037c0b999f2fa3d50,Sven Klemm,Sat Aug 15 08:02:13 2020 +0200,Fix negative bitmapset member not allowed in decompression,Due to a typo in the code the relid of the relation instead of the rangetable index was used when building the pathkey for the DecompressChunk node. Since relids are unsigned but bitmapsets use signed int32 this lead to 'negative bitmapset member not allowed' being thrown as error when the relid is greater than INT32_MAX. This patch also adds an assert to prevent this from happening again. 7d230290b96ba1f1fdabbdfbf5bae333cb640ce7,Sven Klemm,Sun Aug 16 15:11:48 2020 +0200,Remove unnecessary exports in tsl library,Since almost all the functions in the tsl library are accessed via cross module functions there is no need to export the indivial functions. 6aea39147757b4a9e1d9a30bc7e396d99bb4cb62,Sven Klemm,Sat Aug 15 10:56:53 2020 +0200,Fix signature of cagg_watermark,This patch changes the signature from cagg_watermark(oid) to cagg_watermark(int). Since this is an API breaking change it couldn't be done in an earlier release. 91f64b8177173d1306b6ff77db890cdbb0c10386,Sven Klemm,Fri Aug 14 15:10:51 2020 +0200,Run regression tests on PG 11.9 and 12.4,This patch changes all regression tests that were running on 11.8 and 12.3 to 11.9 and 12.4. 9bc5c711f4d790aa1cacd8aa019285ad7e8b8a41,Mats Kindahl,Thu Aug 6 09:06:31 2020 +0200,Fix retention policy on distributed hypertables,"If a retention policy is set up on a distributed hypertable, it will not propagate the drop chunks call to the data nodes since the drop chunks call is done through an internal call. This commit fixes this by creating a drop chunks call internally and executing it as a function. This will then propagate to the data nodes. Fixes timescale/timescaledb-private#833 Fixes #2040 " c3ac8fa19300d5008bbb93a3eda97060ab65fe18,Erik Nordström,Mon Aug 10 13:30:20 2020 +0200,Allow use of infinite refresh windows,"When clearing invalidations during a refresh of a continuous aggregate, one should use the bucketed refresh window rather than the ""raw"" user-defined window. This ensures that the window is capped at the allowable timestamp range (e.g., when using an infinite window) and that the window used to clear invalidations actually matches what gets materialized. Note, however, that, since the end of the refresh window is not inclusive, the last possible time value is not included in the refresh. To include that value, there needs exist a time-type agnostic definition of ""infinity"" and both the invalidation code and the materialization code must be able to handle such windows. " e1c94484cf429a9726e1d29640a374832292ad43,Erik Nordström,Thu Aug 6 22:03:13 2020 +0200,Add support for infinite timestamps,"The internal conversion functions for timestamps didn't account for timestamps that are infinite (`-Infinity` or `+Infinity`), and they would therefore generate an error if such timestamps were encountered. This change adds extra checks to the conversion functions to allow infinite timestamps. " 418f283443dc34438427b1717dd19672d456e6bb,Erik Nordström,Fri Aug 7 23:25:22 2020 +0200,Merge continuous aggregate invalidations,"This change implements deduplication and merging of invalidation entries for continuous aggregates in order to reduce the number of reduntant entries in the continuous aggregate invalidation log. Merging is done both when copying over entries from the hypertable to the continuous aggregate invalidation log and when cutting already existing invalidations in the latter log. Doing this merging in both steps helps reduce the number of invalidations also for the continuous aggregates that don't get refreshed by the active refresh command. Merging works by scanning invalidations in order of the lowest modified value, and given this ordering it is possible to merge the current and next entry into one large entry if they are overlapping. This can continue until the current and next invalidation are disjoint or there are no more invalidations to process. Note, however, that only the continuous aggregate that gets refreshed will be fully deduplicated. Some redundant entries might exist for other aggregates since their entries in the continuous aggregate log aren't cut against the refresh window. Full deduplication for the refreshed continuous aggregate is only possible if the continuous aggregate invalidation log is processed last, since that also includes ""old"" entries. Therefore, this change also changes the ordering of how the logs are processed. This also makes it possible to process the hypertable invalidation log in the first transaction of the refresh. " 6874076a85f5cac91f78915fded166451c76614e,Sven Klemm,Wed Aug 12 20:53:01 2020 +0200,Split ordered_append test into 2 tests,This patch separates the ordered append join tests from the other ordered append tests. It also adds additional constraints to some queries to speed them up. These changes result in a 3x speedup for regresscheck-shared. f61818f3a75ba2ca92ddf326bc978ed30d0c0950,Sven Klemm,Wed Aug 12 14:45:20 2020 +0200,Limit resultsets for constraint_exclusion_prepared test,This patches limits the resultset for the constraint exclusion test with prepared statements to make them run in a more reasonable time. 02d715f216018d77218f5fd338cf35619c457299,Sven Klemm,Tue Aug 11 12:09:34 2020 +0200,Add distributed hypertable to regresscheck-shared,This patch sets up a distributed hypertable in the regresscheck-shared environment to enable running distributed tests. af0ed90f8541316f000a6b3042a8a162f62a5409,Erik Nordström,Mon Aug 10 10:25:46 2020 +0200,Move invalidation threshold code,This change moves the code to set and get the invalidation threshold for continuous aggregates to a separate source file for better code structure. c01faa72f0bcd7a54f942d9bfb541509862b5672,Erik Nordström,Mon Aug 10 09:31:23 2020 +0200,Set invalidation threshold during refresh,"The invalidation threshold governs the window of data from the head of a hypertable that shouldn't be subject to invalidations in order to reduce write amplification during inserts on the hypertable. When a continuous aggregate is refreshed, the invalidation threshold must be moved forward (or initialized if it doesn't previously exist) whenever the refresh window stretches beyond the current threshold. Tests for setting the invalidation threshold are also added, including new isolation tests for concurrency. " 80720206dfa710d22e305eb11543a3aae7803a01,Erik Nordström,Sun Aug 2 11:52:32 2020 +0200,Make refresh_continuous_aggregate a procedure,"When a continuous aggregate is refreshed, it also needs to move the invalidation threshold in case the refresh window stretches beyond the current threshold. The new invalidation threshold must be set in its own transaction during the refresh, which can only be done if the refresh command is a procedure. " b8ce74921a9df3cfa436a7c3005c50a85bdc30c7,Erik Nordström,Tue Aug 4 20:51:48 2020 +0200,Fix refresh of integer-time continuous aggregates,"The calculation of the max-size refresh window for integer-based continuous aggregates used the range of 64-bit integers for all integer types, while the max ranges for 16- and 32-bit integers are lower. This change adds the missing range boundaries. " e939b7e603673cef87afb04bcfa8af5d4c2d02b7,Sven Klemm,Fri Jul 31 11:47:12 2020 +0200,Add policies to update test,This patch adds policies to the update test to ensure their configuration is properly migrated during updates. This patch also fixes the inconsistent background job application_name and adjusts them in the update script. cb801fb215e372ff58e6209a056f86f41d2a191f,Sven Klemm,Wed Aug 12 01:28:30 2020 +0200,Run update test on PG 11.8 and 12.3,This patch changes the update test to run on PG 11.8 and 12.3 instead of 11.0 and 12.0. This patch also adds additional diagnostic output when errors occur during the update test. d547d61516b03586581c01fae83ada7abeb12ddb,Sven Klemm,Fri Aug 7 07:16:33 2020 +0200,Refactor continuous aggregate policy,This patch modifies the continuous aggregate policy to store its configuration in the jobs table. 530cb8296ebad158baa179df272ad2a010a143de,Sven Klemm,Sun Aug 9 13:48:42 2020 +0200,Add check for unreferenced test files,This patch adds a check for test files not referenced in the CMakeLists.txt file to CI. 1a8d0eae06aaf73ab8fafc037ebadcbc3c40b20a,Dmitry Simonenko,Tue Aug 11 15:57:43 2020 +0300,Add check for distributed hypertable to reorder/move_chunk,Ensure that move_chunk() and reorder_chunk() functions cannot be used with distributed hypertable f510a39a742ea13e9b20e5b3506928eb199881bd,Sven Klemm,Mon Aug 10 20:30:51 2020 +0200,Make application name for bgw jobs unique,This patch changes the application name for background worker jobs to include the job_id which makes the application name unique and allows joining against pg_stat_activity to get a list of currently running background worker processes. This change also makes identifying misbehaving jobs easier from the postgres log as the application name can be included in the log line. eecc93f3b658647d5df7cb98cba14d7e10a3ec77,gayyappan,Tue Aug 4 11:49:16 2020 -0400,Add hypertable_index_size function,Function to compute the size for a specific index of a hypertable e40d70716e741b31d03c277dbe55e01ce7fce8ee,Sven Klemm,Fri Aug 7 15:21:15 2020 +0200,Ignore result of loader test on windows,The loader test fails extremely often on windows so this patch makes the window test not fail on failed loader test. 4409bff02537db7359ad9681f7c50cc03dfd04e3,Sven Klemm,Fri Aug 7 08:25:24 2020 +0200,Add unreferenced test files to CMakeLists,The with_clause_parser and continuous_aggs_drop_chunks tests were not referenced in the CMakeLists leading to those tests never being run. This patch adds them to the appropriate file and adjusts the output. 0f60b5b33baf3c89d74273f305df0bd6b9560fe1,Dmitry Simonenko,Fri Aug 7 15:12:05 2020 +0300,Add check for distributed hypertable to continuous aggs,Show an error message in case if a distributed hypertable being used. 656d3a4ef4b80e8a1f5e8cae6107c696666f1add,Sven Klemm,Fri Aug 7 08:56:46 2020 +0200,Update package lists before installing packages in CI, 3a119e066a5205bffff8918bd8c32cf3a1f592fc,Sven Klemm,Thu Jul 30 17:25:56 2020 +0200,Fix telemetry installed_time format,This patch changes the telemetry code to always send the installed_time timestamp AS ISO8601. Previously it was depending on local settings leading to timestamps not processable by the telemetry receiver. 56b4c10a745cce9273365bcffb772eca138b2570,Ruslan Fomkin,Thu Aug 6 09:37:22 2020 +0200,Fix error messages to compression policy,Error messages are improved and formulated in terms of compression policy. 02dae3a5fb64d5cbbf990dc79584625441b97503,Sven Klemm,Sun Aug 2 19:42:13 2020 +0200,Fix background worker scheduler memory consumption,This patch changes how the scheduler handles memory contexts. Previously only memory allocated during transactions would get freed and everything else remained allocated. The scheduler now uses 2 memory contexts for its operation: scheduler_mctx for long-lived objects and scratch_mctx for short-lived objects. After every iteration of the scheduling main loop scratch_mctx gets reset. Special care needs to be taken in regards to memory contexts since StartTransactionCommand creates and switches to a transaction memory context which gets deleted on CommitTransactionCommand which switches CurrentMemoryContext back to TopMemoryContext. So operations wrapped in Start/CommitTransactionCommit will not happen in scratch_mctx but will get freed on CommitTransactionCommand. 393e5b9c1adebd263e57ce565a524224637722d1,Ruslan Fomkin,Wed Aug 5 11:08:55 2020 +0200,Remove enabling enterprise from compression test,Compression is not enterprise feature anymore. Thus enabling enterprise is not needed in tests. 9a7b4aa00361386089e4347ddf01a167d5e3ab0a,Erik Nordström,Sat Jul 25 18:18:53 2020 +0200,Process invalidations when refreshing continuous aggregate,"This change adds intitial support for invalidation processing when refreshing a continuous aggregate. Note that, currently, invalidations are only cleared during a refresh, but not yet used to optimize refreshes. There are two steps to this processing: 1. Invalidations are moved from hypertable invalidation log to the cagg invalidation log 2. The cagg invalidation entries are then processed for the continuous aggregate that gets refreshed. The second step involves finding all invalidations that overlap with the given refresh window and then either deleting them or cutting them, depending on how they overlap. Currently, the ""invalidation threshold"" is not moved up during a refresh. This would only be required if the refresh window crosses that threshold and will be addressed in a future change. " 675eb7dd7313a2fc67f6a461611c8a43e21bd8cf,Erik Nordström,Mon Jul 27 11:38:03 2020 +0200,Allow setting snapshot in Scanner,"This change adds the ability to set a snapshot to use with scans executed with the Scanner module. The Scanner uses SnapshotSelf by default, but this isn't appropriate for certain scans that, e.g., don't want to see their own changes. An option to keep the lock on the scanned relation after the scan is also added. " bb891cf4d251c0c705fbec595287a8b7f1457e9b,Sven Klemm,Sat Aug 1 10:08:42 2020 +0200,Refactor retention policy,This patch changes the retention policy to store its configuration in the bgw_job table and removes the bgw_policy_drop_chunks table. 9049a5d3cbf4e6dea94c238004606c0714116d6d,Mats Kindahl,Fri Jul 31 11:49:35 2020 +0200,Remove requirement of CASCADE from DROP VIEW,"To drop a continuous aggregate it was necessary to use the `CASCADE` keyword, which would then cascade to the materialized hypertable. Since this can cascade the drop to other objects that are dependent on the continuous aggregate, this could accidentally drop more objects than intended. This commit fixes this by removing the check for `CASCADE` and adding the materialized hypertable to the list of objects to drop. Fixes timescale/timescaledb-private#659 " 9f13fb9906c804d861cc032539838ccefd50a701,gayyappan,Thu Jul 30 16:52:56 2020 -0400,Add functions for compression stats,Add chunk_compression_stats and hypertable_compression_stats functions to get before/after compression sizes 417908f19b25194ab463b0b8edb73e2247c2ba74,Sven Klemm,Sat Aug 1 15:54:55 2020 +0200,Fix macos build,A recent change changed the macos build to run in release mode which also changed postgres to be built without assertions. Since we inherit the assertion setting from postgres this leads to assertions being disabled for our code as well. With assertions disabled clang errors on detecting null pointer dereferences so this patch turns assertions for macos back on. Since the postgres build is cached this took not effect immediately and remained unnoticed in the CI run against the PR introducing the change. 13e0a5f4c7c53b6180a251ff55c713820c85b12c,Sven Klemm,Sat Aug 1 15:28:33 2020 +0200,Sort test list in pg_regress,This patch changes pg_regress to sort the test list when tests are controlled with either TESTS or SKIPS. This makes it more consistent with the unfiltered test run which gets a sorted list from cmake. 4c051689095fa0fd41fc71b4ca070f17bd2ddb0e,Sven Klemm,Thu Jul 30 02:12:26 2020 +0200,Remove obsolete sql update files,Since the minimum version we can update from on PG11 is 1.1.0 we can get rid of all the previous versions update files as they are not a valid update source for any current version. f62fd957b7e9c6c6f8a9e534940e3448ff1c9d08,Ruslan Fomkin,Fri Jul 31 12:28:44 2020 +0200,Test both debug and release on MacOS in CI,"Switches MacOS test in PR to run release, so the build time is reduced. Adds the test in Debug to scheduled job. " 590446c6a7f4e06bc2b8e3e2c181535ba7834e4c,Mats Kindahl,Thu Jul 30 11:50:30 2020 +0200,Remove cascade_to_materialization parameter,The parameter `cascade_to_materialization` is removed from `drop_chunks` and `add_drop_chunks_policy` as well as associated tables and test functions. Fixes #2137 c93f9637094b72df656ddfb7a884b0927d78bce1,gayyappan,Wed Jul 29 16:55:53 2020 -0400,Remove chunk_relation_size,Remove chunk_relation_size and chunk_relation_size_pretty functions Fix row_number in chunks view 03d2f32178fde9f6c0fa6f04469ac478548877e6,Mats Kindahl,Tue Jul 28 10:47:42 2020 +0200,Add self-reference check to add_data_node,"If the access node is adding itself as a data node using `add_data_node` it will deadlock since transactions will be opened on both the access node and data node both trying to update the metadata. This commit fixes this by updating `set_dist_id` to check if the UUID being added as `dist_uuid` is the same as the `uuid` of the node. If that is the case, it raises an error. Fixes #2133 " 0d5f1ffc8318e4fa32c335b0de62db8ecf156590,Sven Klemm,Tue Jul 28 11:28:17 2020 +0200,Refactor compress chunk policy,This patch changes the compression policy to store its configuration in the bgw_job table and removes the bgw_policy_compress_chunks table. 68aee5144c90119fa09c2d273727e9b01ae15185,Brian Rowe,Tue Jul 28 16:43:06 2020 -0700,Rename add_drop_chunks_policy,"This change replaces the add_drop_chunks_policy function with add_retention_policy. This also renames the older_than parameter of that function as retention_window. Likewise, the remove_drop_chunks_policy is also being renamed remove_retention_policy. Fixes #2119 " 56966685001eba8e5985f93e589034440aef8577,Ruslan Fomkin,Wed Jul 29 15:29:48 2020 +0200,Test detach_tablespaces on distributed hypertable,"Adds a test to call detach_tablespaces on a distributed hypertable. Since no tablespaces can be attached to distributed hyperatbles, the test detaches 0 tablespaces. Also a test to detach tablespaces on a data node is added. " 6a3d31d0454ff7398c27ba985f06c9e78f49b3c2,Sven Klemm,Thu Jul 30 03:29:41 2020 +0200,Rename variable to be according to our code style,This changes the parseState variable in the telemetry code to parse_state to conform with our code style. 7527a7debac695ed056e4e4d7dce0ca74f473fee,Sven Klemm,Thu Jul 23 10:48:25 2020 +0200,Add support for calling custom functions to bgw scheduler,This patch adds internal support for calling user-defined functions and procedures to the background worker scheduler. 7c4247c3fb86e1c0859e714bf009bfb1f0f3bada,Erik Nordström,Wed Jul 29 02:08:42 2020 +0200,Add test for concurrent continuous aggregate refresh,"This change adds an new isolation test for concurrent refreshing on a continuous aggregate. Although a bucket (group) in a continuous aggregate should be unique on the GROUP BY columns there is no unique constraint on the materialized hypertable to protect against duplicate buckets. Therefore, concurrent refreshes can result in duplicate rows in the materialized hypertable although such duplicates should not be possible by the underlying query's definition. " 84fd3b09b4529102ae87f598508da055bbe9f58c,Erik Nordström,Thu Jul 9 15:29:29 2020 +0200,Add refresh function for continuous aggregates,"This change adds a new refresh function called `refresh_continuous_aggregate` that allows refreshing a continuous aggregate over a given window of data, called the ""refresh window"". This is the first step in a larger overhaul of the continuous aggregate feature with the goal of cleaning up the API and separating policy from the core functionality. Currently, the refresh function does a brute-force refresh of a window and it bypasses the whole invalidation framework. Future updates intend to integrate with this framework (with modifications) to optimize refreshes. An exclusive lock is take on the continuous aggregate's internal materialized hypertable in order to protect against concurrent refreshing. However, as this serializes refreshes, we might want to relax this locking in the future to allow, e.g., concurrent refreshes of non-overlapping windows. The new refresh functionality includes basic tests for bad input and refreshing across different windows. Unfortunately, a bug in the optimization code for `time_bucket` causes timestamps to overflow the allowed MAX time. Therefore, refresh windows that are close to the MAX allowed size are not yet supported or tested. " a3a668e654b4807ba79bfade41963c8611f57750,Sven Klemm,Wed Jul 29 23:38:22 2020 +0200,Fix formatting of query in extension test, 5a410736a9d033270442244aa83df92c2afadaa4,Sven Klemm,Wed Jul 29 23:05:42 2020 +0200,Only run chunk_api test on debug build,The chunk_api test requires a debug build for certain test functions this patch changes the chunk_api test to only run for debug builds. 7d3b4b54425084de1188e586fbd34c4ce4876cab,gayyappan,Wed Jul 8 18:29:14 2020 -0400,New size utils functions,"Add hypertable_detailed_size , chunk_detailed_size, hypertable_size functions. Remove hypertable_relation_size, hypertable_relation_size_pretty, and indexes_relation_size_pretty Remove size information from hypertables view. " 3e83577916937b8fff2c2a78d4b3e9181687bb9b,Sven Klemm,Mon Jul 27 22:43:47 2020 +0200,Refactor reorder policy,This patch changes the reorder policy to store it's configuration in the bgw_job table and removes the bgw_policy_reorder table. 09d37fa4f762f35f2e679ee00ce8da4fe0afc40e,Erik Nordström,Tue Jul 14 12:32:19 2020 +0200,Fix memory issues when scanning chunk constraints,"A function to lookup the name of a chunk constraint returned a pointer to string without first copying the string into a safe memory context. This probably worked by chance because everything in the scan function ran in the current memory context, including the deforming of the tuple. However, returning pointers to data in deformed tuples can easily cause memory corruption with the introduction of other changes (such as improved memory management). This memory issue is fixed by explicitly reallocating the string in the memory context that should be used for any returned data. Changes are also made to avoid unnecessarily deforming tuples multiple times in the same scan function. " a311f3735d2a4422e185192a3ffb31bc39875c06,Erik Nordström,Mon Jul 13 17:26:45 2020 +0200,Adopt table scan methods for Scanner,"This change makes the Scanner code agnostic to the underlying storage implementation of the tables it scans. This also fixes a bug that made it impossible to use non-heap table access methods on a hypertable. The bug existed because a check is made for existing data before a table is made into a hypertable. And, since this check reads data from the table using the Scanner, it must be able to read the data irrespective of the underlying storage. As a result of the more generic scan interface, resource management is also improved by delivering tuples in reference-counted tuple table slots. A backwards-compatibility layer is used for PG11, which maps all table access functions to the heap equivalents. " aec8758b0653bd6597357f0c81f6fe23756d8b86,Sven Klemm,Tue Jul 28 02:04:43 2020 +0200,Improve update test github output,This patch adds the diff output of the update test in a separate step in the workflow and also uploads the update test diff as artifact. 6f64f959dbeafa98c19b105dee7e42fc3f991daa,Mats Kindahl,Thu Jul 23 19:42:02 2020 +0200,Propagate privileges from hypertables to chunks,"Whenever chunks are created, no privileges are added to the chunks. For accesses that go through the hypertable permission checks are ignored so reads and writes will succeed anyway. However, for direct accesses to the chunks, permission checks are done, which creates problems for, e.g., `pg_dump`. This commit fixes this by propagating `GRANT` and `REVOKE` statements to the chunks when executed on the hypertable, and whenever new chunks are created, privileges are copied from the hypertable. This commit do not propagate privileges for distributed hypertables, this is in a separate commit. " 43f2c31b3e5061fc8dbc973f2935633ac6e88787,Sven Klemm,Mon Jul 27 19:35:05 2020 +0200,"Add proc, hypertable index to bgw_job","This patch adds a proc_name, proc_schema, hypertable_id index to bgw_job. 3 functions using the new index are added as well: ts_bgw_job_find_by_proc ts_bgw_job_find_by_hypertable_id ts_bgw_job_find_by_proc_and_hypertable_id These functions are required for migrating the existing policies to store their configuration in bgw_job directly. " 82bf1c458fbbdf8a8c99f49dadb11962849739ea,Sven Klemm,Mon Jul 27 19:00:32 2020 +0200,Refactor jsonb_utils,Adjust jsonb utility functions to always take and return char * as string types to reduce conversion requirements on caller. cde5af31b8d06cb005f82967b5966cffd2ba33b5,Sven Klemm,Thu Jul 23 15:10:11 2020 +0200,Make scheduler ignore jobs where scheduled = false,This patch makes the scheduler ignore all jobs where scheduled = false. The schedule flag can be used to disable jobs without deleting the job entry. This patch also changes the job retrieval to use an index scan to satisfy the assumption of the scheduler to receive a list ordered by job_id. dc61466aef4340363d0abcd2df50341d227a8d4c,gayyappan,Fri Jul 17 19:05:10 2020 -0400,Add chunks and dimensions view,timescaledb_information.chunks view shows metadata related to chunks. timescaledb_information.dimensions shows metadata related to hypertable's dimensions. fca7e36898652f177e45aa454ad985cb381c6567,Dmitry Simonenko,Fri Jul 24 19:01:57 2020 +0300,Support moving compressed chunks,Allow move_chunk() to work with uncompressed chunk and automatically move associated compressed chunk to specified tablespace. Block move_chunk() execution for compressed chunks. Issue: #2067 4506c1e6fe2da64f381b48d1e1b76be4caa90bde,प्रथमेश Sonpatki,Mon Jul 20 18:13:57 2020 +0530,Fix typo in detach tablespace error message, 926a1c9850b949915f908fd6ded6eaf564343e38,gayyappan,Wed Jul 15 13:06:05 2020 -0400,Add compression settings view,Add informational view that lists the settings used while enabling compression on a hypertable. 88f693887abdc879e79672a8b758f162dc36cbde,gayyappan,Wed Jul 22 17:48:17 2020 -0400,Cleanup index on hypertable catalog table,Reorder schema_name + table_name index. Remove unnecessary constraint. 40d284fe764a735b6ec602ba9c23320cec3fb45f,Mats Kindahl,Wed Jul 22 17:38:17 2020 +0200,Remove include of pg_config.h,"The `pg_config.h` file in the PostgreSQL include directory does not contain any include guards, so including it will generate errors when building from PGDG-distributed packages. Since `pg_config.h` is included from `c.h` (which has include guards) which in turn is included from `postgres.h` (which also has include guards), we remove usage of `pg_config.h` from all places where `postgres.h` is included and replace it with `postgres.h` where it is not included. " 2f2e5ae68bd8b32ad18247ca54d335d9864dd20b,Sven Klemm,Mon Jul 20 18:26:20 2020 +0200,Change bgw_job catalog table to enable custom jobs,This patch adds the columns required for custom jobs to the bgw_job catalog table. 6b62ed543cfce5fb415ae5b917988314ad183fbb,Brian Rowe,Thu Jul 16 19:04:46 2020 -0700,Fetch collations from data nodes during ANALYZE,This change fixes the stats collecting code to also return the slot collation fields for PG12. This fixes a bug (#2093) where running an ANALYZE in PG12 would break queries on distributed tables. bdced2b722cab09c24bf71e9c115b5fa55b31825,Ruslan Fomkin,Fri Jul 17 12:43:26 2020 +0200,Add test of drop_chunks on distributed hypertable,Testing that drop_chunks works correctly on a distributed hypertable. Tests of different arguments are assumed to be done on a usual hypertable previously. 5c70dfcc0eac50cdae9a9398f90b6661c9e424f4,Sven Klemm,Mon Jul 20 12:37:17 2020 +0200,Add isolation test files to license check,This patch enhances the license check script to check for a license header in isolation test files 70408cd32516ab34abc670827ad3f1ad64dff3ab,Sven Klemm,Mon Jul 20 12:35:23 2020 +0200,Add missing license header to spec files, 3d1a7ca3ac3ac589d31eaeb94543613f0177770b,Sven Klemm,Thu Jul 16 23:13:19 2020 +0200,Fix delete on tables involving hypertables with compression,The DML blocker to block INSERTs and UPDATEs on compressed hypertables would trigger if the UPDATE or DELETE referenced any hypertable with compressed chunks. This patch changes the logic to only block if the target of the UPDATE or DELETE is a compressed chunk. 13aa729f6837852a6b2951665341caf47792e4e6,Ruslan Fomkin,Fri Jul 17 22:25:32 2020 +0200,Add license note to TSL isolation test,Adding Timescale License header to specs of isolation TSL tests. dd51b09f3e3b4d4b1d198e37ad965bd456aa2feb,Sven Klemm,Thu Jul 16 15:16:48 2020 +0200,Fix index creation with IF NOT EXISTS for existing indexes,When creating an index with IF NOT EXISTS we still tried to create indexes for all the chunks but since no index was created on the parent table the meta data did not have the object id of the main table index leading to an error when trying to open the main table index. This patch adjusts the logic to check for IF NOT EXISTS and does an early return when no index was created on the parent table. 8801781fd62cd1129e5bb143594a55b905da59e7,Sven Klemm,Wed Jul 15 11:22:20 2020 +0200,Move coverity scan to github actions,Migrate the travis coverity config to github actions. This new workflow will depend on the cached postgres build of the main regression workflow so we don't have to duplicate the build matrix in this workflow. Since the main regression workflow runs at least once a day there should always be a cache hit. In case of a cache miss this workflow will abort. Since there is a quota on coverity runs this workflow is configured to run once a week every saturday. Additional runs can be triggered by pushing to the coverity_scan branch. add97fbadfd3d4d73953fa436390e2fee157badf,Dmitry Simonenko,Wed Jul 15 13:41:55 2020 +0300,Apply SET TABLESPACE for compressed chunks,"Allow ALTER SET TABLESPACE on an uncompressed chunk and automatically execute it on the associated compressed chunk, if any. Block SET TABLESPACE command for compressed chunks. Issue #2068 " 795e050d5f2a012742fec2086f82aa957e606733,Mats Kindahl,Tue Jul 14 18:28:29 2020 +0200,Fix update tests,"The update test checks that default jobs are the same before and after an upgrade, but does not order the jobs in any specific way. This means that if the database order is different before and after the update, it will result in a false negative even if the jobs are identical. This commit fixes this by ordering the jobs by id. " 665d56be472edc3c544100bd55e2dee7fcf1f1bb,Sven Klemm,Sat Jul 4 11:14:23 2020 +0200,Improve infering start and stop arguments from gapfill query,Infering start and stop parameter used to only work for top level constraints. However even when constraints are at the toplevel in the query they might not end up at the top-level of the jointree depending on the plan shape. This patch changes the gapfill code to traverse the jointree to find valid start and stop arguments. c4aa8cf7cfed64fdebda4c8c9ab47414a7040945,Sven Klemm,Tue Jul 7 00:09:39 2020 +0200,Move cron tests to GitHub actions,"This patch moves the sanitizer, abi breakage and memory leak tests from travis to github actions. " 5bb4bfdf5041764fc300fece22118e91fcdd5afb,Sven Klemm,Wed Jul 8 17:41:09 2020 +0200,Ignore results of flaky tests, dc5ed1a92d8378c0131f99deaf464dae5da370cf,Sven Klemm,Wed Jul 8 12:30:55 2020 +0200,Remove debian 8 from package test,This patch removes Debian 8 from the package test since its no longer supported. This patch also fixes the repository URL for centos 8. fa223ae7017ad9130325f82f60ebe01f735087dd,Sven Klemm,Mon Jul 6 22:45:40 2020 +0200,Use macros to define crossmodule wrapper functions,This patch adds a helper macro to define cross module wrapper functions to reduce code repetition. It also changes the crossmodule struct names to match the function names where this wasn't the case already. 41c89b56486f70aed5551e4630a1905a60cf0081,Sven Klemm,Mon Jul 6 07:50:39 2020 +0200,Add 1.7.2 to update test scripts, d1fda0347335d6e4b448cea6e54b1be5ae7b6c40,Sven Klemm,Mon Jul 6 13:15:08 2020 +0200,Fix github action cache handling on macOS,On macOS the path used is depending on the runner version leading to cache failure when the runner version changes or is different from the one used to build the cache. This patch extracts the runner version and adds it as suffix to the cache key on macOS. 666369ea2eb2bd3cd94541fbf3f746805b0d2e04,Sven Klemm,Wed Jun 24 16:23:07 2020 +0200,Migrate update test to GitHub actions,This patch migrates the update tests from Travis to GitHub actions. cb789a5efffe6918c26b3f3bf30f677cbbc4eea7,Erik Nordström,Tue Jun 30 13:15:18 2020 +0200,Cleanup unused code paths,This change removes some code paths that are no longer used after the removal of PG10 and PG9.6 support. 76648557bbad5b2420a23dff29d8a904d35cb3e5,Mats Kindahl,Mon Jul 6 12:05:29 2020 +0200,Remove ts_dimension_slice_insert_multi parameter,One parameter was always true. This commit remove the parameter and update the logic to match the existing usage. e6020e88d81a5eb5f5cfe46e39e4ef26efd5d4b2,Sven Klemm,Mon Jul 6 10:42:33 2020 +0200,Fix syntax error in apt package test,The notification configuration for the apt package test was missing a dollar sign in front of the variables leading to failing variable interpolation. 9c026b2194f825b3195b2716bb14b993a1bcc170,Sven Klemm,Sun Jul 5 19:32:43 2020 +0200,Add missing ORDER BY clause to chunk_api test,The chunk_api test fails sometimes because of inconsistent resultset ordering in one of the queries. This patch adds the missing ORDER BY clause to that query. 7fc7b2c360a979025c96c495c7c6a98679c0d641,Sven Klemm,Thu Jul 2 22:33:55 2020 +0200,Release 1.7.2,"This maintenance release contains bugfixes since the 1.7.1 release. We deem it medium priority for upgrading. In particular the fixes contained in this maintenance release address bugs in continuous aggregates, drop_chunks and compression. **Features** * #1877 Add support for fast pruning of inlined functions **Bugfixes** * #1908 Fix drop_chunks with unique constraints when cascade_to_materializations is false * #1915 Check for database in extension_current_state * #1918 Unify chunk index creation * #1932 Change compression locking order * #1938 Fix gapfill locf treat_null_as_missing * #1982 Check for disabled telemetry earlier * #1984 Fix compression bit array left shift count * #1997 Add checks for read-only transactions * #2002 Reset restoring gucs rather than explicitly setting 'off' * #2028 Fix locking in drop_chunks * #2031 Enable compression for tables with compound foreign key * #2039 Fix segfault in create_trigger_handler * #2043 Fix segfault in cagg_update_view_definition * #2046 Use index tablespace during chunk creation * #2047 Better handling of chunk insert state destruction * #2049 Fix handling of PlaceHolderVar in DecompressChunk * #2051 Fix tuple concurrently deleted error with multiple continuous aggregates **Thanks** * @akamensky for reporting an issue with telemetry and an issue with drop_chunks * @darko408 for reporting an issue with decompression * @dmitri191 for reporting an issue with failing background workers * @eduardotsj for reporting an issue with indexes not inheriting tablespace settings * @fourseventy for reporting an issue with multiple continuous aggregrates * @fvannee for contributing optimizations for pruning inlined functions * @jflambert for reporting an issue with failing telemetry jobs * @nbouscal for reporting an issue with compression jobs locking referenced tables * @nicolai6120 for reporting an issue with locf and treat_null_as_missing * @nomanor for reporting an issue with expression index with table references * @olernov for contributing a fix for compressing tables with compound foreign keys * @werjo for reporting an issue with drop_chunks and unique constraints " 43edbf86393e3e59bf28911df214378650c02bdb,gayyappan,Wed Jul 1 18:10:49 2020 -0400,Fix concurrent tuple deletes for continuous aggregates,"When we have multiple continuous aggregates defined on the same hypertable, they could try to delete the hypertable invalidation logs concurrently. Resolve this by serializing invalidation log deletes by raw hypertable id. Fixes #1940 " d649d493f666a569e13d942cffa7c95a7c38150e,Sven Klemm,Wed Jul 1 20:31:10 2020 +0200,Add testing for RPM packages,This workflow will install our rpm packages and then try to enable timescaledb in the database and also check the version installed from the package against the expected version. 68ba6e073cb1cf0e5c20e710faf0cc8adaed1185,Sven Klemm,Thu Jul 2 15:16:22 2020 +0200,Add regresscheck-shared sql files to license check,This adds the directories used by regresscheck-shared to the license check and adds a license where it is currently missing. 73ffc466cbfdeaadca7b1cf8194382f3b56e5854,Mats Kindahl,Fri Jun 26 15:29:46 2020 +0200,Add isolation test for drop_chunks and insert,"If a new chunk is created as part of an insert and drop_chunks runs concurrently with the insert, there is a risk of a race. This is a test for this. Add locks for dimension slice tuples If a dimension slice tuple is found while adding new chunk constraints as part of a chunk creation it is not locked prior to adding the chunk constraint. Hence a concurrently executing `drop_chunks` can find a dimension slice unused (because there is no chunk constraint that references it) and subsequently remove it. The insert will the continue to add the chunk constraint with a reference to a now non-existent dimension slice. This commit fixes this by locking the dimension slice tuple with a share lock when creating chunks and locking the dimension slice with an exclusive lock prior to scanning for existing chunk constraints. The commit also contains a script that repair the `dimension_slice` table if it is broken by extracting information about dimension slices that are mentioned in `chunk_constraint` table but not present in `dimension_slice` table and re-create the rows from the constraints on the chunks. " cfb3331cab9325a8742e178d51d234bda5e62dde,Ruslan Fomkin,Tue Jun 30 16:18:41 2020 +0200,Use index tablespace during chunk creation,"If a tablespace is provided for an index on a hypertable, it will be also used for the index on new chunks. This is done when constraints are created on new chunk from the hypertable constraints. Fixes #903 " 9c49a47fbb88d0cb39df5c8e3a13ed20428a6726,Sven Klemm,Wed Jul 1 18:21:28 2020 +0200,Fix handling of PlaceHolderVar in DecompressChunk,When the relation targetlist of the uncompressed chunk contained PlaceHolderVars the construction of the relation targetlist of the compressed chunk would fail with an error. This patch changes the behaviour to recurse into those PlaceHolderVar. 0e9f1ee9f549242a594e583ce5135331ded940a2,Oleg Smirnov,Sun Jun 28 13:40:48 2020 +0300,Enable compression for tables with compound foreign key,"When enabling compression on a hypertable the existing constraints are being cloned to the new compressed hypertable. During validation of existing constraints a loop through the conkey array is performed, and constraint name is erroneously added to the list multiple times. This fix moves the addition to the list outside the conkey loop. Fixes #2000 " 4cd90789da76f1c5c64b453e9ed65867ef5eeb4b,Mats Kindahl,Wed Jul 1 17:31:01 2020 +0200,Add tests for hypertable unique constraint crash,"If a unique constraint is created on a hypertable, it could under some circumstance crash. This commit adds a test for this situation even though it was already fixed (but was reported on the 1.7 branch). " 3823db12188c53881a5b268aa8291b4fe3b313c9,Sven Klemm,Wed Jul 1 10:36:22 2020 +0200,Merge pg_regress.sh and pg_isolation_regress.sh,Since pg_regress.sh and pg_isolation_regress.sh were almost identical this patch combines them into a single script. This patch also changes the dynamic schedule generation so make installcheck TESTS='foo' is supported again which was broken by previous refactoring and you needed to specify the exact suite your test was in if you wanted to use TESTS. d85309d26826740fabbec53d4e2ef0186713ac56,Sven Klemm,Sun Jun 28 11:35:23 2020 +0200,Add testing for released apt packages,This workflow will install our apt package and then try to enable timescaledb in the database and also check the version installed from the package against the expected version. 029b4e867e7da5fdb15aac31d8b51bb3ae3f4dcd,Brian Rowe,Mon Jun 29 16:36:59 2020 -0700,Fix potential mem corruption in distributed copy,This change copies the chunk object into the distributed copy's memory context before caching it in the ChunkConnectionList. This resolves an issue where the chunk was being modified after being stored which was resulting in rows being sent to the incorrect data node. This fixes github issue #2037 3e5e5255f28a8e84ce0e62fc36c401b62ae01e51,Sven Klemm,Tue Jun 30 16:38:29 2020 +0200,Fix segfault in cagg_update_view_definition,Since relation_close will decrease the reference counter this might lead to the relation being freed while we are still using the view query. This patch changes cagg_update_view_definition to release the relations later and also keeps the locks till the end of the transaction. 4e416721616636cee101531672c144dae8e72243,Sven Klemm,Sun Jun 28 14:48:23 2020 +0200,Remove unused functions,Remove functions that are no longer used due to refactorings. Removes the following functions: - hypertable_tuple_match_name - ts_hypertable_get_all_by_name - ts_hypertable_has_tuples 586f057ad07b5a78f003374fecd7d7f1125ffc3c,Sven Klemm,Fri Jun 26 11:51:56 2020 +0200,Retain parallel schedule when using IGNORES,"When either TESTS, SKIPS or IGNORES is set for a regression check run we would generate a new temporary schedule based on those variables without any parallelity. This patch changes the behaviour to keep the original test schedule when only IGNORES is specified and just prepend the ignore lines to a copy of the original schedule. " 0e8adc97f4c07d52cc1275cf093463ffa76e8905,Sven Klemm,Tue Jun 30 10:30:16 2020 +0200,Fix segfault in create_trigger_handler,The TriggerDesc from rel->trigdesc seems to be modified during iterations of the loop and sometimes gets reallocated leading to a situation where the local variable trigdesc no longer matches rel->trigdesc leading to a segfault in create_trigger_handler. Fixes #2013 8a1976a6b9459aed256455b754990f7664a2b4a1,Sven Klemm,Mon Jun 29 20:11:48 2020 +0200,Upload coredumps as artifact,This patch adds collecting coredumps to the regression workflow in addition to the binary all shared libraries used by the coredump are collected as well. c96216e631dc0e0427432100d2bc056a53891d7e,Sven Klemm,Fri Jun 26 17:14:26 2020 +0200,Improve github action stacktrace handling,Check for coredumps and only execute the stracktrace if there are actually coredumps. This patch also changes the log handling to always collect logs and upload them cause they might have useful information even when all steps succeed. Additionally a list of all failed tests is shown before the regression diff. This patch also disables fail-fast so a single failed job does not cancel other jobs still in progress. 23eeecc4e5ec4ef659bd969f717e3d3176220521,Dmitry Simonenko,Mon Jun 29 15:11:47 2020 +0300,Do not forward relcache invalidation as syscache invalidation,"It is incorrect to forward relcache invalidations as syscache invalidations, like cacheid = InvalidOid which is not possible condition at the moment. Allow syscache invalidations only for FOREIGNSERVEROIDs. " b1eeb56c12c699c336245bb75dc1a843b19f949b,Sven Klemm,Thu Jun 25 20:40:07 2020 +0200,Execute regresscheck-t tests in parallel,This patch adds parallel execution capabilities to regresscheck-t. 48b969ecc04c61b0380cc88673cd5d148f6651e4,Sven Klemm,Thu Jun 25 14:56:14 2020 +0200,Disable fsync for appveyor,"In our normal regression tests fsync is already disabled because the cluster is initialized by pg_regress which turns fsync off, but for all tests using regresschecklocal the setting will be missing because the cluster is initialized outside of pg_regress. " 0c1f967ef8029b8b463fa376e16739237b12dd1a,Sven Klemm,Thu Jun 25 01:29:26 2020 +0200,Refactor transparent_decompression test,This PR reduces the dataset size in the transparent_decompression test to make it finish in a more reasonable time. It also splits the test and modifies queries that used now(). Due to the change in dataset size the resulting diff is rather large but is mostly row count changes in the plan but the actual plan shapes don't change. In addition to those changes to some LATERAL queries additional constraints have been added to reduce the number of loops. a0034e44db55a3c03001d93245657bda32a388bf,Ruslan Fomkin,Thu Jun 25 12:02:13 2020 +0200,Add status badge for Linux i386 build, 9ec944a4dce263b5462d099df4fb54e40ce6fe39,Dmitry Simonenko,Thu Jun 25 14:08:31 2020 +0300,Refactor process utility functions result type,Introduce and use explicit enum type for process utility functions return value instead of using bool type. 65b3a0a99119c914351cf88061345acfb5282291,Sven Klemm,Tue Jun 23 01:05:20 2020 +0200,Constify TIMESTAMPTZ OP INTERVAL in constraints,"Constify expressions of the following form in WHERE clause: column OP timestamptz - interval column OP timestamptz + interval column OP interval + timestamptz Iff interval has no month component. Since the operators for timestamptz OP interval are marked as stable they will not be constified during planning. However, intervals without a month component can be safely constified during planning as the result of those calculations do not depend on the timezone setting. " 0d2cc046c219eb0fc0f68a18b779cb6f07caa80b,Sven Klemm,Wed Jun 24 20:17:49 2020 +0200,Ignore continuous_aggs_insert and continuous_aggs_multi on PG11.0,The log output in the isolation tests when running on PG 11.0 seems to be suppressed and not be recorded leading to a diff in the test output. ccbe67f94f9c449b5cb8280b36fe7b6bc71f977c,Sven Klemm,Mon Jun 22 18:25:22 2020 +0200,Run regresscheck-shared tests in parallel,This patch changes regresscheck-shared to no longer run individual tests serially but runs them in parallel instead. 0e532a165f2e02c4f1f2aebd94d29a106add11a2,Sven Klemm,Tue Jun 23 12:22:46 2020 +0200,Include local variables in github action stacktrace,This patch adds local variables to github action stacktraces to add more context and make investigating issues easier. 1f93536704d144f2161567c296541aa8c38e6131,Sven Klemm,Tue Jun 23 09:13:35 2020 +0200,Change testsupport.sql to not reference tsl code,testsupport.sql had a reference to TSL code which will fail in apache-only code since this file is included in every test run for every build configuration. 71ed4547794d6cc4736027431df2a5e53ce5b8c0,Sven Klemm,Mon Jun 22 16:41:11 2020 +0200,Give regression artifacts unique names,Since within a workflow artifacts share a namespace we need to make sure artifacts have a unique name within the workflow so other runs in the same workflow don't overwrite artifacts. ae0f17705b67ae55fb992de786bc210004e24c75,Sven Klemm,Mon Jun 22 16:24:28 2020 +0200,Improve behaviour of test runner when being run in parallel,The test runner used to use a lockfile to decide whether initial cluster setup has to be done. Unfortunately checking for existance and creating the lockfile are 2 distinct operations leading to race conditions. This patch changes the runner to use a directory instead because with a directory the operation is atomic. a959281e485748500215cff3f57d99087454d81b,Sven Klemm,Fri Jun 19 21:59:14 2020 +0200,Replace Travis badge with GitHub badge,Since most of our regression tests run on GitHub actions now this patch replaces the build status badge from Travis with a GitHub actions one. 66544c75647e5935faeb901e3d8ab07602b44907,David Kohn,Thu Jun 18 12:37:44 2020 -0400,Reset restoring gucs rather than explicitly setting 'off',Setting the `timescaledb.restoring` guc explicitly to 'off' for the db meant that the setting got exported in `pg_dumpall` and some other cases where that setting would then conflict with the setting set by the pre_restore function causing it to be overridden and causing errors on restore. This changes to `RESET` so that instead it will take the system default and not be dumped separately as an override. 04bcc949c166d1c6e118efe0375dfc1758c99d23,Dmitry Simonenko,Mon Jun 22 15:52:41 2020 +0300,Add checks for read-only transactions,This change ensures that API functions and DDL operations which modify data respects read-only transaction state set by default_transaction_read_only option. ab293976f29fd196c7ff4295b115182cd4c3e100,Sven Klemm,Mon Jun 22 11:24:58 2020 +0200,Make TSL_MODULE_PATHNAME available in testsupport.sql,This patch makes TSL_MODULE_PATHNAME available when executing testsupport.sql in the regression test runner. This fixes an error that happened in the test runner that was suppressed because it happens before the actual test run. 157764c70e7a74b01d2e732286ae8fd7f4c829fa,Sven Klemm,Fri Jun 19 13:55:30 2020 +0200,Remove dynamic version checks from SQL tests,The sql tests still had version checks and would run EXPLAIN with different parameters depending on postgres version. Since all supported postgres versions now support all the parameters we use we can safely remove the version check. b93b30b0c2e169fb0658d16eb5c3955982bcfb4f,gayyappan,Wed Jun 10 12:34:18 2020 -0400,Add counts to compression statistics,Store information related to compressed and uncompressed row counts after compressing a chunk. This is saved in compression_chunk_size table. 5aaa07b9ee9d22f4fc3cbcf286fae3230e9f0a93,Sven Klemm,Fri Jun 19 19:05:59 2020 +0200,Fix flaky reorder_vs_insert isolation test,This patch makes the lock_timeout values for the 2 sessions distinct so they will always fire in the same order leading to reproducable results. 5b7f2fec181c30f072fc180b7e57cde66b5726ea,Sven Klemm,Fri Jun 19 17:07:09 2020 +0200,Disable CI fail-fast for code style and i386 test,When fail-fast is true a single failing job will cancel the entire workflow which is not desirable for scheduled runs and the code style tests. This patch changes fail-fast to false for the code style tests and the scheduled i386 tests. It also changes the different code style check steps to always run. 5659cb7515fa6dd6eada1d51940f730838a917b5,Sven Klemm,Fri Jun 19 16:52:40 2020 +0200,Remove postgres 10 test files,Remove files that got introduced by rebase error or were missed during removal of PG9.6 and PG10 support. 6d84751ebb4d700481236d59656c44cdb4762db0,Sven Klemm,Fri Jun 19 11:50:45 2020 +0200,Remove handling for pre PG10 version format,The telemetry code still had code to handle the version format used by postgres before PG10 which is dead code now since PG11 is the minimum version. This patch removes that code path. b810dd3780b71c14bf811f22519c376969e993bb,Sven Klemm,Mon Jun 15 12:53:28 2020 +0200,Add MacOS regression tests, 5b8de4710ebe9cc9119c7b7ebf2894f685eb35fc,Sven Klemm,Thu Jun 18 22:24:07 2020 +0200,Fix compression_ddl isolation test,The compression_ddl test had a permutation that depended on the PGISOLATIONTIMEOUT to cancel the test leading to unreasonably long running and flaky test. This patch changes the test to set lock_timeout instead to cancel the blocking much earlier. 4a6bdb7f1bfb8ceef508b0623212670d4ad9148b,Sven Klemm,Thu Jun 18 10:57:05 2020 +0200,Increase lock_timeout for isolation tests,In slower environments the isolation tests are extremely flaky because lock_timeout is only 50ms this patch changes lock_timeout to 500ms for the isolation tests leading to much more reliable tests in those environments. 32ed2c02d5f6f67415d12e348184ee03e161c156,Sven Klemm,Mon Jun 15 17:51:07 2020 +0200,Adjust export prefix check for MacOS, 596515eb0f9b04f6a44be296a653bca8589e7b2f,Erik Nordström,Mon Jun 15 21:48:32 2020 +0200,Fix ANALYZE on replicated distributed hypertable,"With replicated chunks, the function to import column stats would experience errors when updating `pg_statistics`, since it tried to write identical stats from several replica chunks. This change fixes this issue by filtering duplicate stats rows received from data nodes. In the future, this could be improved by only requesting stats from ""primary"" chunks on each data node, thus avoiding duplicates without having to filter the result. However, this would complicate the function interface as it would require sending a list of chunks instead of just getting the stats for all chunks in a hypertable. " 72d9def6196d89ddfb6c8aee19a55f05bb41fea8,Erik Nordström,Mon Jun 15 23:01:48 2020 +0200,Fix output in show_connection_cache function,"The ""invalidation"" column was accidentally set as the ""processing"" column in `show_connection_cache`. This made the output of these columns show the wrong values. This change fixes this issue. " 195bba114d16e1870eb02cbb4de37519acf94f40,Ruslan Fomkin,Tue Jun 16 12:25:21 2020 +0200,Remove flakiness of cagg materializaion test,"Replaces now() with mocked timestamp value in continuous_aggs_materialize test, so the test doesn't produce unexpected result when now() is close to a chunk border. " 175edfa66919f9fb99e18ae884286dc85c959885,Ruslan Fomkin,Wed Jun 17 10:05:47 2020 +0200,Force installing PostgreSQL in GH Actions on Windows,"PostgreSQL 12 is preinstalled, while 11 is not. To unify different paths of PG 11 and 12 binaries, this commit implements workaround by forcing installation of PostgreSQL 12, so it is in the same path as PostgreSQL 11. " 5d188267e427c088003ff269e448c26e5758d8dc,Mats Kindahl,Tue Jun 16 11:32:05 2020 +0200,Fix crash with create_distributed_hypertable,"Function `ts_hypertable_create_from_info` will error if the hypertable already exists unless the if-not-exists flag is not set. If we reach this point, either if-not-exists-flag was set or the hypertable did not exist and was created above. In `ts_hypertable_create_from_info`, a call to `ts_dimension_info_validate` with `space_dim_info` will be made if (and only if) the table did not exist. The function does not only validate the dimension, it also set `dimension_id` field. If the table already existed, `created` will be false and the `dimension_id` will be set to the invalid OID, which means that `ts_hypertable_check_partitioning` will crash since it it expect a proper OID to be passed. This commit fixes that by checking if the hypertable exists prior to calling `ts_hypertable_create_from_info` in `ts_hypertable_create_internal` and aborting with an error if the hypertable already exists. Fixes #1987 " a089843ffd91dd3fe0914f904dbae6ab5e8a4816,Mats Kindahl,Thu May 7 16:14:50 2020 +0200,Make table mandatory for drop_chunks,"The `drop_chunks` function is refactored to make table name mandatory for the function. As a result, the function was also refactored to accept the `regclass` type instead of table name plus schema name and the parameters were reordered to match the order for `show_chunks`. The commit also refactor the code to pass the hypertable structure between internal functions rather than the hypertable relid and moving error checks to the PostgreSQL function. This allow the internal functions to avoid some lookups and use the information in the structure directly and also give errors earlier instead of first dropping chunks and then error and roll back the transaction. " 0bff7dcac326c809b9abab6f1e9f334338fc960f,Sven Klemm,Sun Jun 7 09:12:21 2020 +0200,Run linux regression tests as GitHub action,Since we want to run additional test configurations when triggered by a push to prerelease_test or by cron but github actions don't allow a dynamic matrix via yaml configuration we generate the matrix with a python script. While we could always have the full matrix and have if checks on every step that would make the actual checks harder to browse because the workflow will have lots of entries and only by navigating into the individual jobs would t sit be visible if a job was actually run. Additionally we set fail-fast to true for pull requests and false for other event types. ca9b37d9de8fc244bff5acc2b8d1dbd1a6861ce6,Sven Klemm,Sun Jun 14 20:47:43 2020 +0200,Improve i386 slack failure message,Change i386 slack notification to set border color to red and to include last commit message. 129aa4798710dd1a68c54439bd5df704ae32ce8c,Sven Klemm,Sun Jun 14 20:39:28 2020 +0200,Adjust export prefix check to work with clang,This patch changes the export prefix check to ignore symbols present in clang builds. 7d132b3a37d3d6be1e60a79c71612e0eaec6a03e,Erik Nordström,Tue Jun 9 21:09:29 2020 +0200,Improve error handling in async library,"The async library for remote connections has multiple ways to deal with errors. For instance, errors can both be thrown internally and be returned as port of an `AsyncResult`. In other cases, it is possible to pass in an ""elevel"" to avoid throwing errors, but this only works for some functions and not for all errors. This change tries to make error handling more consistent by avoiding throwing errors while waiting for results, instead returning any error as `AsyncResponseError`. This makes error handling more predictible and the caller has to explicitly check if the returned response is an error or valid result without having to try-catch any errors. A number of issues with potentially leaking PQresult:s have also been fixed. " 47a4d931f3ad921eed382f9169a13b46d0b4ae37,Erik Nordström,Mon Jun 8 12:48:45 2020 +0200,Remove need to pin connection cache,"Since the connection cache is no longer replaced on a transaction rollback, it is not necessary to pin the connection cache (this wasn't done correctly in some places in any case, e.g., `data_node_get_connection`). " 9d533f31c2f142baf545942b388dcb8d110c3444,Erik Nordström,Fri Jun 5 20:19:36 2020 +0200,Improve connection handling during transactions,"This change refactors how connections are handled during remote transactions. In particular, the connection cache now stays consistent during transactions, even during rollbacks. Previously, the connection cache was replaced on every rollback, even if the rollback was intentional (i.e, not due to an error). This made it hard to debug connections since the cache became completely empty. Connections could also be left in the cache in a bad state after failed transactions. This has been fixed by moving connection checks to the cache and tying transaction state changes to each connection. This ensures that such checks are done in one canonical place instead of being spread out throughout the code. Given how tightly coupled a remote transaction is with its connection, it might make sense to remove the separate remote transaction store and instead put this information in each connection. This is left to a future change, however. In addition to the above changes, this commit includes: * Showing transaction depth and invalidation in the transaction store * Invalidation on individual connections instead of replacing the whole cache * Closing of connections to a local database that is being dropped to prevent ""in use"" errors. * Ability to add callbacks to async requests that are executed when a response is received. This is used by remote transactions to mark connections as having successfully completed a transaction. Thus, on errors, it is easy to detect connections that are in bad states. * Error checks on each connection instead of having global error tracking for each remote transaction. This change removes the global error state for distributed transactions. " 31d5254c2ef1461a4d5150f434e7d60146796455,Erik Nordström,Wed Jun 3 18:09:41 2020 +0200,Add internal function to show connection cache,"The connection cache for remote transactions can now be examined using a function that shows all connections in the cache. This allows easier debugging and validation both in tests and on live systems. In particular, we'd like to know that connections are in good state post commit or rollback and that we don't leave bad connections in the cache. The remote transaction test (`remote_txn`) has been updated to show the connection cache as remote transactions are executed. Unfortunately, the whole cache is replaced on every (sub-)transaction rollback, which makes it hard to debug the connection state of a particular remote transaction. Further, some connections are left in the cache in a bad state after, e.g., connection loss. These issues will be fixed with an upcoming change. " 0fdfcdb9d35bbfe5ae386e536a18d26802d5e8e0,Mats Kindahl,Thu Jun 11 14:36:08 2020 +0200,Fix bit array left shift count,"If `bits_used` is not exactly 64, a shift will be attempted, even when `bits_used > 64`. According to the standard ""[...] if the value of the right operand is negative or is greater or equal to the number of bits in the promoted left operand, the behavior is undefined."" Hence we change the code to return `PG_UINT64_MAX` if a request to shift more bits than the number of bits in the type is requested, otherwise we perform the shift. " ead588f97116f986ccb46a84297528505b97ac07,Mats Kindahl,Wed Jun 10 08:04:45 2020 +0200,Check for disabled telemetry earlier,"If telemetry is disabled, it is checked inside the `ts_telemetry_main` function, which starts to execute after a background worker has been scheduled to execute the function. This means that any errors occurring when starting the job will trigger errors and generate lines in the log. This commit moves the check to before a background worker is scheduled for executing the job and allow the telemetry job to trivially succeed without actually invoking the worker, hence no errors will be generated for the telemetry job if it is disabled. Also, the privacy test result changes and it is updated to check that the telemetry job cannot leak information regardless of whether telemetry is on or off. Fixes #1934 #1788 " c0f66a4e1f2f3fca528999d0e1fbde8d8f72cb28,Sven Klemm,Mon Jun 1 18:42:40 2020 +0200,Move code style checks to github action,"Move code formatting, license and git commit hook checks to GitHub action. " 835ec5af10723f303cb1b53db3c566f505656603,Sven Klemm,Wed Jun 10 14:05:14 2020 +0200,Change export prefix check to work with gcov,This patch adjusts the export prefix check to exclude some symbols present when using older toolchain and also includes gcov symbols so export prefix check can be used on coverage enabled builds. db617bf1d6f8dd7691a8814cb79e55bb2bc5a64b,Sven Klemm,Wed Jun 10 13:54:20 2020 +0200,Fix typos in comments and documentation, 6e3fa54a7c78643e53f0ca32df0cd78cbf4fc44b,Sven Klemm,Wed Jun 3 22:47:27 2020 +0200,Add slack notification to i386 tests,This patch adds a slack notification on failed i386 tests. Unfortunately most slack actions don't work with the 32bit image so our options for better styling/costumisation are somewhat limited. f3b790777813f1159476f979a1c965575e8b2487,Dmitry Simonenko,Wed Jun 10 14:02:16 2020 +0300,Cleanup remote_txn on data node delete,Remove any 2PC transaction records associated with the data node from _timescaledb_catalog.remote_txn on delete_data_node() call. 5bbab82eca4f65d5686f8622dc44059083b7ce3d,Sven Klemm,Mon Jun 8 19:58:35 2020 +0200,Initialize variables complained about by older compilers,GCC 7.5 warns that invalidation_range in continuous aggregate materialization may be used uninitialized. Older clang versons warn that res in calculate_next_start_on_failure may be used uninitialized. This patch changes the code to always initialize those two variables. d3ed288d2a8dd5370324c8055983765ee1baa729,Mats Kindahl,Mon Jun 8 08:58:54 2020 +0200,Do not use NOTICE in CMake files,"The `NOTICE` level for `message` doesn't exist in CMake 3.11, which is our minimum supported version, so removing it. " 182f6f14fdaaa699bd87c3b27de863969ecebb58,Sven Klemm,Thu Jun 4 22:17:33 2020 +0200,Release 2.0.0-beta5,"This release adds new functionality on distributed hypertables, including (but not limited to) basic LIMIT pushdown, manual chunk compression, table access methods storage options, SERIAL columns, and altering of the replication factor. This release only supports PG11 and PG12. Thus, PG9.6 and PG10 are no longer supported. Note that the 2.0 major release will introduce breaking changes to user functions and APIs. In particular, this beta removes the cascade parameter from drop_chunks and changes the names of certain GUC parameters. Expect additional breaking changes to be introduced up until the 2.0 release. **For beta releases**, upgrading from an earlier version of the extension (including previous beta releases) is not supported. **Features** * #1877 Add support for fast pruning of inlined functions * #1922 Cleanup GUC names * #1923 Add repartition option on detach/delete_data_node * #1923 Allow ALTER TABLE SET on distributed hypertable * #1923 Allow SERIAL columns for distributed hypertables * #1923 Basic LIMIT push down support * #1923 Implement altering replication factor * #1923 Support compression on distributed hypertables * #1923 Support storage options for distributed hypertables * #1941 Change default prefix for distributed tables * #1943 Support table access methods for distributed hypertables * #1952 Remove cascade option from drop_chunks * #1955 Remove support for PG9.6 and PG10 **Bugfixes** * #1915 Check for database in extension_current_state * #1918 Unify chunk index creation * #1923 Fix insert batch size calculation for prepared statements * #1923 Fix port conversion issue in add_data_node * #1932 Change compression locking order * #1938 Fix gapfill locf treat_null_as_missing **Thanks** * @dmitri191 for reporting an issue with failing background workers * @fvannee for optimizing pruning of inlined functions * @nbouscal for reporting an issue with compression jobs locking referenced tables * @nicolai6120 for reporting an issue with locf and treat_null_as_missing * @nomanor for reporting an issue with expression index with table references " d08e19931cf576617cf353f1da1860f04132ff84,Mats Kindahl,Fri Jun 5 12:29:24 2020 +0200,Include internal dependencies in delete of chunks,"If a hypertable is created with an index on it and a continuous aggregate is further defined on the hypertable, it will create an internal dependency between the chunks of the hypertable and the chunks of the continuous aggregate. When dropping chunks with `cascade_to_materialization` set to `FALSE`, this will generate an error since the delete is not cascaded to the internal dependencies. This commit fixes this by collecting the internal dependencies and using `performMultipleDelete` rather than `performDelete` to delete several objects as one operation. Fixes #1889 " c39989bca91a9ae9570e107e0283eb88c91f263f,Sven Klemm,Fri Jun 5 12:53:35 2020 +0200,Remove check for PG 10 in update script generation,Since PG 9.6 is no longer supported version not less than 10 is always true now and this check can be removed and remote_txn.sql can always be added. eef1e16ef5f8b0ae2c8f73f8c7486c611ffe0f44,Sven Klemm,Thu Jun 4 23:56:35 2020 +0200,Enable linter only for clang compiler,"Enabling clang-tidy on builds not using clang compiler will not compile due to incompatible compiler flags, so this patch changes the current behaviour to only enable clang-tidy when using clang compiler. You can still overwrite the behaviour by passing -DLINTER=ON when not using clang. " 36d43503c1aeceb66d39e41d0768afd3bde420ad,Sven Klemm,Thu Jun 4 11:59:58 2020 +0200,Change update script generation to not use scratch files,This patch changes the update script generation to not use scratch files and removes the sql fragments to set and unset the post_update_stage from CMakeLists.txt and puts them into dedicated files. 487c0e07032cfc0e9b6cf61fd6d138d35cd5c4e6,Ruslan Fomkin,Thu Jun 4 10:23:34 2020 +0200,Use correct null device on Windows,Fixes null device on Windows to work in both CMD and PowerShell. 5c6940d734e1a18316c8d4d90868808b3ffbf7a7,Floris van Nee,Wed May 6 18:45:00 2020 +0200,Add support for fast pruning of inlined functions,"Inlineable functions used to be slow to plan, because the query preprocessing function could not find the relations inside the functions, as they haven't been inlined yet at that point. This commit adds a separate check in the get_relation_info_hook to optimize pruning of hypertables. " b5915f5eabac12f40a4654af45ea6a4d726dfb45,Sven Klemm,Thu Jun 4 00:02:34 2020 +0200,Skip dist_hypertable and dist_query in release builds,The dist_hypertable and dist_query tests need timestamp mock functionality which is only available in debug builds. afb6c1a78b1e29a9c54920376b3ebce9c992a58c,Stephen Polcyn,Wed Jun 3 09:47:22 2020 -0400,Update CHANGELOG.md,"Updates CHANGELOG with changes from PR ""Change compression locking order""(#1932). " 315a680c725f1e2b996fef642afd7c1b6b292e8e,Sven Klemm,Wed May 27 00:37:37 2020 +0200,Fix gapfill locf treat_null_as_missing,"When using time_bucket_gapfill with the treat_null_as_missing option we did not properly set the number of valid values when generating our own virtual tuples leading to ""cannot extract attribute from empty tuple slot"" when the number of values got reset. This patch changes the gapfill code to always set the number of valid values when generating virtual tuples. " c90397fd6a0cfb5cc6487207424eeac5e6920c52,Sven Klemm,Mon Jun 1 22:18:11 2020 +0200,Remove support for PG9.6 and PG10,This patch removes code support for PG9.6 and PG10. In addition to removing PG96 and PG10 macros the following changes are done: remove HAVE_INT64_TIMESTAMP since this is always true on PG10+ remove PG_VERSION_SUPPORTS_MULTINODE 663463771b9815a2dc7682c76cdb64fa83b9711f,Sven Klemm,Tue Jun 2 13:17:32 2020 +0200,Use EXECUTE FUNCTION instead of EXECUTE PROCEDURE,Replace EXECUTE PROCEDURE with EXECUTE FUNCTION because the former is deprecated in PG11+. Unfortunately some test output will still have EXECUTE PROCEDURE because pg_get_triggerdef in PG11 still generates a definition with EXECUTE PROCEDURE. 92b6c03e4314abd800d224512a256a3165733db5,Mats Kindahl,Thu May 7 09:40:48 2020 +0200,Remove cascade option from drop_chunks,"This commit removes the `cascade` option from the function `drop_chunks` and `add_drop_chunk_policy`, which will now never cascade drops to dependent objects. The tests are fixed accordingly and verbosity turned up to ensure that the dependent objects are printed in the error details. " 92bdcf02d1c4a67c67ba7d3b2bf8e270d045c9c4,Sven Klemm,Tue Jun 2 02:09:30 2020 +0200,Fix license check return code,"This patch fixes the return code of the license check, previously it would always return the return code of the apache license check. Additionally we check for the license in versioned sql base files too so we dont miss files without license when the versioned sql file has not been generated. This patch also adds a missing newline to the jit_load.sql file so the license check properly detects the license and fixes the license for the continuous_aggs_union_view test. " d1aacdccad1055d202ddd1e99bf5d2cb5b5c5674,Stephen Polcyn,Sun May 24 13:49:17 2020 -0400,Change compression locking order,"This patch changes the order in which locks are taken during compression to avoid taking strong locks for long periods on referenced tables. Previously, constraints from the uncompressed chunk were copied to the compressed chunk before compressing the data. When the uncompressed chunk had foreign key constraints, this resulted in a ShareRowExclusiveLock being held on the referenced table for the remainder of the transaction, which includes the (potentially long) period while the data is compressed, and prevented any INSERTs/UPDATEs/DELETEs on the referenced table during the remainder of the time it took the compression transaction to complete. Copying constraints after completing the actual data compression does not pose safety issues (as any updates to referenced keys are caught by the FK constraint on the uncompressed chunk), and it enables the compression job to minimize the time during which strong locks are held on referenced tables. Fixes #1614. " b3d1075ffeef4de1d584161539a264bf02f94006,Dmitry Simonenko,Mon Jun 1 14:37:28 2020 +0300,Support table access methods for distributed hypertables,This change adds support for CREATE TABLE USING clause and includes it for tables created on the data nodes. 0477f40fb84741b34557f709006d77cdfa2737e2,Sven Klemm,Sat May 30 07:50:34 2020 +0200,Remove PG 9.6 and PG10 test output,This patch removes PG 9.6 and PG 10 test output files and merges the output files where PG11 output does not differ from PG12 output. Removing code support for these versions will be in a subsequent patch. bdbdfd5ca961bfbb60f895bdb700c2b832521a20,Mats Kindahl,Mon Jun 1 08:03:28 2020 +0200,Remove $null file,The `$null` file was accidentally added to the repository. This commit removes it. 9e452aa1587d5193283279a4ff795ad13bb23e92,Sven Klemm,Thu May 28 19:05:33 2020 +0200,Make travis coverity run not depend on other tests,This patch changes the travis configuration to make the coverity run not depend on the result of other test runs and runs it on the coverity_scan branch only since there is a quota on the number of coverity runs we can do and we dont want to exceed it by multiple pushes to prerelease_test. 6569b5256c89cffe78448e1db8ce370263608d89,Mats Kindahl,Mon May 25 07:51:31 2020 +0200,Increase number of spins.,Increase the number of spins for spin waits in tests. This is to avoid test flakiness. e16662a94bcf0663363b0b7fe6bf19dba01cdba1,Sven Klemm,Thu May 28 18:58:34 2020 +0200,Set version in master to 2.0.0-dev,We want to have the version in the master branch to be the version of the next major release with a -dev suffix. Actual release versions of the code will get their own respective branches. c7478f04b3607d348790452101011b4899a5ebb1,Brian Rowe,Wed May 27 16:57:17 2020 -0700,Change default prefix for distributed tables,"This change modifies the default 'associated_table_prefix' string for compressed hypertables from ""_hyper_n"" to ""_dist_hyper_n"" where n is the hypertable id of the table. This change makes it so that when the backend hypertable for a distributed hypertable is created on a data node, it won't have an 'associated_table_prefix' that may conflict with locally created tables. As part of this change, chunk names for distributed tables no longer append the string ""dist"" as this is now part of the prefix for those chunk names (i.e. instead of ""_hyper_1_1_dist_chunk"" we will now have ""_dist_hyper_1_1_chunk""). " 66bf3a272c6080343f675043623d15ea9fd71d1b,Erik Nordström,Fri Apr 3 13:14:29 2020 +0200,Make compiler warnings into errors,"This change makes compiler warnings into errors by default on all builds. This setting used to be applied on Travis builds, but was removed (assumingly by accident) in a previous commit. The new behavior can be disabled by including `-DWARNINGS_AS_ERRORS=OFF` when running cmake. " a2cc321700c6377fa610f88f1c02c5981f17d512,Erik Nordström,Wed Apr 8 21:03:46 2020 +0200,Fix Windows compiler warnings,This change fixes various Windows-specific compiler warnings. 589aa898f717eb483a60e466e86b39a199fabd18,Sven Klemm,Wed May 20 22:11:15 2020 +0200,Run 32 bit linux test as github action, ccc1018f441f7d5018785b79ce5069f154acbd4f,Erik Nordström,Thu May 28 17:11:49 2020 +0200,Fix various linter-found issues,"This fixes variuous issues found by clang-tidy. A number of compression-related issues still remain, however. " 1dd9314f4d5c466427e6683cabc6bf15406f48ef,Erik Nordström,Thu May 28 12:37:16 2020 +0200,Improve linting support with clang-tidy,"This change replaces the existing `clang-tidy` linter target with CMake's built-in support for it. The old way of invoking the linter relied on the `run-clang-tidy` wrapper script, which is not installed by default on some platforms. Discovery of the `clang-tidy` tool has also been improved to work with more installation locations. As a result, linting now happens at compile time and is enabled automatically when `clang-tidy` is installed and found. In enabling `clang-tidy`, several non-trivial issues were discovered in compression-related code. These might be false positives, but, until a proper solution can be found, ""warnings-as-errors"" have been disabled for that code to allow compilation to succeed with the linter enabled. " 1c1b3c856e3f9b48d065d4ddfada1a73a119c57d,Sven Klemm,Wed May 20 01:33:55 2020 +0200,Cleanup GUC names,Change our GUC names to use enable-prefix for all boolean GUCs similar to postgres GUC names. This patch renames disable_optimizations to enable_optimizations and constraint_aware_append to enable_constraint_aware_append and removes optimize_non_hypertables. 6bc4765f4d887dda1299ac2140d9b89dfbe31cc7,Ruslan Fomkin,Thu May 28 13:56:35 2020 +0200,Remove regression tests on PG 9.6 and 10,"The first step of removing support for PG 9.6 and 10 is to remove the regression tests, which run against PostgreSQL versions 9.6 and 10. " 14492cc562b4ab771007a2a7f2adc83afd3c61f6,Erik Nordström,Mon May 25 15:08:59 2020 +0200,Add AppVeyor configuration for multinode,"This change updates the AppVeyor configuration for multinode-related tests. These changes include, but are not limited to: * Set `max_prepared_transactions` for 2PC. * Add SSL/TLS configuration (although this is off for now due to failing `loader` test when SSL is on). * Update port settings since `add_data_node` outputs port. * Ignore `remote_connection` and `remote_txn` since they use a ""node killer"" which does not work on Windows (SIGTERM not supported). * Set timezone and datestyle " c2744e13ad44921b53270ce05a66dff53976c59f,Mats Kindahl,Mon Apr 6 08:59:49 2020 +0200,Show error message on unavailable extension,"If the extension is not available on the data node, a strange error message will be displayed since the extension cannot be installed. This commit check for the availability of the extension before trying to bootstrap the node and print a more helpful informational message if the extension is not available. " ac94947199706e52abae47729146994b2b72b368,Dmitry Simonenko,Mon Mar 2 16:42:31 2020 +0300,Fix insert batch size calculation for prepared statements,"Related to the issue #1702. Current insert batching implementation depends on a number of table columns and a batch size. It does not take into account a maximum number of prepared statements arguments, which by default can be exceeded with tables having a large number of columns. This PR has two effects: 1) It automatically recalculates insert batch size instead of using fixed TUPLE_THRESHOLD value, if the expected total number of prepared statement arguments will exceed the limit. 2) If fixes integer overload in INSERT statement deparsing if the number of arguments is greater then 16k. " f214b64b3185c33dfc1c9e03aec107967abb0008,Mats Kindahl,Fri Jan 17 10:46:26 2020 +0100,Add test for grant propagation,"Add test for grant propagation when attaching a data node to a table. Function `data_node_attach` already calls `hypertable_assign_data_nodes`, which assigns data nodes, so grants are properly propagated to data nodes when they are attached. " 5044f5d115cd00a0626c20c0e5530b89c04cea30,Dmitry Simonenko,Tue Feb 25 16:02:52 2020 +0300,Allow SERIAL columns for distributed hypertables,This change introduce fix and adds support for serial columns on distributed hypertables. It fixes issue #1663. Basically a SERIAL type is a syntax sugar which automatically creates a SEQUENCE object and makes it dependable on the column. It also sets DEFAULT expression for using the sequence. The idea behind the fix is to avoid using the default expression when deparsing and recreating tables on data nodes in case if the column has a dependable sequence object. 7a93a2f80504145e13f49aa91caee6ee5a1e0cb0,Mats Kindahl,Fri Jan 31 11:06:18 2020 +0100,Change location of user certificates and keys,"User certificates and keys for logging into data nodes are stored at the top level of the `ssl_dir` or in the data directory. This can cause some confusion since a lot of files with user names resembling existing configuration files will be created as users are added, so this commit change the location of the user certificates and keys to be in the `timescaledb/certs` subdirectory of either the `ssl_dir` or data directory. In addition, since user names can contain strange characters (quoted names are allowed as role names, which can contain anything) the commit changes the names for certificates and keys to use the MD5 sum as hex string as base name for the files. This will prevent strange user names from accessing files outside the certificate directory. The subdirectory is currently hardcoded. " 7b5275e540e73d9329501220206b61a29d612614,Dmitry Simonenko,Mon Feb 17 17:49:49 2020 +0300,Allow ALTER TABLE SET on distributed hypertable,"This change allows to use ALTER TABLE SET/RESET, SET OIDS and SET WITHOUT OIDS clauses with a distributed hypertable. This PR has two effects: 1. It prevents having to copy storage options for foreign table chunks when their objects are created on the AN. The command updates only root table options on the AN and passes it for execution on the data nodes. 2 It prevents distributed hypertable chunks to be updated in the 'ddl_command_end' event trigger on AN, because PostgreSQL does not support altering storage options for foreign tables. " 9b4aae813fa04f01be921bb4e9f58e7fbead1fb2,Dmitry Simonenko,Tue Feb 4 17:30:52 2020 +0300,Support storage options for distributed hypertables,This change allows to deparse and include a main table storage options for the CREATE TABLE command which is executed during the create_distributed_hypertable() call. 26c6e156d7b56a98d9a5d02f9f043a97368661f1,Erik Nordström,Tue Jan 28 13:33:00 2020 +0100,Fix port conversion issue in add_data_node,"This change fixes an issue with port conversion in the `add_data_node` command that results in an error when a port is not explicitly given and PostgreSQL is configured to use a high port number. Note that this issue does _not_ occur when the port number is given as an explicit argument to `add_data_node`. The underlying issue is that, without an explicit port number, the remote port is assumed to be the same as the port configured for the local server instance. The conversion of that port number was done using a _signed_ two-byte integer, while the valid port range fits within an _unsigned_ two-byte integer. To test higher port ranges without an explicit argument to `add_data_node`, the default port for test instances has been updated to a high port number to test integer range overflow for small signed integers. " 0f2d7251cf0abbf5680a150f822115d411bcf7ec,Dmitry Simonenko,Tue Jan 28 16:39:24 2020 +0300,Basic LIMIT push down support,This initial implementation allows to deparse LIMIT clause and include it in the push down query sended to the data nodes. Current implementation is quite restrictive and allows to use LIMIT only for simple queries without aggregates or in conjunction with the ORDER BY clause. effdc478ae56bafe71b7c13368192fa6e5ca70a2,Ruslan Fomkin,Thu Jan 23 10:41:26 2020 +0100,Check replication factor for exceeding data nodes,set_replication_factor will check if the replication factor is bigger than the amount of attached data nodes. It returns an error in such case. 686860ea23abf49445f4213fff76238dc676bdd3,Erik Nordström,Fri Dec 6 16:43:44 2019 +0100,Support compression on distributed hypertables,"Initial support for compression on distributed hypertables. This _only_ includes the ability to run `compress_chunk` and `decompress_chunk` on a distributed hypertable. There is no support for automation, at least not beyond what one can do individually on each data node. Note that an access node keeps no local metadata about which distributed hypertables have compressed chunks. This information needs to be fetched directly from data nodes, although such functionality is not yet implemented. For example, informational views on the access nodes will not yet report the correct compression states for distributed hypertables. " bf343d77189f5fa06765ded856714ecbd72bd110,Brian Rowe,Mon Jan 20 16:49:39 2020 -0800,Add a test to verify attach_node behavior,This change adds a new case to the data_node test that verifies that attaching a data node to a hypertable on a data node will fail (as hypertables are not marked as distributed on data nodes). c44a202576630f7d64c648248df0705805f688d3,Ruslan Fomkin,Wed Jan 15 10:58:58 2020 +0100,Implement altering replication factor,"Implements SQL function set_replication_factor, which changes replication factor of a distributed hypertable. The change of the replication factor doesn't affect existing chunks. Newly created chunks are replicated according to new replication factor. " d49e9a5739b330515e3ca532cbb45d1b5b0ad236,Brian Rowe,Wed Jan 15 16:33:51 2020 -0800,Add repartition option on detach/delete_data_node,This change adds a new parameter to the detach_data_node and delete_data_node functions that will allow the user to automatically shrink their space dimension to match the number of nodes. 32f3d17cde0532f803824d475a7722c673d6b167,Erik Nordström,Wed Jan 15 09:59:39 2020 +0100,Rename hypertable_distributed test,The `hypertable_distributed` test is now renamed to `dist_hypertable` for consistency with other distributed tests that have the `dist_` prefix. 00172083683ea6f52ea3ced62416cd90f51d9afe,Brian Rowe,Fri Jan 3 15:21:07 2020 -0800,Test dimension add on distributed hypertables,Prior to this change attempting to add a dimension to a distributed hypertable which currently or previously contained data would fail with an opaque error. This change will properly test distributed hypertables when adding dimensions and will print appropriate errors. 8d28fad66dfa3c2635b0e60f44ac83b40fb5b5f0,Mats Kindahl,Thu Dec 19 14:58:50 2019 +0100,Error on reference from distributed hypertable,It is not possible to properly reference another table from a distributed hypertable since this would require replication of the referenced table. This commit add a warning message when a distributed hypertable attempt to reference any other table using a foreign key. 32bdf642056777981eaf636af49752c97353a2f9,Erik Nordström,Fri Dec 20 16:25:25 2019 +0100,Fix compiler warning in release builds,This fixes a couple of warnings about unused variables used for assert checking that appear in release builds. The `PG_USED_FOR_ASSERTS_ONLY` attribute has been applied to the variable declarations to quench the warnings. f20ad8231d9e04439e55b29775ef65c350b2a3b9,Erik Nordström,Thu Dec 19 13:58:31 2019 +0100,Release 2.0.0-beta4,"This release includes user experience improvements for managing data nodes, more efficient statistics collection for distributed hypertables, and miscellaneous fixes and improvements. " 55803125f3bee240262a7a5a500d69d1299a42f3,Erik Nordström,Thu Dec 19 17:26:13 2019 +0100,Better handling of chunk insert state destruction,"Previously, the memory context for the chunk insert state was freed using a reset callback on the per-tuple context. This created an unfortunate cyclic dependency between memory contexts, since both the per-tuple context and chunk insert state shared the same memory context parent (the query memory context). Thus, when deletion happens by calling MemoryContextDelete on the parent, without having deleted the children first, the parent could first delete the chunk insert state child, followed by the per-tuple context which then tried to delete the chunk insert state again. A better way to handle this is to simply switch the parent of the chunk insert state's memory context to be the per-tuple context as long as it is still valid, thus breaking the cycle. " 2cec573213182ceb2b51a2ed4713678e616da50f,Dmitry Simonenko,Thu Dec 19 16:30:03 2019 +0300,Fix crash when cancelling long distributed insert,"Problem lies in incorrect handling of reset callback for the ChunkInsertState during the transaction abort procedure, which frees parent memory context. Because reset callbacks are executed only after all child memory context got deleted, it is possible to end up in the sutiation when the context is already deleted before this callback function being called. " 150041566c1c73787d2b305f2869f19bbc0d12b2,Erik Nordström,Thu Dec 19 09:26:59 2019 +0100,Fix non-determinism in distributed query tests,"Tests that run `ANALYZE` on distributed hypertables are susceptible to non-deterministic behavior because `ANALYZE` does random sampling and might have a different seed depending on data node. This change fixes this issue by running `setseed()` on all data node sessions before `ANALYZE` is run. Unfortunately, while this makes the behavior on a specific machine more deterministic, it doesn't produce the exact same statistics across different machines and/or C-libraries since those might have different PRNG implementations for `random()`. " 8887f26baf2572abc81884e838d06b297151430d,Erik Nordström,Thu Dec 19 12:48:03 2019 +0100,Fix array construction issue for remote colstats,"When fetching remote column statistics (`pg_statistic`) from data nodes, the `stanumbers` field was not turned into an array correctly. This caused values to be corrupted when importing them to the access node. This issue has been fixed along with some compiler warning issues (e.g., mixed declaration and code). " fad33fe954a8f31f65e805df03ef64ecbe3f1d80,Brian Rowe,Thu Nov 7 08:22:15 2019 -0800,Collect column stats for distributed tables.,"This change adds a new command to return a subset of the column stats for a hypertable (column width, percent null, and percent distinct). As part of the execution of this command on an access node, these stats will be collected for distributed chunks and updated on the access node. " 222bf759104f29e66cd22aee6a27101d66bcdc92,Mats Kindahl,Mon Dec 16 16:55:41 2019 +0100,Use template1 as secondary connection database,"The `postgres` database might not exists on a data node, but `template1` will always exist so if a connection using `postgres` fails, we use `template1` as a secondary database. This is similar to how `connectMaintenanceDatabase` in the PostgreSQL code base works. " 7a25d4bfb3c3cb98e7bc352b551355b2911f3619,Erik Nordström,Mon Dec 16 13:58:53 2019 +0100,Fix mixed declaration and code warning,"This change fixes a ""mixed declaration and code"" warning in the remote chunk estimation code. " f747e9df8bae8bcd4839b0016fdc6a2931949307,Erik Nordström,Fri Dec 6 12:11:45 2019 +0100,Remove the partitionwise_distributed test,"The partitionwise_distributed test is now superseeded by dist_query, which is a much cleaner and better test for the same things. " 597d04a77a70b81d0872871fe7225b790b6846a5,Erik Nordström,Fri Oct 18 13:36:33 2019 +0200,Refactor distributed query tests,"Tests for queries on distributed hypertables are now consolidated in the `dist_query` test. Not only does this test provide more consistent EXPLAIN output, but it also runs all queries against different types of tables holding the same data, including comparing the result output with `diff`. The different types of tables compared are: - Regular table (for reference) - One-dimensional distributed hypertabe - Two-dimensional distributed hypertabe (which is partially repartitioned) EXPLAINs are provided on the two-dimensional table showing the effect on plans when quering repartitioned time ranges. In most case, FULL push-down is not possible in such cases. In addition to test refactoring, this change includes a fix for handling `HAVING` clauses in remote partialized queries. Such clauses should not be sent to the remote end in case of partial queries since any aggregates in the `HAVING` clause must be returned in the result target list. Fortunately, modification of the target list is already taken care of by the main planner. " 88d59735f90a5ef04e72bf7cf296363415f1230f,Erik Nordström,Fri May 8 19:27:53 2020 +0200,Make dist_query test PG version specific,"This change makes the dist_query test PG version-specific in preparation for test changes that will produce different output between, e.g., PG11 and PG12. " 6f5da9b5eb01b94e31ddb7add95d5711760fa7ab,Dmitry Simonenko,Tue Dec 10 16:42:34 2019 +0300,Fix memory leak during long distributed insert,"Tuple expression context memory context is not properly reset during chunk dispatch execution which eventually consumes all available memory during the query execution: INSERT INTO test_table SELECT now() - random() * interval '2 years', (i/100)::text, random() FROM generate_series(1,700000) AS sub(i); This problem does not reproduces for a distributed hypertables with disabled batching and for a regular hypertables. Because luckly the tuple expression context got freed during the ModifyTable node execution. " 71e2c35d48a4f42d66c7355335e602c26b42a5ee,Dmitry Simonenko,Fri Dec 6 18:01:58 2019 +0300,Run distributed VACUUM/ANALYZE without FDW API,Run VACUUM/ANALYZE and automatically import the updated stats using the distributed DDL functionality instead of FDW analyze wrappers. 59b35db9c92f32358ae1fc881d9318cba8501b13,Erik Nordström,Thu Dec 5 14:46:12 2019 +0100,Print only pruned paths in debug output,"When printing paths in debug mode, the ""considered"" paths saved in the private rel info is a superset of the paths in the rel's pathlist. The result of this is that many paths are printed multiple times. This change makes sure we only print the ""diff"", i.e, the pruned paths that were considered but are no longer in the pathlist. A number of other issues with the debug output has also been addressed, like consistent function naming and being more robust to printing rels that might not have `fdw_private` set. " c60cabd76848458a305d056b03390adade354056,niksa,Wed Nov 6 22:24:34 2019 +0100,Improve relation size estimate,In case when there are no stats (number of tuples/pages) we can use two approaches to estimate relation size: interpolate relation size using stats from previous chunks (if exists) or estimate using shared buffer size (shared buffer size should align with chunk size). 29ce1510a5f3809cc6930a2a89fdd1231b6665f6,Mats Kindahl,Fri Nov 29 14:13:09 2019 +0100,Allow extension on data node,"Before this commit, an existing extension would cause an error and abort the addition of a data node if `bootstrap` was `true`. To allow extension to already exist on the data node, this commit will first check if the extension exists on the data node. If the extension exists, it will be validated, otherwise the extension will be created on the data node. " 66255eb5cb912e10306078c81787233aaf1d615d,niksa,Fri Nov 22 12:25:33 2019 +0100,Improve planner debug output,To better understand choices that planner makes we need to print all the paths (with costs) that planner considered. Otherwise it might be hard to understand why certain path is not picked (eg. due to high startup/total costs) since it will never show up in relation path list that we print. This should help while working on improving distributed cost mode. This fix focuses only on paths that involve data nodes. 3bd1f914f1ac9bfe210c08d5f680b80f3e863c02,niksa,Tue Dec 3 15:47:31 2019 +0100,Use qualified table name for chunks_in,If a table contains a column with a same name as table name then query parser will get confused when parsing `chunks_in` function. The parser would think that we are passing in column instead of a table. Using qualified table name fixes this problem. Note that we needed to expand table using .* in order to avoid parser confusion caused by schema.table syntax. 267a13ec98ff3a98623a4b91f5432d1bbcefd1b8,Mats Kindahl,Thu Nov 28 10:09:54 2019 +0100,Fix data node extension version check,"Currently, if the major version of the extension on the access node is later than the version of the extension on the data node, the data node is accepted. Since major versions are not compatible, it should not be accepted. Changed the check to only accept the data node if: - The major version is the same on the data node and the access node. - The minor version on the data node is same or earlier than than access node. In addition, the code will print a warning if the version on the data node is older than the version on the access node. " 11ef10332ef8491b4b81e161808fe07f179202b5,Dmitry Simonenko,Mon Dec 2 16:29:26 2019 +0300,Add number of compressed hypertables to stat,"This change includes telemetry fixes which extends HypertablesStat with num_hypertables_compressed. It also updates the way how the number of regular hypertables is calculated, which is now treated as a non-compressed and not related to continuous aggregates. " d5f5d92790b88c93b0c8a94d8b804ab44834bcf2,Mats Kindahl,Thu Nov 28 11:17:18 2019 +0100,Refactor to add database representation,"Adding a `Database` structure to keep track of database name, collation, encoding, and character type. " aa327518d67b9f2747a3cb2d6b8f14e6ea19bb7e,niksa,Wed Nov 27 09:38:07 2019 +0100,Row-by-row fetcher hardening,Fix dangling pointers when closing async response results. Remove unnecessary data fetch call. fdc7138bdaeeb7ab0a8cde16cb70bfc57c31f4a9,Mats Kindahl,Tue Nov 26 15:43:30 2019 +0100,Validate database on data node,"When a data node needs bootstrapping and the database to be bootstrapped already exists, it was blindly assumed to be configured correctly. With this commit, we validate the database if it already existed before proceeding and raise an error if it is not correctly configured. When validating the data node and bootstrap is `true`, we are connected to the `postgres` database rather than the database to validate. This means that we cannot use `current_database()` and instead pass the database name as a parameter to `data_node_validate_database`. " c14948ad986b40922bcfaeaa54165d1c8ac68c79,Mats Kindahl,Tue Nov 19 14:09:36 2019 +0100,Propagate grants to data nodes,"Before this commit, grants and revokes where not propagated to data nodes. After this commit, grant and revokes on a distributed hypertable are propagated to the data nodes of the hypertable. " 96dd266a0b624eba47c22025db761a82fa10c351,Mats Kindahl,Mon Nov 11 13:44:58 2019 +0100,Propagate grants when creating hypertables,"When creating a hypertable, grants were not propagated to the table on the remote node, which causes later statements to fail when not executed as the owner of the table. This commit deparse grant statements from the table definition and add the grants to the deparsed statement to send when creating the table on the data node. " ef823a30605a836c087bef4e5b93133f3ffea472,Ruslan Fomkin,Wed Nov 20 15:06:26 2019 +0100,Remove unnecessary check from distributed DDL,"Since NULL value for replication factor in SQL DDL corresponds to HYPERTABLE_REGULAR now, which is different from HYPERTABLE_DISTRIBUTED_MEMBER, there is no need to check for non-NULL value and comparing with HYPERTABLE_DISTRIBUTED_MEMBER is enough. " 6aec69f9c4527d098113633516bc7ade9e3f6277,Ruslan Fomkin,Tue Nov 19 21:23:28 2019 +0100,Rename exported test functions to follow convention,"Rename exported functions used in distributed tests to follow the convention of ts_ prefix, which was recently forced in non-distributed tests. " 78a5ba5bf2e068c3382f2ecb3bee33dc8475eeb9,Ruslan Fomkin,Wed Nov 6 14:55:56 2019 +0100,Fix uninitialized warning in test help function,"Test code in remote_exec fails to build due to the maybe uninitialized error on 32-bit alpine package on a string variable. This fix moves initialization to the string variable declaration, refactors a loop to have a single place with exit condition, which checks for both NULL value and empty string. " 5933c1785ea58a461981af3486a10806c1763a75,Erik Nordström,Mon Oct 7 09:35:55 2019 +0200,Avoid attaching data nodes without permissions,"This change will make `create_distributed_hypertable` attach only those data nodes that the creating user has USAGE permission on instead of trying to attach all data nodes (even when permissions do not allow it). This prevents an ERROR when a user tries to create a distributed hypertable and lacks USAGE on one or more data nodes. This new behavior *only* applies when no explicit set of data nodes is specified. When an explicit set is specified the behavior remains unchanged, i.e., USAGE is required on all the explicitly specified data nodes. Note that, with distributed hypertables, USAGE permission on a data node (foreign server) only governs the ability to attach it to a hypertable. This is analogous to the regular behavior of foreign servers where USAGE governs who can create foreign tables using the server. The actual usage of the server once associated with a table is not affected as that would break the table if permissions are revoked. The new behavior introduced with this change makes it simpler to use `create_distributed_hypertable` in a multi-user and multi-permission environment where users have different permissions on data nodes (DNs). For instance, imagine user A being allowed to attach DN1 and DN2, while user B can attach DN2 and DN3. Without this change, `create_distributed_hypertable` will always fail since it tries to attach DN1, DN2, and DN3 irrespective of the user that calls the function. Even worse, if only DN1 and DN2 existed initially, user A would be able to create distributed hypertables without errors, but, as soon as DN3 is added, `create_distributed_hypertable` would start to fail for user A. The only way to avoid the above described errors when creating distributed hypertables is for users to pass an explicit set of data nodes that only includes the data nodes they have USAGE on. If a user is forced to do that, the result would in any case be the same as that introduced with this change. Unfortunately, users currently have no way of knowing which data nodes they have USAGE on unless they query the PostgreSQL catalogs. In many cases, a user might not even care about the data nodes they are using if they aren't DBAs themselves. To summarize the new behavior: * If public (everyone) has USAGE on all data nodes, there is no change in behavior. * If a user has USAGE on a subset of data nodes, it will by default attach only those data nodes since it cannot attach the other ones anyway. * If the user specifies an explicit set of data nodes to attach, all of those nodes still require USAGE permissions. (Behavior unchanged.) " af6ca6e3aea1a0e94acc273b9dcc98e8f2b81ef9,Ruslan Fomkin,Mon Nov 4 13:56:21 2019 +0100,Fix test code to not leak and improve safety,Fixes test help function remote_exec with: - Free always the memory allocated by open_memstream and skip printing no result. Fixes memory leak. - Replacing strtok with strtok_r to avoid risks of a data race. 3cf11b444f74bb194130125726f861f1c26f67ca,Mats Kindahl,Mon Nov 4 14:20:46 2019 +0100,Change collation in data_node_bootstrap test,"The collation used in `data_node_bootstrap.sql` was not available for the installation used by the sanitizer, causing the sanitizer tests to fail. This commit change the collation and ctype to use types available on that platform. " 1afbac9ac8f44b417171c918ac7ab62775431277,Ruslan Fomkin,Mon Nov 4 14:13:35 2019 +0100,Release 2.0.0-beta3,"**For beta releases**, upgrading from an earlier version of the extension (including previous beta releases) is not supported. This release improves performance for queries executed on distributed hypertables, fixes minor issues and blocks few SQL API functions, which are not supported on distributed hypertables. It also adds information about distributed databases in the telemetry. " 255db087ba2cbcf0414b974b85e68faa12f1c7d8,niksa,Thu Oct 31 11:40:19 2019 +0100,Remove remote estimates,Remote estimation is increasing total query time since it involves additional round trips to data nodes. Since now we can more efficiently fetch chunk stats from data nodes (using get_chunk_relstats) - fetching and parsing remote estimates becomes obsolete. 02f7d7aa48c1c52e4164a21ec86f512e034887eb,niksa,Fri Oct 4 14:53:12 2019 +0200,Fetch data using either cursor or row-by-row,This change introduces two ways of fetching data from data nodes: one using cursors and another one using row-by-row mode. The major benefit of row-by-row mode is that it enables running parallel plans on data nodes. The default data fetcher uses row-by-row mode. A new GUC `timescaledb.remote_data_fetcher` has been added to enable switching between these two implementations (rowbyrow or cursor). 0a31e725401260b0ff579dba10d26fd254c5a3ad,Dmitry Simonenko,Wed Oct 30 15:20:22 2019 +0300,Block tablespace api for a distributed hypertable, 296d134a1e1719cbc988787fca21464404fe77c2,Dmitry Simonenko,Tue Oct 29 14:55:36 2019 +0300,Add telemetry to track distributed databases,"This change extends telemetry report and adds new 'distributed_db' section which includes following keys: 'distributed_member', 'data_nodes_count' and 'distributed_hypertables_count'. " af0a75f8fe4c58f23340f3f2707b7d2fec4895b2,Erik Nordström,Mon Oct 28 10:13:59 2019 +0100,Compute basic data node rel stats from chunks,"Planning of a data node rel during a distributed query should use the accumulated stats from the individual chunks that the data node rel represents. Since the data node rel is not a real base rel (i.e., it doesn't correspond to a real relation) it doesn't have any statistics in the `pg_catalog` that can be used for planning. Thus, some functions, such as `set_baserel_size_estimates` will return strange estimates for data node rels when the planner believes it has stats (e.g., after an ANALYZE). This change fixes this issue by not relying on the planner to compute rel estimates for data node rels. Instead the accumulated estimates based on the chunks queried by the data node rel are used. This also obviates the need to compute these stats again. Given the new size estimates that this change enables, some plan/test outputs have changed and tests updated to deal with that. " 00508108038cf98920701deda8b9950c9ffbb546,Mats Kindahl,Thu Oct 24 20:59:11 2019 +0200,Create data node with character set,"The access node and the data node need to have the same encoding, the same ctype, and the same collation, so the following changes where made when bootstrapping the database on the data node: - Explicity use `template0` - Explicitly set the encoding of the database on the access node - Explicitly set the `LC_CTYPE` of the database on the access node - Explicitly set the `LC_COLLATE` to the collation of the database on the access node When not bootstrapping, it is checked that the encoding, `LC_CTYPE`, and collation matches what the access node is using. " d31135e2cc2c59babc4e7da84959dc471b1ab541,Mats Kindahl,Mon Oct 21 10:04:10 2019 +0200,Add reloptkind to debug output,This commit adds the reloptkind to the debug optimizer printout so that we can see the kind of relation that is being built. 97cbaf55a3121fb00e9199f0ba20ff620ecfd341,Erik Nordström,Fri Oct 25 09:31:22 2019 +0200,Fix variuos compiler warnings and nits,This change fixes a number of compiler/code warnings. f923a8ab8e7991411f400bfd0aec7c540928fa9b,Erik Nordström,Fri Oct 25 10:47:24 2019 +0200,Prepare for next development cycle,This change prepares the repo for developing the 2.0.0-beta3. The CMake configuration required an update to handle both a beta and dev suffix in the version string. 57bc2e6fe830afd25206be14cd35956061da143b,Erik Nordström,Mon Oct 21 17:03:12 2019 +0200,Release 2.0.0-beta2,"**For beta releases**, upgrading from an earlier version of the extension (including previous beta releases) is not supported. This release introduces *distributed hypertables*, a major new feature that allows hypertables to scale out across multiple nodes for increased performance and fault tolerance. Please review the documentation to learn how to configure and use distributed hypertables, including current limitations. " 7ff4d4b4e36d592e78bd2295ad3b087b43ae105b,Erik Nordström,Fri Oct 18 18:21:27 2019 +0200,Fix push down when hitting only one node,"For some queries, with only one node ""hit', push down can be forced by matching the partitioning attributes with the group by expression. However, push down didn't happen in some of these cases because of a bug in `force_group_by_push_down` that didn't properly update the number of attributes in the partitioning expression. This change fixes the bug to always set the number of partitioning attributes to match the group by clause. " 7e198c84321c81703f5c8a84377b2fde15afe5c5,Ruslan Fomkin,Wed Oct 16 18:55:09 2019 +0200,Prevent unexpected retry in bgw test,"Set max retries to 0 for the drop chunks background job in the background worker reorder drop chunk test, so it will not do a retry, which is not expected by the test, and thus the test passes on ARM. " 7ef5e2e21c8af34f0feef2057bd51b5c94582ce5,Erik Nordström,Wed Oct 16 16:54:18 2019 +0200,Fix output row estimation for ordered upper rels,"The number of output rows estimated for ""remote"" upper rels could sometimes erroneously be zero. This happened when computing the estimate for upper rels with different pathkeys: in case of several different path keys the estimation was not recalculated and instead relied on cached values from the first calculation on the same rel. However, the number of output rows was never cached so the second pathkey estimated for the upper rel would always produce zero output rows. This has now been fixed by storing the output rows in the upper rel after the first estimation. This fix affects some query plans so a number of tests are affected. " f0d69aa0eb7bda67dbdac592f4370c90b62f7f83,Mats Kindahl,Thu Oct 17 13:37:43 2019 +0200,Don't assert on connection loss,"If `remote_txn_check_for_leaked_prepared_statements` do not have a working connection, it will abort by crashing the server. Since there are tests that kill the remote server in different phases of the 2PC, bad timing might cause the server to crash rather than generate an error. This commit replace the assertion with both a status check and a check that the correct number of rows and columns are returned and will generate an status message with the error message, if any. " 6a9db8a621935aa13622bce4229861fec9b8be69,Erik Nordström,Wed Oct 16 14:47:22 2019 +0200,Add function to fetch remote chunk relation stats,"A new function, `get_chunk_relstats()`, allows fetching relstats (basically `pg_class.{relpages,reltuples`) from remote chunks on data nodes and writing it to the `pg_class` entry for the corresponding local chunk. The function expects either a chunk or a hypertable as input and returns the relstats for the given chunk or all chunks for the given hypertable, respectively. Importing relstats as described is useful as part of a distributed ANALYZE/VACUUM that won't require fetching all data into the access node for local sampling (like the current implemention does). In a future change, this function will be called as part of a local ANALYZE on the access node that runs ANALYZE on all data nodes followed by importing of the resulting relstats for the analyzed chunks. " 3a35b984f8672c3fd0fcdbf48fb5494adbf1478b,Mats Kindahl,Mon Oct 14 12:31:29 2019 +0200,Remove Postgres FDW from test,Removing usage of Postgres FDW from the test `partitionwise_distributed` since that triggers a memory error in memory checkers. 34b5ef4d183abac7015b250085a9b2b8d66e36ca,Mats Kindahl,Wed Sep 25 10:15:51 2019 +0200,Add debug printout for optimizer,"Debug printouts are added at two locations: * Inside `get_foreign_upper_paths` a notice with the existing paths is send back together with the stage if the `show_upper` is set to show the stage. * Inside `set_rel_pathlist` a notice is printed with the existing paths if the `show_rel` debug flag is set. The debug printouts are sent back to the client as debug messages (`DEBUG2`), allowing developers to quickly experiment and interactively see what effect different statements has with the respect to the paths generated. In addition, the handling of `debug_optimizer_flag` was not correct and is fixed here. * If no `=` was provided to `show_upper`, it triggered a segmentation fault. * If the flag was reset, the internal data structure was not updated. * If just one flag was updated in a `SET` command, the other flag was kept intact. " a8ca09b307fb8729daaef7d6e7cd6c6c38924e14,Dmitry Simonenko,Tue Oct 15 16:33:40 2019 +0300,Support VACUUM on distributed hypertables,"This patch upgrades existing dist_cmd functions to support executing commands which cannot be run inside an active transaction, such as the VACUUM command. " 2b1b1bdf873867879dcd54c931846365235df5d8,Dmitry Simonenko,Fri Oct 11 14:31:52 2019 +0300,Show NOTICE message on a distributed DROP DATABASE,"This patch adds a way to check and print a notice message to a user who wants to drop an access node database. Since database drop can only be done using different database connection, this functionality is implemented inside loader extension. Functionality of the security labels are used in order to mark a distributed database and make this information accessible by other databases in pg_shseclabel table. " 5910f6830501d1e54d607fa9f244cedeec081bd6,Brian Rowe,Thu Sep 26 22:40:05 2019 -0700,Improve pushdown handling of time functions,"This change allows certain mutable functions to be whitelisted so that they can be safely pushed down to data nodes. Additionally, this change will no longer prevent queries containing the `now` function from being pushed down to data nodes, but will instead replace the function call with the transaction start time (which is the same value which would be used had the query been run solely on the access node). " a443fe5ba97edf131a47c160c1dc904123398b62,Erik Nordström,Fri Oct 4 13:23:17 2019 +0200,Fix int to datum conversion in connection tests,"This fixes a problem in the connection tests when converting ints to datums on certain platforms (e.g., ARM). The appropriate datum conversion macros weren't used when returning datum results, which caused errors on ARM platforms. " c2366ece594961598d56c0e4c3d6b2c102b55fe3,Mats Kindahl,Fri Oct 4 09:12:06 2019 +0200,Don't clear dist_uuid in delete_data_node,"When deleting a data node it currently clear the `dist_uuid` in the database on the data node, which require it to be able to connect to the data node and would also mean that it is possible to re-add the data node to a new cluster without checking that it is in a consistent state. This commit remove the code that clear the `dist_uuid` and hence do not need to connect to the data nodel. All tests are updated to reflect the fact that no connection will be made to the data node and that the `dist_uuid` is not cleared. " 5fd363da4c5c488aad65127c30609ca0c4f5ea64,Mats Kindahl,Mon Oct 7 18:35:12 2019 +0200,Disable warnings for remote_txn,"For the tests that are using the `remote_node_set_kill_event` function to perform a hard termination and precise stages of the 2PC the connection will close with different warning messages depending on the timing of connection close and SSL close. To avoid a flaky test, this commit set `client_min_messages` to `ERROR` for the duration of the transaction. Since checks are done after the transaction that the transaction is properly rolled back in the event of a crash, the warning messages does not offer any additional benefits. " ad8b70ac12d6dd05bbd5b1edbff177bdbfc4825a,Mats Kindahl,Fri Sep 20 09:45:56 2019 +0200,Add debug_optimizer_flags GUC option,"Add a new `debug_optimizer_flags` option where you can request optimization debug notices. With this commit, the flags `show_upper` and `show_rel` are added. The flag `show_upper` will permit sending back a message with the resulting relation after creating upper paths. Since this is called at different stages, the flag support setting specific stages where printouts should be done using the format `show_upper=window,final`. The flag `show_rel` will permit sending back the resulting relation after executing `set_rel_pathlist` to add new paths for consideration. The actual implementation to sent the notices will be in a separate commit. " 4e004c556463a521a1b08bd856269902bea5e70c,Ruslan Fomkin,Thu Sep 26 11:17:10 2019 +0200,Unify to use a constant in array declarations,"Replaces a variable array length with a constant, which is commonly used in the code. The change is asserted. " e76d4507156398e23c89296cd5540e434cdfbc43,Ruslan Fomkin,Mon Sep 30 11:47:12 2019 +0200,Fix memory access bug in extension check,The fix copies result of remote call to obtain the owner and returns the copied value to the caller. 5cdef0388093531847d27993d0a182a97813ac79,Ruslan Fomkin,Mon Sep 30 09:53:09 2019 +0200,Fix bug in allow or block new chunks,"Process arguments of data node allow or block new chunks SQL API functions separately, since the number of optional arguments is different between allow and block functions. This fixes the bug with memory access. " 380103080de29d0a1f75d33df787c06e73ed130a,Erik Nordström,Mon Sep 23 15:24:21 2019 +0200,Remove collation checks for foreign expressions,"This change removes collation checks for foreign expressions. With distributed hypertables, we assume that all participating nodes have the same collation configuration so there should be no difference (from a collation perspective) between executing an expression locally and remotely. " cfc72be01d93b356a47c1e75a2fed21795d2a1a5,niksa,Thu Sep 26 23:05:21 2019 +0200,Show explain from data nodes,"We want to get more insights about what plans are executed on data nodes. If a user runs explain with verbose option we will connect to each data node, run explain on data node and print output together with existing explain output. " b01cc6ef1b1918917af0ba999e571aebeee784d2,niksa,Fri Sep 20 14:33:54 2019 +0200,Fix hard crash when aborting transaction,If something is wrong with a connection we would not be able to start a transaction on the remote node. So when abort happens we shoud not try to abort a transaction that hasn't started. We use xact_depth to check if transaction actually started. If we find one we remove it from our tx cache which should result in removing the invalid connection as well. 27f3effcb136bb96050bcd52ffbdb980c9bc8f97,niksa,Thu Sep 19 17:43:41 2019 +0200,Fix Result node pruning in AsyncAppend,The new custom AsyncAppend plan was not set for the case when we remove a Result node. ac7456bdf15ee05f91579c9ee1d3ad149a673675,Mats Kindahl,Wed Sep 18 10:27:05 2019 +0200,Add ENABLE_OPTIMIZER_DEBUG option,"In order to debug the optimizer it is necessary to provide the `OPTIMIZER_DEBUG` preprocessor symbol, so added an option to enable this for the code. It still requires the PostgreSQL source code to be built with this flag. " caf9ea80f9dcdd6c1342c7078be40bee5e51e683,Mats Kindahl,Wed Sep 18 10:04:57 2019 +0200,Some minor refactorings in data_node.c,"There is a read of the server identifier into `server_id` in `add_data_node_internal` which is not subsequently used, so it was removed. " 1c00ab28c03536551b9d664d7bc342c0129b0aa9,niksa,Tue Sep 17 12:35:50 2019 +0200,Prevent OOM when analyzing large datasets,Analyze command stores tuples in it's own memory context and we need to make sure to free tuples we don't need. 94979412efa24cb99a4769ce1877610b486da9c5,niksa,Tue Sep 17 11:20:14 2019 +0200,Fix chunks_in function declaration,We need to mark this function as stable and parallel safe so the planner can pick the most optimal plan. f64fd4247f0a2b5b2c8cfdb5b542a41c7a10ba52,Erik Nordström,Mon Sep 16 11:18:03 2019 +0200,Make image build script work with remote Docker,This change tweaks the `docker-build.sh` script to work against remote Docker daemons. This is useful when testing on minikube and you want to quickly build and deploy a new image with the current code without having to push to a remote registry. 7a4929627283a55435b7ad2f326e6a25c734a537,Erik Nordström,Fri Sep 13 09:21:18 2019 +0200,Distribute dimension-related DDL commands,"This change ensures that all dimension-releated DDL commands on hypertables are distributed to its data nodes. Most importantly, `add_dimension()` now applies across all data nodes of a hypertable when issued on the access node. This ensures the dimension configuration is the same on all nodes of a particular hypertable. While the configuration of chunk time interval and number of partitions do not matter that much for data nodes (since the access node takes care of sizing chunks), functions like `set_chunk_time_interval()` and `set_number_partitions()` are distributed as well. This ensures that dimension-related configuration changes apply across all nodes. " fc78a7271adcee8b69a7631eac3163fc4a28b9ca,Erik Nordström,Fri Sep 13 15:08:34 2019 +0200,Fix unused variable in deparsing code,"In the deparse code, a variable is used only in an `Assert`, which makes the compiler complain in release builds. This change fixes this issue. " 67c5f84bc31f9ac867f3f7ae21f5a6666791e8a4,niksa,Fri Sep 13 10:15:52 2019 +0200,Support additional query plans for AsyncAppend,In some plans DataNodeScan can be buried under Aggregate node. We now optimize for that as well. We attempy async optimization only when distributed hypertable is participating in the query. f20b1b219a482841a3a8c9c794d8c9ee68eb533c,niksa,Fri Sep 13 10:13:21 2019 +0200,Add GUC for AsyncAppend,GUC timescaledb.enable_async_append enables users to turn on/off async append optimization. It is on by default. 0d71f952f8531b6b6fa81821603c6d2ec48747da,Mats Kindahl,Tue Sep 10 08:15:39 2019 +0200,Add bootstrap option to add_data_node,"When the access node executes `add_data_node`, bootstrapping the data node is done by: 1. Optionally creating the database on the remote server. 2. Creating a schema for the TimescaleDB extension objects. 3. Creating the TimescaleDB extension in the database. After bootstrapping, the `dist_uuid` of the data node and access node is set to the `uuid` of the access node. If `bootstrap` is `true`, bootstrapping of the data node is done. If `boostrap` is `false`, bootstrapping is not done, but the procedure attempts to connect to the database and verify that the TimescaleDB extension is loaded and that the `dist_uuid` is clear. If it is not possible to connect to the database, or if `dist_uuid` is set, `add_data_node` will fail. " 7f3bc09eb6d4049a47e205bcb709dce56cd38b54,Erik Nordström,Wed Sep 11 09:55:14 2019 +0200,Generalize deparsing of remote function calls,"Certain functions invoked on an access node need to be passed on to data nodes to ensure any mutations happen also on those nodes. Examples of such functions are `drop_chunks`, `add_dimension`, `set_chunk_time_interval`, etc. So far, the approach has been to deparse these ""manually"" on a case-by-case basis. This change implements a generalized deparsing function that deparses the function based on the function call info (`FunctionCallInfo`) that holds the information about any invoked function that can be used to deparse the function call. The `drop_chunks` function has been updated to use this generalized deparsing functionality when it is invoking remote nodes. " 55d205b09ba37fdfa8aa7419837872b6c25346a5,Dmitry Simonenko,Wed Sep 11 14:22:01 2019 +0300,Check timescaledb extension version on a data node,Compare remote connection extension version with the one installed on the access node. Show a warning message if it differs. Check happens during add_data_node() call and after creating new data node connection. 0c0e6b10701d5dfc6a719dbb751583524f000518,niksa,Thu Sep 5 11:26:56 2019 +0200,Use AsyncAppend with min/max & window func,This change makes sure AsyncAppend is applied to the execution of plans that include min/max and window functions. 8145d75c3f068771a1ca2db22c003a7d10c4109b,Mats Kindahl,Wed Sep 4 15:25:46 2019 +0200,Remove bootstrap_user from add_data_node,"This commit changes so that the same user is used both on the access node and the data nodes when executing a `add_data_node`, which means that the `bootstrap_user` parameter is removed. Since most tests assume that you can pass a separate user with superuser privileges to `add_data_node`, this affected a lot of tests. " c233330478cdb773292e31c76ed01291a0787fa0,niksa,Tue Sep 3 15:35:23 2019 +0200,Fix cursor pending requests,In some cases a cursor might have pending data fetch requests while we attempt to close it.We need to make sure to throw away any pending cursor requests before proceeding with sending CLOSE request or cursor rewind. This commit also prevents `ANALYZE disttable` from failing (due to fetching random tuples and not using next_tuple marker) 0998ecf3280cab401b965d80a06b94780c310cb5,Ruslan Fomkin,Mon Sep 2 10:55:36 2019 +0200,Fix transparent formatting int64 in deparser,"int64 is defined differently depending on environment variables. Implementation of deparse, which is used to generate commands to create hypertable on data nodes, has assumed only one possible definition. This is fix to use correct formatting. " 7fc4c869e222aa75f04c3daa7f243927c62dbe1b,niksa,Wed Aug 7 14:20:50 2019 +0200,Add AsyncAppend to asynchronously scan data nodes,"The general idea with AsyncAppend is to do more things in parallel when executing a query on a distributed hypertable. With AsyncAppend, we use TimescaleDB's Async API to asynchronously create cursors and fetch data from data nodes. We modify existing planner code to inject an AsyncAppend node which will take care of sending/processing async requests. An AsyncAppend node is being injected as a parent of Append or MergeAppend and uses their child nodes (DataNodeScan) to perform async calls. " 6e9f6447140b77a85a9b9231be50c1bfa2d6ebe2,Mats Kindahl,Fri Aug 30 11:22:35 2019 +0200,Require host parameter in add_data_node,"Change `add_data_node` so that host parameter is required. If the host parameter is not provided, or is `NULL`, an error will be printed. Also change logic for how the default value for `port` is picked. Now it will by default use the port given in the configuration file. The commit update all the result files, add the `host` parameter to all calls of `add_data_node` and add a few tests to check that an error is given when `host` is not provided. " abd5a9a93939d42c95ff0a157373c2d869bd32c1,Ruslan Fomkin,Mon Sep 2 11:58:21 2019 +0200,Replace assert with error report,"Replaces assert of expected result after completing COPY command with reporting error on unexpected state, since asserts are removed on release builds. " 9438c1c0755f578c0f667eb56e3d149806e07cc4,Ruslan Fomkin,Tue Aug 27 14:19:30 2019 +0200,Verify that connections are to a TimescaleDB node,Checks that a data node connection is created with the TimescaleDB foreign data wrapper. Refactors existing code to remove duplicated code related to TimescaleDB foreign data wrapper. 85ed39b65dde7928cf89128bf91ae3528bab02e3,Erik Nordström,Wed Jul 24 13:25:26 2019 +0200,Block creating and dropping servers,"Creating and dropping server objects with SQL DDL statements are now blocked. The purpose is to avoid confusion with our custom data node APIs and ensure integrity. This restriction only applies to TimescaleDB-related server objects. Some tests use `DROP DATABASE` to simulate a failed data node. This failed node could be cleaned up on the access node with `DROP SERVER`. However, our corresponding `delete_data_node()` fails if it cannot contact the ""remote"" database (which was dropped in tests). Therefore, this PR also makes sure that `delete_data_node()` has a `force` option that allows the local data node object to be removed despite a failure of the remote node (or database). This is, however, only allowed if data integrity concerns allow it. This change also brings back a previously removed connection cache test that used the now blocked APIs to invalidate cache objects. The tests instead uses our custom APIs and/or test-specific hacks to trigger cache invalidation. " bcb8352be2bfc23e21800506499fc6cafd4ee22b,Dmitry Simonenko,Fri Aug 30 14:07:58 2019 +0300,Unset libpq environment variables,"By default libpq uses environment variables as a fallback way to specify connection options, potentially they could be in a conflict with internal implementation and introduce security risks. Introduce new option `timescaledb.passfile` which specifies the name of the file used to store passwords used for a data node connection. It is intended to be used instead of PGPASSFILE variable, which is no longer accessible. " 33923548c78cc7b83e6c03903ef3410685d50275,Mats Kindahl,Thu Aug 29 14:44:03 2019 +0200,Remove cascade option from delete_data_node,"The `cascade` option was added earlier since it was necessary to allow cascading the delete of user mappings when removing the server objects. Since the user mappings are removed from the code, the `cascade` option is not needed any more. This commit remove the option and fix all the tests. " eb72293fdad9950a8501ec99359a57467493330d,Erik Nordström,Mon Aug 19 23:43:34 2019 +0200,Avoid throwing errors in connection library,"This change refactors the connection library to avoid throwing errors and instead returning, e.g., an error status or a failed PGresult object. Avoiding errors is sometimes desirable to avoid failing the transaction, for instance when doing liveness checks (to return a remote node status) or when trying to delete a node (in which case the delete might need to succeed locally despite an unresponsive remote node). Although exceptions can be caught for cleanup purposes in code using connections, it is not safe to proceed with the transaction when errors are thrown. If in doubt, read this pgsql-hackers list post: https://www.postgresql.org/message-id/27190.1508727890" "40sss.pgh.pa.us This change also cleans up some of the connection library APIs, for instance, string formatting can now be handled by query functions, obviating the need to first construct a query string. Also, the number of requests sent to configure connection options has been reduced. Finally, the connection API has been decoupled from the async API to avoid a circular dependency. ",,,, 696a95ded886b8f84f232f336fed6b6ae6dc2fa5,Mats Kindahl,Tue Aug 27 14:23:14 2019 +0200,No error on low max_prepared_transactions,"If `max_prepared_transactions` is set below `max_connections` it is not possible to create a data node because an error is generated from the `validate_as_data_node()` function when executed on the data node. Since it might be necessary to set `max_prepared_transactions` below this, we change this to only print a warning. " 77776faf201a474e75f9713f9df21623725e803b,Mats Kindahl,Mon Aug 26 16:38:50 2019 +0200,Fix port usage for add_data_node(),"For a statement which only specify the database, we expect the data node to be created on the same Postgres instance as the one where the statement is executed. SELECT * FROM add_data_node('data1', database => 'base1'); However, if the port for the server is changed in the configuration file to not use the default port, the command will try to connect to the wrong Postgres server, namly the one listening on port 5432. This commit fixes this by letting `host` and `port` parameters be NULL by default and use the following logic to decide what port should be used. - If a port is explicitly provided, use that. - If a port is not provided but a host is provided, it is assumed that the intention is to connect to a default-installed Postgres server on a different address, so use the default Postgres port (5432). - If neither port nor host is provided, it assumed that the intention is to connect to the same server as where the command is executed, so use the port that was written in the configuration file. The default host to use is still 'localhost', but it is not written explicitly in the function definition in `ddl_api.sql`. The commit also fixes one warning where an uninitialized variable could be used. " e8a43adc260a33d74d17d74d1467b818a393ec78,Dmitry Simonenko,Tue Aug 27 14:59:41 2019 +0300,Block ALTER SERVER command,"This patch prevents changing a timescaledb foreign server settings directly using ALTER SERVER command. Original idea of it is to prevent from adding SSL and connection options, which will be in conflict with internal implementation. " 3cf2e2e4a3b55716557e00bc6f71516b8ea7ec8a,Ruslan Fomkin,Mon Aug 26 10:44:58 2019 +0200,Verify distributed hypertable when attaching node,Add a check that the given hypertable to attach node is distributed. c8563b2d4663ef924e548709965daf5ee788afb3,Dmitry Simonenko,Thu Aug 22 15:30:08 2019 +0300,Add distributed_exec() function,"This function allows users to execute a SQL query on a list of data nodes. The purpose is to provide users a way to, e.g., create roles on data nodes. The current implementation is quite straightforward. Just execute any provided query on a list of data nodes. The query will execute with the current user role. The function does not return or print any result values. In case of error, it will print the data node name and a related error message. " 334604edc73bdd23d72e5dc62ac38115ad63a912,niksa,Wed Aug 21 11:38:01 2019 +0200,Fix race condition for distributed chunk creation,There is a race condition that happens when access node and data node are both trying to create a chunk on a data node. 5e386518a4c9bc8c6cfa4f1430e2f14409c39d06,Erik Nordström,Thu Aug 15 18:51:56 2019 +0200,Tie libpq object lifecycles to transactions,"Objects created by libpq, such as connections (`PGconn`) and results (`PGresult`), live outside PostgreSQL's memory management system (memory contexts) and therefore require manual lifecycle management (explicit freeing). However, not only does this lead to more complicated code, but it also increases the risk of significant memory leaks when scanning large amounts of remote data. To avoid such leaks, this change ties libpq connections and results to PostgreSQL transactions using a combination of transaction event hooks and libpq event hooks. This allows both manual lifecycle management and auto-release of these resources at transaction end. It should be noted that manual lifecycle management is still recommended in case of, e.g., generating large amounts of result objects in a tight loop within one transaction (for instance, when scanning a remote table). Without such manual release of resources, memory usage can still build up quickly within the transaction. In summary, connections and results are released on `PQfinish()` and `PQclear()`, respectively, or at the end of the transaction or sub-transaction that created them. Auto-release can be disabled for connections, however, which is a requirement for cached connections that live beyond a single transaction. However, the result objects created on cached connections are still auto-released at transaction end. " f7a39c6debcedf4312c616c14268620d308a99fc,Dmitry Simonenko,Fri Aug 16 14:03:28 2019 +0300,Support SSL with Certificate Based Authentication,Add a tunable way to support SSL connections to data nodes with optional certificate based authentication. Switch test suite to use pre-generated certificates. 6ec7e300d45b4930b2b3d7dc68e8230c38cce214,Mats Kindahl,Mon Aug 12 15:06:02 2019 +0200,Some minor fixes to add_data_node(),This commit updates a few error messages and adds a few tests to check that add_data_node() behaves as intended. a50db32c1801fa0f7693d1ab12db1c6c9078f183,Brian Rowe,Thu Aug 8 16:22:49 2019 -0700,Check data node for valid postgres version,This change will check if the postgres version of the data node is 11 or greater during the add_data_node call. It will also now print a more meaningful error message if the data node validation fails. d97c32d0c7a3c3ff3e003e51c6b3af327e2aa079,niksa,Tue Aug 6 12:08:12 2019 +0200,Support distributed drop_chunks,Running `drop_chunks` on a distributed hypertable should remove chunks from all its data nodes. To make it work we send the same SQL command to all involved data nodes. 92459e638ea875a163972f6dd1d462295a59fa8c,Ruslan Fomkin,Wed Aug 7 14:46:05 2019 +0200,Fix naming bug in create_distributed_hypertable,Fix `create_distributed_hypertable` to support single quote in schema name 3833a5bb6fae4003dad5c8b04d1f874ad2529d81,Mats Kindahl,Tue Aug 6 11:38:08 2019 +0200,Don't fail add_data_node() when database exists,"When the remote database exists `add_data_node()` currently fails unless `if_not_exists` is true. Typically, `IF NOT EXISTS` mean that an object creation should not generate an error. Regardless of the `IF NOT EXISTS` the object is still created. The current behavior is not in line with what is expected so this commit fixes this by not aborting the command if the remote database exists and only print a notice that this step is skipped. If the database exists, it is checked that the user have `CREATE` privileges on the database before proceeding. " 71991ab8861685913f68b894f70729b69d42585b,Erik Nordström,Tue Jul 23 17:42:45 2019 +0200,Block chunk creation by insert on data nodes,"It is unsafe to insert data directly on data nodes if it would create a new chunk, since chunk creation must be coordinated by the access node. This change blocks such chunk creation but still allows inserting tuples on data nodes that do not create new chunks and otherwise adhere to existing constraints. " 79f62236313ac62dfe78ba693a9f1ce4df31d56e,Dmitry Simonenko,Mon Jul 29 16:03:43 2019 +0300,Replace UserMappings with a connection ID,"This change replace UserMappings with newly introduced TSConnectionId object, which represent a pair of foreign server id and local user id. Authentication has been moved to non-password based, since original UserMappings were used to store a data node user passwords as well. This is a temporary step, until introduction of certificate based authentication. List of changes: * add_data_node() password and bootstrap_password arguments removed * introduced authentication using pgpass file * RemoteTxn format string which represents tx changed to tx-version-xid-server_id-user_id * data_node_dispatch, remote transaction cache, connection cache hash tables keys switched to TSConnectionId instead of user mappings * remote_connection_open() been rework to exclude user options * Tests upgraded, user mappings and passwords usage has been excluded " 31953f0dc65208bcb5af04ebe475bf6f95170b64,Brian Rowe,Tue Jul 23 16:33:22 2019 -0700,Verify configuration before adding data node,"This change will call a function on a remote database to validate its configuration before following through with an add_data_node call. Right now the check will ensure that the data is able to use prepared transactions, but further checks can be easily added in the future. Since this uses the timescaledb extension to validate the remote database, it runs at the end of bootstrapping. We may want to consider adding code to undo our bootstrapping changes if this check fails. " f18f3d640f9ed9feaa4b766918eb31a08e878d70,Erik Nordström,Wed Jul 24 15:50:00 2019 +0200,Support TRUNCATE on distributed hypertables,"Distributed hypertables can now be truncated using the regular TRUNCATE command. To support TRUNCATE, the handling of this command has been refactored into a pre-process and post-process step. In the pre-process step, distributed hypertables are excluded from regular TRUNCATE processing on the access node to avoid having PostgreSQL generate an error when trying to truncate chunks that are foreign tables. Local foreign table chunks and metadata are still cleaned up though. " 2bbe6677e1eac0684410f0d3fc9bf99f9a4175ad,Erik Nordström,Tue Jul 23 18:10:02 2019 +0200,Block creation of stand-alone foreign tables,"There's no reason for us to allow creating stand-alone foreign tables using the ""timescaledb_fdw"" foreign data wrapper. All such foreign tables should be chunks and created automatically by the database system. This change therefore blocks any foreign table creation using timescaledb_fdw servers. The `timescaledb_fdw` test has been removed since this made use of the now blocked functionality. " 3d3824dbc1c3a9164d0352d116b97a51dc1c6b39,Brian Rowe,Tue Jul 16 11:29:23 2019 -0700,Fix some issues with num_dist_tables,This change fixes a couple issues with the num_dist_tables column of the timescaledb_information.data_node view. The first fix will allow the column to correctly report 0 when no tables are yet created (it currently will count a NULL table as 1 in this case). The second fix addresses a bug in the dist_util_remote_hypertable_info function which was causing the code to only see the first hypertable returned. This second bug will also cause incorrect results for many of our usage reporting views and utilities when there are multiple distributed hypertables. 76237a44134d957d457f51586387f275461cea4a,Erik Nordström,Fri Jul 12 12:22:28 2019 +0200,Fix error handling in connection library,This fixes an where the connection library tried to access an error message in a libpq connection handle even though the handle was NULL. 05126dd0a2ada0c1cf512f78a1a3c32cf66402fc,Erik Nordström,Wed Jul 10 10:28:55 2019 +0200,Avoid some bucketing push downs,"This change avoids pushing down some bucketing expressions in certain unsafe circumstances. Bucketing push down is safe when chunks do not overlap in the partitioning dimension used to assign chunks to data nodes (typically the first space dimension). However, when chunk slices overlap in that dimension, bucketing on time is not safe. This change avoids such unsafe push downs in one case, also removing some code necessary to push down bucketing expression that is no longer needed. A new query test `dist_query` is added which includes one query that previously generated different results between a repartitioned and non-repartitioned table, both holding the same data. The `dist_query` test may also, in the future, serve to replace the unwieldy `partitionwise_distributed` test. " ac3f0bcb92b94efa2e2e00fb550e92c1994417c9,Mats Kindahl,Thu Jul 11 11:15:24 2019 +0200,Change order of parameters in attach_data_node,All data node functions except `attach_data_node` take the node name as the first parameter. This commit changes the order of the two first parameters to `attach_data_node` so that the node name is the first parameter and the hypertable is the second parameter. 8259225a020a0d6cb87f10bbab9e8bd2efbe7783,Erik Nordström,Thu Jul 4 10:30:56 2019 +0200,Add ordered paths for upper rels,"Upper relations, e.g., those relations the planner adds for GROUP BY aggregates, now have ordered paths (with pathkeys) added to their pathlist. Without computing sorted paths for upper relations, sort push down to happen when there are aggregates. The costing of sorting has also been tweaked (lowered) since the planner otherwise won't pick pushed-down sorts in many cases. Pushing down sorts to data nodes should be better in almost all cases compared to sorting on the access node. However, the low cost can also lead to sort paths being picked in cases where sorting is not necessary, like sorting before the input to a hash aggregate. This points to a larger issue with the cost estimation that will have to be addressed separately. To avoid such push-downs, we only add sort paths for upper relations if there's an ORDER BY clasue. (Adding sort paths without ORDER BY might be useful for merge joins, group aggregates, etc.) Note that pushing down sorts also has an impact on the `dist_partial_agg` test, which now shows a diff output where none is normally expected. This is because the queries included the `last` and `first` functions, which can give non-deterministic output if there are multiple values with the same timestamp (i.e., the output order depends on the input order). This change fixes this issue by adding any pathkey paths for upper rels, similar to how this is done for regular base rels. " 7b64bf20f5ef246abf2040c134b0b402e5ed412c,Erik Nordström,Tue Jul 9 21:21:21 2019 +0200,Use num data nodes as default num partitions,"For convenience, this adds the option to create a distributed hypertable without specifying the number of partitions in the space dimension even in the case when no data nodes are specified (defaulting to the data nodes added to the database). " 5309cd6c5fe5fb8b1e2786c8a5848f2484546310,Erik Nordström,Sat Jul 6 15:27:45 2019 +0200,Repartition hypertables when attaching data node,"Distributed hypertables are now repartitioned when attaching new data nodes and the current number of partition (slices) in the first closed (space) dimension is less than the number of data nodes. Increasing the number of partitions is necessary to make use of a newly attached data node. However, repartitioning is optional and can be avoided via a boolean parameter in `attach_server()`. In addition to the above repartitioning, this change also adds informational messages to `create_hypertable` and `set_number_partitions` to raise awareness of situations when the number of partitions in the space dimensions is lower than the number of attached data nodes. " 9108ddad15dee9889f12d72a1fc9927cef375c9d,Erik Nordström,Sun Jul 7 12:01:29 2019 +0200,Fix corner cases when detaching data nodes,"This change fixes the following: * Refactor the code for setting the default data node for a chunk. The `set_chunk_default_data_node()` API function now takes a `regclass`/`oid` instead of separate schema + table names and returns `true` when a new data node is set and `false` if called with a data node that is already the default. Like before, exceptions are thrown on errors. It also does proper permissions checks. The related code has been moved from `data_node.c` to `chunk.c` since this is an operation on a chunk, and the code now also lives in the `tsl` directory since this is non-trivial logic that should fall under the TSL license. * When setting the default data node on a chunk (failing over to another data node), it is now verified that the new data node actually has a replica of the chunk and that the corresponding foreign server belongs to the ""right"" foreign data wrapper. * Error messages and permissions handling have been tweaked. " b07461ec0093108930bc955ff526230fc2e1ad8f,Erik Nordström,Sun Jul 7 11:26:15 2019 +0200,Refactor and harden data node management,"This change refactors and hardens parts of data node management functionality. * A number of of permissions checks have been added to data node management functions. This includes checking that the user has proper permissions for both table and server objects. * Permissions checks are now done when creating remote chunks on data nodes. * The add_data_node() API function has been simplified and now returns more intuitive status about created objects (foreign server, database, extension). It is no longer necessary to specify a user to connect with as this is always assumed to be the current user. The bootstrap user can still be specified explicitly, however, as that user might require elevated permissions on the remote node to bootstrap. * Functions that capture exceptions without re-throwing, such as `ping_data_node()` and `get_user_mapping()`, have been refactored to not do this as the transaction state and memory contexts are not in states where it is safe to proceed as normal. * Data node management functions now consistently check that any foreign servers operated on are actually TimescaleDB server objects. * Tests now run with a superuser a regular user specific to clustering. These users have password auth enabled in `pg_hba.conf`, which is required by the connection library when connecting as a non-superuser. Tests have been refactored to bootstrap data nodes using these user roles. " 29f8efc2fb264e5e42a7e3e1aad12e2223eb9319,Mats Kindahl,Sun Jul 7 08:53:11 2019 +0200,Set timezone if changed since last command,"If the timezone changed since the last command was issued, send a new SET TIMEZONE command over the remote connection before executing the actual command. " 90bf98f4ebae467bbb4ca89d9b387ab5e9900e4c,Mats Kindahl,Wed Jul 3 16:59:40 2019 +0200,Recreate cached remote connections when broken,"When a new connection is fetched from the cache, it can potentially have broken since last use, so we extend the connection cache with a check function that can be used to check if the cache entry is valid whenever a connection is fetched from the cache. If the cache entry is not valid, it will be re-created instead of updated. For the connection cache, a connection is invalid if it is still in processing state. In that case, a previous use was aborted early and the connection need to be re-created. " bfefa532f63636e7f34abb0ab2ec99082258e366,Brian Rowe,Mon Jul 29 06:24:24 2019 -0700,Rename ServerScan to DataNodeScan,This change modifies the data_node_scan_plan to appropriately use the name DataNodeScan in place of ServerScan. 79fb46456f9ee583225a8ab7399aef7b5130d6d5,Brian Rowe,Tue Jun 25 13:01:26 2019 -0700,Rename server to data node,"The timescale clustering code so far has been written referring to the remote databases as 'servers'. This terminology is a bit overloaded, and in particular we don't enforce any network topology limitations that the term 'server' would suggest. In light of this we've decided to change to use the term 'node' when referring to the different databases in a distributed database. Specifically we refer to the frontend as an 'access node' and to the backends as 'data nodes', though we may omit the access or data qualifier where it's unambiguous. As the vast bulk of the code so far has been written for the case where there was a single access node, almost all instances of 'server' were references to data nodes. This change has updated the code to rename those instances. " dd3847a7e065dcc46066193bbc22f8934c2669c0,Brian Rowe,Mon Jul 1 09:48:29 2019 -0700,Rename files in preparation for large refactor,This change includes the only rename changes required by the renaming of server to data node across the clustering codebase. This change is being committed separately from the bulk of the rename changes to prevent git from losing the file history of renamed files (merging the rename with extensive code modifications resulted in git treating some of the file moves as a file delete and new file creation). c0daf7b1d094bd35c0b1ebf30b92c105306fe6cf,Erik Nordström,Fri Jul 5 14:55:46 2019 +0200,Refactor estimation code for remote queries,"This change refactors the estimation code for calculating the cost of scanning remote relations. Estimation is now broken up based on the type of relation one is estimating the cost of (e.g., base or upper relations). It also adds better handling of upper relations that include pathkeys (i.e., it is possible to generate sorted paths for them). " de6df5c67d9138458a96a99b90660ca401eaa2cf,Erik Nordström,Mon Jul 1 22:45:51 2019 +0200,Add ANALYZE support for distributed hypertables,"ANALYZE is now supported on distributed hypertables via the FDW interface. The functionality is, for the most part, based on the corresponding `postgres_fdw` code, but refactored for our purposes. When ANALYZE is run, it proceeds recursively to every (foreign table) chunk in the distributed hypertable, which means the operation is run seperately on each chunk. This might not be the most efficient way to fetch statistics from remote data nodes as there is one request sent per chunk and all its data is fetched. Future work to optimize ANALYZE should explore using a TABLESAMPLE query to compute the sample at the remote node (it is unclear why the `postgres_fdw` doesn't already do this), or implement a specific function to get a sample. We might also consider ways to improve how we sample the distributed hypertable as a whole to get en even sample across the entire table and not on a per chunk basis. This might be important if chunk sizes vary, e.g., due to changes in the chunk time interval. In such cases it seems wrong to, e.g., get the same number of samples from a chunk that covers only a day vs. a chunk that covers a week. " fe7f7ef2ef9c49c0d799b4e6664dcd96a63f8ad1,Erik Nordström,Mon Jul 1 17:51:09 2019 +0200,Create cursor and tuplefactory modules,"This change moves the cursor and tuple making code into their own modules. This provides better usability, code readability, and isolation for these components. The ""Cursor"" module now provides an object that represent the cursor on the remote end and can be used to produce new tuples. Internally, the Cursor uses a TupleFactory to turn query result data into internal HeapTuple format. The ""TupleFactory"" can also be used standalone from a ""Cursor"" object. This happens, e.g, on the INSERT path. " 5d4a0db85ef9acd8b500f11d14630727150bbd25,Erik Nordström,Fri Jun 28 13:01:40 2019 +0200,Split FDW code into multiple files,"This change makes the FDW-related code more modular by splitting the code into its component parts. The split is done along two main axes: plan/execute and modify/scan. In addition, code and utilites that are common across the code along these axes are also split into their own modules and files. " fd75dec223024dc880f7d9049afd3d034c2ba420,Brian Rowe,Fri Jun 7 11:32:27 2019 -0400,Support pushing partial aggregates to data nodes,"This change modifies the timescale_fdw to allow aggregates on a subset of partitioning dimensions to be pushed down to data node. The deparsing code has also been modified to wrap the pushed aggregate in a _timescaledb_internal.partialize_agg call, which will return the proper array of values which postgres can combine into the finalized value. " 112a7cf8d040b57c760f31d17444fc320653d8eb,Erik Nordström,Fri Jun 7 16:28:14 2019 +0200,"Fully push down more aggs, including bucketing","GROUP BY aggregates are now fully pushed down in more cases. 1. GROUP BY expressions that cover all partitioning dimensions are always fully pushed down. This is safe to do regardless of repartitioning issues. 2. All GROUP BYs on (just) the closed ""space"" dimension will be fully pushed down as long as no slices in that dimension overlap across servers. 3. GROUP BYs that include a bucketing expression on time (e.g., `date_trunc` or `time_bucket`) will be fully pushed down like case (1), as bucketing expressions are now treated as a ""compatible"" partitioning key. 4. GROUP BYs are always pushed down--irrespective of partitioning--if only one server is involved in the query. Special handling has been implemented for pushing down bucketing functions like `date_trunc`. The first parameter of the `date_trunc` functions are collatable, which normally prohibits push down. This has been handled specifically for bucketing functions until a more cohesive handling of collations across servers has been implemented. Further, the `date_trunc(text, timestamptz)` function is marked STABLE, due to the timezone-dependent second parameter, which also prohibits push-down. This has been handled by implementing our own `contain_mutable_functions` check that filters these functions, thus allowing push down. Note, to make this safe, we have to guarantee that the connection session uses the same time zone setting as the access node. NOTE, however, that we currently do not handle changes to time zone within a session. A couple of costing issues for server rels have also been fixed. " ab38460a35a728b3a5903a6c6290383be3ce5c7a,Brian Rowe,Fri Jun 21 15:25:26 2019 -0700,Update ordinal calculation to handle cut chunks,The ts_dimension_get_closed_slice_ordinal function was implemented to use the size of the target slice to estimate its ordinal. This could lead to some very odd values if the hypercube containing the slice had been cut due to having to fit in with other cubes around a repartioning event. The new approach works by determining which slice in the current partition configuration most overlaps the target slice and returns the ordinal for that slice. d5b4a48cdaf5d4e61e68ad960885635f8b3daed2,Mats Kindahl,Mon Jun 24 11:56:36 2019 +0200,Fix crash when creating distributed hypertable,"If a distributed hypertable is created but one or more of the servers are not defined, a crash will result. The reason for this is that an attempt is made to print an error message using fromctx.cstate in ChunkCopyState. Since the error message is printed out in the last phase of the command, the fromctx union is not set to a cstate and instead to a scandesc, which causes a segmentation fault. This commit fixes the issue by using different ErrorContextCallback for the different phases and pass that to timescaledb_CopyFrom. For the initial CopyFrom phases, the default CopyFromErrorCallback is used, while a newly defined error callback is used for the last phase. " 3638ca3a35393f426333246bbc910879174edfd6,Erik Nordström,Fri Jun 21 00:21:27 2019 +0200,Use a custom ServerScan node to scan servers,"A new custom ServerScan node is now used to scan remote server hypertables. Previously, a foreign scan was used for this purpose, but this proved to have some issues that couldn't handle queries like, e.g.: ``` SELECT device, avg(temp) FROM hyper WHERE time > '2019-06-01 01:00' GROUP BY 1; ``` When scanning a foreign server this failed with an error saying that ""time"" is not in the target list. The reason this happened is that we treated the server scan as an upper relation, while it really is more akin to a base relation. Upper relations invoke code with certain expectations on target lists and restriction clauses (setrefs.c: set_foreignscan_references), leading to the error. The alternative is to treat the per-server scan as a base rel (non-zero `scan_relid`), but this doesn't work with the ForeignScan executor node because it expects any base rels to map to a ""real"" foreign table in the database; thus the foreign scan fails with an error when trying to lookup the foreign table. A CustomScan implementation gives better control for handling query optimzations and custom functionality that goes the beyond the capabilities and expectations of ForeignScans. This also gives us a better setup for future modifications to remote hypertable scans. " 545c830ec0aaf824162944b4f726e3257c5007db,Erik Nordström,Thu Jun 20 16:52:46 2019 +0200,Refactor FDW to support custom scans,"This refactor decouples the code to scan remote tables from the FDW interface. The intention is to allow implementing remote table scans without relying on ForeignScan plans, which have shown to have limitations when doing optimizations that go beyond scanning standard foreign tables. " a99ae05723d23811e2b8ef82a3a6376565303d71,Dmitry Simonenko,Tue Jun 25 14:22:05 2019 +0300,Cleanup PG version checks for multinode,"Since distributed hypertables will only be support on PG11 or greater, ensure that we do not compile multinode-related files on previous versions. Also raise appropriate errors when trying to invoke multinode-related functionality on versions prior to PG11. " e110a42a2b04a0261619a63de3782b6ec802d9b4,Brian Rowe,Fri May 17 14:14:05 2019 -0700,Add space usage utilities to distributed database,"This change adds a new utility function for postgres `server_hypertable_info`. This function will contact a provided node and pull down the space information for all the distributed hypertables on that node. Additionally, a new view `distributed_server_info` has been added to timescaledb_information. This view leverages the new remote_hypertable_data function to display a list of nodes, along with counts of tables, chunks, and total bytes used by distributed data. Finally, this change also adds a `hypertable_server_relation_size` function, which, given the name of a distributed hypertable, will print the space information for that hypertable on each node of the distributed database. " 88a91afc67e97933265490bf40a5326f1c2a151b,Erik Nordström,Fri Jun 14 11:40:42 2019 +0200,Create chunks in non-ordinal order in test,This changes the order of inserted rows in the hypertable_distributed test so that chunks no longer are created across servers in ordinal order. 3943a758a7a7a7f309b07a23235fcde424d44024,Erik Nordström,Tue Jun 11 14:05:58 2019 +0200,Refactor test of distributed hypertables,This refactors the `hypertable_distributed` test to make better use of the `remote_exec` utility function. The refactoring also makes sure we actually use space partitioning when testing distributed hypertables. 9a52a2819fbead504b3aa00ca6239511eaa92940,Erik Nordström,Tue Jun 11 01:16:27 2019 +0200,Make chunk placement consistent across data nodes,"Chunks are placed across data nodes based on the ordinal of the slice in the first space dimension, if such a dimension exists. For instance, if a chunk belongs to the second slice in the space dimension, this ordinal number will be used modulo the number of data nodes to find the data node to place the chunk on. However, the ordinal is calculated based on the existing slices in the dimension, and, because slices are created lazily, the ordinal of a specific slice might vary until all slices are created in the space dimension. This has the result that chunks aren't consistently placed on data nodes based on their space partition, prohibiting some push-down optimizations that rely on consistent partitioning. This change ensures the ordinal of a space slice is calculated as if all slices in the dimension are pre-existing. This might still lead to inconsistencies during times of repartioning, but fixes issues that occur initially when no slices exists. " 0da34e840e6ef7e167e90b6889a91b0acabe788a,niksa,Tue Jun 4 21:17:01 2019 +0200,Fix server detach/delete corner cases,"Prevent server delete if the server contains data, unless user specifies `force => true`. In case the server is the only data replica, we don't allow delete/detach unless table/chunks are dropped. The idea is to have the same semantics for delete as for detach since delete actually calls detach We also try to update pg_foreign_table when we delete server if there is another server containing the same chunk. An internal function is added to enable updating foreign table server which might be useful in some cases since foreign table server is considered a default server for that particular chunk. Since this command needs to work even if the server we're trying to remove is non responsive, we're not removing any data on the remote data node. " b2fde83d2e580ae4c38faf6176510c30d758cd6b,Dmitry Simonenko,Fri Jun 7 16:51:20 2019 +0300,Block DDL operations on a data nodes,"This change adds support for blocking DDL operations on data nodes when not executed via the access node. Specifically, DDL operations can only be executed on data nodes if the request came on a connection from the access node or DDL operations have been explicitly allowed by setting `timescaledb.enable_client_ddl_on_data_servers=true`. " 2fd99c6f4b09e831fee24c25e69141bd854a1ed1,niksa,Thu May 23 14:54:21 2019 +0200,Block new chunks on data nodes,"This functionality enables users to block or allow creation of new chunks on a data node for one or more hypertables. Use cases for this include the ability to block new chunks when a data node is running low on disk space or to affect chunk distribution across data nodes. Sometimes blocking data nodes for new chunks can make a hypertable under-replicated. For that case an additional argument `force => true` can be supplied to force blocking new chunks. Here are some examples. Block for a specific hypertable: `SELECT * FROM block_new_chunks_on_server('server_1', 'disttable');` Block for all hypertables on the server: `SELECT * FROM block_new_chunks_on_server('server_1', force =>true);` Unblock: `SELECT * FROM allow_new_chunks_on_server('server_1', true);` This change adds the `force` argument to `detach_server` as well. If detaching or blocking new chunks will make a hypertable under-replicated then `force => true` needs to used. " 7ca992f48d7ff69836bd4853a2c5f46f3f88e1c3,Dmitry Simonenko,Thu May 23 16:31:34 2019 +0300,Improve remote connection error message,This patch adds server name mentioning in a remote error messages d8d13d9475a62497d9184713b40dac6ccba44990,niksa,Mon Apr 29 18:09:20 2019 +0200,Allow detaching servers from hypertables,"A server can now be detached from one or more distributed hypertables so that it no longer in use. We only allow detaching a server if there is no data on the server and detaching it doesn't risk making a hypertable under-replicated. A user can detach a server for a specific hypertable, or for all hypertables to which the server is attached. `SELECT * FROM detach_server('server1', 'my_hypertable');` `SELECT * FROM detach_server('server2');` " f6a829669ab07362a84ee4f5d934c0a394bd8132,Dmitry Simonenko,Mon May 20 16:18:59 2019 +0300,Distinguish data node hypertables from regular ones,"Hypertables created on a data node by an access node (via `create_distributed_hypertable()`) will now have their `replication_factor` set to -1. This makes it possible to distinguish regular data node hypertables from those that are part of a larger distributed hypertable. This functionality will be needed for decision making based on the connection type, for example allow or block a DDL commands on a data node. " 2f43408eb566057a8491a65d6e034ef54eceb633,Erik Nordström,Fri Apr 12 16:23:35 2019 +0200,Push down partitionwise aggregates to servers,"This change adds support for pushing down FULL partitionwise aggregates to remote servers. Partial partitionwise aggregates cannot yet be pushed down since that requires a way to tell the remote server to compute a specific partial. NOTE: Push-down aggregates are a PG11 only feature as it builds on top of partitionwise aggregate push-down only available in PG11. Therefore, a number of query-running tests now only run on PG11, since these have different output on PG10. To make push downs work on a per-server basis, hypertables are now first expended into chunk append plans. This is useful to let the planner do chunk exclusion and cost estimation of individual chunks. The append path is then converted into a per-server plan by grouping chunks by servers, with reduced cost because there is only one startup cost per server instead of per chunk. Future optimizations might consider avoiding the original per-chunk plan computation, in order to increase planning spead. To make use of existing PostgreSQL planning code for partitionwise aggregates, we need to create range table entries for the server relations even though these aren't ""real"" tables in the system. This is because the planner code expects those entries to be present for any ""partitions"" it is planning aggregates on (note that in ""declarative partitioning"" all partitions are system tables). For this purpose, we create range table entries for each server that points to the root hypertable relation. This is in a sense ""correct"" since each per-server relation is an identical (albeit partial) hypertable on the remote server. The upside of pointing the server rel's range table entry to the root hypertable is that the planner can make use of the indexes on the hypertable for planning purposes. This leads to more efficient remote queries when, e.g., ordering is important (i.e., we get push down sorts for free). " e517d1d1a990188c3a66704d53c43b6fe7ab7695,Erik Nordström,Tue Apr 9 07:33:45 2019 +0200,Add code related to partitionwise aggregate push-down,This adds code related to partitionwise push-down from the postgres_fdw. c5adf7e4e0a8def548aa70187b99398c602ae125,Erik Nordström,Thu May 23 12:12:28 2019 +0200,Make TSConnection malloced like PGConn,"Since our remote connection object `TSConnection` wraps `PGconn`, we should make sure their memory lifecycles are managed in the same way. However, `PGconn` is malloced while `TSConnection` is currently palloced, which means the latter is managed via PostgreSQLs memory context system. This discrepancy means that `TsConnection` could be auto freed by deleting a memory context while PGconn would not. This change makes sure the memory handling is the same for the two objects. " 86858e36e9fb6243d52191bcedc6be05ee2ab734,niksa,Wed Apr 10 12:03:15 2019 +0200,Support multiple async requests per connection,The idea here is to allow multiple async requests to be created for the same connection. Since connection can process only one request at the time only that means that one request can be running and the rest needs to be deferred. The deferred async request will run on get response if the connection is not in use by running async request. This support should pave the way for async creation of cursors. 96727fa5c4e2d0e084303d7a839156c5147b8323,Dmitry Simonenko,Mon May 13 15:52:26 2019 +0300,Add support for distributed peer ID,"This change makes it possible for a data node to distinguish between regular client connections and distributed database connections (from the access node). This functionality will be needed for decision making based on the connection type, for example allow or block a DDL commands on a data node. " 59e3d7f1bd71923f6faf0b764296c5292cdbb07d,Brian Rowe,Fri May 3 16:31:06 2019 -0700,Add create_distributed_hypertable command,This change adds a variant of the create_hypertable command that will ensure the created table is distributed. 6f3848e744f4f58c228cb3484fe3c2a2aae5661b,niksa,Wed May 1 12:55:04 2019 +0200,Add function to check server liveness,"Try connecting to a server and running `SELECT 1`. It returns true if succeed. If fails false is returned. There can be many reasons to fail: no valid UserMapping, server is down or failed running `SELECT 1`. More information about failure is written to server log. `timescaledb_information.server` view is updated to show server status. " 5c643e0ac4ce88b686ed7324c6bf816c8b8cf567,Brian Rowe,Thu Apr 25 23:56:09 2019 -0700,Add distributed group id and enforce topology,"This change adds a distributed database id to the installation data for a database. It also provides a number of utilities that can be used for getting/setting/clearing this value or using it to determing if a database is a frontend, backend, or not a member of distributed database. This change also includes modifications to the add_server and delete_server functions to check the distributed id to ensure the operation is allowed, and then update or clear it appropriately. After this changes it will no longer be possible to add a database as a backend to multiple frontend databases, nor will it be possible to add a frontend database as a backend to any other database. " 7aa4ab4ff0601e20416fa4a86c22f4b0a472bc91,Dmitry Simonenko,Mon Apr 29 12:13:24 2019 +0300,Add test.remote_exec() function,This function is meant to be used in tsl tests to execute same query on a set of remote servers 8068ad1a5520a1c93f6099726c28d1dac4cd2eaa,Erik Nordström,Mon May 4 13:13:16 2020 +0200,Rename distributed DDL test,The distributed DDL test is renamed to conform with other tests that have the `dist_` prefix. 11aab55094343e30dd8532870e7540c30aa76bf0,Dmitry Simonenko,Thu Apr 18 14:35:48 2019 +0300,Add support for basic distributed DDL,This is straightforward implementation which allows to execute limited set of DDL commands on distributed hypertable. 24e024c6e211612b8383ff84e186b395ee94acc8,niksa,Wed Apr 10 12:09:22 2019 +0200,Refactor getting cursor number,Connection parameter is not used when getting cursor number 0052d03e978061c246b8d512f1ae41aeac5fe288,niksa,Tue Apr 9 13:13:14 2019 +0200,Fix ctid parameter encoding,We've used binary format to encode ctid value which is wrong when you force text mode to talk to data nodes. bd55e7cdb714d11a989e06cb5a4ac433602fe56c,Erik Nordström,Mon May 4 12:07:00 2020 +0200,Rename files for distributed copy,"The source files that implement distributed COPY has been renamed to `dist_copy.{c,h}` so that it uses the same prefix as other files. " 8c2258aae30511e981b2c5d0c07a2cd0604d100b,Brian Rowe,Mon Apr 1 09:55:27 2019 -0700,Add support for binary transfer of copy data,This change adds the ability for the frontend to forward data to backend nodes using the postgres binary protocol when handling a distributed copy operation. This option can be disabled via the enable_connection_binary_data GUC setting. 106a5a4bc5907ed1a62bf98dec8fe5c78fb245a4,Brian Rowe,Fri Mar 8 16:36:49 2019 -0800,Implement support for remote copy operations,With this change a COPY operation that comes into a timescale frontend for a distributed hypertable will parse the incoming rows and pass them to the backends hosting the chunks into which the data will be written. This will require that the incoming COPY operation is in text or csv format (no support for binary yet). 0cc942c878bc8747c38950646446fd3b83ca5efa,niksa,Wed Mar 20 17:05:48 2019 +0100,Add support for binary response,"We request binary responses from data nodes only if all requested columns have binary IN function, otherwise we fall back to text format. We use binary only when fetching query results or when getting data from RETURNING statements. Maybe in the future we'd like to change all the communication code to use binary. However, it seems that it would not bring significant performance benefit so it was left out for now. " 6ba70029e3295b0c68e095c30a0732725d728d6e,Erik Nordström,Thu Mar 14 17:26:02 2019 +0100,Cleanup chunk servers when dropping dependencies,"Chunk server mappings are now cleaned up when dropping chunk tables and foreign servers. In particular, this works even when such objects are removed as a result of cascading deletions of other objects. Some refactoring has been done to the event trigger handling code in addition to adding support for new event objects. " 22185a8556ebd202b7630510a2924e57a88982a9,niksa,Thu Mar 14 15:45:29 2019 +0100,Refactor parameter handling for remote inserts,ServerDispatch now uses the `StmtParams` module to convert tuples to text or binary output before sending to remote data nodes. a38f51441161f0427936f07074fe041388394d32,niksa,Thu Mar 7 09:22:21 2019 +0100,Binary format support for statement parameters,This change clearly separates handling of statement parameters. By default parameters are represented in binary format but if not supported it can fallback to text format. A GUC timescaledb.enable_cluster_binary_format can be used to force TEXT. ed1b9d19f15168a3ce70c7d02ab32fc7b1764987,Erik Nordström,Thu Feb 14 23:43:07 2019 +0100,Implement per-server batching for remote INSERTs,"This change ensures inserted tuples are sent in batches to servers instead of tuple-by-tuple. This should reduce a lot of back-and-forth communication that otherwise incur significant overhead. The per-server batch size can be set using the GUC `timescaledb.max_insert_batch_size`, which defaults to 1000. Note that this number is the maximum number of tuples stored per server before they are flushed, and that the original INSERT statement's tuples will be split across these servers. That is, if the INSERT statement has 3000 tuples, and there are three backend datanodes, then they will roughly get 1000 tuples each. The batch size can determine latency by, e.g., spreading the work across a number of smaller batches, as opposed to deferring inserts to one big batch at the end of the transaction. Note that batched tuples are flushed at the end of execution irrespective of whether the flush threshold is reached or not. " 9880bc84e7b96659046713c1b60b280c0b5b8234,Matvey Arye,Wed Feb 20 12:14:11 2019 -0500,Query hypertables on a per-server instead of a per-chunk basis,"This optimization enhances the query when querying a hypertable using the timescale fdw. Previously such queries created execution nodes and queries on a per-chunk basis. This PR combines all the chunks belonging to the same hypertable and server together so that only one query and executor node are created per hypertable-server. This is accomplished by first changing the chunk expansion code to not do the table expansion for remote hypertables and instead simply save the chunk oids in a metadata field in TimescaleDBPrivate. Next, we intercept the set rel pathlist hook to create paths for the hypertable-server nodes. This uses the new server chunk assignment module to choose which chunks will be fetched from which servers. For now, we have only one assignment strategy but later we can have multiple strategies each creating it's own paths and having the planner choose the cheapest path using standard methodologies. Finally, during the plan creation phase we pass down the server chunk assignment to the deparser so that it can add a `chunks_in` call to the where clause. This tells the data node which chunks to use and is necessary when we have replicated chunks so that two servers don't return data for the same chunk. An alternative approach for deparsing `chunks_in` could have been to add a clause to remote_exprs and have the deparser include that in the WHERE clause that way. However, the standard way to deparse whole-row expressions is using the ROW syntax (to protect against different schema definitions on local and remote nodes). However, 'chunks_in' requires a record, not row reference so that approach was rejected as being too awkward. Some additional deparsing changes were made to handle base relations that don't have an associated foreign-table (i.e. the hypertable-server case). This commit also changes the way fdw_private is treated on RelOptInfo. Previously, this could have been NULL, a TimescaleDBPrivate object or a TsFwdRelationInfo. This led to some bugs and awkwardness as well as increased lack of type-safety. Now, this field could only be NULL or TimescaleDBPrivate and the TsFwdRelationInfo object is now an optional and type-safe member of TimescaleDBPrivate. " b1c6172d0a0f2aa9d437338b99ec947cb1f12472,Brian Rowe,Fri Feb 22 16:03:08 2019 -0800,Add attach_server function,This adds an attach_server function which is used to associate a server with an existing hypertable. 0b18f669eedcebe83f7c730269c7b2b0274a1385,Dmitry Simonenko,Fri Feb 22 16:03:45 2019 +0300,Fix log level in tsl/test/src/remote/async.c, d8982c3e15086b977041af6b8a7c504626b7a525,Dmitry Simonenko,Fri Feb 22 13:10:21 2019 +0300,Add add_server() support for remote server bootstrapping,"This patch adds functionality for automatic database and extension creation on remote server. New function arguments: bootstrap_database, bootstrap_user and bootstrap_password. " 0edd8cbd2af88508de450bac3bd62a4831529ea2,Brian Rowe,Fri Feb 15 17:05:14 2019 -0800,Create dimensions during remote hypertable create,"During remote hypertable creation, either during initial hypertable creation or when assigning a new backend to a hypertable, this will ensure that all dimensions for the hypertable are created on the remote server. " 77cc311076dbf42f5a13eaced899cebf29a85b81,Brian Rowe,Fri Feb 15 14:38:10 2019 -0800,Update internal server table on hypertable add,"When adding a server as a backend for a hypertable, update the hypertable_server table on the frontend to contain the hypertable id used for the given hypertable on the backend. " d355c0096106bdd537381222c15a1eaa46e82fcf,Brian Rowe,Mon Feb 4 18:26:01 2019 -0800,Distribute hypertable creation to data nodes,"When creating a hypertable with backend servers (replication_factor > 0), this will deparse the table structure and send the commands to create it on all of the backend nodes. It will then send a command to create the hypertable on each backend. There is still some more work needed to handle assigning hypertables with multiple space dimensions and supporting partitioning functions. " 7767a631c7a4730bdd0e494a59c2aa1a9dbded15,Matvey Arye,Thu Jun 7 15:51:07 2018 -0400,Add 2PC handling in distributed txn callbacks,This commit adds the ability to use two-phase commit for distributed txns. 2PC is safer for guaranteeing atomicity of commits when compared to 1PC but require one more round of communication. Whether to use the 1pc or 2pc variant is controlled by a GUC. 2pc is default. For more information look at tsl/src/remote/README.md included in this commit. 59d2f8920eb468a65ca4ffd2af3b60fe51401cd1,Erik Nordström,Fri Feb 8 14:32:20 2019 +0100,Fix prepared statements on distributed hypertables,"A pointer to a fdwroutine object was kept in the `HypertableInsert` plan node, so that it could later be used in the execution stage. However, this caused prepared statements to fail because, apparently, fdwroutine cannot be copied with copyObject() despite being a Node type. This fix removes the fdwroutine from the plan node and instead lets the executor state reresolve the fdwroutine from the list of servers. " f5a09913e1e69aa3d7af7668ebcc106325b0abbb,Matvey Arye,Wed Feb 6 16:14:15 2019 -0500,Make heal safe to non-timescale prepared txns,"To make the heal function safe to non-ts prepared txns we introduce a prefix ""ts"" to our prepared txns. This allows us to separate cases where there is a ts vs non-ts prepared txn and have heal ignore non-ts txns. An alternative would be to consider all txns that don't parse correctly as non-ts transactions. But, that is less robust to bugs in our parsing/printing code. One downside to the current approach is that all prepared txns with a ""ts"" prefix are considered reserved for ts. That should be acceptable. " e7ba327f4c6017de31fcc488a2944a41f2f25505,Matvey Arye,Thu Jun 7 15:51:07 2018 -0400,Add resolve and heal infrastructure for 2PC,"This commit adds the ability to resolve whether or not 2PC transactions have been committed or aborted and also adds a heal function to resolve transactions that have been prepared but not committed or rolled back. This commit also removes the server id of the primary key on the remote_txn table and adds another index. This was done because the `remote_txn_persistent_record_exists` should not rely on the server being contacted but should rather just check for the existance of the id. This makes the resolution safe to setups where two frontend server definitions point to the same database. While this may not be a properly configured setup, it's better if the resolution process is robust to this case. " a3ebdf1f2dd85053cfb3374c479e9f4512f31550,Erik Nordström,Fri Feb 1 20:52:50 2019 +0100,Unify remote UPDATE and DELETE FDW handler,The handlers for executing UPDATEs and DELETEs are almost identical in the foreign data wrapper. This change refactors these handlers to call a common function. 7e615e10a32eb896a1682f9518e314f00a7d53b9,Erik Nordström,Fri Feb 1 15:44:34 2019 +0100,Add FDW code for executing remote DELETEs,This adds the FDW code from `postgres_fdw` for executing foreign deletes. The code has been refactored to use the async API and to do deletes across all chunk replicas. 09dd97e339d0254953e48f4401de14b14463dcae,Erik Nordström,Fri Feb 1 15:20:47 2019 +0100,Add FDW code for excuting remote UPDATEs,This adds the FDW code from `postgres_fdw` for executing foreign updates. The code has been refactored to use the async API and to do updates across all chunk replicas. 1780897d541e3af88dcb1607974a353f12ae58bb,Erik Nordström,Fri Feb 1 14:40:45 2019 +0100,Add FDW code for foreign scans,Add the `postgres_fdw` code for foreign scans to the TimescaleDB foreign data wrapper. The code has been updated to use the async API for remote queries. 6e2ea3be50a3f8ea568d3e9fb8eb9dd47cf3a264,Erik Nordström,Fri Feb 1 09:59:21 2019 +0100,Update FDW-related tests with EXPLAIN queries,This adds EXPLAIN queries to the tests that cover the foreign data wrapper and distributed hypertables. These tests cover the planning code and ensures that EXPLAIN output works and looks reasonable. 9f16c6ca24acc21d8209a0e9e569864e746baa75,Erik Nordström,Fri Feb 1 09:56:39 2019 +0100,Add FDW code for explaining foreign scans,Add code from `postgres_fdw` to explain a scan. bc564cb4b84114368a3cb29cb45ccb31023c3be3,Erik Nordström,Fri Feb 1 09:47:16 2019 +0100,Add FDW planning code for creating foreign plans,"Add code from `postgres_fdw` that implements plan creation in the foreign data wrapper. The parts of the code that relates to JOINs have been omitted, since JOINs won't be supported initially. " 77e7504bbca2968207817ea0cb9c3047e1784e00,Erik Nordström,Fri Feb 1 09:38:49 2019 +0100,Add FDW planning code for getting foreign paths,"Add code from `postgres_fdw` that creates paths for foreign scans. The only changes made to the code is to remove path creation for JOINs, which won't be supported initially. " 9874fb3d27b6aec8b249ae90bb45c2f7ac4014ac,Erik Nordström,Fri Feb 1 09:29:45 2019 +0100,Add relsize estimate to FDW,"Import the code for relsize estimate from `postgres_fdw` into the TimescaleDB foreign data wrapper. A couple of notable, but minor, changes have been applied to the code: - Slight code reorganization - Remove support for the foreign table options `schema_name`, `table_name`, and `column_name` since we mandate that these are always the same across frontend and backend servers - Refactor to use our connection and async APIs - Remove support for JOINs " 45e23d2744d1109ad77a50af937e8985a7002dd6,Erik Nordström,Tue Jan 29 10:11:03 2019 +0100,"Support replicated INSERT, UPDATE, DELETEs","Previously, INSERT/UPDATE/DELETEs only happened to a chunk's ""primary"" server, which is the one set in the chunk's foreign table. However, chunks can have more than one assigned server and this change enables these operations across all the assigned servers. " d5399e9045a161f3ce0c5d64ca3835d8ff4f7525,Erik Nordström,Wed Jan 23 15:32:10 2019 +0100,Add FDW code for executing remote INSERTs,"This implements the execution path for remote INSERTs in the foreign data wrapper (FDW). The code is imported from the `postgres_fdw`, but have been updated to use the TimescaleDB connection cache and asynchronous query API. " 3ddbc386f07feec1449b66e7aff2d965d3889f0d,Erik Nordström,Thu Jan 31 18:32:10 2019 +0100,Only support multinode on PG11 and greater,"Multinode-related APIs now raise errors when called any PostgreSQL version below 11, as these versions do not have the required features to support multinode or have different behavior. Raising errors at runtime on affected APIs is preferred over excluding these functions altogether. Having a different user-facing SQL API would severly complicate the upgrade process for the extension. A new CMake check has been added to disable multinode features on unsupported PostgreSQL versions. It also generates a macro in `config.h` that can be used in code to check for multinode support. " 0e109d209d29ab184abc294e73cd1fa99132af27,Matvey Arye,Thu Jun 7 15:51:07 2018 -0400,Add tables for saving 2pc persistent records,"The remote_txn table records commit decisions for 2pc transactions. A successful 2pc transaction will have one row per remote connection recorded in this table. In effect it is a mapping between the distributed transaction and an identifier for each remote connection. The records are needed to protect against crashes after a frontend send a `COMMIT TRANSACTION` to one node but not all nodes involved in the transaction. Towards this end, the commitment of remote_txn rows represent a crash-safe irrevocable promise that all participating datanodes will eventually get a `COMMIT TRANSACTION` and occurs before any datanodes get a `COMMIT TRANSACTION`. The irrevocable nature of the commit of these records means that this can only happen after the system is sure all participating transactions will succeed. Thus it can only happen after all datanodes have succeeded on a `PREPARE TRANSACTION` and will happen as part of the frontend's transaction commit.. " 4368fcff3c19f2291f6f0276e6942a428c794d42,niksa,Mon Jan 28 12:36:44 2019 +0100,Add function to reconstruct the creating command for a table,"Deparse a table into a set of SQL commands that can be used to reconstruct it. Together with column definiton it deparses constraints, indexes, triggers and rules as well. There are some table types that are not supported: temporary, partitioned, foreign, inherited and a table that uses options. Row security is also not supported. " e2371558f748a4a8794561dfc4e4ec28fff3d1ee,Erik Nordström,Fri Jan 25 18:49:49 2019 +0100,Create chunks on remote servers,"This change ensures that chunk replicas are created on remote (datanode) servers whenever a chunk is created in a local distributed hypertable. Remote chunks are created using the `create_chunk()` function, which has been slightly refactored to allow specifying an explicit chunk table name. The one making the remote call also records the resulting remote chunk IDs in its `chunk_server` mappings table. Since remote command invokation without super-user permissions requires password authentication, the test configuration files have been updated to require password authentication for a cluster test user that is used in tests. " db82c25d44195c22855b3110904132d6792e87d8,Brian Rowe,Thu Jan 24 19:21:23 2019 -0800,Add an API to invoke SQL on backend servers,This change adds a new function that will invoke the passed in command on any specified backend servers (defaulting to all). The command will be run asynchronously with transactional semantics (it either succeeds on all targets or is rolled back). 125f7933073e917e7656795143fc76f1743061f0,Erik Nordström,Fri Jan 25 10:03:57 2019 +0100,Add password parameter to add_server(),"Establishing a remote connection requires a password, unless the connection is made as a superuser. Therefore, this change adds the option to specify a password in the `add_server()` command. This is a required parameter unless called as a superuser. " 652bb26415585e278df77f00bb601eb28a58613a,Matvey Arye,Thu Jan 24 16:55:08 2019 -0500,Make clustering tests not run on PG 9.6,We are not supporting clustering in PG 9.6. 86ac51abbc5d8b01fbaa142fb5aa882effe293bd,Erik Nordström,Sat Apr 25 21:14:22 2020 +0200,Avoid index creation on foreign table chunks,Distributed hypertables should not have indexes on access node chunks. This change adds checks to not recurse to chunks in case of distributed hypertables. 3779af400d9ec6727ad20105fb3437cc0f50f60f,Matvey Arye,Thu Jan 24 16:50:50 2019 -0500,Change license header to new format in SQL files,"The license header for SQL test files has been updated, but some tests haven't had this new header applied. This change makes sure the new header is applied to all test files. " 882c1f17f0da5133f7010464e88bc3dfe1fe1872,Erik Nordström,Fri Jan 18 10:51:12 2019 +0100,Add planning code for remote INSERTs,"This change adds an implementation of the PlanForeignModify FDW handler based on the one from `postgres_fdw`. The planning involves creating a deparsed INSERT/UPDATE/DELETE statement for sending to the remote node. Note that this does not add support for the direct modify interface. This PostgreSQL code is cleaned and a test has been added validate that proper SQL statements are generated. A minor fix has been applied to the deparsing code to allow deparsing INSERT/UPDATE/DELETE statements for a relation that might not be a foreign table. We need to deparse statements for regular relations since modifications on the remote data node will happen on the main hypertable (which is not a foreign table), and not directly on chunks. " d2b4b6e22e23509761b193e2dc9b24e9e2e3439a,Matvey Arye,Sun Jan 20 23:03:53 2019 -0500,Add remote transaction ID module,The remote transaction ID is used in two phase commit. It is the identifier sent to the datanodes in PREPARE TRANSACTION and related postgresql commands. This is the first in a series of commits for adding two phase commit support to our distributed txn infrastructure. 33f1601e6fcbad5c8dde7f175a5f5626409d84ac,Erik Nordström,Mon Jan 21 12:34:29 2019 +0100,"Handle constraints, triggers, and indexes on distributed hypertables","In distributed hypertables, chunks are foreign tables and such tables do not support (or should not support) indexes, certain constraints, and triggers. Therefore, such objects should not recurse to foreign table chunks nor add a mappings in the `chunk_constraint` or `chunk_index` tables. This change ensures that we properly filter out the indexes, triggers, and constraints that should not recurse to chunks on distributed hypertables. " af7d9fe4d63e727e4d326b0c8ee09974f3d214e4,Matvey Arye,Thu Jun 7 15:51:07 2018 -0400,Add remote txns support (1pc),"This commit adds the ability to open remote transactions that share transactional semantics with the ongoing local transaction. This is done by adding txn hooks to the local transaction that performs txn commands on remote nodes. This commit adds the 1pc variant of the commit protocol. A 2-pc variant is going to be added in an upcoming PR. The 1pc variant exists mostly for performance testing (i.e. how much is the 2pc penalty). But, there are also cases where 1pc is good enough (non-critical data and/or data backed by Kafka). Note that the remote_txn test is not run during sanitizer tests due to an apparent bug in PostgreSQL. (The test fails on an assertion of a global variable `AbortStartTime`, which is non-atomic although read across the ServerLoop and signal handlers.) " b221acafdb2299a234a592160aacff46d4f7e063,Erik Nordström,Fri Jan 18 11:13:52 2019 +0100,Remove JOIN-related code in FDW deparsing code,"Since multinode deployments won't support JOINs initially, this change removes JOIN-related code from the FDW deparsing. This avoids having this code linger, being unused and unmaintained. " 1b39924b4a1f4260c691b481cc798ef192324fe2,Erik Nordström,Tue Jan 15 20:08:39 2019 +0100,Add deparsing code for sending queries to data nodes,"This adds the deparsing code from the postgres_fdw to be used for our own FDW implementation. Deparsing is needed to generate the textual SQL statements that are sent to remote data nodes. The following (minor) modifications have been made to the code: - Add appropriate license notices, headers and includes - Break code into appropriate subfiles. For instance, utility functions go into utils.c - Rename some data structures, e.g., PgFdwRelationInfo -> TsFdwRelationInfo " 596be8cda1e6ded3901f229a93e56d227306a686,Erik Nordström,Tue Jan 1 11:56:03 2019 +0100,Add mappings table for remote chunks,"A frontend node will now maintain mappings from a local chunk to the corresponding remote chunks in a `chunk_server` table. The frontend creates local chunks as foreign tables and adds entries to `chunk_server` for each chunk it creates on remote data node. Currently, the creation of remote chunks is not implemented, so a dummy chunk_id for the remote chunk will be added instead for testing purposes. " 02c178d9caceca407431133a8c7f20460403660b,Matvey Arye,Thu Jun 7 15:51:07 2018 -0400,Add connection caching infrastructure,This commit adds the ability to cache remote connections across commands and transactions. This is needed since establishing new connections is expensive. The cache is invalidated when the foreign server or user mapping is changed. Because of this the cache is tied to a user mapping (it is keyed by the user mapping's oid and requires a user_mapping for invalidation). We use the syscache invalidation mechanism since foreign servers and user mappings are already invalidated using this mechanism. This requires some extra plumbing in our cache invalidation handling. This cache will be used in txn callback handling and so the regular auto-release of caches on (sub)txn commits/aborts that happens with most caches is inappropriate. Therefore we added a new flag to the cache called `handle_txn_callbacks` that allows a cache to turn off the auto-release mechanism ece582d458dbf732df4d90094d897248ad384ca3,Erik Nordström,Fri Dec 14 10:57:55 2018 +0100,Add mappings table for remote hypertables,"In a multi-node (clustering) setup, TimescaleDB needs to track which remote servers have data for a particular distributed hypertable. It also needs to know which servers to place new chunks on and to use in queries against a distributed hypertable. A new metadata table, `hypertable_server` is added to map a local hypertable ID to a hypertable ID on a remote server. We require that the remote hypertable has the same schema and name as the local hypertable. When a local server is removed (using `DROP SERVER` or our `delete_server()`), all remote hypertable mappings for that server should also be removed. " ae587c9964839eb27c3b2d89de8da97d3a46bf2a,Erik Nordström,Fri Apr 17 17:51:39 2020 +0200,Add API function for explicit chunk creation,This adds an internal API function to create a chunk using explicit constraints (dimension slices). A function to export a chunk in a format consistent with the chunk creation function is also added. The chunk export/create functions are needed for distributed hypertables so that an access node can create chunks on data nodes according to its own (global) partitioning configuration. 26cad3b03790349cbe7f4be950e2f91239e78b37,Matvey Arye,Thu Jan 3 18:54:16 2019 -0500,Change license header statement,Update to the new license header statement 5f13d751337da01dc1786596386f21247eb9ea1b,Erik Nordström,Sat Dec 22 20:24:38 2018 +0100,Use consistent formatting for header files in TSL module,This fixes a number of inconsistencies in how headers are formatted in the TSL module: - Consistent naming for include guards - Consistent #include of non-local headers 0a12a678d5c062939ac7fabed7535b0aa04b7a2a,Erik Nordström,Sat Dec 22 20:21:04 2018 +0100,Add missing libpq-fe.h include for remote connection module,The remote connection module fails to build on systems where the PostgreSQL client headers (in particular `libpq-fe.h`) are not in a standard header include path. This change fixes this by adding `pg_config --includedir` to the include directories for the remote connection module and its tests. 79544e8919b8381a37cb3445a2b7e91cd0e6351c,Matvey Arye,Thu Jun 7 15:51:07 2018 -0400,Add remote connection code,This PR adds two modules: for connection to remote databases and for asynchronous communication with those nodes. 538e27d1406b86cba267af4d7d9295942de632e6,niksa,Tue Dec 18 13:10:10 2018 +0100,Add Noop Foreign Data Wrapper,"This adds a skeleton TimescaleDB foreign data wrapper (FDW) for scale-out clustering. It currently works as a noop FDW that can be used for testing, although the intention is to develop it into a full-blown implementation. " eca7cc337ae07bfff13f8b2b0ecb7ebf5817a38d,Erik Nordström,Fri Dec 14 15:36:02 2018 +0100,Add server management API and functionality,"Servers for a scale-out clustering setup can now be added and deleted with `add_server()` and `delete_server()`, providing a convenience API for server management. While similar functionality can be achieved using the standard PostgreSQL `CREATE SERVER` and `CREATE USER MAPPING` commands, this new API makes it easier to add clustering servers and user mappings consistent with the needs of TimescaleDBs particular clustering setup. The API currently works with the `postgres_fdw` foreign data wrapper. It will be updated to use our own foreign data wrapper once it is available. " a37332e2146f4acf4c8f7acdeb94b8c67024296e,Ruslan Fomkin,Tue May 26 09:49:23 2020 +0200,Update tests to run on newest PG,"PostgreSQL released 12.3, 11.8, 10.13 and 9.6.18. So tests, which uses latest PostgreSQL, are updated to these versions. " 43603e83fadd96b136451bb4d6526fd8d8bd1982,Sven Klemm,Thu May 21 16:34:15 2020 +0200,Only enable codecoverage on specific travis jobs,Because add_link_options is not supported on older cmake versions available in older postgresql docker images we only enable code coverage on the codecov jobs. 965404a7fb86cdbe1e6a8da77e964246102705d4,Sven Klemm,Sat May 23 21:45:24 2020 +0200,Ignore compression_qualpushdown test generates files, fd07772daf019e648fa5729a9c982fb5953c8bf5,Sven Klemm,Sat May 23 12:11:11 2020 +0200,Fix multiple definitions of ts_mock_timer, 30dbda9fbb575090c361c0e65282fef7e5d67a6c,Sven Klemm,Tue May 19 18:33:35 2020 +0200,Unify chunk index creation,This patch changes chunk index creation to use the same functions for creating index in one transaction and using multiple transactions. The single transaction index creation used to adjust the original stmt and adjusted it for the chunk which lead to problems with table references not being adjusted properly for the chunk. e1b9e02e121c19748005b9c298e107ee52e6adaf,Erik Nordström,Thu May 14 18:08:33 2020 +0200,Cleanup code coverage and make it work locally,"The code coverage options for CMake have been cleaned up and fixed where they were broken. Build targets for local coverage reports now work. The option to enable code coverage has been changed to `-DCODECOVERAGE=ON` and local HTML-based reports can be generated using the `lcov` program through the following steps: 1. `make install` 2. `make installcheck` 3. `make coverage` The previous CMake options and targets didn't properly work, and often included redundant and confusing options. For instance, the only compiler option needed for code coverage is `--coverage` as this is an alias for `-fprofile-arcs -ftest-coverage`. Previously, however, these options were added multiple times in various places, often all of them, but sometimes only a subset for no apparent reason. They were also added using the wrong CMake commands, for instance `add_definitions`, which is deprecated and reserved for preprocessor definitions and not compiler options. The `lcov` program is used to generate local HTML-based code coverage reports. Although CMake was setup to look for `lcov`, it didn't check for failure cases in case it wasn't found and further had incomplete target configurations that really didn't do much. No additional documentation was provided on how to generate local coverage reports. All of this has been fixed, including documentation in `codecov/README.md`. " 739631d6a71f0deeb8f0a77285d98743e8f2405d,Sven Klemm,Tue May 19 00:23:47 2020 +0200,Check for database in extension_current_state,"The extension_current_state is called in the cache_invalidate_callback which might be called in a background worker when the database has not been initialized yet leading to a ""cannot read pg_class without having selected a database"" error. This patch adds a check for this condition to prevent this error. " 1df97acfd234a9162b4d1e86372054fda2ac1721,Sven Klemm,Mon May 18 17:01:37 2020 +0200,Add 1.7.1 to update test scripts, b57d2ac388ac505087e2f301fe4b360fdd446e84,Stephen Polcyn,Mon Apr 6 17:16:34 2020 -0400,Cleanup TODOs and FIXMEs,"Unless otherwise listed, the TODO was converted to a comment or put into an issue tracker. test/sql/ - triggers.sql: Made required change tsl/test/ - CMakeLists.txt: TODO complete - bgw_policy.sql: TODO complete - continuous_aggs_materialize.sql: TODO complete - compression.sql: TODO complete - compression_algos.sql: TODO complete tsl/src/ - compression/compression.c: - row_compressor_decompress_row: Expected complete - compression/dictionary.c: FIXME complete - materialize.c: TODO complete - reorder.c: TODO complete - simple8b_rle.h: - compressor_finish: Removed (obsolete) src/ - extension.c: Removed due to age - adts/simplehash.h: TODOs are from copied Postgres code - adts/vec.h: TODO is non-significant - planner.c: Removed - process_utility.c - process_altertable_end_subcmd: Removed (PG will handle case) " dab50aef19f1353b563be1d8e25ed9e39695a701,Sven Klemm,Sat May 16 01:04:14 2020 +0200,Add real-time aggregation test to update test,This patch adds a continuous aggregate with real-time aggregation enabled to the update test suite since we rebuild view definition for real time aggregation during extension update. The continuous aggregate is in its own schema because all view definitions in the public schema are dumped and those view definitions will change between versions. 0ea509cc48d06d2de2f0b1406ccf3b03c88fc88c,Sven Klemm,Fri May 15 13:17:39 2020 +0200,Release 1.7.1,"This maintenance release contains bugfixes since the 1.7.0 release. We deem it medium priority for upgrading and high priority for users with multiple continuous aggregates. In particular the fixes contained in this maintenance release address bugs in continuous aggregates with real-time aggregation and PostgreSQL 12 support. **Bugfixes** * #1834 Define strerror() for Windows * #1846 Fix segfault on COPY to hypertable * #1850 Fix scheduler failure due to bad next_start_time for jobs * #1851 Fix hypertable expansion for UNION ALL * #1854 Fix reorder policy job to skip compressed chunks * #1861 Fix qual pushdown for compressed hypertables where quals have casts * #1864 Fix issue with subplan selection in parallel ChunkAppend * #1868 Add support for WHERE, HAVING clauses with real time aggregates * #1869 Fix real time aggregate support for multiple continuous aggregates * #1871 Don't rely on timescaledb.restoring for upgrade * #1875 Fix hypertable detection in subqueries * #1884 Fix crash on SELECT WHERE NOT with empty table **Thanks** * @airton-neto for reporting an issue with queries over UNIONs of hypertables * @dhodyn for reporting an issue with UNION ALL queries * @frostwind for reporting an issue with casts in where clauses on compressed hypertables * @fvannee for reporting an issue with hypertable detection in inlined SQL functions and an issue with COPY * @hgiasac for reporting missing where clause with real time aggregates * @louisth for reporting an issue with real-time aggregation and multiple continuous aggregates * @michael-sayapin for reporting an issue with INSERTs and WHERE NOT EXISTS * @olernov for reporting and fixing an issue with compressed chunks in the reorder policy * @pehlert for reporting an issue with pg_upgrade " 3c06278d5c3b88dea74dd215f612d8839610ef51,Erik Nordström,Fri May 15 00:51:05 2020 +0200,Use test-specific assertions in C test code,"Test code in C should use test-specific assertions that throw errors instead of exiting the program with a signal (crash). Not only does this provide more useful and easily accessible information of the failing condition, but it also allows running the test suite without assertions (`USE_ASSERT_CHECKING`) enabled. Having assertions disabled during tests also provides more accurate test coverage numbers. Note that these test-specific assertions are not intended to replace regular assertions (`Assert`), which are used in non-test code. The way to enable (or disable) assertions in CMake has also been simplified and cleaned up. The option `-DASSERTIONS=[ON|OFF]` can be used to enable assertions for a build, unless already enabled in the PostgreSQL one is building against (in which case that setting takes precedence). Note that the `ASSERTIONS` option defaults to `OFF` since it is no longer necessary to have assertions enabled for tests. " ed64af76a54c2980f8d5dfd51b6b0d5fa2eefd43,gayyappan,Fri May 1 17:43:32 2020 -0400,Fix real time aggregate support for multiple aggregates,We should compute the watermark using the materialization hypertable id and not by using the raw hypertable id. New test cases added to continuous_aggs_multi.sql. Existing test cases in continuous_aggs_multi.sql were not correctly updated for this feature. Fixes #1865 d0c92dd433175b4a4f5209752e52dfc20a2f4805,Dmitry Simonenko,Fri May 15 12:08:56 2020 +0300,Fix crash on SELECT WHERE NOT with empty table,"Modify table state is not created with an empty tables, which lead to NULL pointer evaluation. Starting from PG12 the planner injects a gating plan node above any node that has pseusdo-constant quals. To fix this, we need to check for such a gating node and handle the case. We could optionally prune the extra node, since there's already such a node below ChunkDispatch. Fixes #1883. " 74dcff5807ee4bc80e5337aa79dfb3d877499887,Ruslan Fomkin,Tue May 12 15:47:44 2020 +0200,Run Windows regression tests on PG12,Change appveyor script to run regression tests against PG12 on Windows. It is short term solution with official PG alpine image. No EXEC_BACKEND is enabled. 8ae8bd3848089d25df5f2b3a72e7d3e192d70ded,Ruslan Fomkin,Tue May 12 19:17:19 2020 +0200,Remove unused static declaration of a function,This removes warning of a function declared static but never defined. d4c655fb2f1c5f80503bb0a409f4733e9fdbc849,Brian Rowe,Mon May 11 13:02:41 2020 -0700,Change ChunkAppend leader to use worker subplan,"When running a parallel ChunkAppend query, the code would use a subplan selection routine for the leader that would return an empty slot unless it determined there were no workers started. While this had the desired effect of forcing the workers to do the work of evaluating the subnodes, returning an empty slot is not always safe. In particular, Append nodes will interpret an empty slot as a sign that a given subplan has completed, so a plan resulting in a parallel MergeAppend under an Append node (very possible under a UNION of hypertables) might fail to execute some or all of the subplans. This change modifies the ChunkAppend so that the leader uses the same subplan function as the workers. This may result in the leader being less responsive as it try to fetch a batch of results on its own if no worker has any results yet. However, if this isn't the desired behavior, PostgresQL already exposes a GUC option, parallel_leader_participation, which will prevent the leader from executing any subplans. " a95308e91737cf9bd6b39975ab303bc6b22d2cbb,Erik Nordström,Thu Apr 9 12:44:28 2020 +0200,Run Windows builds and tests using GitHub Actions,"Windows builds, for Debug and Release configurations, are now tested using GitHub Actions. The build test covers all currently supported PostgreSQL versions (9.6, 10, 11, 12) using a build matrix. Currently, the TimescaleDB regression test suite is not run since it requires a Windows-specific test client that we do not support. Instead, the binaries are tested by doing a `CREATE EXTENSION` to verify that they load and run without, e.g., missing symbols or similar. The test configuration is optimized for speed by using a cache for the PostgreSQL installation. This avoids repeated downloads and installations of PostgreSQL from EnterpriseDB's servers. But it also means that we won't have a full installation on cache hits, which means no running PostgreSQL service. This requires manual PostgreSQL starts, which is what we want anyway since we need to preload our extension. It's anticipated that this build configuration can be extended to produce release binaries in the future. " db80ca713695e8593cbe47b28e1dc329cef20877,Erik Nordström,Sat May 9 21:03:24 2020 +0200,Remove DEBUG logging in extension check,"This change removes a `DEBUG1` level log message that was added to the `ts_extension_is_loaded` check. The problem with this message is that it is called very frequently, e.g., in planning, and it will flood the output log. " 647657404e46864602b42175b1ea0bb2136a5fff,Joshua Lockerman,Thu Feb 6 13:21:38 2020 -0500,Improve time_bucket_gapfill optimizations,"This commit adds the optimizations we do for regular time bucket, pushing down sort orders to the underlying time column, to time_bucket_gapfill. This is made slightly more invasive by the fact that time_bucket_gapfill is marked as VOLATILE (to prevent postgres from moving it in inappropriate ways). We handle this by time_bucket_gapfill EC as not volatile, and letting the ususal sort machinery do its job. This should be safe since we've already planned all the per-relation paths and have installed the appropriate metadata to know we will gapfill. " db1fc4083d4e77c1b80fc4b30b0a6469f8e6567a,Erik Nordström,Sat May 9 21:19:09 2020 +0200,Fix flaky hypertable cache plan test,"The `plan_hypertable_cache` test used `current_date` to generate data, which is inherently flaky since it can create a different number of chunks depending on which date you start at. When the number of chunks differ, the test output changes too. " c13ef6df9791fc7d2d3f0563223e85635c567fcc,gayyappan,Wed Apr 29 09:23:38 2020 -0400,Support WHERE and HAVING clauses for real time aggregation,Continuous aggregates with real time support did not propagate WHERE and HAVING clauses to the direct query on the table. Fixes #1860 b368563e3f62fbc51007b93366aa5554b94fa438,Oleg Smirnov,Sun Apr 26 15:49:18 2020 +0300,Fix reorder policy job to skip compressed chunks,Reorder policy does not skip compressed chunks when selecting next chunk to apply reordering. This causes an error in job execution since it's not possible to reorder compressed chunks. With this fix the job excludes compressed chunks from selection. Fixes #1810 853e37efbfac542daf1d4b3b30c8281b76be33f0,Sven Klemm,Tue May 5 14:51:56 2020 +0200,Fix hypertable detection in subqueries,When classify_relation is called for relations of subqueries it would not be able to correctly classify the relation unless it was already in cache. This patch changes classify_relation to call get_hypertable without CACHE_FLAG_NOCREATE when the RangeTblEntry has the inheritance flag set. 5e1c4e614dcb57642c75d3cdf0f90ddc5d87c371,Ruslan Fomkin,Tue May 5 09:58:40 2020 +0200,Ignore Windows build files,Adds build output on Windows and CMake settings for VS to gitignore. 730822127d3acda1319918f75d55965528b147f0,Mats Kindahl,Mon May 4 12:46:01 2020 +0200,Don't rely on timescaledb.restoring for upgrade,"If a binary upgrade is in progress (when using `pg_upgrade`) the per-database setting of `timescaledb.restoring` can be included in the dump, which causes `pg_upgrade` to fail. This commit fixes this by checking the global variable `IsBinaryUpgrade` and not refreshing cache if we are in the middle of doing a binary upgrade. Fixes #1844 " 2574e5a5639fc59c0357b68f5d6fe3d805c3e04d,Ruslan Fomkin,Tue May 5 11:03:25 2020 +0200,Add 12 folder to search for PG bin in CMake,CMake file is missing the path suffix for PG 12 when searching for PostgreSQL binaries. This commit adds the path suffix for 12. 49a56f7826048006a1b34d425db50f80ef9655cd,gayyappan,Mon Apr 27 23:36:02 2020 -0400,Fix qual pushdown for compression,Operators used for qual pushdown for compressed chunks should take the RHS expression's type into account. Fixes #1855 28e9a443b3a17ab889fd7dbd874e1cc6aee667bb,Erik Nordström,Wed Apr 22 12:55:51 2020 +0200,"Improve handling of ""dropped"" chunks","The internal chunk API is updated to avoid returning `Chunk` objects that are marked `dropped=true` along with some refactoring, hardening, and cleanup of the internal chunk APIs. In particular, apart from being returned in a dropped state, chunks could also be returned in a partial state (without all fields set, partial constraints, etc.). None of this is allowed as of this change. Further, lock handling was unclear when joining chunk metadata from different catalog tables. This is made clear by having chunks built within nested scan loops so that proper locks are held when joining in additional metadata (such as constraints). This change also fixes issues with dropped chunks that caused chunk metadata to be processed many times instead of just once, leading to potential bugs or bad performance. In particular, since the introduction of the “dropped” flag, chunk metadata can exist in two states: 1. `dropped=false` 2. `dropped=true`. When dropping chunks (e.g., via `drop_chunks`, `DROP TABLE `, or `DROP TABLE `) there are also two modes of dropping: 1. DELETE row and 2. UPDATE row and SET dropped=true. The deletion mode and the current state of chunk lead to a cross-product resulting in 4 cases when dropping/deleting a chunk: 1. DELETE row when dropped=false 2. DELETE row when dropped=true 3. UPDATE row when dropped=false 4. UPDATE row when dropped=true Unfortunately, the code didn't distinguish between these cases. In particular, case (4) should not be able to happen, but since it did it lead to a recursing loop where an UPDATE created a new tuple that then is recursed to in the same loop, and so on. To fix this recursing loop and make the code for dropping chunks less error prone, a number of assertions have been added, including some new light-weight scan functions to access chunk information without building a full-blown chunk. This change also removes the need to provide the number of constraints when scanning for chunks. This was really just a hint anyway, but this is no longer needed since all constraints are joined in anyway. " b34363edc222b4e2931dab4ae78a31b86d5b7573,Mats Kindahl,Mon Apr 20 14:33:24 2020 +0200,Change lookup of clang-format version,"The formatting requires `clang-format` version 7 or 8, but if a later distro is used it will find a version that cannot be used and default to using docker even if the user installs an earlier version of `clang-format` in parallel with the default version. This commit fixes this by looking for `clang-format-8` and `clang-format-7` before `clang-format`. " 52e23773dc829733e902f8890a4ecb047f5d1a37,gayyappan,Mon Apr 20 17:20:34 2020 -0400,Validate job time to avoid error thrown by scheduler,Trying to use an invalid time for a job raises an error. This case should be checked by the scheduler. Failure to do so results in the scheduler being killed. 0e9461251bee2405889ec7def9607a63c7ef5a66,Erik Nordström,Mon Apr 27 10:01:10 2020 +0200,Silence various compiler warnings,"This change fixes various compiler warnings that show up on different compilers and platforms. In particular, MSVC is sensitive to functions that do not return a value after throwing an error since it doesn't realize that the code path is not reachable. " 7d78540a222429acbefcc945612e03d1780bd936,Mats Kindahl,Wed Apr 22 22:53:28 2020 +0200,Fix segfault on COPY to hypertable,"When copying from standard input the range table was not set up to handle the constraints for the target table and instead is initialized to null. In addition, the range table index was set to zero, causing an underflow when executing the constraint check. This commit fixes this by initializing the range table and setting the index correctly. The code worked correctly for PG12, so the code is also refactored to ensure that the range table and index is set the same way in all versions. Fixes #1840 " 0a7e9722f6f7d9ed43ff2245f9f435316523996a,Sven Klemm,Thu Apr 23 20:54:57 2020 +0200,Fix hypertable expansion for UNION ALL,When doing a UNION ALL query between a hypertable and a regular table the hypertable would not get expanded leading to no data of the hypertable included in the resultset. 966c34fdba8e85b08ddba3d82a9be482eb1eda04,Ruslan Fomkin,Mon Apr 20 14:31:44 2020 +0200,Update build from source instructions for 1.7,"PostgreSQL 12 is supported since 1.7, while 9.6 and 10 are deprecated. This PR updates the building from source instruction with this change and uses 1.7.0 in examples. " 685f8bf9f0d96f9d9b52d9c3e1e448f85dd457d0,Mats Kindahl,Thu Apr 16 14:17:42 2020 +0200,Enable clang-tidy,"Adding a custom target `clang-tidy` that will run `run-clang-tidy` if it is available and the compiler database is enabled. If the compiler database is not enabled but `run-clang-tidy` is found, a warning will be printed and the custom command will not be added. " 5a6fd603e6c402f3029b0c75afb94cf3d2540b07,Sven Klemm,Tue Apr 21 00:21:19 2020 +0200,Add 1.7.0 to update test scripts,Due to the changes of the default view behaviour of continuous aggregates we need a new testsuite for the update tests for 1.7.0 This patch also changes the update test for 9.6 and 10 to run on cron and 11 and 12 on pull request. 8804b57cfbe40a861663f55c26eaf5ed52c63489,Sven Klemm,Mon Apr 20 23:37:24 2020 +0200,Move catalog missing column check into separate file, ed3eda2ceb86084576c9e84a51a164e08e2fab1f,Sven Klemm,Mon Apr 20 21:40:53 2020 +0200,Add missing ORDER BY clause to update test queries, d9bf54c604034c434621191af7342a5aaa63da03,Sven Klemm,Mon Apr 20 20:24:06 2020 +0200,Use OID const in cagg_watermark call,"The view definition for the realtime aggregation union view would use a INT Const as argument to cagg_watermark, but the function argument is defined as OID leading to a cast being inserted in a view definition restored from backup. This leads to a difference between the original view definition and a view definition from a restored view. " 8e94875d85204b36f07adac2a4f15692db5973d8,Mats Kindahl,Mon Apr 20 09:54:10 2020 +0200,Define strerror() for Windows,"The symbol `pgwin32_socket_strerror` was undefined on windows builds at PG12 and later. This because the function was removed and instead `pg_strerror` was introduced to be used on all platforms. This commit fixes the issue by ensuring to use `pg_strerror` on PG12 and later, and `pgwin32_socket_strerror` on Windows builds before PG12. Found using `clang-tidy` " ed12257810c589d9832fdecbacb877544a069b30,Dmitry Simonenko,Fri Apr 17 16:26:02 2020 +0300,Improve cmake PostgreSQL version check,This change improves cmake check for PostgreSQL minor/patch versions and makes it consistent with src/compat.h version check 5e678610bfb95d4929a682b8218061cf95f907bc,Sven Klemm,Thu Apr 16 11:07:33 2020 +0200,Release 1.7.0,"This release adds major new features and bugfixes since the 1.6.1 release. We deem it moderate priority for upgrading. This release adds the long-awaited support for PostgreSQL 12 to TimescaleDB. This release also adds a new default behavior when querying continuous aggregates that we call real-time aggregation. A query on a continuous aggregate will now combine materialized data with recent data that has yet to be materialized. Note that only newly created continuous aggregates will have this real-time query behavior, although it can be enabled on existing continuous aggregates with a configuration setting as follows: ALTER VIEW continuous_view_name SET (timescaledb.materialized_only=false); This release also moves several data management lifecycle features to the Community version of TimescaleDB (from Enterprise), including data reordering and data retention policies. **Major Features** * #1456 Add support for PostgreSQL 12 * #1685 Add support for real-time aggregation on continuous aggregates **Bugfixes** * #1665 Add ignore_invalidation_older_than to timescaledb_information.continuous_aggregates view * #1750 Handle undefined ignore_invalidation_older_than * #1757 Restrict watermark to max for continuous aggregates * #1769 Add rescan function to CompressChunkDml CustomScan node * #1785 Fix last_run_success value in continuous_aggregate_stats view * #1801 Include parallel leader in plan execution * #1808 Fix ts_hypertable_get_all for compressed tables * #1828 Ignore dropped chunks in compressed_chunk_stats **Licensing changes** * Reorder and policies around reorder and drop chunks are now accessible to community users, not just enterprise * Gapfill functionality no longer warns about expired license **Thanks** * @t0k4rt for reporting an issue with parallel chunk append plans * @alxndrdude for reporting an issue when trying to insert into compressed chunks * @Olernov for reporting and fixing an issue with show_chunks and drop_chunks for compressed hypertables * @mjb512 for reporting an issue with INSERTs in CTEs in cached plans * @dmarsh19 for reporting and fixing an issue with dropped chunks in compressed_chunk_stats " 88773323f4d06cd7565f89d4bfe2cb49bfebd9d1,Derek Marsh,Fri Apr 10 22:43:09 2020 -0500,Ignore dropped chunks in compressed_chunk_stats, 57327b52c8baff58a89cff209a398316b700de07,Sven Klemm,Wed Apr 15 17:08:10 2020 +0200,Update CHANGELOG, 1f37c1813f8d56c151d9f14f18a6dc7bf6d26cc3,Mats Kindahl,Thu Apr 16 09:25:57 2020 +0200,Fix null pointer deref in continuous aggregates,There is a potential null pointer dereference in that `raw_hypertable` might be NULL when counting the number of continuous aggregates attached. This commit fixes this by assuming that no continuous aggregates are attached if `raw_hypertable` is NULL. Found using `clang-tidy`. e7f70e354e2f798785914396834c98a6f274c29b,Oleg Smirnov,Fri Apr 10 11:26:39 2020 +0300,Fix ts_hypertable_get_all for compressed tables,When calling show_chunks or drop_chunks without specifying a particular hypertable TimescaleDB iterates through all existing hypertables and builds a list. While doing this it adds the internal '_compressed_hypertable_*' tables which leads to incorrect behaviour of ts_chunk_get_chunks_in_time_range function. This fix filters out the internal compressed tables while scanning at ts_hypertable_get_all function. be47dd0020821ce3d35e0b4c2e7f5af727a1e632,Sven Klemm,Wed Apr 15 11:12:35 2020 +0200,Simplify postgres version check, c5e983e4226a0a8849e9b4ec24bc27ec8c3e36d9,Sven Klemm,Tue Apr 14 15:43:50 2020 +0200,Add test for INSERT in cached plans,This adds a test for INSERTs with cached plans. This test causes a segfault before 1.7 but was fixed independently by the refactoring of the INSERT path when adding PG12 support. 5b5240c00cee188e91a2d22c060264f88f63c3d7,Michael J. Freedman,Fri Mar 27 11:26:05 2020 -0400,Point to Timescale Code of Conduct,Create file that points to Timescale Code of Conduct on webpage 3efc59e8ac2be05d55b30d58fd37901b2211cab7,Ruslan Fomkin,Tue Apr 14 17:35:11 2020 +0200,Update change log with support for PG12, fc92651738f7fce219ba4c33d42a78ec41523942,Ruslan Fomkin,Wed Apr 8 12:20:55 2020 +0200,Fix uninitialized warnings due to different scans,Fixes uninitialized warnings due to disjoint paths of scans. eaf0211012e9b3abc725086ed550bef51d70bf5d,Ruslan Fomkin,Tue Apr 7 11:39:47 2020 +0200,Run coverity test on PG11 and PG12,Changes coverity test to run on PostgreSQL 11 and 12 instead of 9.6. 777606263887ab47c263f73bfc1a70591c972ad9,Mats Kindahl,Mon Apr 6 14:45:28 2020 +0200,Remove pull of TSDB dev tools,"The memory leak job pulls TSDB dev tools from bitbucket but only two files from that repository are needed. This commit copy the files from the `tsdb-dev-tools` repository, remove the need to clone the repository, update the memory leak job to use these files, and remove the two secrets containing the environment variables `USR` and `PASSWORD`. " bd9a755298094082775855da90727bb9f8521c69,Ruslan Fomkin,Tue Apr 7 13:49:12 2020 +0200,Remove unnecessary null check,"Removes NULL check on variable, which has already been used. Fixes an issue reported by coverity scan. " 5df7946e8883dca361b995bd197c5f19f34f67d1,Ruslan Fomkin,Tue Apr 7 10:24:53 2020 +0200,Make code logic clear for coverity test,Homogenizes condition for using correct scan method and resolving coverity test complain. dd753f2a3c6569f48893d37b7fed39dc5a094c30,Erik Nordström,Tue Apr 7 10:46:45 2020 +0200,Remove INSTEAD OF trigger in copy path,"This change removes some place-holder code for supporting `INSTEAD OF` triggers in the code that implements `COPY` support on hypertables. The code was introduced when updating the copy path with PostgreSQL 12 changes. However, such triggers are only supported on views, and, since the hypertable copy path only inserts on chunks, the code isn't needed. " 5cc91e34dedb7ce07eee8558d669c36ca8ce749a,Ruslan Fomkin,Fri Apr 3 14:26:56 2020 +0200,Add CRON jobs for PG12,Adds several jobs to run on PG12 as they run on all other PG major versions in CRON builds. Fixes 32-bits jobs. 21d81ff60227e1a6204e8bcd84fde063856d60e3,Ruslan Fomkin,Thu Apr 2 20:36:56 2020 +0200,Add test to contrast MATERIALIZED in WITH queries,"PG12 by default optimizes CTEs from WITH clause with the rest of the queries. MATERIALIZED is used in rowsecurity tests to produce the same query plans as in PG11. This commit adds query plans tests with default behavior, which is equivalent to NOT MATERIALIZED. " a0d320ae2ffac23c94c3878a47443de33ae2a271,Ruslan Fomkin,Fri Apr 3 13:29:59 2020 +0200,Run CodeCov on PG12 for every PR,Switching mandatory CodeCov test from PG11.0 to PG12.0. 6724b132bb31c51f489c685ddafa631af074d1f3,Ruslan Fomkin,Fri Apr 3 14:49:01 2020 +0200,Fix version comparison in CMake to work on 9.6.6,CMake version in PG image 9.6.6 doesn't support VERSION_GREATER_EQUAL in its CMake version. Fixing with two separate conditions. 3b88e10d513cc587bcb1da28a131e072f256f83a,Sven Klemm,Fri Apr 3 14:51:28 2020 +0200,Add ordered_append test output for PG12,This patch also fixes one query in the ordered append test that had non-deterministic output. d31184b13ae8dffa28f9c41de574e98295daf4bb,Sven Klemm,Thu Apr 2 14:37:04 2020 +0200,Fix constraint propagation for PG12,Since hypertable expansion happens later in PG12 the propagated constraints will not be pushed down to the actual scans but stay as join filter. This patch adds the constraint to baserestrictinfo or as joinfilter depending on the constraint type. 8f84e35e5d02339c1a8682fd1cb7014235be21c5,Ruslan Fomkin,Thu Apr 2 14:48:15 2020 +0200,Run rowsecurity test on PG12,"Runs a modification of rowsecurity test on PG12. The differences are: - WITH OIDS is removed from PG12. - OID column is replaced with CTID in queries, which is expected to be stable enough. - MATERIALIZED is used on WITH to get the same plan. - Detail of an error message is slightly different in PG12. Improved ORDER BY in number of queries in PG11 and PG12 tests to avoid result permutations. " b340c4118efdfca0cf83e16ffdeb4013d9de6c99,Ruslan Fomkin,Thu Apr 2 14:51:13 2020 +0200,Run vacuum_multi test also on PG12,Moves vacuum_multi test from running just on PG11 to be executed on PG11 and higher. 94af6b88cd21d3b93aeb00ec5f5b9ce17aac7504,Sven Klemm,Thu Apr 2 13:17:21 2020 +0200,Update append-12 output after rebase,"Update the PG12-specific append test. Earlier, the test produced a different chunk order in plans across PostgreSQL versions because the ordering of the inserted data was not deterministic. The ordering was made deterministic and this change updates the PG12 output after these changes, now making the plan output consistent with other PostgreSQL versions. " 0be5cb4d9ddd4a227afbb6af92aeea8e90947725,Erik Nordström,Thu Mar 26 09:10:40 2020 +0100,Improve target list handling for INSERTs,"`INSERT`s on a hypertable requires wrapping the top-level `ModifyTable` plan node with a `CustomScan` node (`HypertableInsert`) that setups the tuple routing that occurs during execution. However, using `CustomScan` nodes at the top-level requires implementing a number of work-arounds for assumptions made in the planner on how top-level nodes deal with targetlists. The existing way this was handled wasn't robust enough, and problems occur when enabling JIT compilation for expression evaluation. This change improves the handling to work also for JIT. Intuitively, our `HypertableInsert` node should adopt the target list of the `ModifyTable` subplan without further projection. For a `CustomScan` this means setting the ""input"" `custom_scan_tlist` to the `ModifyTable`'s target list and having an ""output"" `targetlist` that references the `TupleDesc` that is created from the `custom_scan_tlist` at execution time. Now, while this seems straight-forward, there are several things with how `ModifyTable` nodes are handled in the planner that complicates this: - First, `ModifyTable` doesn't set a `targetlist` when the node is created. It is only set later in `set_plan_references` (`setrefs.c`) if there's a `RETURNING` clause. Thus, there's no `targetlist` available when the `HypertableInsert` plan is created. - Second, top-level plan nodes, except for `ModifyTable` nodes, need to have a `targetlist` matching `root->processed_tlist`. This is asserted in `apply_tlist_labeling`, which is called in `create_plan` (`createplan.c`) immediately after the `HypertableInsert` plan node is created. `ModifyTable` is exempted because it doesn't always have a `targetlist` that matches `processed_tlist`. So, even if we had access to `ModifyTable`'s targetlist at plan creation we wouldn't be able to use it since the `HypertableInsert` is a `CustomScan` and thus not exempted. - Third, a `CustomScan`'s `targetlist` should reference the attributes of the `TupleDesc` that gets created from the `custom_scan_tlist` at the start of execution. This means we need to make the `targetlist` into all `Var`s with attribute numbers that correspond to the `TupleDesc` instead of result relation in the `ModifyTable`. To get around these issues, we set the `CustomScan`'s `targetlist` to `root->processed_tlist` when we create the plan node, and at the end of planning when the `ModifyTable`'s `targetlist` is set, we go back and fix up the `CustomScan`'s `targetlist`. " afa003007065da322ebd69ed56bb80b427974162,Erik Nordström,Wed Apr 1 18:06:10 2020 +0200,Add real-time aggregate test for PG12,This change adds the missing PG12-specific output file for the `continuous_aggs_union_view` test. 72253ac222448df93eb2c6c4e0398a2ba07e7606,Mats Kindahl,Wed Apr 1 13:02:04 2020 +0200,Add debug printout for malformed telemetry,"If the telemetry response is malformed, strange errors will be generated in the log because the use of `DirectFunctionCall2` expect the result of function calls to not be NULL and will throw an error if it is not. By printing the response in the log we can debug what went wrong. " f947e865e209dd1d48cffa7d058a5f400095fc3d,Mats Kindahl,Wed Apr 1 09:29:30 2020 +0200,Add tests for telemetry response parsing,"The telemetry response processing is handled in the function `process_response`, which was an internal function and cannot be tested using unit tests. This commit rename the function to follow conventions for extern functions and add test functions and tests to check that it can handle well-formed responses. No tests for malformed responses are added since the function cannot currently handle that. " d0cde563c5083e55177c592f9f5b27484fdd1d51,Mats Kindahl,Fri Mar 27 14:16:21 2020 +0100,Add collation when comparing versions,"When comparing versions in the telemetry module when processing the HTTP response, a direct call of `texteq` is made, but without using a collation. This generate unnecessary errors in the log and also cause the telemetry job to abort. This commit fixes that by using the ""C"" collation when comparing the version strings. " e2da5606cf649a47bf933cb2e107661ce10d8e64,Dmitry Simonenko,Wed Apr 1 14:15:44 2020 +0300,Add test for jit compilation support,This test also covers a failure case descibed in the issue #1262. 21ebd9e68f23896450ffe6da7d3f6230c844c1ba,Erik Nordström,Tue Mar 31 16:52:07 2020 +0200,Support custom table access methods,"PG12 introduced support for custom table access methods. While one could previously set a custom table access method on a hypertable, it wasn't propagated to chunks. This is now fixed, and a test is added to show that chunks inherit the table access method of the parent hypertable. " 1f5b463acf87b71820e20f7c1e801db8cf4d8f53,Brian Rowe,Mon Mar 30 17:05:45 2020 -0700,Make query test versioned and add PG12 output,"This change modifies the query test to use the test template mechanism so that we can capture the plan differences introduced by Postgres 12 pruning append nodes. The PG12 output file also contains a plan change which seems to result in a GroupAggregate being replace by a less efficient Sort plus HashAggregate. This new behavior is still being investigated, but it is still a correct plan (just potentially suboptimal). " 38c9f17a0e1864d2be0ff54ea42d08f1c126078e,Mats Kindahl,Mon Mar 30 13:50:52 2020 +0200,Fix error in compat/CMakeList.txt,"The `CMakeLists.txt` in `src/compat` was conditionally reusing the variable `SOURCES` inherited from above, which contained relative paths not related to this directory. This caused `target_sources` to generate a warning. This commit fixes this by setting `SOURCES` to be empty first in the file. " 082a9ffedca88926865061ec44e9e5f86cb9fbef,gayyappan,Thu Mar 26 10:54:37 2020 -0400,Add PG12 specific output for isolation tests,PG12 isolation tests have labelled output - the label being the session name. Add .out files for PG12 16897d223842bdd5a1997f69c0503c3ff0c095ee,Ruslan Fomkin,Fri Mar 13 09:21:25 2020 +0100,Drop FK constraints on chunk compression,Drop Foreign Key constraints from uncompressed chunks during the compression. This allows to cascade data deletion in FK-referenced tables to compressed chunks. The foreign key constrains are restored during decompression. c3d1c51054705a7bede65e95ea3ae2d2cd40fa23,Brian Rowe,Thu Mar 19 16:09:17 2020 -0700,Fix multi_transaction_index test on PG12,This test changes the multi_transaction_index test to sort the chunk_index selection commands by index_name. This fixes an issue where the order would vary depending on the version of postgres. This also made the multi_transaction_index test a test template so that we can capture the explain difference for a plan that has an append node pruned in PostgresQL 12. ed32d093dc8c9dbcc53d6394d98c5d865c90dea6,Ruslan Fomkin,Mon Mar 23 18:20:38 2020 +0100,Use table_open/close and PG aggregated directive,Fixing more places to use table_open and table_close introduced in PG12. Unifies PG version directives to use aggregated macro. b27e883edd1f837a581db8539268cc727ed0af27,Brian Rowe,Wed Mar 18 16:25:06 2020 -0700,Fix some failing isolation tests for PG12,"This just addresses some minor test changes for PostgresQL version 12. Specifically it changes the tests to set client_min_message to error, as fatal is no longer a valid setting here. This also hides a new NOTICE message in bgw_job_delete, which was including a PID and was thus nondeterministic. " f0b396eec2bc58c7985a9a33a5f92c18fb7b07cf,Mats Kindahl,Thu Mar 19 09:06:13 2020 +0100,Remove warnings about unused variables,"Some variables are not used any more in PG12, so it generates warnings. This commit remove the variables from the code and also all uses of the variables. " f633b1926670357525d45a851144291900dbfa17,Erik Nordström,Wed Mar 18 12:28:47 2020 +0100,Make adaptive chunking test work for PG12,"PostgreSQL 12 introduced space optimizations for indexes, which caused the adaptive chunking test to fail since its measure of chunk size includes indexes that now report different sizes. To fix this, the adaptive chunking test now has version-specific output files. " 54c92562569bf044c1abefb466247856e1a73ab6,Brian Rowe,Fri Mar 13 16:58:23 2020 -0700,Add version specific golden files to some tests,"This change moves the custom_type, ddl, ddl_single, insert, and partition tests under test/sql, as well as the move and reorder tests under /tsl/test/sql to our test template framework. This allows them to have different golden files for different version of Postgres. With the exception of the reorder test, all of these tests produced new output in PG12 which only differed by the exclusion of append nodes from plan explanations (this exclusion is a new feature of PG12). The reorder test had some of these modified plans, but also included a test using a table with OIDs, which is not valid in PG12. This test was modified to allow the table creation to fail, and we captured the expected output in the new golden file for PG12. " 09616da90eb46bd471de7484912a1be469450b42,Brian Rowe,Wed Mar 4 16:18:35 2020 -0800,Add support for PG12 COPY WHERE statements,PG12 allows users to add a WHERE clause when copying from from a file into a table. This change adds support for such clauses on hypertables. Also fixes a issue that would have arisen in cases where a table being copied into had a trigger that caused a row to be skipped. d747e8d68af8d9110c62e661b106e3f3e21f4a7c,Erik Nordström,Tue Mar 10 12:04:58 2020 +0100,Make pg_dump tests use wrapper scripts,"The `pg_dump` command has slightly different informational output across PostgreSQL versions, which causes problems for tests. This change makes sure that all tests that use `pg_dump` uses the appropriate wrapper script where we can better control the output to make it the same across PostgreSQL versions. Note that the main `pg_dump` test still fails for other reasons that will be fixed separately. " a04fc4b570e5e35edeb1d042fe934cc9f6045fc2,Erik Nordström,Mon Mar 9 15:27:23 2020 +0100,Make cluster test have predictible output,"When `CLUSTER` is run in verbose mode on a hypertable, it prints the chunks that are being clustered. The ordering of these chunks differs in PG12 compared to previous versions, which caused the 'cluster' test to fail. The chunk index mappings that decide which chunks to cluster is now sorted on chunk OID to make the output predicitible across PostgreSQL versions. This comes at a slight cost, however, but shouldn't be much overhead if one considers that the actual clustering is a lot more costly. One could optionally do the sorting only in case ""verbose"" output is requested, but this doesn't seem worthwhile as it requires additional code logic to handle both cases. It can always be done later as an optimization. " 72d52bcbd6aa8d66286165af7c4d1eb6ee1fd38b,Erik Nordström,Mon Mar 9 12:57:54 2020 +0100,Avoid constraint-aware append with one child,"PostgreSQL 12 removes Append and MergeAppend nodes when there is only one child node to scan. This removal happens when creating the plan from the selected path, which confused the constraint-aware append node since it expected an Append as child. To fix this issue, constraint-aware append is no longer applied when there is only one child in the append. Note that ChunkAppend is not affected since it completely replaces the Append path, thus PostgreSQL won't remove the Append since it is no longer there. In addition, this change updates expected file for the append test for PG12, since it didn't seem to be updated. " 47c5fa3a1893cab5fc2f6e353bee7fa746859969,Dmitry Simonenko,Mon Mar 9 15:01:41 2020 +0300,Block generated columns for hypertable partitioning,PG12 introduced new feature which allows to define auto generated columns. This PR adds check which prevents using such columns for hypertable partitioning. 7351c5d377fd005f6674d7a5e5eea13002f47ee0,Erik Nordström,Fri Mar 6 15:08:16 2020 +0100,Make DDL hook test predictable across PG versions,"The `ddl_hook` test failed on PG12 because dropped objects in the DDL hook had a different order compared to previous versions. This change therefore adds a sort of the dropped objects list before processing so that the order objects when printed in the test is predictable. The ordering happens on object type, which assumes the order of objects with the same type is predictable. This seems sufficient for the test to pass. " dad7eaa7905900429723c043799bc764213d521c,Erik Nordström,Fri Mar 6 15:42:20 2020 +0100,Update gapfill test for PG12,The `plan_gapfill` test for PG12 wasn't updated after having removed the notice about expired license that was previously in the test. This change updates the test to account for removal of the license message. dfcb6afb39ed2a9b91b0e4dd5029088f14f37918,Ruslan Fomkin,Thu Mar 5 22:17:08 2020 +0100,Move out WITH OIDS test to be run on PG12_LT,"WITH OIDS option doesn't exist in PG12. This fix moves the test of blocking compression for tables WITH OIDS into a separate file, which is run only for PG version before 12. " 5432cd043de67ac190d264741d7f63fa0c65506c,Erik Nordström,Thu Mar 5 17:58:30 2020 +0100,Fix warning of unused value used for assertion,This change quenches a warning about an unused variable when assertions aren't enabled. 21dc7b9f7bee23153845662e06af2f19122bfd0a,Erik Nordström,Thu Mar 5 15:35:12 2020 +0100,Make tests using pg_dump work for PG12,"PostgreSQL 12 changed the log level in client tools, such as `pg_dump`, which makes some of our tests fail due to different log level labels. This change filters and modifies the log level output of `pg_dump` in earlier PostgreSQL versions to adopt the new PostgreSQL 12 format. " 313507c9c9adaafe5e4cc572961e9c7a377ba5ba,Erik Nordström,Thu Mar 5 16:36:04 2020 +0100,Set backwards compatible float rounding for tests,"PostgreSQL 12 changed how rounding of floating point values are displayed by default. This caused different output for PostgreSQL 12 in some tests. Setting `extra_float_digits=0` in our test configuration adopts the old behavior for all PostgreSQL versions, thus fixing the test output issues. " d9cada4e99035716eea07fbf6b6c2d2d7158fa1b,Erik Nordström,Thu Mar 5 12:03:06 2020 +0100,Move planner imports to import directory,This change moves `planner_import.c` to the `import` subdirectory and renames it to `planner.c`. The code can now be included as `import/planner.h`. 36af23ec94028cc5103ef1ae14fa7c7fc90e737d,Erik Nordström,Wed Mar 4 12:54:01 2020 +0100,Use flags for cache query options,"Cache queries support multiple optional behaviors, such as ""missing ok"" (do not fail on cache miss) and ""no create"" (do not create a new entry if one doesn't exist in the cache). With multiple boolean parameters, the query API has become unwieldy so this change turns these booleans into one flag parameter. " afb4c7ba51a195e70307aee1b0035f9131133a40,Erik Nordström,Tue Feb 25 12:47:46 2020 +0100,Refactor planner hooks,"This change refactors our main planner hooks in `planner.c` with the intention of providing a consistent way to classify planned relations across hooks. In our hooks, we'd like to know whether a planned relation (`RelOptInfo`) is one of the following: * Hypertable * Hypertable child (a hypertable can appear as a child of itself) * Chunk as a child of hypertable (from expansion) * Chunk as standalone (operation directly on chunk) * Any other relation Previously, there was no way to consistently know which of these one was dealing with. Instead, a mix of various functions was used without ""remembering"" the classification for reuse in later sections of the code. When classifying relations according to the above categories, the only source of truth about a relation is our catalog metadata. In case of hypertables, this is cached in the hypertable cache. However, this cache is read-through, so, in case of a cache miss, the metadata will always be scanned to resolve a new entry. To avoid unnecessary metadata scans, this change introduces a way to do cache-only queries. This requires maintaining a single warmed cache throughout planning and is enabled by using a planner-global cache object. The pre-planning query processing warms the cache by populating it with all hypertables in the to-be-planned query. " f284a691b2dc409c0c5ed0929e687d81b5bae695,Erik Nordström,Mon Feb 24 21:58:39 2020 +0100,Avoid extra tuple table slots in CustomScan nodes,"The `CustomScan` node is oddly hard-coded to use fixed tuple table slots in its initialization, thus expecting both the in and out slots (scan and result slots) to always have the same type (underlying ops). However, when implementing an append-type node on `CustomScan`, the children that supply tuples can have slots of any type. In PG12, the regular `Append` node just passes on these slots, signaling this by setting `resultopsfixed = false`. This change adopts the behavior of `Append` for out custom `ChunkAppend` and `ConstraintAwareAppend` nodes. An alternative would be to copy the contents of the child's slot into the `CustomScan`'s own scan slot to get it to the ""expected"" type. But this would require materializing the tuple, which seems unnecessary. " 7a0d6e7c23e88c739f68f19d9524acb2413ef647,Ruslan Fomkin,Tue Feb 25 12:41:37 2020 +0100,Fix compression order in a test,Add missing ORDER BY clause to sort chunks for compression in deterministic order for the constraint exclusion test. cc5a2764c51f4443fde7fc6294686433df31347e,Ruslan Fomkin,Tue Feb 25 11:24:20 2020 +0100,Add expected output for cagg query 12,Expected output for cagg query 12 is different to 11 due to removed unnecessary nodes in the query plans. Also the cost is slightly different between the query plans of 12 and 11. fe87ac40bea4232df4a4b66b1795d42dc56bd9b1,Ruslan Fomkin,Wed Feb 12 11:36:51 2020 +0100,Replace ExecBRInsertTriggersCompat with fixed code,"PG12 implementation of ExecBRInsertTriggers changes given slot in place and return Boolean indicating success, which is different from earlier implementations where it returned new slot value. This commit fixes a bug in the compatability macro and its usage. " d2d5a484cedb8bb0e3f80c7e04c6fa9841427df7,Erik Nordström,Mon Feb 24 12:01:31 2020 +0100,Fix whitespace error in parallel tests,This change fixes a whitespace error in the test reference files for the `parallel` test in PostgreSQL 9.6 and 10. e97216b36e7ab48c10913b39c71dd22b3252aaf5,Erik Nordström,Mon Feb 24 11:06:49 2020 +0100,Only compile planner imports for PG12,"New planner code was imported for PG12, but this code isn't needed for other versions and resulted in compilation errors. This change fixes these errors by not compiling the code for versions where it is not needed. " 6d12b89b1e44d40febf2bce20100d9b21767e13f,Erik Nordström,Fri Feb 21 15:57:40 2020 +0100,Refactor adaptive chunking scan code for PG12,This change adopts the PG12 table/index scan API for the adaptive chunking functions that scan for min/max values in a table. PostgreSQL versions older than 12 use compatibility functions and wrappers for the new API. 053040e3fd41a2bb9225edd7990f47c6a209063d,Erik Nordström,Thu Feb 20 13:27:11 2020 +0100,Move planner imports to separate directory,A lot of planner code was imported to support hypertable expansion in PG12. This code is now moved to a `import` directory to avoid having this code mix with regular TimescaleDB code. 1304b59d9a6c2352a7e7b1bd6077ce637009ddbf,Erik Nordström,Thu Feb 13 13:27:03 2020 +0100,Avoid creating range table entry for chunks during insert,"It is not necessary to create a new range table entry for each chunk during inserts. Instead, we can point to the range table entry of the hypertable's root table. " 176b616e372a0dfa955f946569b225864c01c4fc,Erik Nordström,Wed Feb 5 08:28:03 2020 +0100,Refactor insert and copy paths,"The INSERT and COPY paths have been refactored to better handle differences between PostgreSQL versions. In particular, PostgreSQL 12 introduced the new table access mangement (AM) API which ties tuple table slots to specific table AM implementations, requiring more careful management of those data structures. The code tries to adopt the new (PG12) API to the extent possible, providing compatibility layers and functions for older PostgreSQL versions where needed. " 68e64f654012304cc9d1931fc7f1af408a2bcda2,Ruslan Fomkin,Fri Feb 14 13:45:57 2020 +0100,Refactor calculating count in GetLockConflicts,"Refactors implementation of calculating the result count in GetLockConflictsCompat in PG 11 and earlier, so it follows the implementation as in PG 12. " 9a3211f3aa415b97b508629e7e1c28dada4a36e8,Ruslan Fomkin,Tue Feb 11 15:03:59 2020 +0100,Refactor chunk append,"Improve readability of code, which takes care of pruned children in PG12. Abstract away difference in a function cardinality between PG96 and later. " 2c7b42d0445f421cd88733d3728792617a2cbc60,Ruslan Fomkin,Wed Feb 12 09:57:45 2020 +0100,Fix and refactor small issues,Replace relation_close with appropriate specific functions. Remove todo comment. Remove unnecessary headers. Capitalize new macro. Remove code duplication in compatibility macros. Improve comment. 1ddc62eb5f2691874c0344e028ae93396ac58a3a,Ruslan Fomkin,Mon Feb 10 11:39:11 2020 +0100,Refactor header inclusion,"Correcting conditions in #ifdefs, adding missing includes, removing and rearranging existing includes, replacing PG12 with PG12_GE for forward compatibility. Fixed number of places with relation_close to table_close, which were missed earlier. " e57ee45fcf1ef2e6f06f9b474a0d1a4be41f65b0,Ruslan Fomkin,Thu Feb 6 17:29:59 2020 +0100,Replace general relation_open with specific,"relation_open is a general function, which is called from more specific functions per database type. This commit replaces them with the specific functions, which control correct types. " 89d32c0682fd4da22adac000028a402b7fe85bb2,Ruslan Fomkin,Thu Feb 6 17:41:58 2020 +0100,Fix ts prefix to export functions,One exported function is fixed with ts_ prefix as required. bddcf2a78ae7ee819cede8c20fdde4d5b90ae537,Ruslan Fomkin,Thu Feb 6 15:17:56 2020 +0100,Enable collation test with compression,Enables back the test of collation error as part of the compression test. 9b0104f57fb16ded986f007c37d407670fc95558,Ruslan Fomkin,Thu Feb 6 11:59:57 2020 +0100,Replace heap_open with table_open for RangeVar,Replaces heap_openrv with table_openrv for PG12 and routes to heap_openrv for earlier versions of PG. c71c1c3e7be46f345f6270b1fd456d26093c2568,Ruslan Fomkin,Thu Feb 6 10:26:30 2020 +0100,Use heap_close for table_close in PG12_LT,Defines table_close as heap_close for PG versions before PG12. 489bff2a6d2541dbe27b37ee3b1a9abbdac7699b,Ruslan Fomkin,Thu Feb 6 10:31:00 2020 +0100,Add Travis job for PG12,Adds a Travis job to run regression tests on PG12.1. 949b88ef2eafded2472275e8a0107261e2135535,Joshua Lockerman,Wed Nov 6 15:41:06 2019 +0100,Initial support for PostgreSQL 12,"This change includes a major refactoring to support PostgreSQL 12. Note that many tests aren't passing at this point. Changes include, but are not limited to: - Handle changes related to table access methods - New way to expand hypertables since expansion has changed in PostgreSQL 12 (more on this below). - Handle changes related to table expansion for UPDATE/DELETE - Fixes for various TimescaleDB optimizations that were affected by planner changes in PostgreSQL (gapfill, first/last, etc.) Before PostgreSQL 12, planning was organized something like as follows: 1. construct add `RelOptInfo` for base and appendrels 2. add restrict info, joins, etc. 3. perform the actual planning with `make_one_rel` For our optimizations we would expand hypertables in the middle of step 1; since nothing in the query planner before `make_one_rel` cared about the inheritance children, we didn’t have to be too precises about where we were doing it. However, with PG12, and the optimizations around declarative partitioning, PostgreSQL now does care about when the children are expanded, since it wants as much information as possible to perform partition-pruning. Now planning is organized like: 1. construct add RelOptInfo for base rels only 2. add restrict info, joins, etc. 3. expand appendrels, removing irrelevant declarative partitions 4. perform the actual planning with make_one_rel Step 3 always expands appendrels, so when we also expand them during step 1, the hypertable gets expanded twice, and things in the planner break. The changes to support PostgreSQL 12 attempts to solve this problem by keeping the hypertable root marked as a non-inheritance table until `make_one_rel` is called, and only then revealing to PostgreSQL that it does in fact have inheritance children. While this strategy entails the least code change on our end, the fact that the first hook we can use to re-enable inheritance is `set_rel_pathlist_hook` it does entail a number of annoyances: 1. this hook is called after the sizes of tables are calculated, so we must recalculate the sizes of all hypertables, as they will not have taken the chunk sizes into account 2. the table upon which the hook is called will have its paths planned under the assumption it has no inheritance children, so if it's a hypertable we have to replan it's paths Unfortunately, the code for doing these is static, so we need to copy them into our own codebase, instead of just using PostgreSQL's. In PostgreSQL 12, UPDATE/DELETE on inheritance relations have also changed and are now planned in two stages: - In stage 1, the statement is planned as if it was a `SELECT` and all leaf tables are discovered. - In stage 2, the original query is planned against each leaf table, discovered in stage 1, directly, not part of an Append. Unfortunately, this means we cannot look in the appendrelinfo during UPDATE/DELETE planning, in particular to determine if a table is a chunk, as the appendrelinfo is not at the point we wish to do so initialized. This has consequences for how we identify operations on chunks (sometimes for blocking and something for enabling functionality). " cbda1acd4f656130b58a6544e018f4f4b1df9f5e,Sven Klemm,Mon Apr 13 11:39:29 2020 +0200,Record cagg view state in catalog,Record materialized_only state of continuous aggregate view in catalog and show state in timescaledb_information.continuous_aggregates. 3e03ca0b02c9ac1dae885919d28a4e5faa0b33c1,Sven Klemm,Mon Apr 6 22:33:17 2020 +0200,Include parallel leader in plan execution,We have to let the leader participate in parallel plan execution if for some reason no parallel workers were started. This commit also changes the parallel EXPLAINs to not run with ANALYZE because the output is not stable as it depends on worker assignment. 403782a5899c754db67c41ea859cf34611fa5bde,Ruslan Fomkin,Tue Mar 24 21:13:52 2020 +0100,Run regression test on latest PG versions,"Docker images are build with latest versions of PostgreSQL, thus updating regression tests to run on the latest PG versions. Fixing authentication for PG docker images, since it is required after a recent change in docker-library/postgres@42ce743. Running isolation tests on new versions of PG produces additional output: - Notifications are not missed to print postgres/postgres@ebd4992 - Notifications are prefixed with session name postgres/postgres@a28e10e - Timeout cancellation is printed in isolation tests postgres/postgres@b578404 The expected outputs are modified to succeed on latest PG versions, while the affected isolation tests are disabled for earlier versions of PG. " c63685632207646b55fda7b7b97c7320a6645cb5,Sven Klemm,Wed Apr 1 19:10:30 2020 +0200,Fix use-after-free in cagg union view creation, d9cad86dc5263db0efe5b3082b22295cda0e50af,Sven Klemm,Wed Apr 1 13:51:47 2020 +0200,Add missing cast to cagg union view,The view definition for the union view was missing a cast from int8 to the actual int type used for hypertables with int partitioning column this was not a problem on 64 bit systems but on 32 bit systems int4 and int8 are represented differently. 2f6a7d5e28ca4d38af56338b2a0142f5045f4c17,Erik Nordström,Wed Apr 1 08:59:47 2020 +0200,Fix use of uninitialized variables in cont. aggs,"The function to get the materialization end point for a continuous aggregate could use uninitialized min and max time values in its calculations when a hypertable has no data. This change ensures that the min and max time is initialized to `INT64_MIN` and `INT64_MAX`, respectively, if no min and max values are found. This will mute warnings on some compilers. " 2ae4592930be1df40cc490bfc6e8ef3f18af6b4e,Sven Klemm,Fri Jan 24 11:09:59 2020 +0100,Add real-time support to continuous aggregates,This PR adds a new mode for continuous aggregates that we name real-time aggregates. Unlike the original this new mode will combine materialized data with new data received after the last refresh has happened. This new mode will be the default behaviour for newly created continuous aggregates. To upgrade existing continuous aggregates to the new behaviour the following command needs to be run for all continuous aggregates ALTER VIEW continuous_view_name SET (timescaledb.materialized_only=false); To disable this behaviour for newly created continuous aggregates and get the old behaviour the following command can be run ALTER VIEW continuous_view_name SET (timescaledb.materialized_only=true); baaa41bf91c6481c9d06cacb72f9fd083638dc1e,gayyappan,Mon Mar 30 14:51:52 2020 -0400,Modify continuous_aggregate_stats view definition,"last_run_success value is reset when a job is started. So mask the value if the status of a job is running, otherwise it will show an incorrect state. Fixes #1781 " ce2237accd1a34773082c063826a38e0869f95a0,Mats Kindahl,Wed Mar 25 14:18:05 2020 +0100,Treat dropped chunks as not found,"This commit fixes `chunk_scan_find` to not return chunks that are marked as dropped, since the callers don't expect such chunks to be returned as found. " 19f417ce92ad504339f3c832b3af625da8f34754,gayyappan,Thu Mar 26 15:25:02 2020 -0400,Fix compiler error,Fix uninitialized variable error with cc for materialize.c. ce624d61d37c3ddffd07345380d39c79b6d62811,gayyappan,Thu Mar 12 10:28:27 2020 -0400,Restrict watermark to max for continuous aggregates,"Set the threshold for continuous aggregates as the max value in the raw hypertable when the max value is lesser than the computed now time. This helps avoid unnecessary materialization checks for data ranges that do not exist. As a result, we also prevent unnecessary writes to the thresholds and invalidation log tables. " 9b3d2e6d52f437800e7631bd858fa6091ae02128,Sven Klemm,Tue Mar 24 17:38:19 2020 +0100,Add explicit ORDER BY to append test data generation,The queries to produce test data for space partitioned hypertables in the append test did not have an explicit ORDER BY clause leading to a different ordering for the chunks created on PG12. 039607dc1aa0f871fb578ee0a84924718d49fc43,Sven Klemm,Mon Mar 23 12:15:16 2020 +0100,Add rescan function to CompressChunkDml CustomScan node,The CompressChunkDml custom scan was missing a rescan function leading to a segfault in plans that required a rescan. d9aa40d36d03eff1230945eb92b69989e0b55c24,Sven Klemm,Sat Mar 21 18:47:59 2020 +0100,Merge 9.6 specific update test files, b38b0d05b0e8bd3dd7f8b455e34df858ec16dba1,Sven Klemm,Fri Mar 20 12:12:22 2020 +0100,Add 1.6.1 to update test scripts, 44fd22a92a61fa38426a3bc4e3c0f56695134d74,Sven Klemm,Sat Mar 21 18:28:18 2020 +0100,Add missing DROP VIEW to update script,The definition change for timescaledb_information.continuous_aggregates requires a drop view in the update script because the update might be from a version that has a incompatible view definition. 70e23d3d28ecd4c8ce185a3f351a5a73e51ab7de,gayyappan,Wed Mar 18 17:20:24 2020 -0400,Modify isolation test makefile rules,PG12 needs version specific output for some isolation tests. Modify the makefile to make it consistent with use of TEST_TEMPLATES in other regression suites and add version specific output files. cbe06c3b728c1d4312373072c6579cbe4e7b88d3,Mats Kindahl,Fri Mar 13 10:15:21 2020 +0100,Handle undefined ignore_invalidation_older_than,"If `ignore_invalidation_older_than` is undefined, it is set to maximum for `BIGINT` type. This is not handled in `continuous_aggregates` information schema so the column shows up as a very strange value. This commit fixes this by checking if `ignore_invalidation_older_than` is set to maximum, and uses `NULL` in the view in that case, which will show up as empty. " 6749cb7fa33d20e30de8d2d801168dfc197ee3ef,Ruslan Fomkin,Wed Mar 18 10:04:39 2020 +0100,Update Travis build fail notification with commit,Timescale is notified internally about failed builds on Travis. This update adds information about the last commit. d465c81e6adf8e494d6207738dde02d57eb5e9da,Mats Kindahl,Mon Mar 16 15:54:23 2020 +0100,Do not compress chunks that are dropped,"The function `get_chunks_to_compress` return chunks that are not compressed but that are dropped, meaning a lookup using `ts_chunk_get_by_id` will fail to find the corresponding `table_id`, which later leads to a null pointer when looking for the chunk. This leads to a segmentation fault. This commit fixes this by ignoring chunk that have are marked as dropped in the chunk table when scanning for chunks to compress. " e818f3af5a1228cbca91c686604f71e4914c628e,Sven Klemm,Fri Mar 13 09:45:06 2020 +0100,Release 1.6.1,"This maintenance release contains bugfixes since the 1.6.0 release. We deem it medium priority for upgrading. In particular the fixes contained in this maintenance release address bugs in continuous aggregates, time_bucket_gapfill, partial index handling and drop_chunks. **For this release only**, you will need to restart the database after upgrade before restoring a backup. **Minor Features** * #1666 Support drop_chunks API for continuous aggregates * #1711 Change log level for continuous aggregate materialization messages **Bugfixes** * #1630 Print notice for COPY TO on hypertable * #1648 Drop chunks from materialized hypertable * #1668 Cannot add dimension if hypertable has empty chunks * #1673 Fix crash when interrupting create_hypertable * #1674 Fix time_bucket_gapfill's interaction with GROUP BY * #1686 Fix order by queries on compressed hypertables that have char segment by column * #1687 Fix issue with disabling compression when foreign keys are present * #1688 Handle many BGW jobs better * #1698 Add logic to ignore dropped chunks in hypertable_relation_size * #1704 Fix bad plan for continuous aggregate materialization * #1709 Prevent starting background workers with NOLOGIN * #1713 Fix miscellaneous background worker issues * #1715 Fix issue with overly aggressive chunk exclusion in outer joins * #1719 Fix restoring/scheduler entrypoint to avoid BGW death * #1720 Add scheduler cache invalidations * #1727 Fix compressing INTERVAL columns * #1728 Handle Sort nodes in ConstraintAwareAppend * #1730 Fix partial index handling on hypertables * #1739 Use release OpenSSL DLLs for debug builds on Windows * #1740 Fix invalidation entries from multiple caggs on same hypertable * #1743 Fix continuous aggregate materialization timezone handling * #1748 Fix remove_drop_chunks_policy for continuous aggregates **Thanks** * @RJPhillips01 for reporting an issue with drop chunks. * @b4eEx for reporting an issue with disabling compression. * @darko408 for reporting an issue with order by on compressed hypertables * @mrechte for reporting an issue with compressing INTERVAL columns * @tstaehli for reporting an issue with ConstraintAwareAppend * @chadshowalter for reporting an issue with partial index on hypertables * @geoffreybennett for reporting an issue with create_hypertable when interrupting operations * @alxndrdude for reporting an issue with background workers during restore * @zcavaliero for reporting and fixing an issue with dropped columns in hypertable_relation_size * @ismailakpolat for reporting an issue with cagg materialization on hypertables with TIMESTAMP column " b7aacb673746eb72419e48d8ca40dec2e2281cd7,Mats Kindahl,Fri Mar 13 13:53:20 2020 +0100,Use current user for docker version of clang-format,"If `clang-format` with the correct usage does not exist on the machine, the `clang_format_all.sh` script is executed in a docker image. Since a `--user` is not provided, the script is executed with the wrong user, which gives wrong user for formatted files. This commit fixes this by providing the user id and group id of the current user when using the docker version of `clang-format`. " d88be2772bdda4fffc1566e4f26cf69fdc4a16f8,Ruslan Fomkin,Thu Mar 12 14:59:52 2020 +0100,Fix returning error on chunk get and exporting,"Fixes to return error in ts_chunk_get_by_relid if fail_if_not_found is true. Removes TSDLLEXPORT macro from definitions in few C files, which helps VS Code to work properly and find function definitions. " 414463e998257e457fab96370436427df42daf00,gayyappan,Wed Mar 4 16:54:43 2020 -0500,Make scheduler more resilient to failures,Invalidate cache state when jobs are detected as potentially missing. This will update the jobs list used by the scheduler. Restrict job start times to a finite interval when there are consecutive job failures. Additional logging. 4f8912f6fdcab69c8deb14011bbd084b6f155794,Mats Kindahl,Thu Mar 5 13:06:40 2020 +0100,Use release OpenSSL DLLs for debug builds,"When building TimescaleDB with the EnterpriseDB package installed, it does not include the debug versions of the DLLs, so a debug build of TimescaleDB fails. This commit fix the issue by replacing the debug DLLs in `OPENSSL_LIBRARIES` with release DLLs even for a debug build when compiling using MSVC. " 2a79baae0e4f8f5322c6e49a30a92e2fcd583eed,Mats Kindahl,Thu Mar 12 16:08:20 2020 +0100,Fix remove_drop_chunks_policy for continuous aggregates,Function `remove_drop_chunks_policy` did not work if a continuous aggregate was provided as input. This commit fixes that by looking for a continuous aggregate if a hypertable is not found. Fixes timescale/timescaledb-private#670 c61e799d1ff81a896ed2aa49cb9ee82a121a4c12,Sven Klemm,Wed Mar 11 01:40:33 2020 +0100,Fix continuous aggregate materialization timezone handling,For hypertables with a TIMESTAMP column materialization would not honor the local timezone when determining the range for materialization and instead treat times as UTC. 656bd3f6cccef72c645c3d7394ecd059a7a7edda,zcavaliero<61030067+zcavaliero@users.noreply.github.com>,Mon Feb 17 09:30:09 2020 -0500,Add logic to ignore dropped chunks in hypertable_relation_size,Function hypertable_relation_size includes chunks that were dropped which causes a failure when looking up the size of dropped chunks. This patch adds a constraint to ignore dropped chunks when determining the size of the hypertable. 474db5e44829e530769520e7ee8b0b7817df2dff,Erik Nordström,Thu Mar 12 12:40:46 2020 +0100,Fix continuous aggs DDL test on PG9.6,"The test `continuous_aggs_ddl` failed on PostgreSQL 9.6 because it had a line that tested compression on a hypertable when this feature is not supported in 9.6. This prohibited a large portion of the test to run on 9.6. This change moves the testing of compression on a continuous aggregate to the `compression` test instead, which only runs on supported PostgreSQL versions. A permission check on a view is also removed, since similar tests are already in the `continuous_aggs_permissions` tests. The permission check was the only thing that caused different output across PostgreSQL versions, so therefore the test no longer requires version-specific output files and has been simplified to use the same output file irrespective of PostgreSQL version. " a4fb0cec3f9288fe7a7a2b51eb4d64a6178d0d14,Erik Nordström,Wed Mar 11 12:38:25 2020 +0100,Cleanup compression-related errors,"This change fixes a number of typos and issues with inconsistent formatting for compression-related code. A couple of other fixes for variable names, etc. have also been applied. " 3ae9d6ca184b7b7cb4b278263f7451bf35207a6a,Sven Klemm,Mon Mar 9 12:02:31 2020 +0100,Fix flaky gapfill test,The gapfill test assumed 1 day is always <= 24 hours which is not true during DST switch leading to a failing test when run in that time. This PR fixes the test to have reproducable output even when run during DST switch. ad97d266c022df4679f0260283ccbfc62cc4d7f1,David Kohn,Thu Feb 27 10:28:59 2020 -0500,Fix restoring/scheduler entrypoint to avoid BGW death,"There was a race condition between the post_restore function restarting the background worker and the setting of the restoring flag to ""off"". If the worker started before the change to the restoring flag had been committed, it would not see the change and then die because the worker should exit when the db is in a restoring state. This modifies the post_restore function to use a restart instead of a start so that it waits on the commit to start up. It also adds logic to the entrypoint to reload config changes caused by an `ALTER DATABASE SET` command. These changes are normally only seen at connection startup but we have to wait until after our lock on the modifying transaction is released to know whether we should adopt them. " a19be04d7b646c0a6dddab0b550264380a564c47,Sven Klemm,Fri Mar 6 23:32:45 2020 +0100,Handle Sort nodes in ConstraintAwareAppend,"When a MergeAppendPath has children that do not produce sorted output a Sort node will be injected during plan creation, those plans would trigger an error about invalid child nodes in ConstraintAwareAppend. This PR makes ConstraintAwareAppend handle those plans correctly. " d1700711cf2b5688eaa7ff8c5912eb478f74388a,gayyappan,Mon Mar 9 14:54:54 2020 -0400,Fix copy of continuous agg invalidation entries,"When we copy the invalidation logs for individual continuous aggregates, the lowest-value was globally overwritten. Fix this so that the change is specific to a continuous aggregate. This bug could result in missing invalidations. " bfab289121352331bdb78bf5e6a50f4dc5bdc983,Matvey Arye,Tue Feb 11 21:59:06 2020 -0500,Handle many BGW jobs better,"This PR improves the scheduling of jobs when the number of jobs exceeds the amount of background workers. Previously, this was not a case the scheduler handled well. The basic strategy we employ to handle this case better is to use a job's next_start field to create a priority for jobs. More concretely, jobs are scheduled in increasing order of next_start. If the scheduler runs out of bgw's it waits to until bgws become available and then retries again, also in increasing next_start order. The first change this PR implements is start jobs in order of increasing next_start. We also make sure that if we run out of BGWs, the scheduler will try again in START_RETRY_MS (1 second by default). This PR also needed to change the logic of what happens when a job fails to start because BGWs have run out. Previously, such jobs were marked as failed and their next_start was reset using the regular post-failure backoff logic. But, this means that a job looses its priority every time we run out of BGWs. Thus, we changed this logic so that next_start does not change when we encounter this situation. There are actually 2 ways to run out of BGWs: 1) We run out of the timescale limit on BGWs - in this case the job is simply put back into the scheduled state, and it will be retried in START_RETRY_MS. The job is not marked started or failed. This is the common error. 2) We run out of PostgreSQL workers. We won't know if this failed until we try to start the worker, by which time the job must be in the started state. Thus if we run into this error we must mark the job as failed. But we don't change next_start. To do this we create a new job result type called JOB_FAILURE_TO_START. " 99381fc2ccaeb9855246aaaf189b6bcab400e7d3,Sven Klemm,Sat Mar 7 01:08:48 2020 +0100,Adjust index predicate attnos when creating chunk index,When the index for a chunk was created the attnos for the index predicate were not adjusted leading to insert errors on hypertables with dropped columns that had indexes with predicates. This PR adjust index predicate attnos when creating the chunk index to match the chunk attno. d3966cead3449af12f1d6b647fa7f05af1dd1f23,Mats Kindahl,Mon Mar 9 09:10:41 2020 +0100,Remove usage of uninitilized value,"Function `timescaledb_CopyFrom` created a variable `errcallback` to save away the previous error callback and restoring it afterwards. However, if `ccstate->cstate` was false, the variable would not be set and the later restore of the error callback would use a random value. This commit fixes the issue by initializing `errcallback` to all zeroes and checking if a callback has been saved away before restoring it. " 18c9cf1c0b7896eb59297451de8fb05ee41d1037,Mats Kindahl,Fri Mar 6 10:04:41 2020 +0100,"Fix typo of ""continuous""", 25eb98c0ecca09f98233bd07e164faaaa2033430,Brian Rowe,Mon Feb 24 16:55:02 2020 -0800,Prevent starting background workers with NOLOGIN,"This change will check sql commands which start a background worker on a hypertable to verify that the table owner has permission to log into the database. This is necessary, as background workers for these commands will run with the permissions of the table owner, and thus immediately fail if unable to log in. " 030443a8e2918f9aa12c52ec11396bd18e5b0389,Sven Klemm,Fri Mar 6 19:36:32 2020 +0100,Fix compressing interval columns,When trying to compress a chunk that had a column of datatype interval delta-delta compression would be selected for the column but our delta-delta compression does not support interval and would throw an errow when trying to compress a chunk. This PR changes the compression selected for interval to dictionary compression. 0cc22ad278901c07a9fe6444a00fa20d95db717b,Sven Klemm,Fri Mar 6 12:36:01 2020 +0100,Stop background worker in tests,"To make tests more stable and to remove some repeated code in the tests this PR changes the test runner to stop background workers. Individual tests that need background workers can still start them and this PR will only stop background workers for the initial database for the test, behaviour for additional databases created during the tests will not change. " 08c3d9015f9f5e8ee083eed13dc579bb65a4a382,Sven Klemm,Wed Feb 26 10:46:04 2020 +0100,Change log level for cagg materialization messages,"The log level used for continuous aggregate materialization messages was INFO which is for requested information. Since there is no way to control the behaviour externally INFO is a suboptimal choice because INFO messages cannot be easily suppressed leading to irreproducable test output. Even though time can be mocked to make output consistent this is only available in debug builds. This patch changes the log level of those messages to LOG, so clients can easily control the ouput by setting client_min_messages. " 416cf13385b313fe1ac9bd440fa20639c12f41f5,Michael J. Freedman,Fri Feb 14 17:32:14 2020 -0500,Clarify supported intervals in error msg,"Error message used to specify that interval must be defined in terms of days or smaller, which was confusing because we really meant any fixed interval (e.g., weeks, days, hours, minutes, etc.), but not an interval that is not of fixed duration (e.g., months or years). " 7de3564d188e0bde1d0fc9292eedd7cb86c0f125,gayyappan,Wed Feb 26 12:07:05 2020 -0500,Catch elog errors in scheduler code,"Direct function calls to PG functions can throw errors for bad inputs. If these are not handled when called by the scheduler, the process dies and background workers cannot be scheduled. " b363e8a37957e5d491baa78321d181e163bbf281,gayyappan,Tue Feb 25 13:11:37 2020 -0500,Fix cache invalidation code,Cache invalidation code does not call the bgw invalidate cache callback when all caches have to be invalidated. 84f0c8977e465ea0313689dc50acae27e7c095e4,Erik Nordström,Fri Feb 28 12:27:37 2020 +0100,Fix incorrect chunk exclusion and broken test,"This change fixes the `plan_expand_hypertable` test, which was broken and never ran the output comparison due to prematurely ending in an uncaught error. The test appeared to succeeded, however, since also the broken test ""expected"" files were committed to the repo. Fixing the test revealed that the query output with our optimizations enabled is incorrect for outer joins (i.e., the output from the query differed from regular PostgreSQL). Restriction clauses were too aggressively pushed down to outer relations, leading to chunk exclusion when it shouldn't happen. This incorrect behavior has also been fixed. " 8d81a837170173f5db0ba27710400ada25fc1ec0,Michael J. Freedman,Sat Feb 29 13:50:28 2020 -0500,Update copyright year to 2020, 91fe723d3aaaf88b53ffffebf8adc3e16a68ec45,gayyappan,Tue Jan 28 17:30:17 2020 -0500,Drop chunks from materialized hypertables,Add support for dropping chunks from materialized hypertables. drop_chunks_policy can now be set up for materialized hypertables. f49d00c4ec4196e4867a4eb260dd466a55aae025,Sven Klemm,Wed Feb 26 08:20:00 2020 +0100,Remove plan costs from test output,Since planning costs are not stable across platforms they should not be included in test output. ca9363af8b9d6ba998ab0de4f29af8d6f915ef1f,gayyappan,Thu Feb 20 11:40:56 2020 -0500,Fix bad plan for materialization,"REFRESH MATERIALIZED VIEW statements sometimes crash because the generated plan has a HashAgg node instead of a Partial Aggregate node when executing INSERT INTO SELECT * FROM Where the view stmt itself is along these lines SELECT time_bucket('1 day'::interval, cpu.""time"") , ...... _timescaledb_internal.partialize_agg(avg(...)) FROM " 9da50cc6861f738b15c685839f26e0a77b6be493,Erik Nordström,Mon Feb 3 15:59:32 2020 +0100,Move enterprise features to community,"As of this change, a number of enterprise features are accessible to community users. These features include: * reorder * policies around reorder and drop chunks The move chunks feature is still covered by enterprise. Gapfill no longer warns about expired enterprise license. Tests have been updated to reflect these changes. " 66f47c08574a06e17298e069372183a883360435,Lacey Butler<56133917+laceybutler@users.noreply.github.com>,Thu Feb 13 14:58:12 2020 -0800,"Update README with addt resources, build instructions to separate file","This commit updates the README with updated resources, including Timescale Cloud, new support options, release notes, and others. For readability, it also moves instructions for building from source to a separate readme file. " 8864239187fecbb0c0cc6014074359e1c11146c3,gayyappan,Mon Feb 10 18:07:45 2020 -0500,Fix segment_by var for decompress chunk node,"Fix order by queries on compressed hypertables that have char segment by column. The segment by var column for decompressed chunks should be created after setting the typmod and collation ids. Otherwise, we get failures with char datatypes while decompressing. Fixes #1650 " 565cca795a7470f29fe2c46d8d5c91ffc033b097,gayyappan,Tue Feb 11 17:21:45 2020 -0500,Support disabling compression when foreign keys are present,Fix failure with disabling compression when no compressed chunks are present and the table has foreign key constraints 2702140fa37739ee262e752b3d66beb0f212a2b8,gayyappan,Fri Jan 31 10:46:36 2020 -0500,Cannot add dimension if table has empty chunks,add_dimension should fail when table has no data but still has empty chunks. Fixes #1623 1880196ab49a366578ec29dbce901573a252d348,Mats Kindahl,Tue Feb 4 09:56:38 2020 +0100,Fix crash when interrupting create_hypertable,"When doing a migrate of table data to a hypertable during a call of `create_hypertable`, any error (including an interrupt signal) would cause the signal handling to call `CopyFromErrorCallback` which would read garbage data because a heap scan descriptor is read as if it was a `CopyState` structure. This commit fixes that by not adding a new context to the error context stack when migrating data, which will ensure that the error context strack does not contain garbage data. Fixes #1651 " df0acb161e322443ea59c5ccd0283b85432b1e88,Joshua Lockerman,Tue Feb 4 12:39:42 2020 -0500,Fix GapFill with ReScan,"The GapFill node was not fully reset on a ReScan, so if there was a GapFill within a NestedLoop, only the first iteration would return results. This commit fixes this issues. " 0ae37ba864c76a10e9ce0b7fcfc41156cbbb1747,Mike Freedman,Sat Feb 1 12:51:47 2020 -0500,Small grammar nits to CHANGELOG, a2629cbff29969632d4406d381fbe49d1a3faab2,gayyappan,Thu Jan 30 10:06:26 2020 -0500,Add ignore_invalidation_older_than to view,Add ignore_invalidation_older_than parameter to timescaledb_information.continuous_aggregates view Fixes #1664 4dc0693d1f8fc5a7d70d7a1bd372250650ecc6f8,Ruslan Fomkin,Mon Jan 27 10:41:37 2020 +0100,Unify error message if hypertable not found,"Refactors multiple implementations of finding hypertables in cache and failing with different error messages if not found. The implementations are replaced with calling functions, which encapsulate a single error message. This provides the unified error message and removes need for copy-paste. " b1b840f00ef16b8ab0fb7bbe12017eee6d1eca9a,gayyappan,Thu Jan 23 09:43:14 2020 -0500,Use timescaledb prefix for compression errors,Modify compression parameter related error messages to make them consistent. 783c8e80eaabfdceb0a382f140d5bb018c6666a4,gayyappan,Wed Jan 22 17:18:26 2020 -0500,Drop chunks for materialized hypertable,"When drop_chunks is called with cascade_to_materialization = true, the materialized data is deleted from the materialization hypertable, but the chunks are not dropped. This fix drops chunks if possible and deletes the data only if the materialized chunk cannot be dropped (which is the case if the materialzied chunk contains data from multiple raw chunks and some of the raw chunks are not dropped). Fixes #1644 " 8f2d6b1d2f7e4a5088e72dec0c3a02a90b9ca2fd,Sven Klemm,Thu Jan 23 11:36:24 2020 +0100,Increase wait time for bgw tests,TEST_SPINWAIT_ITERS is used as maximum number of iterations to wait for background worker results. Each iteration has a 0.1 second sleep so a setting of 10 makes the test wait 1 second for the desired result. This PR changes the iterations to 100 to wait at most 10 seconds which should help with some flaky bgw tests. ab5d06245fd0774290c87d8dae3c558bd56e6599,Erik Nordström,Thu Jan 23 10:36:05 2020 +0100,Add CODEOWNERS file to auto-assign reviewers,"A `CODEOWNERS` file allows assigning default reviewers for a branch, obviating the need to add reviewers manually. " f298ff0079c6c3256f0aef14f4591428780485e1,Sven Klemm,Mon Jan 20 12:37:48 2020 +0100,Exclude costs from test query plans,The continuous aggs query test included costs for the query plans in the test output leading to flaky tests. This PR removes costs from plan output. 442a173ad6cbf9618c33da176dedd180e2065510,Sven Klemm,Sat Jan 18 17:21:55 2020 +0100,Add 1.5.1 and 1.6.0 to update test scripts, bae083823d5909f98041128c587d90db3b1e4e7e,Sven Klemm,Fri Jan 17 10:19:39 2020 +0100,Remove -Wdeclaration-after-statement from PG_CFLAGS,Postgres CFLAGS includes -Wdeclaration-after-statement which leads to problems when compiling with -Werror since we aim for C99 and allow that so we strip this flag from PG_CFLAGS before adding postgres flags to our own 38654b2158174d30248de567474a05188b539145,Mats Kindahl,Tue Jan 14 15:12:26 2020 +0100,Print notice for COPY TO on hypertable,"When using `COPY TO` on a hypertable (which copies from the hypertable to some other destination), nothing will be printed and nothing will be copied. Since this can be potentially confusing for users, this commit print a notice when an attempt is made to copy from a hypertable directly (not using a query) to some other destination. " 2728658a0e33751d210b3ec9559afcfe932fd457,Sven Klemm,Wed Jan 15 00:49:53 2020 +0100,Release 1.6.0,"This release adds major new features and bugfixes since the 1.5.1 release. We deem it moderate priority for upgrading. The major new feature in this release allows users to keep the aggregated data in a continuous aggregate while dropping the raw data with drop_chunks. This allows users to save storage by keeping only the aggregates. The semantics of the refresh_lag parameter for continuous aggregates has been changed to be relative to the current timestamp instead of the maximum value in the table. This change requires that an integer_now func be set on hypertables with integer-based time columns to use continuous aggregates on this table. We added a timescaledb.ignore_invalidation_older_than parameter for continuous aggregatess. This parameter accept a time-interval (e.g. 1 month). if set, it limits the amount of time for which to process invalidation. Thus, if timescaledb.ignore_invalidation_older_than = '1 month', then any modifications for data older than 1 month from the current timestamp at modification time may not cause continuous aggregate to be updated. This limits the amount of work that a backfill can trigger. By default, all invalidations are processed. **Major Features** * #1589 Allow drop_chunks while keeping continuous aggregates **Minor Features** * #1568 Add ignore_invalidation_older_than option to continuous aggs * #1575 Reorder group-by clause for continuous aggregates * #1592 Improve continuous agg user messages **Bugfixes** * #1565 Fix partial select query for continuous aggregate * #1591 Fix locf treat_null_as_missing option * #1594 Fix error in compression constraint check * #1603 Add join info to compressed chunk * #1606 Fix constify params during runtime exclusion * #1607 Delete compression policy when drop hypertable * #1608 Add jobs to timescaledb_information.policy_stats * #1609 Fix bug with parent table in decompression * #1624 Fix drop_chunks for ApacheOnly * #1632 Check for NULL before dereferencing variable **Thanks** * @optijon for reporting an issue with locf treat_null_as_missing option * @acarrera42 for reporting an issue with constify params during runtime exclusion * @ChristopherZellermann for reporting an issue with the compression constraint check * @SimonDelamare for reporting an issue with joining hypertables with compression " 6590858cd2dbacbc98c8d218198648a4ada10d16,Sven Klemm,Wed Jan 15 15:58:11 2020 +0100,Ignore parallel test result in 32 bit tests,The worker assignment for parallel queries behaves differently from the 64 bit builds leading to differences in plan output and failing tests. This commit ignores the results of the parallel tests for 32 bit builds. 8f25517d4bfec063bda16aff54ee3c80edcfe6af,Sven Klemm,Wed Jan 15 14:01:31 2020 +0100,Check for NULL before dereferencing variable,Move the NULL-check in continuous_agg_execute_materialization before the first dereferencing of the variable. 59c04767a6018047817f49d16696e7fdeed712e4,Sven Klemm,Mon Jan 13 14:27:59 2020 +0100,Fix tests with drop_chunks for ApacheOnly,The drop_chunks function would call continuous_agg_drop_chunks_by_chunk_id even for hypertables without continuous aggregates leading to license errors when compiled as Apache-only. This PR skips the call when the hypertable has no continuous aggregates. 94f3cff7090a2eb77f9597424a9fe19886b42978,Matvey Arye,Fri Jan 3 14:20:14 2020 -0500,Fix bug with parent table in decompression,Fix bug with transparent decompression getting the hypertable parent table. This can happen with self-referencing updates. Fixes #1555 599a87785477750cca9cff142721f4093b7fde8c,Matvey Arye,Thu Jan 2 11:39:20 2020 -0500,Fix constify params during runtime exclusion,Descending into subplans during constification of params seems unsafe and has led to bugs. Turning this off seems to be safe and not regress any tested optimizations. In the future we may want to find a way to optimize this case as well. Fixes #1598. f8d4e7f6414c99edd0a43434ded3b7acb665af9e,Erik Nordström,Fri Nov 22 15:52:44 2019 +0100,Fix crash when partializing agg with HAVING,"This change fixes an assertion-based crash that happened when using the `partialize_agg` function together with HAVING clauses. For instance, ``` SELECT time_bucket('1 day', time), device, __timescaledb_internal.partialize_agg(avg(temp)) GROUP BY 1, 2 HAVING avg(temp) > 3; ``` would crash because the HAVING clause's aggregate didn't have its `Aggref` node set to partial aggregate mode. Regular partial aggregations executed by the planner (i.e., those not induced by the `partialize_agg` function) have their HAVING aggs transparently moved to the target list during planning so that the finalize node can use it when applying the final filter on the resulting groups. However, it doesn't make much sense to transparently do that when partializing with `partialize_agg` since it would be odd to return more columns than requested by the user. Therefore, the caller would have to do that manually. This, in fact, is also done when materializing continuous aggregates. For this reason, HAVING clauses with `partialize_agg` are blocked, except in cases where the planner transparently reduces the HAVING expression to a simple filter (e.g., `HAVING device > 3`). Apart from fixing this issue, this change also refectors some of the related code and adds tests for some error cases. " e44be9c03a4f99bd0e3d89802dabbdd7d9524c8b,Matvey Arye,Thu Jan 2 16:26:16 2020 -0500,Add jobs to timescaledb_information.policy_stats,Add the compression and continuous aggs job types to the timescaledb_information.policy_stats view. It is a bug that it wasn't there before. d52b48e0c3a52f7046d54894ea3701461c6e4e0a,Matvey Arye,Thu Jan 2 15:41:49 2020 -0500,Delete compression policy when drop hypertable,Previously we could have a dangling policy and job referring to a now-dropped hypertable. We also block changing the compression options if a policy exists. Fixes #1570 ef77c2ace89e43105e73bf218e8424f16a7449cd,Matvey Arye,Wed Dec 18 13:19:43 2019 -0500,Improve continuous agg user messages,Switch from using internal timestamps to more user-friendly timestamps in our log messages and clean up some messages. 6122e08fcb2f70a3dc4f515eb81f5df052a4d076,Matvey Arye,Thu Dec 19 16:56:58 2019 -0500,Fix error in compression constraint check,The constraint check previously assumed that the col_meta offset for a column was equal to that columns attribute offset. This is incorrect in the presence of dropped columns. Fixed to match on column names. Fixes #1590 6dad1f246a961444b21f9fdbcac51bc5b9196491,gayyappan,Tue Dec 24 13:51:49 2019 -0500,Add joininfo to compressed rel,"If the joininfo for a rel is not available, the index path cannot compute the correct filters for parameterized paths as the RelOptInfo's ppilist is setup using information from the joininfo. Fixes 1558 " 533df9645ae80cb8dae2fe73df005820d674f985,gayyappan,Mon Dec 30 10:28:12 2019 -0500,Run continuous_aggs_query.sql in debug build,Test case uses current_timestamp_mock which is available only in debug builds 1a387963465cb97e88fb362a271afaf3e7f319d2,Matvey Arye,Mon Dec 16 16:47:20 2019 -0500,Add test dropping chunks & keeping cagg,Add test for changing chunk interval when recreating chunks. Fix tests for pg10 and pg9.6. 2c594ec6f90eb10ccda45375c74f7985f28e2277,Matvey Arye,Mon Dec 16 15:31:42 2019 -0500,Keep catalog rows for some dropped chunks,"If a chunk is dropped but it has a continuous aggregate that is not dropped we want to preserve the chunk catalog row instead of deleting the row. This is to prevent dangling identifiers in the materialization hypertable. It also preserves the dimension slice and chunk constraints rows for the chunk since those will be necessary when enabling this with multinode and is necessary to recreate the chunk too. The postgres objects associated with the chunk are all dropped (table, constraints, indexes). If data is ever reinserted to the same data region, the chunk is recreated with the same dimension definitions as before. The postgres objects are simply recreated. " 5eb047413b28f34bb48ccbf04f98ed1a00fb7f5c,Matvey Arye,Thu Dec 5 13:37:22 2019 -0500,Allow drop_chunks while keeping continuous aggs,"Allow dropping raw chunks on the raw hypertable while keeping the continuous aggregate. This allows for downsampling data and allows users to save on TCO. We only allow dropping such data when the dropped data is older than the `ignore_invalidation_older_than` parameter on all the associated continuous aggs. This ensures that any modifications to the region of data which was dropped should never be reflected in the continuous agg and thus avoids semantic ambiguity if chunks are dropped but then again recreated due to an insert. Before we drop a chunk we need to make sure to process any continuous aggregate invalidations that were registed on data inside the chunk. Thus we add an option to materialization to perform materialization transactionally, to only process invalidations, and to process invalidation only before a timestamp. We fix drop_chunks and policy to properly process `cascade_to_materialization` as a tri-state variable (unknown, true, false); Existing policy rows should change false to NULL (unknown) and true stays as true since it was explicitly set. Remove the form data for bgw_policy_drop_chunk because there is no good way to represent the tri-state variable in the form data. When dropping chunks with cascade_to_materialization = false, all invalidations on the chunks are processed before dropping the chunk. If we are so far behind that even the completion threshold is inside the chunks being dropped, we error. There are 2 reasons that we error: 1) We can't safely process new ranges transactionally without taking heavy weight locks and potentially locking the entire sytem 2) If a completion threshold is that far behind the system probably has some serious issues anyway. " ff78290718787ca52859edd725457d099a787a6e,gayyappan,Fri Nov 22 15:16:25 2019 -0500,Modify group by clause on continuous aggr views,"Reorder the group by clause to match the ordering of the ORDER BY by clause. SELECTs on continuous aggregate queries often have an order by clause on the time_bucket column. If the order-by does not match the grouping clause, the planner inserts an additional Sort node after the view evaluation (Aggregate node). preprocess_groupclause does this reordering if the ORDER BY clause is part of the current subquery being processed. But when we have a continuous aggregate view, the order by needs to be derived from the outer query. This PR allows the order by from outer query to propagate to group clause of continuous aggr. view. The rewrite is : select * from (select a, b, max(c), min(d) from ...group by a, b) order by b; is transformed as SELECT * from (select a, b, max(c), min(d) from .. group by b, a ) order by b; " e6700907273c37ba442917f3381da07969310e79,Sven Klemm,Wed Dec 18 02:02:59 2019 +0100,Fix locf treat_null_as_missing option,The locf treat_null_as_missing option would not trigger the lookup query if there was a row for the first bucket and value in that row was NULL. This patch fixes the behaviour and triggers the lookup query for the first row too. bb8f59a65467db4004d7bdc7c394b2d1b9bfbbeb,Sven Klemm,Wed Dec 11 22:09:05 2019 +0100,Disable background workers in cagg ddl tests,The continuous agg ddl test is flaky and occasionally fails with a deadlock which might be caused by background workers running. d9d1a44d2e64e4cef6fcbdab5fe0a85322cdf84d,Matvey Arye,Fri Dec 6 00:33:03 2019 -0500,Refactor chunk handling to separate out stub,"Previously, the Chunk struct was used to represent both a full chunk and the stub used for joins. The stub used for joins only contained valid values for some chunk fields and not others. After the join determined that a Chunk was complete, it filled in the rest of the chunk field. The fact that a chunk could have only some fields filled out and not others at different times, made the code hard to follow and error prone. So we separate out the stub state of the chunk into a separate struct that doesn't contain the not-filled-out fields inside of it. This leverages the type system to prevent errors that try to access invalid fields during the join phase and makes the code easier to follow. " e6b215b8c0bb997394d08e1a944c75dd2487dc95,Ruslan Fomkin,Thu Dec 5 12:53:25 2019 +0100,Remove duplicate allocation during planning,Removes duplicate call to setup_append_rel_array and avoids allocating another append_rel_array with the same values during planning queries with hypertables. 08ad7b6612aaf23aae9c86b2a745ebbca92a257b,Matvey Arye,Tue Dec 3 21:07:41 2019 -0500,Add ignore_invalidation_older_than to continuous aggs,"We added a timescaledb.ignore_invalidation_older_than parameter for continuous aggregatess. This parameter accept a time-interval (e.g. 1 month). if set, it limits the amount of time for which to process invalidation. Thus, if timescaledb.ignore_invalidation_older_than = '1 month' then any modifications for data older than 1 month from the current timestamp at insert time will not cause updates to the continuous aggregate. This limits the amount of work that a backfill can trigger. This parameter must be >= 0. A value of 0 means that invalidations are never processed. When recording invalidations for the hypertable at insert time, we use the maximum ignore_invalidation_older_than of any continuous agg attached to the hypertable as a cutoff for whether to record the invalidation at all. When materializing a particular continuous agg, we use that aggs ignore_invalidation_older_than cutoff. However we have to apply that cutoff relative to the insert time not the materialization time to make it easier for users to reason about. Therefore, we record the insert time as part of the invalidation entry. " 3848c95c1cdd5b24929474dc65d8c4519b7fc814,Matvey Arye,Wed Dec 4 10:36:33 2019 -0500,Fix isolation test on older PG versions,"On older point releases (e.g. 10.2) the step size in isolation tests is smaller leading to ""SQL step too long"" errors. This PR splits up the setup step to avoid this error. " 4ecc96509dad565c8bebe60b2276de2bfddaf657,gayyappan,Mon Dec 2 16:07:34 2019 -0500,Fix partial select query for continuous aggregate,"continuous aggregate views like select time_bucket(), sum(col) from ... group by time_bucket(), grpcol; when grpcol is missing from the select targetlist, the partialize query's select targetlist is incorrect and the view cannot be materialized. This PR fixes this issue. " 513ad6de6a740101c300aa04afc6f1b6fe96975c,Matvey Arye,Mon Dec 2 16:22:03 2019 -0500,Fix tests errors,Fix some minor compiler warnings and errors. Move some tests that require `timescaledb.current_timestamp_mock` from release to debug-only. 09051df8cd271e3d6c044513c3b5b074cfe0d4ff,Matvey Arye,Tue Nov 26 13:32:22 2019 -0500,Apply max_interval_per_job to invalidations,"This change the continuous aggregate materialization logic so that the max_interval_per_job applies to invalidation entries as well as new ranges in the materialization. The new logic is that the MIPJ setting limits the sum of work done by the invalidations and new ranges. Invalidations take precedence so new ranges are only processed if there is time left over in the MIPJ budget after all invalidations are done. This forces us to calculate the invalidation range during the first transaction. We still delete and/or cut the invalidation entries in the second transaction. This change also more neatly separates concerns: all decisions on work to be done happens in the first txn while only execution happens in the second. Further refactoring could make this more clear by passing a list of InternalRanges to represent the work. But this PR is big enough, so that's left to a future refactor. Note: There is remaining work to be done in breaking up invalidation entries as created during inserts to constrain the length of the entries. But that's a separate issue to be addressed in the future. " 4ad40931f13179c546867100b7dfd32ead8c4a12,Ruslan Fomkin,Thu Nov 28 10:13:57 2019 +0100,Notify internal Slack an Travis CRON build fail,Adds notification from Travis CI to an internal Slack channel when a CRON build fails. e82b94a658f238ca2e2b8bf5f4df1504ab82761c,Sven Klemm,Tue Nov 26 13:03:24 2019 +0100,Use function cache to check for valid bucketing functions,Refactor the continuous aggregate validation to use our function cache to check for bucketing function. This simplifies the code and allows adding support for other bucketing functions like date_trunc later on. 1e5d0d442453741ffc37a49fafda7fc45aebea18,niksa,Wed Nov 20 11:13:58 2019 +0100,Fix CMake condition,In some cases _temp variable will not be set due to pg_config not returning any output for a specific flag. This results in an error when doing comparison using STREQUAL and build failure. Wrapping variable in double quotes fixes the problem. 2f7d69f93b53eadc2c58518e6d10c38646ae2d58,Matvey Arye,Wed Nov 20 17:03:53 2019 -0500,Make continuous agg relative to now(),"Previously, refresh_lag in continuous aggs was calculated relative to the maximum timestamp in the table. Change the semantics so that it is relative to now(). This is more intuitive. Requires an integer_now function applied to hypertables with integer-based time dimensions. " 1ec16869f806d9e9ca357ae8dc5c9c6ef167b9ef,Sven Klemm,Mon Nov 11 22:29:36 2019 +0100,Release 1.5.1,This maintenance release contains bugfixes since the 1.5.0 release. We deem it low priority for upgrading. In particular the fixes contained in this maintenance release address potential segfaults and no other security vulnerabilities. The bugfixes are related to bloom indexes and updates from previous versions. **Bugfixes** * #1523 Fix bad SQL updates from previous updates * #1526 Fix hypertable model * #1530 Set active snapshots in multi-xact index create **Thanks** * @84660320 for reporting an issue with bloom indexes 122856c1bdc8f429262d6fa0cd2ccf3c1cb546e7,Matvey Arye,Mon Nov 11 14:07:56 2019 -0500,Fix update scripts for type functions,"Type functions have to be CREATE OR REPLACED on every update since they need to point to the correct .so. Thus, split the type definitions into a pre, functions, and post part and rerun the functions part on both pre_install and on every update. " 707bb9d5844af1d8fc43816d27b88287dc9f9002,Matvey Arye,Fri Nov 8 13:52:25 2019 -0500,Add compression to update tests,Add a test using cmpressed tables to our extension update test suite. 713ceb2a2344a97d5cea8c46d619d7c55a5e72af,Matvey Arye,Fri Nov 8 14:10:52 2019 -0500,Set active snapshots in multi-xact index create,Set active snapshots when creating txns during index create with timescaledb.transaction_per_chunk. This is needed for some index types like `bloom`. Tests not added since we don't want dependencies on contrib modules like bloom. Fixes #1521. 85d35af14073222e1c9ac836b54f378b099016db,Matvey Arye,Thu Nov 7 15:47:24 2019 -0500,Fix shared tests on OSX,Fix the test runner to work with the OSX version of tests. 2cf66fdf44eebc4dacbc4a3a967961f734afd0bc,Matvey Arye,Thu Nov 7 15:41:47 2019 -0500,Fix hypertable model handling,The hypertable model now has NULL fields but was still using GETSTRUCT. This is unsafe. Switch to using the proper model access methods. 3e5e0de1643f6bc2eea1f7dee0e1629dac1277b7,gayyappan,Wed Oct 30 14:29:17 2019 -0400,Add constraint aware tests for compression,1. This commit introduces changes to existing plans due to the addition of new chunks to metrics_ordered_idx. 2. Add tests for constraint aware appends on compressed tables. 99f862198ed9c2db1fbc6127e6ed8ec8136ee022,Matvey Arye,Thu Nov 7 09:10:44 2019 -0500,Fix update logic from 1.4.2 to 1.5.0,The update logic from 1.4.2 to 1.5.0 had an error where the _timescaledb_catalog.hypertable table was altered in such a way that the table was not re-written. This causes bugs in catalog processing code. A CLUSTER rewrites the table. We also backpatch this change to the 1.4.2--1.5.0 script to help anyone building from source. Also fixes a similar error on _timescaledb_catalog.metadata introduced in the 1.3.2--1.4.0 update. e575cc2b88d5297b462241d2d7dfd6700604776e,Matvey Arye,Wed Nov 6 17:56:56 2019 -0500,Add tests for updates of catalog tables,"PG11 added an optimization where columns that were added by an ALTER TABLE that had a DEFAULT value did not cause a table re-write. Instead, those columns are filled with the default value on read. But, this mechanism does not apply to catalog tables and does not work with our catalog scanning code. This tests makes sure we never have such alters in our updates. " d34731a4e3062a94f93f987b13324eced2079c41,Matvey Arye,Wed Nov 6 17:01:01 2019 -0500,Add update test on PG11,Change travis to to run update tests on PG11. 6fe5ef3957f18d53b063ab0070e246799271c92f,Sven Klemm,Thu Nov 7 11:30:49 2019 +0100,Move prepared statement queries to shared tests,This patch moves the prepared statement test to the shared tests so they can be run on compressed hypertables as well. 41209fd629ebdae68bd68f4e036a9ec068ae2f02,Sven Klemm,Thu Nov 7 11:51:17 2019 +0100,Add 1.5.0 to update test scripts, 04041b06c74045ebc55fb248752adff742d0b28f,Sven Klemm,Thu Nov 7 11:29:52 2019 +0100,Ignore bgw_db_scheduler failure in 32 bit test,Ignore results of bgw_db_scheduler test because it is not reliable. 156da6cf75d795c0bb95c12abf144728a3e0c207,Sven Klemm,Thu Nov 7 11:27:45 2019 +0100,Refactor transparent decompression sort pushdown,"The construct used for pushing down produces a warning on certain older compiler, so while it was correct this patch changes it to get rid of the warning and to prevent introducing an imbalance later. " b2918e58fefb52475a83af3fbcc8f56337e38c2d,Mats Kindahl,Tue Nov 5 11:19:00 2019 +0100,Fix permission issue in test_sanitizer,The `test_sanitizer.sh` test failed because source code was being copied from the host to the container as user `postgres` and this user did not have read permissions on the mounted directory. This is fixed by copying the files as `root` and then changing the owner to `postgres`. The commit also removes `wait_for_pg` since PostgreSQL server status is not relevant for the tests since they start their own temporary instance. The commit also switches to use here-is documents for the execution for readability purposes. 0ca9e30198421a7210f58f26950848011b37e6ba,Sven Klemm,Tue Nov 5 12:22:21 2019 +0100,Run 32 bit tests on i386 instead of ARM emulation,The main reason to run ARM tests was not to identify issues with ARM but to identify 32 bit issues e.g. int64 as pointer instead type by value. Those issues don't need ARM emulation but can be tested with i386 which is much faster. 5427bccf0155c43a43965779d1a84c9edd5745cb,Sven Klemm,Tue Nov 5 12:22:55 2019 +0100,Remove byte sizes from chunk_adaptive test,The byte sizes for tables/indexes differ between 32 bit and 64 bit so this patch removes them from the test output. c6350aaaea9f1e212710993aed5271fc41e9feb0,Sven Klemm,Fri Nov 1 16:35:50 2019 +0100,Move decompress_chunk to nodes/decompress_chunk,Code for all custom nodes should live under the nodes directory to make different parts of the code easier to distinguish. 874b4512b2578b7cfbb1731df8abc1dabed1bc6d,gayyappan,Thu Oct 31 17:44:46 2019 -0400,Fix test failure with custom type,Fix tests that fail like so: test=# CREATE CAST (customtype AS bigint) test-# WITHOUT FUNCTION AS ASSIGNMENT; ERROR: source and target data types are not physically compatible 1e75c27c160532127c97edfdb95d60b1a8206527,Erik Nordström,Mon Nov 4 09:34:35 2019 +0100,Fix build flags for Apple builds,"A previous change made `UNIX` and `APPLE` build flags mutually exclusive instead of complementary. This broke builds on, e.g., Mac OS X. The changes in this commit will make builds work on Mac OS X again. " 90cab8e61cf5e5106a05bad9d39999aa080f7df4,Mats Kindahl,Mon Oct 28 11:44:17 2019 +0100,Use Postgres linker flags when linking,"When linking the extensions as shared libraries, the linker flags from `pg_config` is not used. This means that if `PG_PATH` is provided and refer to a locally compiled Postgres installation, shared libraries from that installation will not be used. Instead any default-installed version of Postgres will be used. This commit adds `PG_LDFLAGS` to `CMAKE_SHARED_LINKER_FLAGS` and `CMAKE_MODULE_LINKER_FLAGS`. To handle that Windows set some fields to ""not recorded"" when they are not available, it introduces a CMake function `get_pg_config` that will replace it with `-NOTFOUND` so that it is treated as undefined by CMake. " d6b1c52d71eeca50da7c9a8accbc8530cc8314f2,Joshua Lockerman,Thu Oct 31 13:43:10 2019 -0400,Set CMAKE_C_STANDARD on TSL,We should have this since we set it on the rest of the DB. 7b2519eb44ea8ba20903ce0b24ffab2386cdc0f1,Sven Klemm,Thu Oct 31 14:21:37 2019 +0100,Release 1.5.0,"This release adds major new features and bugfixes since the 1.4.2 release. We deem it moderate priority for upgrading. This release adds compression as a major new feature. Multiple type-specific compression options are available in this release (including DeltaDelta with run-length-encoding for integers and timestamps; Gorilla compression for floats; dictionary-based compression for any data type, but specifically for low-cardinality datasets; and other LZ-based techniques). Individual columns can be compressed with type-specific compression algorithms as Postgres' native row-based format are rolled up into columnar-like arrays on a per chunk basis. The query planner then handles transparent decompression for compressed chunks at execution time. This release also adds support for basic data tiering by supporting the migration of chunks between tablespaces, as well as support for parallel query coordination to the ChunkAppend node. Previously ChunkAppend would rely on parallel coordination in the underlying scans for parallel plans. " 825645b2f5c3685ce50d643f657e48d35677d015,Matvey Arye,Tue Oct 29 14:14:17 2019 -0400,Fix bug with histogram function in parallel,Histogram's combine function threw a segfault if both state1 and state2 were NULL. I could only reproduce this case in PG 10. Add a tests that hits this with PG 10.4 Fixes #1490 db23139b3ce0dcafefd03a3e36ad97d02183a0aa,Matvey Arye,Sun Oct 27 17:02:20 2019 -0400,Fix error for exported_uuid in pg_restore,"When restoring a database, people would encounter errors if the restore happened after telemetry has run. This is because a 'exported_uuid' field would then exist and people would encounter a ""duplicate key value"" when the restore tried to overwrite it. We fix this by moving this metadata to a different key in pre_restore and trying to move it back in post_restore. If the restore create an exported_uuid, that restored value is used and the moved version is simply deleted We also remove the error redirection in restore so that errors will show up in tests in the future. Fixes #1409. " 0cf0e11923af5dc56bf3e4128f49ddd1616d4a61,Matvey Arye,Tue Oct 29 22:27:14 2019 -0400,Fix telemetry for ARM32,Fix telemetry logic for ARM32. d2db84fd98d4d5d3dfd464917082886f6f4d1856,Matvey Arye,Tue Oct 29 18:12:33 2019 -0400,Fix windows compilation,Some minor fixes for compilation in Windows. 5241053c4a0ad0ad0dbbfbf78a656dab28cf2a86,Matvey Arye,Tue Oct 29 17:27:12 2019 -0400,Fixes to test setup,Several fixes: - Change incorrect variable name in CmakeLists that prevented tests from running. - Add a PG 10.10 test to codecov - Remove unused CODECOV_FLAGS in travis.yml 1e9bc6895b877dcef2e9a246da4139514ea2fc40,Matvey Arye,Thu Oct 17 13:59:53 2019 -0400,Add telemetry fields to track compression,"The following fields are added: -num_compressed_hypertables -compressed_KIND_size -uncompressed_KIND_size Where KIND = heap, index, toast. `num_hypertables` field does NOT count the internal hypertables used for compressed data. We also removed internal continuous aggs tables from the `num_hypertables` count. " 48ef701fa9d14504870aefc75f5847efbfc495b4,Joshua Lockerman,Fri Oct 4 15:01:20 2019 -0400,Set toast_tuple_target to 128B when able,"We want compressed data to be stored out-of-line whenever possible so that the headers are colocated and scans on the metadata and segmentbys are cheap. This commit lowers toast_tuple_target to 128 bytes, so that more tables will have this occur; using the default size, very often a non-trivial portion of the data ends up in the main table, and only very few rows are stored in a page. " e9e7c5f38e1880b2b8d8042f08108cfde8ca9628,Joshua Lockerman,Tue Oct 29 14:38:25 2019 -0400,Add missing tests discovered by Codecov 3,"Tests for continuous aggregates over compressed data, which also tests selecting tableoids from compressed tables. " 7ae97088f30915bda986f927e808963b90da27bd,Matvey Arye,Tue Oct 29 13:20:27 2019 -0400,Fix ordered append test for PG <10.4,"Prior to PG 10.4, the costing for hashaggs was different (see PG commit `1007b0`). We fix the tests not to change between the old and new versions by disabling hashagg. We were not testing for that anyway. " efb131dd6f225d7fb437096f215531a5ce130cd1,Joshua Lockerman,Tue Oct 29 10:03:54 2019 -0400,Add missing tests discovered by Codecov 2,"This commit adds tests for DATE, TIMESTAMP, and FLOAT compression and decompression, NULL compression and decompression in dictionaries and fixes a bug where the database would refuse to decompress DATEs. This commit also removes the fallback allowing any binary compatible 8-byte types to be compressed by our integer compressors as I believe I found a bug in said fallback last time I reviewed it, and cannot recall what the bug was. These can be re-added later, with appropriate tests. " 3070a89f0101699e30f612ccfd6ea5325ed0e299,Matvey Arye,Tue Oct 29 11:57:06 2019 -0400,Fix tests after concurrent merge,The merges of some recent PRs occurred concurrently and did not update the test output. This fixes that issue. aaa45df369ec638185d107c0166b2d8ffdf14187,Matvey Arye,Wed Oct 23 13:56:24 2019 -0400,Fix varoattno when creating ec for segment by.,"We reset the varoattno when creating the equivalence member for segment by columns. varoattno is used for finding equivalence members (em) when searching for pathkeys (although, strangely not for indexclauses). Without this change the code for finding matching ems differs in the case where attnos have changed and where they haven't. Fixing this, allows the planner to plan more different types of paths for several tests, Because of the way the cost fuzzer in `compare_path_costs_fuzzily` interacts with disabling seqscans, some choices the planner makes have changed (pretty much the cost is dominated by the penalty of the seqscan and so it picks the first available path). We've changed some enable_seqscan clauses to get around this and have the planner show what we want in tests. Also delete transparent_decompression-9.6.out since compression is disabled on 9.6. " c39091e740053e0ae1ff88a3192f474635e26be8,Matvey Arye,Tue Oct 29 09:59:44 2019 -0400,Fix pre_release tests,2 fixes: - The no-ssl telemetry tests should always run in debug tests - Run explain on a query instead of the query because this query runs out of memory in our underpowered arm tests and the bug we are looking is in the planner anyway. 819414df026a966d3f49471ff3fa0879572e513f,Sven Klemm,Tue Oct 22 18:19:22 2019 +0200,Add test infrastructure with shared tables,"This PR adds test infrastructure for running tests with shared tables. This allows having hypertables with specific configurations usable for all tests. Since these tests also don't require creating a new database for each test case some of the overhead of the normal tests is removed. While this will lead to much faster query tests some tests will still require their own database to test things, but most queres could be moved to this infrastructure to improve test coverage and speed them up. " e2df62c81ce1dc9e76347fb468cfd16157c4cacd,Sven Klemm,Mon Oct 28 12:36:20 2019 +0100,Fix transparent decompression interaction with first/last,Queries with the first/last optimization on compressed chunks would not properly decompress data but instead access the uncompressed chunk. This patch fixes the behaviour and also unifies the check whether a hypertable has compression. 405c65fb99fae6a8a615b7da5a4e26fcb082ef1f,Joshua Lockerman,Mon Oct 28 15:37:46 2019 -0400,Add missing tests discovered by Codecov 1,This commit adds tests for: 1. Fixed-size pass-by-ref types. 2. Types whose alignment is greater than their size. 3. NULLs in segmentby columns All of which were reported as missing by Codecov 516736d1a35fcc1cbc1f07d34ae6535f4bb2d7c7,Joshua Lockerman,Wed Oct 16 14:26:27 2019 -0400,Improve the reporting accuracy of test coverage,"This commit fixes a number of issues that were affecting the accuracy of our test coverage measurements. It makes two changes to the environment in which we collect coverage information: 1. It collects coverage with optimizations disabled. Optimizations, most significantly inlining, can move code in ways that gcov cannot account for, which caused many lines to be spuriously marked as never executed. 2. It collects coverage with assertions disabled. Every assertion adds a branch which will never be taken in any valid run of the database these branches pollute the coverage information, and make it difficult to determine whether regressions in coverage are legitimize, due to adding code which is not tested, or spurious, due to adding assertions whose code paths should not be taken in the tests anyway. This does not affect our tests reliability since we already run all tests in a instance with assertions enabled; the test-coverage metrics are used to test our tests, not the database. This commit also changes the way we handle -DCMAKE_C_FLAGS in our CMakefiles so that flags passed in through that override (almost) all of the flags passed by the CMakefiles, as was originally intended. " a0a9fdaae5e0644283fe0a85ec7588ee8b358132,gayyappan,Thu Oct 24 13:20:54 2019 -0400,Block compression for postgres 9 versions,Block compression for PG versions < 10 and disable compression tests as well. fab50e31681479e8d6f99bc197d8e583b60ba9c7,Matvey Arye,Tue Oct 22 20:41:50 2019 -0400,Optimize planning decompress_chunk,Change the eclasses and emembers for compressed chunks to be prepended instead of appended to their lists. Also optimize the checking of relids on eclasses in `add_segmentby_to_equivalence_class` to use checking the varno on the var instead of bms_overlap. This reduced planning by 35 on some queries. ,,,, 5ee5c39215a85438167845c4922b930de3e6f10b,Matvey Arye,Tue Oct 22 04:22:15 2019 -0400,Prevent planning indexes on uncompressed chunk,"Planning indexes is expensive in terms of planning time. Since uncompressed chunks are never scanned, their indexes are never used. This patch prevents the planner from doing useless work. " 1c4dbbb78dd06f1251c1c5dea4b6f886ec21bf20,gayyappan,Wed Oct 23 14:05:43 2019 -0400,Grant tests for compression,Check DML operations work when user has the correct permissions for a compressed table. Permissions for compressed hypertable are carried over from original hypertable -so remove ACL_SELECT permission while creating the RangeTblEntry for the compressed hypertable. 87786f1520e4aca40c0d87aaa9e349a8213984b4,gayyappan,Thu Oct 17 09:18:16 2019 -0400,Add compressed table size to existing views,Some information views report hypertable sizes. Include compressed table size in the calculation when applicable. 940d5aa3ac190349b97f681b41d856d9b0760516,gayyappan,Mon Oct 21 12:35:26 2019 -0400,Compression related ddl tests,Trigger tests with compress/decompress_chunk 83601550a72dab9a543dec544b30ed980ceaed26,gayyappan,Wed Oct 16 10:19:55 2019 -0400,Concurrency test for compress_chunk,Test compress_chunk concurrently with other operations on the table. 8d97ab1d3959871491edb7a3a65cc2a9adcf91f8,Sven Klemm,Wed Oct 16 11:25:06 2019 +0200,Refactor pathkey handling,This patch also sets reloptkind of the compressed rel to RELOPT_DEADREL to prevent postgresql from replanning the relation. 03f8b597cb56058518c5de35500bef7bfcd16aff,Matvey Arye,Wed Oct 16 13:12:40 2019 -0400,Fix tests for ARM32,A few fixes: - Turn off bgw workers during tests - Increase timeout - Cleanup function signature to get rid off never-used parameter 07841670a7a5606e0f96dad29fca041c82a71913,Joshua Lockerman,Wed Oct 16 13:28:45 2019 -0400,Fix issues discovered by coverity,"This commit fixes issues reported by coverity. Of these, the only real issue is an integer overflow in bitarray, which can never happen in its current usages. This also adds a PG_USED_FOR_ASSERTS_ONLY for a variable only used for Assert. " c4efacdc926aa541dbc68f9aa5179c639998bd7a,Matvey Arye,Tue Oct 15 21:27:48 2019 -0400,Fix tests and function naming after rebase,Small fixup after rebase on the master branch. Fixes test output and function naming (even test functions should have the ts_ not the tsl_ prefix for exported C functions). 28653640af59c1ab5b3ca49ccbd0650313995df3,Joshua Lockerman,Sun Oct 6 13:41:02 2019 -0400,Fix redundant sorts and other planner fixes,"This commit changes the sort-before-decompression logic from always sorting if possible to only sorting if it is possible, and the underlying scan of the compressed chunk does not already retun tuples in the correct order. The pathkeys used by the decompression node are the query_pathkeys if ordering can be pushed down. The equivalent compressed_pathkeys are saved in a separate field. At plan time, a sort is put in between the scan and the decompress node if the scan's pathkeys do not satisfy the ordering of the decompressed path's compressed_pathkey field. Other planner fixes: Allow using ordered paths when ordering by a subset of segment_by columns (and no other columns). Adjust costing of ordered paths to add costs for sorting when it is needed. That prevents the planner from using ordered paths when it doesn't make use of the ordering. EC members of compressed rels are marked as children. In order to correctly find them when building compressed scans, we have to correctly pass down the relids being searched instead of NULL, otherwise child ems are not considered. This improves index pushdowns. " 85d30e404d06c24599be887444573b32d50bbab1,Matvey Arye,Sun Oct 13 18:56:06 2019 -0400,Add ability to turn off compression,"Since enabling compression creates limits on the hypertable (e.g. types of constraints allowed) even if there are no compressed chunks, we add the ability to turn off compression. This is only possible if there are no compressed chunks. " 887d0271e87ae7c07ed2dc10497a8072028bbadb,gayyappan,Thu Oct 3 18:18:13 2019 -0400,Enable Index Backward Scan,enable index backward scan and backward scan of chunk data by checking if the compress_orderby columns can satisfy the required sort order 37d132e80f9a9410082fd37a9932ba015d42006f,Sven Klemm,Wed Oct 9 18:44:09 2019 +0200,Mark EquivalenceMember as child and add it to ec_relids,This fixes a planner regression for queries on compressed hypertables with many chunks. 2fe51d2735db457919ed38a697647316eb3befe0,Matvey Arye,Tue Oct 8 18:32:37 2019 -0400,Improve (de)compress_chunk API,This commit improves the API of compress_chunk and decompress_chunk: - have it return the chunk regclass processed (or NULL in the idempotent case); - mark it as STRICT - add if_not_compressed/if_compressed options for idempotency 92aa77247ae43c6e0b9a082f5c684310bd085691,Matvey Arye,Tue Oct 8 14:39:09 2019 -0400,Improve minor UIUX,Some small improvements: - allow alter table with empty segment by if the original definition had an empty segment by. Improve error msgs. - block compression on tables with OIDs - block compression on tables with RLS 2335fc58378c2b0390edf0adc79612f5f13e7ed6,Matvey Arye,Sun Oct 6 13:41:02 2019 -0400,Adjust hypertable row count on decompression,Adjust the row count on the hypertable (append) relation. This improves plans. It is especially important to make sure that parallel queries use split aggregate (partial/finalize) instead of aggregating after gather. Also adjust the test to show parallel plans. c48ada15577677b406d0caa8529b769a43cbb4d0,Matvey Arye,Fri Oct 4 16:49:56 2019 -0400,Fix use-after-free on paths,"Whenever paths are added to a rel, that path or another path that previously was on the rel can be freed. Previously, the compressed rel's paths could be freed when it was re-planned by the postgres planner after being created and planned by us. The new path the postgres planner added was cheaper and overwrote and pfreed the old path which we created and saved as a child path of the decompress node. Thus we ended up with a dangling reference to a pfreed path. This solution prevents this bug by removing the path we create from the compressed rel. Thus, the chunk rel now ""owns"" the path. Note that this does not prevent the compressed rel from being replanned and thus some throw-away planner work is still happening. But that's a battle for another day. The backtrace for the core planner overwriting our path is: ``` frame #4: 0x0000000105c4ed0f postgres`pfree(pointer=0x00007fe20d01a628) at mcxt.c:1035 * frame #5: 0x000000010594c998 postgres`add_partial_path(parent_rel=0x00007fe20d01ae10, new_path=0x00007fe20f800298) at pathnode.c:844 frame #6: 0x00000001058ede4b postgres`create_plain_partial_paths(root=0x00007fe2113fc668, rel=0x00007fe20d01ae10) at allpaths.c:753 frame #7: 0x00000001058edb93 postgres`set_plain_rel_pathlist(root=0x00007fe2113fc668, rel=0x00007fe20d01ae10, rte=0x00007fe20d0198c0) at allpaths.c:727 frame #8: 0x00000001058ed78b postgres`set_rel_pathlist(root=0x00007fe2113fc668, rel=0x00007fe20d01ae10, rti=13, rte=0x00007fe20d0198c0) at allpaths.c:452 frame #9: 0x00000001058e8e16 postgres`set_base_rel_pathlists(root=0x00007fe2113fc668) at allpaths.c:310 frame #10: 0x00000001058e8b49 postgres`make_one_rel(root=0x00007fe2113fc668, joinlist=0x00007fe20d0121c8) at allpaths.c:180 frame #11: 0x000000010591ee77 postgres`query_planner(root=0x00007fe2113fc668, tlist=0x00007fe2113fcb58, qp_callback=(postgres`standard_qp_callback at planner.c:3492), qp_extra=0x00007ffeea6ba2b8) at planmain.c:265 frame #12: 0x00000001059229cb postgres`grouping_planner(root=0x00007fe2113fc668, inheritance_update=false, tuple_fraction=0) at planner.c:1942 frame #13: 0x0000000105920546 postgres`subquery_planner(glob=0x00007fe218000328, parse=0x00007fe218000858, parent_root=0x0000000000000000, hasRecursion=false, tuple_fraction=0) at planner.c:966 frame #14: 0x000000010591f1e7 postgres`standard_planner(parse=0x00007fe218000858, cursorOptions=256, boundParams=0x0000000000000000) at planner.c:405 frame #15: 0x000000010642d9b4 timescaledb-1.5.0-dev.so`timescaledb_planner(parse=0x00007fe218000858, cursor_opts=256, bound_params=0x0000000000000000) at planner.c:152 ``` " 7380efa0fefa200cb8c8c960332ff885839e1268,Matvey Arye,Fri Oct 4 15:52:10 2019 -0400,Add tests that constraint adding is blocked,Adding constraints to tables that have compression enabled should be blocked for now. b8a98c1f18c50b37c899b1f885edc6d24ebe226d,Matvey Arye,Thu Oct 3 17:59:29 2019 -0400,Make compressed chunks use same tablespace as uncompressed,"For tablepaces with compressed chunks the semantics are the following: - compressed chunks get put into the same tablespace as the uncommpressed chunk on compression. - set tablespace on uncompressed hypertable cascades to compressed hypertable+chunks - set tablespace on all chunks is blocked (same as w/o compression) - move chunks on a uncompressed chunk errors - move chunks on compressed chunk works In the future we will: - add tablespace option to compress_chunk function and policy (this will override the setting of the uncompressed chunk). This will allow changing tablespaces upon compression - Note: The current plan is to never listen to the setting on compressed hypertable. In fact, we will block setting tablespace on compressed hypertables " 4d65a01a5765ebb7c9aee56b2d45cdad8331f91b,Matvey Arye,Fri Sep 27 16:46:20 2019 -0400,Handle change owner on ht with compressio,Pass down the change owner command to the compressed hypertable. c5d4ce7f90a836ff503e048016a9f2f202408f3c,Matvey Arye,Fri Sep 27 09:29:43 2019 -0400,Handle set tablespace on ht with compression,Pass down the set tablespace command to the compressed hypertable. a399a57af90c383e072e9a09bc3e3fbdbf1a0ba5,Matvey Arye,Thu Sep 26 16:16:03 2019 -0400,Block most DDL on hypertable with compression,"Block most DDL commands on hypertables with compression enabled. This restriction will be relaxed over time. The only alter table commands currently allowed are: Add/Remove index, Set storage options, clustering index, set statistics and set tablespace. We also disallow reseting compression options. " 91a73c3e173865d2dd477e62c426d42333882f82,Joshua Lockerman,Thu Oct 3 14:17:48 2019 -0400,Set statistics on compressed chunks,"The statistics on segmentby and metadata columns are very important as they affect the decompressed data a thousand-fold. Statistics on the compressed columns are irrelevant, as the regular postgres planner cannot understand the compressed columns. This commit sets the statistics for compressed tables based on this, weighting the uncompressed columns greatly, and the compressed columns not-at-all. " 72588a2382f09fbba476652e67c543a5b4d927af,gayyappan,Mon Sep 30 10:23:42 2019 -0400,Restrict constraints on compressed hypertables.,Primary and unqiue constraints are limited to segment_by and order_by columns and foreign key constraints are limited to segment_by columns when creating a compressed hypertable. There are no restrictions on check constraints. 0f3e74215ae24f7678a276ae49cc3dc608f9e9b5,Matvey Arye,Mon Sep 30 14:25:47 2019 -0400,Split segment meta min_max into two columns,"This simplifies the code and the access to the min/max metadata. Before we used a custom type, but now the min/max are just the same type as the underlying column and stored as two columns. This also removes the custom type that was used before. " 4d12f5b8f3778b3f378ad7b334a31350e27a7fd1,Sven Klemm,Mon Sep 30 10:20:58 2019 +0200,Fix transparent decompression sort interaction,The sort optimization adds a new index path to the pathlist of rel with the modified pathkeys. This optimization needs to happen before the DecompressChunk paths get generated otherwise those paths will survive in pathlist and a query on a compressed chunk will target the empty chunk of the uncompressed hypertable. 7b570b0daba5df644f0eff5f2d5035700ca919a0,Joshua Lockerman,Fri Sep 27 15:46:09 2019 -0400,Forward TRUNCATE from to compressed hypertable,TRUNCATEs of uncompressed hypertables should be forwarded to their equivalent compressed tables. This currently does not occur for chunks. a2ea01831a962dd82610373e810d19f3392c7100,Matvey Arye,Fri Sep 27 17:56:04 2019 -0400,Fix compression_bgw test flakiness,"Previously we were creating multiple rows using generate_series and now(), depending on the time of day the test was run, this could create one or two chunks, causing flakiness. We changed the test to only create one row and thus one chunk " 6687189a6c9c6b4b1b57dfe1e2e7c958a259af49,Joshua Lockerman,Fri Sep 27 15:16:51 2019 -0400,Free memory earlier in decompress_chunk,"This was supposed to be part of an earlier commit, but seems to have been lost. This should reduce peak memory usage of that function. " 43aa49ddc019f3f0a2bfcd2732305a50bdd34dd7,gayyappan,Tue Sep 24 15:56:07 2019 -0400,Add more information in compression views,Rename compression views to compressed_hypertable_stats and compressed_chunk_stats and summarize information about compression status for chunks. b79db68b3a420d3e0db6aa977a1093c862c90ed2,Sven Klemm,Thu Sep 26 13:49:00 2019 +0200,Improve qual pushdown error handling,Improve transparent decompression qual pushdown error handling by preventing pushdown if the expression is volatile or when operators are not member of btree family. 64f56d50888d6b4e9cbb77e22071ea41cb03ebf5,Joshua Lockerman,Wed Sep 25 14:16:28 2019 -0400,Create indexes on segmentby columns,This commit creates indexes on all segmentby columns of the compressed hypertable. 46b4a9d3c231c901f02f90816157c71e080d218d,Joshua Lockerman,Mon Sep 23 13:01:46 2019 -0400,Fix tests,This commit contains all of the miscellaneous fixes needed to pass pre-release tests. It: 1. Switches the custom type in tsl/tests/compression_segment_meta.sql to determine its alignment and pass-by-ref'ness based on what TIMESTAMPTZ has. This is needed due to the fact that 8-byte values are pass-by-val in 64-bit platforms and pas-by-ref on 32-bit. 2. Marks some variables that are only used for assertions as PG_USED_FOR_ASSERTS_ONLY 909b0ece78fb4896a40b2ba7b6e53561021a327d,gayyappan,Fri Sep 20 16:55:51 2019 -0400,Block updates/deletes on compressed chunks, edd3999553ffb292e737360dad299b8551e269b0,gayyappan,Fri Sep 13 15:40:47 2019 -0400,Add trigger to block INSERT on compressed chunk,Prevent insert on compressed chunks by adding a trigger that blocks it. Enable insert if the chunk gets decompressed. 12929fc8130a1d756bea3b0adb67dc96f81d7ab5,Matvey Arye,Sun Sep 22 23:15:36 2019 +0200,Use DatumSerialize for binary strings,This is a refactor of the array and dictionary code to use binary string functions in datum serialize to consolidate code. We also made the datum serialize more flexible in that it no longer must use a byte to store the encoding type (binary or text) but instead can get that as input. This makes the encoding use less data in the array case. 14f02f423e640dddbba2e958157321dfc3919b98,Matvey Arye,Wed Sep 18 17:15:00 2019 +0200,Switch the array code to use DatumSerializer,This commit switches the array compressor code to using DatumSerializer/DatumDeserializer to reduce code duplication and to add in some more efficiency. 300db8594a433909d80c8a1839762e9094111033,Matvey Arye,Wed Sep 18 17:04:14 2019 +0200,Fix detoasting bug and add tests,"Previously, the detoasting in Array was incorrect and so the compressed table stored pointers into the toast table of the uncomoressed table. This commit fixes the bug and also add logic to the test to remove the uncompressed table so such a bug would cause test failures in the future. " fac8eca0b31495345024e3443f339457fa650576,Joshua Lockerman,Fri Sep 20 13:30:13 2019 -0400,Free Memory Earlier in decompress_chunk,This commit alters decompress_chunk to free memory as soon as possible instead of waiting until the function ends. This should decrease peak memory usage from roughly the size of the dataset to roughly the size of the a single compressed row. 32cb4a6af842087ee4ba41c4dd31b2bfe1cdd018,Sven Klemm,Thu Sep 19 19:48:48 2019 +0200,Add tableoid support for transparent decompression,This patch adds support for the tableoid system column to transparent decompression. tableoid will be the relid of the uncompressed chunk. All other system columns will still throw an error when queried. 0606aeba9e83852d943b040a06d701d10cf6ad21,Joshua Lockerman,Thu Sep 19 12:57:21 2019 -0400,Reduce Peak Memory Usage for compress_chunk,"Before this PR some state (most notably deTOASTed values) would persist across compressed rows during compress_chunk, despite the fact that they were no longer needed. This increased peak memory usage of compress_chunk. This commit adds a MemoryContext that is reset after each compressed row is inserted, ensuring that state needed for only one row does not hang around longer than needed. " d2d21abbcc2e20c9b90e9ab7981bfdeb52570d48,Sven Klemm,Tue Sep 17 20:49:27 2019 +0200,Improve pathkeys pushdown for transparent decompression,Valid orderings that can be produced by the DecompressChunk node require all segmentby columns to be prefix of pathkeys. This patch relaxes this requirement and allows segmentby columns to be left out of pathkeys iff they have an equality constraint in baserestrictinfo. 2bf126a0e7230b2445ef158a10d05486ac68d6e3,Joshua Lockerman,Tue Sep 17 12:39:27 2019 -0400,Add a per-batch MemoryContext to DecompressChunkState,This commit adds a per-batch MemoryContext to DecompressChunkState that is freed every time row in the compressed table is finished. This is the earliest time we can free such memory (some decompressors return Datums that point directly into the DecompressionIterator) and allows us to use much less memory. 47da7292366ca432f0b8a408b0a38d76af303396,Joshua Lockerman,Mon Sep 16 15:07:20 2019 -0400,Allow pushdown of quals containing params,"This commit enables the pushdown of quals containing a param. This allows us to filter on such expression before decompression, speeding up eg lastpoint approximately 2x " 15028421340e11a6f267dda90c34ee0ad9969f4b,Joshua Lockerman,Thu Sep 12 17:21:40 2019 -0400,Improve JOIN handling for compressed chunks,"This commit improves the JOIN handling of compressed hypertables, specifically enabling NestLoop JOINs that filter on segmentby columns before the table is decompressed. It does this via the following changes: 1. It adds equivalence-members relating each segmentby on the decompressed-chunk to the equivalent column on the compressed chunk. 2. It pulls the ParamInfo and has_eclass_joins from the compressed path to the DecompressedPath. 3. It fixes up the quals in the DecompressPlan to remove ones redundant with IndexScans, and to ensure that any remaining quals refer to columns on the decompressed-chunk. " 939489d2d7e9e5cc70ca4ad0efcc4aba545a666f,gayyappan,Tue Sep 3 17:24:56 2019 -0400,Permission tests for compression commands,Add tests to check that users without adequate privileges cannot execute compression related commands. 4c3bb6d2d6ef85de6a393afdc47d42fc0f43ebcb,Sven Klemm,Mon Sep 16 18:12:38 2019 +0200,Add JOIN tests for transparent decompression, 6465a4e85a0ceab7bd64c9b437c8f94ef6fb8f73,Matvey Arye,Mon Sep 16 17:39:59 2019 +0200,Switch to using get_attnum function,This is a fix for a rebase on master since `attno_find_by_attname` was removed. 4140c58f1be327c1b86c2a1ec717d93223d41523,Matvey Arye,Thu Sep 12 12:06:36 2019 -0400,Update postgresql.conf used for testing,The windows tests apply the non-tsl config options to the TSL tests. Update the linux postgresql.conf files to use the same semantics. This caused a change in the transparent_decompression golden file because of the change to random_page_cost for the tsl test. 965054658ecbb699b71aaed1b865d9b06c29b001,Joshua Lockerman,Wed Sep 11 12:15:26 2019 -0400,Enable IndexScans on compressed chunks,"This commit enables IndexScans on the segmentby columns of compressed chunks, if they have an index. It makes three changes to enable this: 1. It creates a DecompressChunkPath for every path planned on the compressed chunk, not only the cheapest one. 2. It sets up the reltargetlist on the compressed RelOptInfo accurately reflect the columns of the compressed chunk that are read, instead of leaving it empty (needed to prevent IndexOnlyScans from being planned). 3. It plans IndexPaths, not only SeqScanPaths. " e2c03e40aa08245dbc007addeab3394a527b3911,Sven Klemm,Mon Sep 9 20:20:11 2019 +0200,Add support for pathkey pushdown for transparent decompression,This patch adds support for producing ordered output. All segmentby columns need to be prefix of pathkeys and the orderby specified for the compression needs exactly match the rest of pathkeys. 8250714a299d3f25722f26d228b4aa006cd269bb,Matvey Arye,Tue Sep 10 17:09:24 2019 -0400,Add fixes for Windows,- Fix declaration of functions wrt TSDLLEXPORT consistency - Empty structs need to be created with '{ 0 }' syntax. - Alignment sentinels have to use uint64 instead of a struct with a 0-size member - Add some more ORDER BY clauses in the tests to constrain the order of results - Add ANALYZE after running compression in transparent-decompression test be946c436d5fefbdd92637529a1b9412f20121a2,Matvey Arye,Fri Sep 6 18:41:34 2019 -0400,Block add_drop_policy on internal compressed table,Add drop policy can only be added to the public uncompressed hypertable. This blocks a call when a policy is added to the internal hypertable with compressed data. df4c4445513de501624607368f17fca64842a84d,Matvey Arye,Fri Sep 6 17:20:33 2019 -0400,Delete related rows for compression,"This fixes delete of relate rows when we have compressed hypertables. Namely we delete rows from: - compression_chunk_size - hypertable_compression We also fix hypertable_compression to handle NULLS correctly. We add a stub for tests with continuous aggs as well as compression. But, that's broken for now so it's commented out. Will be fixed in another PR. " 06557257f55b0083e3eda5599046016402c87e22,Matvey Arye,Fri Sep 6 12:49:28 2019 -0400,Fix the chunk model to handle NULLs correctly,"Since chunks now have NULL fields, some cleanup was necessary. Namely we remove all direct GETSTRUCT usage and instead move to a method that uses heap_form/deform_tuple. We also cleanup some naming. The catalog_insert function for tuples was made public to ease unifying interfaces for going from formdata->tuples. " 0db50e7ffc7e5626349ea835f995ad0ef32db4d8,Matvey Arye,Wed Aug 28 21:20:25 2019 -0400,Handle drops of compressed chunks/hypertables,This commit add handling for dropping of chunks and hypertables in the presence of associated compressed objects. If the uncompressed chunk/hypertable is dropped than drop the associated compressed object using DROP_RESTRICT unless cascading is explicitly enabled. Also add a compressed_chunk_id index on compressed tables for figuring out whether a chunk is compressed or not. Change a bunch of APIs to use DropBehavior instead of a cascade bool to be more explicit. Also test the drop chunks policy. a4773adb58fd91cf83d0a0220fc56c2d0523a288,Matvey Arye,Fri Sep 6 14:20:02 2019 -0400,Make compression feature use the community license,"Compression is a community, not enterprise feature. " 2bf97e452d1b9659e6d770e4e2ddcb77b83c72ad,Matvey Arye,Fri Aug 30 09:46:28 2019 -0400,Push down quals to segment meta columns,"This commit pushes down quals or order_by columns to make use of the SegmentMetaMinMax objects. Namely =,<,<=,>,>= quals can now be pushed down. We also remove filters from decompress node for quals that have been pushed down and don't need a recheck. This commit also changes tests to add more segment by and order-by columns. Finally, we rename segment meta accessor functions to be smaller " 6e60d2614cf000de83d11e2cd29d46d53d036ce0,gayyappan,Wed Aug 21 13:01:05 2019 -0400,Add compress chunks policy support,Add and drop compress chunks policy using bgw infrastructure. 5c891f732e20a9b10745393e41a1d870ea8a54d3,Matvey Arye,Tue Aug 27 12:27:43 2019 -0400,Add sequence id metadata col to compressed table,"Add a sequence id to the compressed table. This id increments monotonically for each compressed row in a way that follows the order by clause. We leave gaps to allow for the possibility to fill in rows due to e.g. inserts down the line. The sequence id is global to the entire chunk and does not reset for each segment-by-group-change since this has the potential to allow some micro optimizations when ordering by a segment by columns as well. The sequence number is a INT32, which allows up to 200 billion uncompressed rows per chunk to be supported (assuming 1000 rows per compressed row and a gap of 10). Overflow is checked in the code and will error if this is breached. " 6d0dfdfe1a5c5ff8c5cd04a7cb04b2e681c158ca,Joshua Lockerman,Wed Aug 28 11:01:22 2019 -0400,Switch Timestamptz to use deltadelta and bugfixes,"Timestamptz is an integer-like type, and thus should use deltadelta encoding by default. Making this change uncovered a bug where RLE was truncating values on decompression, which has also been fixed. " f2e4266aa033b2dbfaa1cf8ecccbf8e1f4a736f6,Joshua Lockerman,Wed Aug 28 15:30:19 2019 -0400,Don't de-toast compressed values multiple times,"This is a performance fix, as detoasting multiple times is expensive. " b4a71084920fa7de8a0e337f008bd52fe9b6f324,Matvey Arye,Mon Aug 26 12:54:21 2019 -0400,Integrate segment meta into compression,This commit integrates the SegmentMetaMinMax into the compression logic. It adds metadata columns to the compressed table and correctly sets it upon compression. We also fix several errors with datum detoasting in SegmentMetaMinMax be199bec70eedcd79a1fb7daf00de7bcfe73afa0,Matvey Arye,Sun Aug 25 20:51:28 2019 -0400,Add type cache,Add a type cache to get the OID corresponding to a particular defined SQL type. 42a2c8666ef31b8c717f2d74ba3287c7a654f99f,Sven Klemm,Mon Aug 26 07:47:30 2019 +0200,Fix DecompressChunk parallel execution,When DecompressChunk is used in parallel plans the scan on the compressed hypertable chunk needs to be parallel aware to prevent duplicating work. This patch will change DecompressChunk to always create a non parallel safe path and if requested a parallel safe partial path with a parallel aware scan. abbe5c84fdd05f2087cf9d544d3371721c492843,Joshua Lockerman,Mon Aug 26 13:02:39 2019 -0400,Test all compressors with single-value tables,Single-value tables have previously had bugs in deltadelta and is a good edge case to have in general. 2b1e950df300aa43790cd4db05b25224657647ae,Joshua Lockerman,Mon Aug 26 12:09:00 2019 -0400,Store first deltadelta element in simple8b,"This commit changes deltadelta compression to store the first element in the simple8b array instead of out-of-line. Besides shrinking the data in some cases, this also ensures that the simple8b array is never empty, fixing the case where only a single element is stored. " 3d55595ad0d3fc665d83fc09f78761fc1fb8312e,Sven Klemm,Mon Aug 26 10:42:44 2019 +0200,Fix error hint for compress_chunk,The errror hint for compress_chunk misspelled the option to use for enabling compression. This patch changes the error hint and also makes the hint a proper sentence. b9674600ae4771ec6552c6154fea30069515920f,Matvey Arye,Fri Aug 23 11:25:24 2019 -0400,Add segment meta min/max,Add the type for min/max segment meta object. Segment metadata objects keep metadata about data in segments (compressed rows). The min/max variant keeps the min and max values inside the compressed object. It will be used on compression order by columns to allow queries that have quals on those columns to be able to exclude entire segments if no uncompressed rows in the segment may match the qual. We also add generalized infrastructure for datum serialization / deserialization for arbitrary types to and from memory as well as binary strings. b1a5000b5c8451ad62c8c5d5498dc7d2cd859dfb,Sven Klemm,Thu Aug 22 08:53:48 2019 +0200,Improve qual pushdown for transparent decompression,"This patch adds support for pushing down IS NULL, IS NOT NULL and ScalarArrayOp expression to the scan on the compressed chunk. " 8b273a5187feede8295dd8cb94f52a85b97ce3fc,Joshua Lockerman,Thu Aug 22 11:22:01 2019 -0400,Fix flush when num-rows overflow,"We should only free the segment-bys when we're changing groups not when we've got too many rows to compress, in that case we'll need them. " dcc1d902d1953c2a42d53cc45790b22e9daf519d,Matvey Arye,Wed Aug 21 11:50:53 2019 -0400,Add more compression tests,Adds several test features: - Add a tests on compression of an altered hypertable - Adds tests for transparent decompression in hypertables tests - Add dump/restore in hypertable test 45fac0ebe6bfe8d486b07327ea9590d4ed44d789,Sven Klemm,Wed Aug 21 14:41:22 2019 +0200,Add test for compress_chunk plan invalidation,This patch adds a testcase for prepared statement plan invalidation when a chunk gets compressed. ea7d2c7e60a485edc6eeae5df74b98ee1b60bfff,Matvey Arye,Tue Aug 20 16:55:32 2019 -0400,Enforce license checks for compression,"Enforce enterprise license check for compression. Note: these checks are now outdated as compression is now a community, not enterprise feature. " 70b43482e9506053c41b7d6ac9631dab650032ef,Sven Klemm,Wed Aug 21 13:53:36 2019 +0200,Fixup for rebase against master, 7c52e82aafc129e3016a80d9e38c79111b8e0a25,Sven Klemm,Tue Aug 20 22:29:12 2019 +0200,Use hypertable selectedCols for building scan on compressed chunk,Since we do not adjust selectedCols for the Chunk RangeTblEntry attribute numbers in selectedCols will be wrong if attribute numbers on hypertable and chunk differ. This patch changes the target list creation to use the hypertable selectedCols and look up names on the hypertable to work around this. 6832ed2ca5e33897fa2b9fdfd7309381fae5fcc6,gayyappan,Mon Aug 19 17:23:08 2019 -0400,Modify storage type for toast columns,This PR modifies the toast type for compressed columns based on the algorithm used for compression. b1a34496938a6649d2ecba9773a6d0fccd454ac8,Matvey Arye,Tue Aug 20 11:15:00 2019 -0400,Add missing test to CMakeLists, bce292a64f3fbef62cbd473cfa80bd22a06533d9,Matvey Arye,Mon Aug 19 16:16:46 2019 -0400,Fix locking when altering compression options,Take an exclusive lock when taking compression options as it is safer. 0059360522a4dfeda5d4db6ca3272f8dfe449fe8,Matvey Arye,Sun Aug 18 19:19:46 2019 -0400,Fix indexes during compression and decompression,"This rebuilds indexes during compression and decompression. Previously, indexes were not updated during these operations. We also fix a small bug with orderby and segmentby handling of empty strings/ lists. Finally, we add some more tests. " cdf6fcb69a806ba3caab2566e030c536fcdb0fd9,Matvey Arye,Fri Aug 16 14:58:48 2019 -0400,Allow altering compression options,We now allow changing the compression options on a hypertable as long as there are no existing compressed chunks. eba612ea2edd5c148dd4e85c230449ca8bec9494,Matvey Arye,Thu Aug 15 15:17:34 2019 -0400,Add time column to compressed order by list,Add the column to the order by list if it's not already there. This is never wrong and might improve performance. This also guarantees that we have at least one ordering column during compression and therefore can always use tuplesort (o/w we'd need a non-tuplesort method of getting tuples). 4cc1a4159af68a5bfdab4880e392f809652cd681,Sven Klemm,Thu Aug 15 15:18:16 2019 +0200,Add DecompressChunk custom scan node,"This patch adds a DecompressChunk custom scan node, which will be used when querying hypertables with compressed chunks to transparently decompress chunks. " 6f22a7a68cfd2af45a29242a5f51f147275875f8,Matvey Arye,Thu Aug 15 17:05:27 2019 -0400,Improve parsing of segment by and order by lists,"Replace custom parsing of order by and segment by lists with the postgres parser. The segment by list is now parsed in the same way as the GROUP BY clause and the order by list in the same way as the ORDER BY clause. Also fix default for nulls first/last to follow the PG convention: LAST for ASC, FIRST for DESC. " f6573f92475ffea72dd40315d863f92161d04b53,Matvey Arye,Thu Aug 15 15:17:34 2019 -0400,Add a metadata count column to compressed table,"This is useful, if some or all compressed columns are NULL. The count reflects the number of uncompressed rows that are in the compressed row. Stored as a 32-bit integer. " a078781c2e6f6a14604c18cf2d2cf758ebeace24,Matvey Arye,Wed Aug 14 23:09:11 2019 -0400,Add decompress_chunk function,This is the opposite dual of compress_chunk. bdc599793c615115af48b49fbbbef3e884d9afd2,Sven Klemm,Thu Aug 15 13:29:57 2019 +0200,Add helper function to get decompression iterator init function, a5a3dca517f54b22a9ad47b587446563c92fbfa2,Sven Klemm,Sat Jul 13 20:17:27 2019 +0200,Add GUC for transparent decompression,This GUC will control whether the planner automatically queries compressed data when some chunks in a query are compressed. 47c1d7e32341fa267e3b68e720eb96a0536d8352,Sven Klemm,Mon Jul 8 22:43:53 2019 +0200,Add set_rel_pathlist hook for tsl code,Will be needed for compression. 9223f08d68aada12f934862353820d093577a4c5,Matvey Arye,Wed Aug 14 13:21:28 2019 -0400,Truncate chunks after (de-)compression,This commit will truncate the original chunk after compression or decompression. 5bdb29b8f75cd87dd6e30b34dfc28e601ddf1933,Matvey Arye,Wed Aug 14 15:52:06 2019 -0400,Fix compression for PG96,Fixes some compilation and test errors. 7a728dc15f8c2b8d9e982e122268fa39bebb5001,gayyappan,Mon Aug 12 17:24:11 2019 -0400,Add view for compression size,View for compressed_chunk_size and compressed_hypertable_size 1f4689eca92851523c8c854f28c00108a5711ecb,gayyappan,Mon Aug 12 15:31:30 2019 -0400,Record chunk sizes after compression,Compute chunk size before/after compressing a chunk and record in catalog table. 44941f7bd291698c4e7fc412295656ecc53ce734,gayyappan,Thu Jul 25 18:30:39 2019 -0400,Add UI for compress_chunks functionality,Add support for compress_chunks function. This also adds support for compress_orderby and compress_segmentby parameters in ALTER TABLE. These parameteres are used by the compress_chunks function. The parsing code will most likely be changed to use PG raw_parser function. bb89e62629ca05961da55c37f17727bf063b2322,Joshua Lockerman,Wed Aug 7 13:42:23 2019 -0400,Add fallback from dictionary compressor to array,"Dictionary compression can be a pessimization if there aren't many repeated values. Since we want to have a single fallback compressor we can recommend when one of the more specialized compressors aren't appropriate, this commit adds a fallback where, if it would be more efficient to store data as an array instead of dictionary-compressed, the dictionary compressor will automatically return the value as an array. " fa26992c4ca4142b968aab17c5ac8bb37e69d9eb,Joshua Lockerman,Wed Aug 7 13:04:46 2019 -0400,Improve deltadelta and gorilla compressors,- Add fallback compressors for deltadelta/gorilla - Add bool compressor for deltadelta 6d55f6f615e9b71827fd3dbda1e2a81863b64bae,Joshua Lockerman,Wed Aug 7 15:53:13 2019 -0400,Add decompress_chunk,"This function is the inverse of compress_chunk: it takes a table containing compressed data decompresses it, and writes it out to another table. " da7bcedc3ea741c0b37b58609e7feb5816a27c9e,Joshua Lockerman,Wed Aug 7 10:08:10 2019 -0400,Fix name of compression test, 2f16d84c395f7e81316a8cf10be1eec6fca23570,Joshua Lockerman,Fri Jul 26 15:08:38 2019 -0400,Add ability to compress tables,"We eventually want to be able to compress chunks in the background as they become old enough. As an incremental step in this directions, this commit adds the ability to compress any table, albeit with an unintuitive and brittle interface. This will eventually married to our catalogs and background workers to provide a seamless experience. This commit also fixes a bug in gorilla in which the compressor could not handle the case where the leading/trailing zeroes were always 0. " 1c6aacc37414464f8156fff8df0a54076ab06925,gayyappan,Wed Jul 17 16:12:57 2019 -0400,Add ability to create the compressed hypertable,This happens when compression is turned on for regular hypertables. 584f5d1061eb0f4221766499aa26cf40cf105835,Joshua Lockerman,Mon May 6 16:06:46 2019 -0400,Implement time-series compression algorithms,This commit introduces 4 compression algorithms as well as 3 ADTs to support them. The compression algorithms are time-series optimized. The following algorithms are implemented: - DeltaDelta compresses integer and timestamp values - Gorilla compresses floats - Dictionary compression handles any data type and is optimized for low-cardinality datasets. - Array stores any data type in an array-like structure and does not actually compress it (though TOAST-based compression can be applied on top). These compression algorithms are are fully described in tsl/src/compression/README.md. The Abstract Data Types that are implemented are - Vector - A dynamic vector that can store any type. - BitArray - A dynamic vector to store bits. - SimpleHash - A hash table implementation from PG12. More information can be found in src/adts/README.md 3edc016dfcad58d9214df8e3ebd00fd307be03f4,gayyappan,Thu Jul 11 17:58:12 2019 -0400,Add catalog tables to support compression,This commit adds catalog tables that will be used by the compression infrastructure. 3ad6a5cdc0dbb17b2727956162edff4d3e86ef4b,Matvey Arye,Mon Oct 28 16:50:14 2019 -0400,Fix interpolate for ARM32,Interpolate was trying to use a datum as an int64. This is a bug that was flagged on arm. bf9eea05950df2925eeab7ed4ad355064281ea5b,Matvey Arye,Sun Oct 27 14:07:30 2019 -0400,Fix overflow in gapfill's interpolate,All integer types must use numeric-based interpolation calculations since they are multiplied by int64 and this could cause an overflow. numerics also interpolate better because the answer is rounded and not truncated. We can't use float8 because that doesn't handle really big ints exactly. We can't use the Postgres INT128 implementation because it doesn't support division. In the future we can optimize this for cases where overflow doesn't occur. Fixes #1491. f82d83783e8c69b169a7d81deb1b4d3b30add573,Matvey Arye,Wed Oct 23 22:06:51 2019 -0400,Prevent recursion in invalidation processing,"This patch prevents recursing into the cache invalidation code. Since this code's result is dependent on the transaction snapshot, processing it multiple times recursively won't change the result. Fixes #1486 " 267451f2ca5eb34e0163e9f38850a0df59bf98a6,Björn Esser,Tue Oct 22 23:03:54 2019 +0200,Allow PG_*DIR to be set during build (#1469),"When cross-compiling TimescaleDB, having more fine-grained macros to set is helpful. This commit allows defaults from `pg_config` such as the libdir, etc to be overridden. Signed-off-by: Björn Esser " 7ea492f29e4e096aa3e1f8d32104343c423db0c2,Matvey Arye,Fri Oct 11 14:01:30 2019 -0400,Add last_successful_finish to bgw_job_stats,This allows people to better monitor the bgw job health. It indicates when the last time the job made progress was. f36d68bc3746a64fa4fa4d8a5804482f496be580,Sven Klemm,Tue Oct 15 20:18:23 2019 +0200,Only use local clang-format 7 or 8,"clang-format > 8 has slightly different oppinion about source code formatting than earlier versions, unfortunately some of the new options cannot be set in a backwardscompatible way so we fall back to docker for newer versions. " d82ad2c8f6e0759bba48b72b1e013623109c5054,Sven Klemm,Tue Oct 15 02:39:58 2019 +0200,Add ts_ prefix to all exported functions,This patch adds the `ts_` prefix to exported functions that didnt have it and removes exports that are not needed. 81d5db662af47aba34c2b396c07c0faffb1f91ec,Sven Klemm,Mon Oct 14 11:25:50 2019 +0200,Fix export_prefix_check script,The export_prefix_check script would not set proper exit code depending on the found exported functions. 53753abdf9e20b0bf8752bd63f9db39528ee7e07,Sven Klemm,Sun Oct 13 20:43:47 2019 +0200,Make bgw_reorder_drop_chunks test output reproducable,The test output from bgw_reorder_drop_chunks depends on timing making the test output differ in slower test environments. This patch truncates the timing information with time_bucket. 3082c5bb8bf654f23cdd5997b9c8f509590a1172,Sven Klemm,Sat Oct 12 09:31:50 2019 +0200,Bump the postgres version PR tests run against,"Changes the PR tests to run against 9.6.15, 10.10 and 11.5 " 77ac7ccd89a1519f5013ba2d2eeeee143ceabfa1,Sven Klemm,Sun Oct 13 17:19:30 2019 +0200,Fix flaky bgw_db_scheduler test,"Change handling of background workers on rename database, the previous implementation would restart background workers on RENAME DATABASE unfortunately this leads to a race condition where its possible that the background workers are restarted while the RENAME DATABASE has not yet finished leading to the command not completing successfully. This patch changes the restart to a stop and generating a warning that background jobs need to be manually restarted. This patch also fixes 2 segfaults in the background worker test code. " 4ad2a2f46ae50fdd4c1b0c0c224d0c3b6cb8fa06,Sven Klemm,Sun Oct 13 13:35:08 2019 +0200,Add database to log_line_prefix used for tests,This patch adds the database to log_line_prefix to make it easier to correlate test output with entries in the postgres log. 43e7fd6aa5843206f37668cb6053edc15c50cbc2,Sven Klemm,Sat Oct 12 05:24:17 2019 +0200,Fix negative array index access in ChunkAppend worker assignment,When all subplans are excluded during runtime exclusion negative array index access might be possible on the ChunkAppend finished state. This patch fixes the check for a valid subplan index. d61efef0b93458c29ce7f929269b1e20bce69fb7,Sven Klemm,Sat Oct 12 04:26:44 2019 +0200,Fix coverity travis task,Since the base image for travis changed from ubuntu trusty to xenial the package name for cmake has to be adjusted to make the coverity test work. 9b2b5da9170158bd93dffd9d9a73fa78670f1cf2,Sven Klemm,Fri Oct 11 14:25:26 2019 +0200,Fix gapfill treat_null_as_missing,When locf was used with the treat_null_as_missing option and the query ORDER BY did not match gapfill ordering the treat_null_as_missing option was not working correctly because the values from orignal tuple would be returned and not our modified ones. 13f7984e8f5ec027aec5f5483ed6ffb53a2a7525,Sven Klemm,Sat Oct 12 00:17:53 2019 +0200,Make continuous_aggs_bgw test output consistent,One of the test functions in the continuous_aggs_bgw test would produce an output message for each 0.1 seconds it took to wait for the result which leads to changing output depending on test environment the test is run on. This patch removes the warning to make the output consistent. d85368b2f60d56a0fc130303b2f727eca8bcc3d6,Sven Klemm,Fri Oct 11 23:58:07 2019 +0200,Remove extern from ChunkAppend functions that don't need exporting, 8034486d00838db5a5f472666ff3a30df9c41584,David Kohn,Fri Oct 11 13:25:22 2019 -0400,Set default retry_period for jobs to 5 minutes,"The default retry_period for jobs was set too high, causing jobs that failed because of lack of workers to be restarted too infrequently, this sets it significantly lower, to 5 minutes, though exponential backoff will cause that to be larger as multiple failures stack up. " 2209133781d0c371f6a794fdcfbc036f7d23f4ae,Matvey Arye,Fri Oct 11 11:08:57 2019 -0400,Add next_start option to alter_job_schedule,Add the option to set the next start time on a job in the alter job schedule function. This also adds the ability to pause jobs by setting next_start to 'infinity' Also fix the enterprise licence check to only activate for enterprise jobs. 01f2bbaf5aa2de9278fcd5023a2d5102e4b45779,Matvey Arye,Wed Oct 9 15:07:26 2019 -0400,Add better errors for no permission in callbacks,Have better permission errors when setting the integer now func and the partitioning func. Also move tests from tsl to apache2 for the now func. d2f68cbd64d43dc1aa2f41384ac9ae8568f3c37a,Matvey Arye,Wed Oct 9 12:33:05 2019 -0400,Move the set_integer_now func into Apache2,We decided this should be an OSS capability. 37c5bfdf965eed6a874d611ca0a1e56b7ce78109,Matvey Arye,Thu Oct 10 16:03:01 2019 -0400,Fix logic for BGW rescheduling,"This fixes the logic for what happens when the next start time is changed on the bgw job: - When changing the schedule_interval, the new next start time is set to last_finish + new_schedule_interval instead of now() + new_schedule_interval. This makes it easier to test and is semantically more correct - New logic is added in the scheduler to recalculate the scheduler's next start time - Better tests are added - System bgw scheduler is disabled in tests Fixes #1462 " b0c75d3106db7dd5bd2cef52cebbcd804868fa2d,Sven Klemm,Thu Oct 10 15:29:33 2019 +0200,Speed up parallel test,Since we can now set reloptions on chunks we don't have to rely on data volume to trigger parallel plans. 0d9e6425dc4d2fee161a6df6a78e7742d603f188,Ruslan Fomkin,Wed Oct 9 16:23:47 2019 +0200,Disable autovacuum in tests,"The tests do not expect that autovacuum is run. The most of the tests finishes before it runs, but some ARM tests might fail due if autovacuum is executed. " 36bff37daf68b37de3bc757d6820d86df5d82130,Sven Klemm,Tue Oct 8 11:12:35 2019 +0200,Add support for non-partial paths to parallel ChunkAppend,PostgreSQL will produce plans where a parallel append has children that are not partial paths. This patch adds support for those plans to ChunkAppend. fc9a3e8bc6de9e27faf52cdf6eac964c74c2d64a,Sven Klemm,Tue Oct 8 21:42:06 2019 +0200,Allow setting reloptions on chunks,This patch unblocks setting reloptions on chunks. This was originally blocked out of an overabundance of caution. It is now enabled for better control over parallel operations. 53384112ef40dccecb1f71697e75643080723fc2,Ruslan Fomkin,Tue Oct 8 15:58:27 2019 +0200,Disable running 11.0 on every branch on Travis-CI,"Since a build job for PG 11.4 is run on every PR and every feature branch, there is no need to run PG 11.0 on every commit on every feature branch. " 57777bc9645203531166ef7e83a2127223828c8d,Mats Kindahl,Wed Aug 21 16:36:13 2019 +0200,Enable some compiler warnings and fix code,"It is not possible to enable both `-Wall` and `-Wextra` since that generate too many warnings, so enabling some additional warnings to get a decent level of checking. Warnings enabled are: `-Wempty-body` and `-Wvla`. For GCC version 7 and above and Clang version 3.3 and later: `-Wimplicit-fallthrough` " 524ba815780db669f0a9830438349ffe5a4b4bcd,Sven Klemm,Tue Oct 8 12:41:10 2019 +0200,Fix clang-format and licence check on travis,"The ASAN_OPTIONS to disable leak check for this test were not properly passed down into the container, this patch sets them as environment variables in the container as well. " 4cd645eb72f4999f4042c75b499385475397609b,Sven Klemm,Sun Oct 6 19:05:11 2019 +0200,Improve ChunkAppend parallel worker assignment,"The initial implementation of assigning multiple workers per subplan would always assign a fixed amount of workers per child. This patch changes the logic to assign each child one worker and then moving to the next subplan, iterating until all subplans are finished. This will lead to a much better distribution of workers especially when the children are not well balanced. " ddb8f46b5f0137bf6d65b3b743ff62b7416a2860,Sven Klemm,Fri Oct 4 15:19:27 2019 +0200,Fix ordered append with space partitioning,Ordered append for space partitioned hypertable would lead to an error when the ORDER BY clause was not a column reference on PG 9.6 and PG 10. This patch fixes ordered append for space partitioned hypertable and allows arbitary expressions to be used in the ORDER BY clause. 3fa3822d08e74d5045edec22c8b5b900105b8c4c,Mats Kindahl,Thu Oct 3 15:43:51 2019 +0200,Only update reformatted files,"When running `make format` it will update all files. If a file does not actually require reformatting, it will trigger a re-build of the entire code base. This commit instead compare the re-formatted file with the existing one and does not update it unless it has actually changed. " 3fe9f13d445bbc935320c6dbe1c502ead6f93f69,Joshua Lockerman,Mon Sep 23 16:21:24 2019 -0400,Fix memory leaks part 2: Licenses,This commit fixes memory leaks in the license-deserialization infrastructure. 1cc1b7c14e793d13f7f047f54babcc231caa36d4,Joshua Lockerman,Mon Sep 23 16:19:58 2019 -0400,Fix memory leaks part 1: BGWs,"This commit fixes memory leaks in the Background Worker infrastructure. Notably, it fixes a leak in the launcher and BGW scheduler where shared message queue handles would not be freed, and it fixes a leak where BackgroundWorkerHandles were not freed in the scheduler. " 7553cdcd5417a457a821090481dc3f73635feb9b,Sven Klemm,Wed Sep 25 18:03:12 2019 +0200,Enable multiple worker per child for ChunkAppend,This patch changes ChunkAppend to assign multiple worker per child if the number of worker exceeds the number of children. a3a49703aababc1fea7b016f87dd061a3dcb27d6,Sven Klemm,Tue Sep 24 20:23:36 2019 +0200,Remove get_function_oid from utils.c,The get_function_oid function was a reimplementation of PostgreSQL LookupFuncName. This patch removes the function and switches all callers to use LookupFuncName instead. caf2ab439b1dfda535ddf355dd8cc08e7e1cfb01,Sven Klemm,Tue Sep 24 01:02:51 2019 +0200,Fix postgres version reported by telemetry,Telemetry would send the postgres version the extension was compiled against as postgres_version instead of the postgres version that is currently running. This patch fixes telemetry to send the version actually in use. a1a74f9530ecf00f015ab875895c95e78f48ad89,Sven Klemm,Fri Sep 20 18:50:52 2019 +0200,Filter Memory Usage from EXPLAIN ANALYZE output,Since memory usage is not stable and leads to test failures we filter the memory usage before comparing test results. e41aea6f22015dc50d1518ce3f2fa3b9bcbecfe9,Sven Klemm,Fri Sep 20 14:34:38 2019 +0200,Fix runtime exclusion memory leak,The intermediate expressions used for runtime exclusions would only be freed at the end of a query leading to excessive memory usage for nested loop joins with many loops. This patch changes runtime exclusion to free intermediate expressions as soon as they are no longer needed. 38b0c538e3ebe4fd3a905bc24d4af13bd00f5ae5,Sven Klemm,Wed Sep 18 17:10:11 2019 +0200,Fix append test flakyness,Some of the append test queries relied on ANALYZE not having run yet to produce the desired plan. This patch changes the test to generate more test data and runs ANALYZE explicitly so the tests are reproducible. 441dc241da0cf157804f1f7a16570b350264a27d,Sven Klemm,Fri Aug 9 15:20:55 2019 +0200,Make ChunkAppend parallel aware,This patch makes ChunkAppend parallel aware allowing ChunkAppend to coordinate parallel workers. Since setting up the shared memory for the LWLock has to happen in the library loaded with shared_preload_libraries this change requires a server restart. e2ce574934464d33063cb932f59a9f4c4220335d,Sven Klemm,Mon Sep 16 13:15:55 2019 +0200,Prevent LIMIT pushdown in JOINs,Pushing down limit in ChunkApppend is not safe in the JOIN case because the decision about required tuples can happen above ChunkAppend node. This patch disables LIMIT pushdown for all JOINs. b86e47a8a19216e582a51e75e57c2b93394d69e3,Sven Klemm,Mon Sep 16 08:25:27 2019 +0200,Fix microsoft compiler warnings,The microsoft compiler can't figure out that elog(ERROR) doesn't return and warns about functions not returning a value in all code paths. This patch adds pg_unreachable calls to those functions. 468c205a4f79481c684fecdef50d1a3ab13a0dc5,Sven Klemm,Fri Sep 13 13:55:47 2019 +0200,Remove attno_find_by_attname and use get_attnum instead,The patch removes the custom implementation to find the attribute number for a column and uses PostgreSQL get_attnum function instead. c81248391f0b30c5e75e23021bc2fc007551a2e6,Sven Klemm,Fri Sep 13 13:52:08 2019 +0200,Use ts_get_appendrelinfo instead of static function,Remove the static get_appendrelinfo function from ConstraintAwareAppend and use the function defined in utils.c instead. b2d7be31bbd8539dac2f0cf8e61fc9e9cba908ab,Sven Klemm,Thu Sep 12 08:54:24 2019 +0200,Add 1.4.2 to update test scripts, 33a75ab09d851915d1938be1420eef5db07a8d1b,Sven Klemm,Mon Sep 9 17:21:34 2019 +0200,Release 1.4.2,"This maintenance release contains bugfixes since the 1.4.1 release. We deem it medium priority for upgrading. In particular the fixes contained in this maintenance release address 2 potential segfaults and no other security vulnerabilities. The bugfixes are related to background workers, OUTER JOINs, ordered append on space partitioned hypertables and expression indexes. " d793357323eb42c072586dd5a4bf4f0a36d20ce2,Sven Klemm,Mon Sep 9 10:13:24 2019 +0200,Disable check for pg_isolation_regress for coverity, 462bd20a6dd2f8d29058769e32e42e8d0777451c,Sven Klemm,Mon Sep 9 09:09:04 2019 +0200,Ignore bgw_job_delete test result on ARM,Ignore the result of the bgw_job_delete test on ARM because it is flaky. 32629ca2a1542f3558066057247c1754bafc3b71,Sven Klemm,Sat Sep 7 14:14:17 2019 +0200,Fix expression index creation,Creating an expression index on a hypertable with dropped columns would lead to an error on the next INSERT into the hypertable because the attribute numbers in the expression were not properly adjusted. This patch fixes the adjusting of the attribute numbers for expression indexes. 08d9f84736aa0f7c7f981f6378a524d5e341fa59,Matvey Arye,Wed Aug 14 09:46:16 2019 -0400,Add testing for bgw delete,Adds test infrastructure for deleting bgw jobs. We do this using isolation tests. We add infrastructure for running bgws in isolation tests. We also add infrastructure to control which user runs the bgw scheduler and jobs during the tests. We test the bgw delete as non-superuser. c29b3edf23048ec6f187c63c6641e18cc0b627f6,Matvey Arye,Wed Aug 7 21:42:35 2019 -0400,Fix deletes of background workers,"Previously there were at least 2 problems with deletes of BGWs: 1) A delete of a job or job_stats row while a BGW was running would often cause ""tuple concurrently updated"" errors in the deleting backend, worker, or, worst of all, scheduler. Sometimes the scheduler would abort because of these errors. 2) A delete of a job did not cause the BGW to quit, thus this operation could take a long time and would not help with misbehaving BGWs. This commit solves both problems. We introduce a new job-row advisory lock that is taken by any running worker and that prevents concurrent job deletes. We also use this lock to determine the PID of the worker and cancel (SIGINT) the worker during job deletes. Furthermore we make the scheduler more gracefully handle unexpected job deletions and take share locks on the job row when performing operations on the row. Fixes #1386 " a986a75afa27b0edf1e27d01ed90b30046503738,Sven Klemm,Wed Sep 4 10:48:51 2019 +0200,Improve ARM tests,"Output postgres error log after ARM test failure The append test is flaky on ARM and sometimes switches to use a Sort -> ChunkAppend instead of MergeAppend. This patch disables ChunkAppend for this test since we want to test interaction with MergeAppend in the test. On ARM the continuous_aggs_bgw test is flaky and fails quite often, so this patch changes CI to ignore the result of that test on ARM. " b784a5f4102e0d28d4a48faba5b8f6c32d0e0b2a,Sven Klemm,Mon Sep 2 17:44:44 2019 +0200,Fix ChunkAppend LIMIT pushdown,"Do not push down limit to subplan when the query has operations that modify the number of tuples returned. Even if we know a hard limit overall, it doesn't apply if the query has any grouping/aggregation operations, or SRFs in the tlist. " a5c5a1b493cb658e8ac8493fe1645f7706a81b0d,Sven Klemm,Sat Aug 31 06:51:48 2019 +0200,Move gapfill code to nodes/gapfill,Move the gapfill code to tsl/nodes/gapfill to give our directory layout some more structure. Future custom nodes should all live under the nodes directory. 41878e735b4b99af14d224ac619327c99a0d1b2c,Sven Klemm,Sun Sep 1 11:52:37 2019 +0200,Fix background worker segfaults,The background worker function to check the owner of a job did not have proper error handling when the object referenced by a job did not exist leading to a segfault in that case. This patch adds proper checking of return values and errors when the object cannot be found. eaf93b5fc8bb556823eecb9957d413c98c4e5ec0,Sven Klemm,Sun Sep 1 17:15:55 2019 +0200,Add ORDER BY clause to unstable JOIN test queries, 8bb14d7fd2745e61a02ff27b60a5a61037df3f76,Sven Klemm,Sun Sep 1 17:04:20 2019 +0200,Exclude EXPLAIN costs from partitionwise tests,Remove the cost from the partitionwise test explain output to make the output reproducable. This patch also changes the test to use a psql variable instead of repeating the explain part for every query. 2ac9eca484d85129258cda06bbda4d4d70ae0c36,Sven Klemm,Sun Sep 1 15:04:31 2019 +0200,Fix segfault on ARM/32-bit builds,"In ts_interval_form_heaptuple the wrong macro was used to get a datum, the macro is a noop on 64 bit builds but it resulted in a segfault on 32 bit builds. " 7d310f272e8837df3aad683e8e7516484ff069ae,Sven Klemm,Fri Aug 30 12:05:36 2019 +0200,Run postgres JOIN tests on hypertables,This patch adds the postgres join tests and adjusts them to run on hypertables. We run the tests once with all optimizations and diff the result with optimization against the result without optimizations to ensure results do not change with optimizations enabled. 9b6bcc077beaa6a58643296e305bf1f615688ada,Sven Klemm,Fri Aug 30 11:48:53 2019 +0200,Fix OUTER JOIN qual propagation,The code for propagating quals to JOINs was too aggressive and propagated quals for OUTER JOINs when it was not safe to do so. This patch fixes this behaviour. This patch also refactors the qual processing and splits the processing of JOIN related information into a separate function. e1310a76b847b78dc1973128df9ffa84cc618c1a,Sven Klemm,Thu Aug 29 23:05:59 2019 +0200,Echo query description into plan_expand_hypertable test,When using psql output redirection only the query results get written to the output file which makes it hard to identify misbehaving queries by looking at the output. This patch changes the query script to use \qecho to output query comments in the output file as well to make identifying queries in the results output easier. This patch also moves the queries that test error behaviour into a separate file to keep that output out of the results diff. 6dddcaf4d0ec37c474a2f703b633a3fe5bae9d54,Sven Klemm,Tue Aug 27 13:33:33 2019 +0200,Fix duplicate #includes, a1666544a2ed945ef2fe9f7b1b2c9029e9648a1d,Sven Klemm,Mon Aug 26 00:48:01 2019 +0200,Ignore .psqlrc in test_fatal_command,When a local .psqlrc sets options that modify output of psql the loader test will fail. This patch changes the test to ignore the .psqlrc file. 9447617ee5779659de992aa6f2cacf3ddb74656d,Joshua Lockerman,Thu Aug 22 12:36:35 2019 -0400,Add random jitter to BGW failure backoff,"Since we backoff by a fixed amount, if multiple jobs fail at the same time, if they have the same retry-interval, they will be rescheduled to run at the same time again. Since running out of background workers is considered a failure case by our scheduling code, this can cause cascading backoffs, until the workers are scheduled at the maximum. This commit fixes this issue by adding random jitter to the re-scheduled time: a reschedule time of T will instead be scheduled at T +- 12.5" ,,,, 7c434d4914367989cbd04e7ce59a2a078cc87623,Sven Klemm,Wed Aug 21 21:45:23 2019 +0200,Fix ChunkAppend space partitioning support for ordered append,When ordered append tried to push down targetlist to child paths it assumed childs would be scans on rels which is not true for space partitioning where children might be MergeAppend nodes. This patch also no longer applies the ordered append optimization to partial paths because its not safe to do so. This patch also adds more tests for space partitioned hypertables. 53f45864b680b207f3fedc3f1254a5e77c208104,David Kohn,Wed Aug 21 13:36:40 2019 -0400,Make version mismatch errors FATAL,"We currently check and throw an error if the version loaded in the client is different from the installed extension version, however there is no way to recover from this state in a backend. (There is no way to load the new version as we cannot unload the old and no commands can be effectively run). Now, we instead throw a FATAL error which will cause the client to reconnect so it can load the proper extension version. " 897fef42b60c95c0f87c36b3b3bed286a0f124fe,David Kohn,Wed Jun 19 00:43:33 2019 -0400,Add support for moving chunks to different tablespaces,"Adds a move_chunk function which to a different tablespace. This is implemented as an extension to the reorder command. Given that the heap, toast tables, and indexes are being rewritten during the reorder operation, adding the ability to modify the tablespace is relatively simple and mostly requires adding parameters to the relevant functions for the destination tablespace (and index tablespace). The tests do not focus on further exercising the reorder infrastructure, but instead ensure that tablespace movement and permissions checks properly occur. " 62de29987baef0f9102060ce4ea0cfc6bd3ba7e6,Narek Galstyan,Thu Jun 27 00:12:19 2019 -0400,Add a notion of now for integer time columns,This commit implements functionality for users to give a custom definition of now() for integer open dimension typed hypertables. Such a now() function enables us to talk about intervals in the context of hypertables with integer time columns. In order to simplify future code. This commit defines a custom ts_interval type that unites the usual postgres intervals and integer time dimension intervals under a single composite type. The commit also enables adding drop chunks policy on hypertables with integer time dimensions if a custom now() function has been set. 53860e40ca9026b52de6573d9b271343c65c6e06,Sven Klemm,Thu Aug 8 13:15:43 2019 +0200,Fix cagg_agg_validate expression handling,The expression tree walker function used to validate the definition did not check for NULL values. This patch adds checking for NULL and also fixes the expression_tree_walker call. 22dc39bfd44f1bea96218bad895e1578225a57e5,Sven Klemm,Sun Aug 4 21:03:25 2019 +0200,Refactor ChunkAppend plan selection,Move ChunkAppend selection of next subplan into separate function. This is in preparation for making ChunkAppend parallel aware. 75cb1a5e1b709c7a5293490dd87266c24d8f39ae,Sven Klemm,Fri Aug 2 11:00:35 2019 +0200,Run all tests on cron triggered runs,Change cron triggered travis runs to include all tests. This patch also adds tagging for coverage information to prevent faulty coverage differences. ee9fc6af4cf7cc347d0f600542bb356a1b7c9874,Sven Klemm,Thu Aug 1 23:38:08 2019 +0200,Add 1.4.1 to update test scripts, 03d4ae03d63e105fb24b825c9802e7183903c0c8,Sven Klemm,Wed Jul 31 16:52:50 2019 +0200,Release 1.4.1,"This maintenance release contains bugfixes since the 1.4.0 release. We deem it medium priority for upgrading. In particular the fixes contained in this maintenance release address 2 potential segfaults and no other security vulnerabilities. The bugfixes are related to queries with prepared statements, PL/pgSQL functions and interoperability with other extensions. More details below. **Bugfixes** * #1362 Fix ConstraintAwareAppend subquery exclusion * #1363 Mark drop_chunks as VOLATILE and not PARALLEL SAFE * #1369 Fix ChunkAppend with prepared statements * #1373 Only allow PARAM_EXTERN as time_bucket_gapfill arguments * #1380 Handle Result nodes gracefully in ChunkAppend **Thanks** * @overhacked for reporting an issue with drop_chunks and parallel queries * @fvannee for reporting an issue with ConstraintAwareAppend and subqueries * @rrb3942 for reporting a segfault with ChunkAppend and prepared statements * @mchesser for reporting a segfault with time_bucket_gapfill and subqueries * @lolizeppelin for reporting and helping debug an issue with ChunkAppend and Result nodes " 519ef8ebf569eb1409f8ec91ac23b9f3e3a55875,Sven Klemm,Tue Jul 30 22:33:13 2019 +0200,Handle Result nodes gracefully in ChunkAppend,In some occasions (other extensions modifying plans) MergeAppend nodes might be wrapped in Result nodes when Projection is needed and since MergeAppend cannot do projection a Result node is put on top of MergeAppend. Since ChunkAppend can do projection we can remove the Result node if it has no one-time filter. c59bae0b6224aac3aecf5bab8d56ca8b669495f7,Sven Klemm,Mon Jul 29 20:06:03 2019 +0200,Run all tests on branch prerelease_test,We want to run all tests on the prerelease_test branch to make sure we don't miss any required changes when cherry-picking. 8ee2391f651934e5f449a2aca854b8451552a21d,Sven Klemm,Mon Jul 29 00:45:27 2019 +0200,Run update tests on prerelease branch, b80f8f3d06507af62992e6649a031d4d383b3270,Sven Klemm,Thu Jul 25 11:03:41 2019 +0200,Only allow PARAM_EXTERN as time_bucket_gapfill arguments,"Since bucket_width, start and stop of the gapfill node are initialized during executor startup only PARAM_EXTERN params are safe to be used. This patch will make the gapfill node throw an error if any argument is a param that is not PARAM_EXTERN. This patch also improves the error message for negative or 0 bucket_width. " 9551f4a5f643385ebc04ef7171f36034f978b65d,Sven Klemm,Wed Jul 24 10:55:54 2019 +0200,Fix ChunkAppend with prepared statements,ChunkAppend replaced the list of child plans on the plan node with a local copy which corrupted the child plans when the plan was turned into a generic plan. c6000e9839a3b8b3ad6cf564cb93b3d63622e6df,Sven Klemm,Mon Jul 22 20:03:26 2019 +0200,Fix alternate_users test,"The alternate_users tests relied on other tests to create tablespace ""tablespace1"" and would fail if it was not there. The output of the tests had captured the incorrect state when ""tablespace1"" did not exist. This patch fixes the expected output and makes sure the tablespace required for the test actually exists. " c8bbcbae22f3e5150db228994e606ecb20e5128a,Sven Klemm,Mon Jul 22 17:57:59 2019 +0200,Remove POST_TESTS list from regression test,We don't have any tests anymore that rely on output of previous tests anymore so this functionality is no longer needed. The alternate_users test is moved to SOLO_TESTS because it is flaky when run in parallel. f4f0edad358cbb6db06ae776f5eea23b3fcc0cb1,Sven Klemm,Mon Jul 22 17:48:22 2019 +0200,Merge different agg_bookends tests into 1 test,Change agg_bookends tests to be similar to other tests where the plan output and diff for unoptimized and optimized results are all done in the same file. Unlike in those other tests we do include output of results in expected files as well to make sure the aggregate functions produce correct output. 05c6b9db2c02e4d88af660032d57610f0cbc355d,Sven Klemm,Mon Jul 22 11:23:03 2019 +0200,Clean up agg_bookends test,Remove unnecessary quotes from identifiers and wrap modifying queries in transaction to enable multiple executions. 3737a6bc166d2a0f86fb196213486f8fa0dd059b,Sven Klemm,Fri Jul 19 20:41:31 2019 +0200,Fix ConstraintAwareAppend subquery exclusion,The RangeTblEntry index used during planning might be different from the index used during execution due to flattening. This patch changes ConstraintAwareAppend to handle queries where the index changed between planning and execution. a11910b5d5c9be006aab8ef6893eb69cb19ee9b6,Sven Klemm,Sun Jul 21 09:08:29 2019 +0200,Mark drop_chunks as VOLATILE and PARALLEL UNSAFE,The drop_chunks function was incorrectly marked as stable and parallel safe this patch fixes the attributes. 1dc1850793752df3167caaeb9ed01cfecc760593,Stephen Polcyn,Fri Jun 28 12:00:05 2019 -0400,Drop_chunks returns list of dropped chunks,"Previously, drop_chunks returned an empty table, giving the user no indication of what (if anything) had happened. Now, drop_chunks returns a list of the chunks identifiers in the same style as show_chunks, with the chunk's schema and table name. Notably, when show_chunks is called directly before drop_chunks, the output should be the same. " f77374982c2473c3cc440416ab7264c25ca79a44,Sven Klemm,Sun Jul 14 10:38:30 2019 +0200,Handle CustomScan node with scanrelid similar to other Scans,This patch changes ChunkAppend to treat CustomScan nodes which have scanrelid set similar to postgres native Scan nodes. This enables constraint exclusion to work for CustomScan nodes that scan a relation. f8f5a73fd2ec9352be10708a80f075f1d71bd06b,Sven Klemm,Wed Jul 17 18:38:01 2019 +0200,Add 1.4.0 to update test scripts, 6c72de54242e40e8b319abdaef6f658bca6c8852,Narek Galstyan,Mon Jul 8 12:26:19 2019 -0400,Avoid starting postmaster when there are no tests, a2f2db9cab0572ff2cdfd255880767889ccb77d2,Sven Klemm,Wed Jul 17 12:06:03 2019 +0200,Release 1.4.0,"This release contains major new functionality for continuous aggregates and adds performance improvements for analytical queries. In version 1.3.0 we added support for continuous aggregates which was initially limited to one continuous aggregate per hypertable. With this release, we remove this restriction and allow multiple continuous aggregates per hypertable. This release adds a new custom node ChunkAppend that can perform execution time constraint exclusion and is also used for ordered append. Ordered append no longer requires a LIMIT clause and now supports space partitioning and ordering by time_bucket. " 5f4dd2bc9fa0631cc051ece5d3835e519d6c2e03,Sven Klemm,Sat Jul 13 21:27:14 2019 +0200,Mark postgres includes as system includes,The postgres includes in reorder.c were not included as system include this patch fixes those includes. dd42beb9e498fb1aa759c72f77b1f3dd7051022c,Matvey Arye,Fri Jun 28 15:10:08 2019 -0400,Add architecture and bit size to telemetry,Add architecture and pointer bit size fields to what is sent in telemetry. This will allow us to evaluate the product usage on different architectures. fe47f10e251f02b2dacb927ad25c85b614a16724,Erik Nordström,Mon Jul 8 20:03:38 2019 +0200,Fix partitioned table check and enhance PG macros,"This change enables a check in `create_hypertable` that prohibits turning partitioned tables into hypertables. The check was only enabled when compiling against PG10, but should be there for PG version 10 and greater. To avoid such disabled code in the future, some extra convenience macros have been added. For instance `PG10_GE` means PG10 and greater. " 70fd8df0157dd3a69844672044db37087a8db507,Sven Klemm,Sat Jul 13 13:14:14 2019 +0200,Add missing license information,"Add license information to files missing it and fix the license check script to honor the return code of both the apache license check and the TSL license check. Previously errors occurring during apache license check would not be reflected in the return code of the script, so only the TSL license check was effective. " 06c174889c4eb17db6cf4da2750ba9a26951de7e,gayyappan,Tue Jul 9 18:20:36 2019 -0400,Update continuous agg bgw job start time,"When the continuous aggregate refresh_interval setting is modified, it does not modify the job schedule until the next scheduled job runs. This fix addresses it by updating the next_start time for the bgw job. " 368a5c80cbdc280a3a5b55aac415b737a26eaaa5,Mats Kindahl,Tue Jul 9 09:19:35 2019 +0200,Disable tool checks for test_sanitizers.sh,"The `test_sanitizers.sh` script do not require regress tools, so turning off regress tools check when calling CMake. " b1a365b0e02f2f5bf684cdc855c2f51a431d050d,gayyappan,Wed Jul 3 11:10:19 2019 -0400,Add warning message for REFRESH MATERIALIZED VIEW,Add a warning when REFRESH does not materialize the entire range due to max_interval_per_job setting. e9df3bc1b6b081376cfb88a735f32e1ded077d62,gayyappan,Mon Jul 8 11:24:45 2019 -0400,Fix continuous agg catalog table insert failure,The primary key on continuous_aggs_materialization_invalidation_log prevents multiple records with the same materialization id. Remove the primary key to fix this problem. 5a0a73eabde4a1b2cd66abe6af4f202889d7671f,gayyappan,Wed Jul 3 14:14:05 2019 -0400,Add columns to continuous_aggregate_stats view,Add more information about job history for continuous aggregate background worker jobs. 12ce2b8803d429d38f838b437b5d8b17056026ed,Erik Nordström,Wed Jul 3 14:13:48 2019 +0200,Fail when adding space dimension with no partitions,Calling `create_hypertable` with a space dimension silently succeeds without actually creating the space dimension if `num_partitions` is not specified. This change ensures that we raise an appropriate error when a user fails to specify the number of partitions. 4e08f140750792ea99eb5bc0b8451ba86442a27c,Erik Nordström,Sat Jun 29 14:32:17 2019 +0200,Fix chunk exclusion with ordered append,"With ordered append, chunk exclusion occur only along the primary open ""time"" dimension, failing to exclude chunks along additional partitioning dimensions. For instance, a query on a two-dimensional table ""hyper"" (time, device), such as ``` SELECT * FROM hyper WHERE time > '2019-06-11 12:30' AND device = 1 ORDER BY time; ``` would only exclude chunks based on the ""time"" column restriction, but not the ""device"" column restriction. This causes an unnecessary number of chunks to be included in the query plan. The reason this happens is because chunk exclusion during ordered append is based on pre-sorting the set of slices in the primary dimension to determine ordering. This is followed by a scan for chunks slice-by-slice in the order of the sorted slices. Since those scans do not include the restrictions in other dimensions, chunks that would otherwise not match are included in the result. This change fixes this issue by using the ""regular"" chunk scan that account for multi-dimensional restrictions. This is followed by a sort of the resulting chunks along the primary ""time"" dimension. While this, sometimes, means sorting a larger set than the initial slices in the primary ""time"" dimension, the resulting chunk set is smaller instead. Sorting chunks also allows doing secondary ordering on chunk ID for those chunks that belong to the same ""time"" slice. While this additional ordering is not required for correct tuple ordering, it gives slightly nicer EXPLAIN output since chunks are also ordered by ID. " eddfa772b68f63998f38e07506551e355f0a6781,gayyappan,Wed Jun 26 13:35:57 2019 -0400,Fix continuous agg isolation test,"The test had a race condition between the Insert and Refresh. When the invalidation log is released, if the Insert got it first it would write a new invalidation that would be picked up by the refresh and if Refresh got the lock first, it would not see any invalidations. Both are valid paths for execution but caused non determinism in the tests. " d1f8f9d32612f6cf8070a77de35b279faab860e2,Mats Kindahl,Thu Jun 27 18:21:42 2019 +0200,CMake check for necessary testing tools,"Adding checks for `pg_isolation_regress` and `pg_regress` and not running the tests if the programs cannot be found either installed in default directories or provided with the `PG_PATH` variable, or the `PG_SOURCE_DIR` variable. " 2768c5db3f99b5a2cc412eb8e2d8f2de0c7ffdb6,Matvey Arye,Wed Jun 26 18:26:56 2019 -0400,Fix SIGHUP handling in scheduler and launcher,"SIGHUP's can be dropped in between background worker startup and the call into the background worker entrypoint. Namely, Postgres calls `pqsignal(SIGHUP, SIG_IGN)` inside of `StartBackgroundWorker`. Thus, SIGHUPs will be ignored before the call to the entrypoint. This creates a possible race condition where a config file change is not correctly processed (and is instead ignored). We prevent this by always processing the config file after we set our own signal handler. We also fix the tests here in two ways: 1) We disable background workers in this test (or, rather, delete the line that starts them back up). 2) We put in a new mock timer option that calls the standard timer wait. This allows us to test proper latch processing in the SIGHUP case. We believe that this resolves some flakiness in our tests as well. " 1194df58c3273f712346812cbdb4fbefac5b5781,Matvey Arye,Mon Jun 10 20:16:00 2019 -0400,Test PRs on the latest stable patch of Postgres,Make Travis test PRs and other branches on the latest released PG patch version for each major version. Cron still tests the earliest supported patch version (and the ABI tests run by cron cover the tip of each major version). 63d1e9f2b3ddd610de394701c577d737eaecbc31,Solar Olugebefola,Mon Jun 24 09:25:25 2019 -0400,Update README links, ff44b3332757822b1ffdbf0da9c9626d3ace6fcd,Stephen Polcyn,Tue Jun 18 11:28:54 2019 -0400,Update get_telemetry_report to expected behavior,"Previously, returns full report even if telemetry is disabled. Now, reassures user telemetry is disabled and provides the option to view the report locally. " 743a22f1fad40ceb509e70958730db572d1aa963,Sven Klemm,Mon Jun 24 19:18:13 2019 -0400,Add 1.3.2 to update test scripts, 7ae044c658490eac6f951d381d6024114b07b18a,Sven Klemm,Mon Jun 24 12:45:36 2019 -0400,Support time bucket functions in Ordered Append,The initial implementation for Ordered Append required the ORDER BY clause expression to match the time partitioning column. This patch loosens that restriction and will apply the Ordered Append optimization for queries with ORDER BY time_bucket and date_trunc as well. 60cfe6cc905c4c53d70281b8c7f1866e7e854a3e,gayyappan,Thu May 30 08:58:13 2019 -0400,Support for multiple continuous aggregates,Allow multiple continuous aggregates to be defined on a hypertable. 5e5625ecaae337ba065af61947c20eccc379adec,Sven Klemm,Mon Jun 24 05:55:59 2019 -0400,Fix Order display in ChunkAppend EXPLAIN,The Order display of the ChunkAppend node used the output of the node instead of the input of the node when resolving the targetlist index to display the order information leading to incorrect display when the Sort column was not passed through or the position changed. 687a08aa46d1c03999127d4e51e17884ce3db9b1,Erik Nordström,Mon Jun 24 16:45:44 2019 +0200,Make test runner output filter work on BSDs,The regexp-based test output filtering in `runner.sh` does not work with the Mac OS/BSD version of `sed` since it doesn't use the extended/modern regexp syntax by default. This makes some tests fail on Mac OS X/BSD. This change makes the filter use non-extended regexp syntax to fix this issue. 9925e69acb72af290098736f35e620b39db8607e,Joshua Lockerman,Mon Jun 24 11:02:30 2019 -0400,Fix lost invalidations in continuous aggs,"We use the InvalidationThreshold as a barrier to ensure that all transactions see an invalidation threshold. Upon transaction commit, all transactions grab a lock on said table, while during the first phase of materialization, the materializer grabs an AccessExclusive lock on the table. This is supposed to ensure that all INSERT/UPDATE/DELETEs are ordered strictly before or after the first phase of materialization, and thus any mutations to newly materialized data will be part the new materialization, or append an invalidation. Unfortunately, the scanner does not hold its locks until the end of the transaction, and since that was the only manner we were taking a lock on the InvalidationLog in mutations, it was not functioning as a barrier. To fix this, we now take an explicit lock. " e9d8b826905069c79b2c36aa14d879dc5b483575,Matvey Arye,Mon Jun 24 10:45:06 2019 -0400,Update release notes for 1.3.2,Preparation for 1.3.2 release b822328dfe190771f1e71d52010655d65774fac0,Matvey Arye,Mon Jun 24 09:00:45 2019 -0400,Remove debug function dependency in tests,Allows the permissions test to run in release mode. 3c385b88338d94fe3e4096771af14d1481b98ae8,Matvey Arye,Wed Jun 19 21:35:31 2019 -0400,Add continuous agg permission tests,Make sure that you can't add a view to a schema without CREATE privileges. Nor can you use function where you don't have EXECUTE privileges. The latter case is also tested with background workers. afdb3269a8257ed9bc7809f8d2c0c22603e81bc1,Matvey Arye,Wed Jun 19 15:22:53 2019 -0400,Require superuser for BGW start/stop/restart,Start requiring superuser privileges to start/stop/restart background workers. e049238a07581bfc5b08c0e6b45598e1e788e54b,Matvey Arye,Wed Jun 19 14:29:16 2019 -0400,Adjust permissions on internal functions,The following functions have had permission checks added or adjusted: ts_chunk_index_clone ts_chunk_index_replace ts_hypertable_insert_blocker_trigger_add ts_current_license_key ts_calculate_chunk_interval ts_chunk_adaptive_set The following functions have been removed from the regular SQL install. They are only installed and used in tests: dimension_calculate_default_range_open dimension_calculate_default_range_closed 8ff9650d5ce5eaea142dfed9cac02500580f43a6,Matvey Arye,Tue Jun 18 21:15:59 2019 -0400,Make BGWs execute under owner permissions,"Since some of our background workers now execute user defined functions, we should make them execute under the roles of the objects that are associated with them (as defined by ts_bgw_job_owner). This prevents attacks such as UDF executing arbitrary code under the default BGW user. Currently, the only possible worry is regarding continuous aggs. But, this solution protects all BGW jobs. " 991ba7afabedd19759ad4cc79147c3c37a8042ed,Matvey Arye,Mon Jun 17 16:15:07 2019 -0400,Reword permission error,Reword the permission error to make clear that the permission issue relates to ownership and to match PG errors. d580abf04f91528836bf15ade3d56c9fa3905ac9,Matvey Arye,Fri Jun 14 15:18:49 2019 -0400,Change how permissions work with continuous aggs,To create a continuous agg you now only need SELECT and TRIGGER permission on the raw table. To continue refreshing the continuous agg the owner of the continuous agg needs only SELECT permission. This commit adds tests to make sure that removing the SELECT permission removes ability to refresh using both REFRESH MATERIALIZED VIEW and also through a background worker. This work also uncovered divergence in permission logic for creating triggers by a CREATE TRIGGER on chunks and when new chunks are created. This has now been unified: there is a check to make sure you can create the trigger on the main table and then there is a check that the owner of the main table can create triggers on chunks. Alter view for continuous aggregates is allowed for the owner of the view. d3e582fd2319f2af276663f1d0831985af4fcd97,Matvey Arye,Fri Jun 14 10:02:17 2019 -0400,Adjust permission checks to ProcessUtility start,"Items that are ""handled"" in process utility start never go through the standard process utility. Thus they may not have permissions checks called. This commit goes through all such items and adds permissions checks as appropriate. " 0670610f1ca584a1de69b81cb97b26a57503a3c3,Matvey Arye,Fri Jun 14 08:18:00 2019 -0400,Improve permission checks,This tests and/or improves permission checks for: -create index multi-transaction -set_adaptive_chunking -timescaledb_pre_restore -timescaledb_post_restore 77abec0d385b506ab9ef108cb747fbada2fefe9c,Matvey Arye,Fri Jun 14 07:45:45 2019 -0400,Improve permission checking for continuous aggs,Checks: - Create View - Drop View - Alter View - Refresh Materialized View e834c2aba82b946f22e6d3211e7d6ebecfea27c7,Matvey Arye,Thu Jun 13 20:50:10 2019 -0400,Better permission checks in API calls,This commit fixes and tests permissions in the following API calls: - reorder_chunk (test only) - alter_job_schedule - add_drop_chunks_policy - remove_drop_chunks_policy - add_reorder_policy - remove_reorder_policy - drop_chunks 9a29f0436d59fd0f2f0876fec2de89512140c752,Erik Nordström,Thu Jun 13 14:06:15 2019 +0200,Expose a function cache for query optimizations,"A number of TimescaleDB query optimizations involve operations on functions. This refactor exposes a function cache that can be used to quickly identify important functions and get access to relevant auxiliary functionality and/or information. In particular, certain functions apply to some type of (time) bucketing expression, e.g., expressions involving our own `time_bucket` function or PostgreSQL's `date_trunc`. This change recognizes the importance of time bucketing and uses the function cache to access custom functionality around time bucketing used in query optimizations. For example, both grouping estimates for hash aggregates and sort transforms can be quickly accessed to make better use of indexes when bucketing on a time column. This refactor is also done with anticipation that this will be useful going forward when other types of optimizations are implemented on time bucketing expressions, or other functions that can benefit from this cache. " 8c2acecbf4c775fbc1972c404c4bfeb4572825df,Sven Klemm,Sat Jun 22 04:00:18 2019 -0400,Skip runtime exclusion when Param is not partitioning column,When the column a Param references is not a partitioning column the constraint is not useful for excluding chunks so we skip enabling runtime exclusion for those cases. 6f936ea2e3f76dbf9b1a995aa3a526c0607e99ce,Sven Klemm,Fri Jun 21 15:37:44 2019 -0400,Fix use-after-free in ChunkAppend,"Fixes use-after-free in chunk_append/exec.c. Also, strip out memory usage from EXPLAIN ANALYZE output of Sort nodes because it is not stable across platforms " 2801c6a5f535f18e95a3b1d77247408f7995b9c2,Joshua Lockerman,Wed Jun 19 13:30:58 2019 -0400,Fix handling of types with custom partitioning,"In various places, most notably drop_chunks and show_chunks, we dispatch based on the type of the ""time"" column of the hypertable, for things such as determining which interval type to use. With a custom partition function, this logic is incorrect, as we should instead be determining this based on the return type of the partitioning function. This commit changes all relevant access of dimension.column_type to a new function, ts_dimension_get_partition_type, which has the correct behavior: it returns the partitioning function's return type, if one exists, and only otherwise uses the column type. After this commit, all references to column_type directly should have a comment explaining why this is appropriate. fixes Gihub issue #1250 " 3d63273c7146ab57d7b933536592913528ae9e6e,Narek Galstyan,Mon Jun 17 14:53:01 2019 -0400,Fix Datum to int cast issue,"Given a query like: SELECT * FROM hyper WHERE time_bucket(10, time) < 100 where time column has type bigint the current `time_bucket` parser assumes the type of (Const*)100 to be the same as the type of the `time` column of the table. This does not have to be the same for integer types: `time` can be a `bigint`, but if the operand fits in an `int`, the relevant (Const*) object will have type `int` (reflected in its `consttype` attribute). This PR makes sure that we use this type information to accordingly extract value from Datum and not rely on the type information of the column. " 1a21e4133244dda9bbc0d2582a3926a75fbe5045,Matvey Arye,Thu Jun 20 14:52:09 2019 -0400,Disable OpenSSL on appveyor,The 1.0.2 version of OpenSSL is broken on appveyor for now. Disable openssl for now. 256c915e02b46ff1061dee2f71215e3c0b26a7e3,Sven Klemm,Thu Jun 20 07:26:05 2019 -0400,Make append test results deterministic, c5c13415f0087879a036d0efc86b6156e7af0011,Sven Klemm,Mon May 27 20:01:39 2019 +0200,Use ChunkAppend to replace Append nodes,"This patch makes TimescaleDB use ChunkAppend in places where it used to used to use ConstraintAwareAppend before. ConstraintAwareAppend will still be used for MergeAppend nodes that cannot be changed to Ordered Append or when ChunkAppend is disabled. When a query on a hypertable is identified as benefitting from execution exclusion Append nodes will be replaced by ChunkAppend nodes. This will enable the use of runtime exclusion for joins, lateral joins, subqueries and correlated subqueries. " 0406ba9a1b7ef61b86850574ebe0629f4a1f200f,Sven Klemm,Wed Jun 19 06:12:27 2019 -0400,Use wildcard in .gitignore, af916cb4845355273aafc75000723e9c36818e47,Sven Klemm,Wed Jun 12 03:06:33 2019 -0400,Enable version specific output for append test,To allow running EXPLAIN ANALYZE on PG10 and PG11 make the append test output version specific. ecc9f8cafbce3caae690441b7db861f07a33f9dc,Matvey Arye,Thu Jun 13 15:49:52 2019 -0400,Cleanup ignored Arm32 test in travis.yml,Remove ignored tests that are now passing. 36a6e014f73be8266837b256470094814d0c10fb,Matvey Arye,Thu Jun 13 14:27:25 2019 -0400,Verify long correctly defined for arm32,Go through the code and verify that no L constants need to be defined as INT64CONST. Add comments as appropriate. c700843660a980031bf7e8ccc8881841809c4020,Matvey Arye,Thu Jun 13 13:40:22 2019 -0400,Fix timer bug for arm32,"Fix 64 bit constants to be correctly defined, and some other logic issues so that the timer logic works correctly under arm32 " 25a72f1547d90689feeb650eb7f55fb1e02ad45c,Ruslan Fomkin,Thu Jun 13 14:57:05 2019 +0200,Remove REQUIRED from find_program in CMake,"Remove option `REQUIRED` from calls to `find_program` in CMake files, since such options doesn't exist and is ignored. " bbe0f70c358bb1efd23f778c8bb17dfda1097b88,Matvey Arye,Wed Jun 12 16:27:03 2019 -0400,Fix spurious test failures due to rounding,Floats sometimes caused differences in output of least significant digits. Fix by truncating. 2e588662210f9fb487c6f1d137daa6c375995f92,Matvey Arye,Sun Jun 2 14:23:57 2019 -0400,Refactor sql_query_results test,"Change the test from using the old x_diff method to the new direct diff method for checking that the results don't differ in the optimized and unoptimized cases. This cleans up the golden file so that in case of success the golden files are nearly empty instead of containing a diff file that must be checked to ensure that it only has explain (and not tuple) output. Also combine several test files into query.sql, and get rid of differences in pg versions. " 33d28d2217df648eb528fa0dca902d7233d2626d,Matvey Arye,Tue Jun 11 15:24:54 2019 -0400,Test continuous aggs with extension updates,Add a v3 update test that makes sure that continuous aggregates continue to work after extensions are updated. 37cdd7f45be61dca34ba8ef6c4f2b1152043617a,Erik Nordström,Wed Jun 12 11:26:51 2019 +0200,Fix compiler warning in ChunkAppend on PG10/9.6,This adds a missing header include to quench a compiler warning about ResetExprContext being undefined. 8eb647d4b1bd2aed4409a597ae6eb9925058f0c9,Erik Nordström,Wed Jun 12 11:02:59 2019 +0200,Fix column name in show_columns test function,"The ""Nullable"" column in the output of `show_columns` actually shows NOT NULL constraints on a column, which is the inverse of what the column name suggests. This changes ""Nullable"" to ""NotNull"" to avoid confusion. " e56e66f39aa9860d37938e5ec24e7b92064c0352,Sven Klemm,Tue Jun 11 02:16:31 2019 -0400,Do not remove Result nodes with one-time filter,ConstraintAwareAppend removes Result node children from the plan because usually Result nodes are injected to do projection on top of nodes that cannot project themselves. ConstraintAwareAppend can do this projection instead. The Result node must not be removed when the Result node has a one-time filter though. 654e683f104b5d01785c9346391eec3f6c837764,Matvey Arye,Mon Jun 10 17:59:39 2019 -0400,Improve travis config definitions,Get rid of the PG_GIT_TAG and instead only rely on PG_VERSION. Download the postgres source based on the definition in the docker file. b860a8784115b98e29f1817b01eaeae75d52e50d,Narek Galstyan,Mon May 27 12:08:51 2019 -0400,Add ARM32 testing on Travis, 4ed379c33fca70a66d3aab428bb0afddb892d481,Narek Galstyan,Mon May 27 12:01:16 2019 -0400,Refactor IGNORES to run tests and ignore result,"Currently make installcheck IGNORES=... skips ignored tests. This commit changes IGNORE functionality to mean ""run tests but ignore the result if the test fails"" which is what ignore refers to in pg_regress framework terminology. The change was necessary to allow running ARM regression tests on travis and account for known failure issues. The commit also adds a make installcheck SKIPS=... argument that replicates the functionality previously refered as IGNORES. " d51d3ee0957f69bfe40195026c636e7ebe269d74,Narek Galstyan,Mon May 27 11:32:14 2019 -0400,Fix long-related type issues for ARM CPUs,1. ld interpolates 32bit numeric values on ARM processors and ,,,, lld is needed to successfully parse int64 types on arm. 2. Postgres TimestampDifference function expects long* for the referene type of time length but we internally keep int64 for arm and x86 interoperability. The change makes the type cast explicit to suppress a compile time warning. ,,,, 70a02b541076a1cc62fbfcd58d4dee272b059e75,Sven Klemm,Mon Jun 10 13:32:45 2019 -0400,Add 1.3.1 to update test scripts, 6a032fe29e23c9971b3806346aa6a1ebdeea5842,Sven Klemm,Wed Jun 5 09:34:45 2019 -0400,Propagate quals to joined hypertables,When a hypertable is joined on any column with an equality operator to another table all constraints that reference the same column will be propagated to both sides of the JOIN. 9aa24efae619eb0faa5ec0c39cc8ce6364126833,Sven Klemm,Sun Jun 9 21:20:47 2019 -0400,Fix constraint exclusion for OUTER JOIN,Using ON clause quals for constraint exclusion is only safe for INNER JOINs for LEFT/RIGHT JOIN only quals referencing the appropriate relation may be used for constraint exclusion. ba2e6b12571a728ca0358223a35a0b09c6419321,Matvey Arye,Fri May 31 12:32:30 2019 -0400,Update release notes for 1.3.1,"Add release description, bugfix PRs and thanks. " 48d9e2ce25caa5afb1c76d0a826f4213ed9f01d0,Matvey Arye,Wed Jun 5 16:59:35 2019 -0400,Add CMAKE option for default telemetry setting,Adds a CMAKE option to turn telemetry off by default by specifying -DSEND_TELEMETRY_DEFAULT=NO. The default is YES or on. 6dbcc852109ee691659e6406fc1765ebaaae8bbe,didier,Mon Jun 3 03:26:37 2019 +0200,Remove duplicate or unused includes,"In PG12 relation.h will be renamed pathnodes.h, remove it if it's not needed. In PG12 tqual.h is no more. " 100771ce3415923ee722ad903b88cabbdfc7cd7a,didier,Sun Jun 2 17:20:00 2019 +0200,Fix 'Parameter hides a global variable' warning, 50f1a1c427df6214dbd996b5023ba80a65fd9d9f,Sven Klemm,Sun Jun 9 09:14:21 2019 -0400,Output postmaster.log on travis failure,To make it easier to diagnose problems that only occur in CI it is useful to have the postgres logs in addition to having the test diffs. This patch adds all the postgres logs and also removes the hardcoded locations for the regression diffs and uses find to locate them instead. 721c08a146015fffca38296ebbe0e0e164b96d22,Sven Klemm,Fri May 31 16:17:36 2019 +0200,Improve ChunkAppend EXPLAIN output,Change EXPLAIN output to include the order if its an ordered append a9a360bc5f34c307d6ad2be2cf42fa0d81ca4e18,Sven Klemm,Sat Jun 1 09:04:20 2019 +0200,Remove space partition restriction from ordered append,The initial support for space partitions for ordered append only supported 1 space dimension. This patch removes that restriction and supports an arbitrary amount of space partitions. 94769854eb48832bbd56e1c5271186141dfe7830,Matvey Arye,Sat Jun 1 11:34:26 2019 -0400,Handle plan node changes in upstream Postgres,Postgres introduced a change in commit `e7eed0` that sometimes creates a Result node above an append/mergeappend node. This commit makes sure that our constraint aware node logic is compatible with this change in behavior. 6f90e4edd8dd5030d0b24e673d89d5102caa5e22,Matvey Arye,Thu May 30 22:22:45 2019 -0400,Run make install right before installcheck,"Previously the version of the extension without OpenSSL was compiled but never used in tests. This is because the version with ssl was compiled and installed before the nossl tests were actually run. This is a bug and is now fixed by making the nossl test run make install right before running the nossl test. The ssl extension is installed after, followed by the ssl test. Note: as before, the only test run in nossl mode is the telemetry test. This is because only the telemetry test is affected by the OpenSSL library. " 2ecb6b2c802a5fc8c72444200e464cef97a48593,Matvey Arye,Thu May 30 22:02:52 2019 -0400,Force nossl tests to always use Debug,"The regression tests with OpenSSL disabled have to always use Debug compilation mode because the telemetry test requires it. So even if the main test uses Release mode, force this nossl tests to use Debug. " c0b1a02859304a204b511fa7dc2badb739196a68,Sven Klemm,Wed May 29 23:43:05 2019 +0200,Fix compilation errors,"In https://github.com/postgres/postgres/commit/05eb923eae46c1698088d555ae590a73d4fc7070 Postgres fixed PARAM_EXEC handling which got backported to 9.6.12, 10.7 and 11.2. To allow TimescaleDB builds built on older versions to work with the latest PG version we need to backport some of the functions added in this commit. " c1e8b341d0f4ea3961a04f46ab53412349a25384,Matvey Arye,Thu May 23 18:28:11 2019 -0400,Fix rowsecurity test output for PG11.3,PG11.3 changes the output of tuple headers slightly during output. Change the rowsecurity test to elide the headers for affected test rows to make the tests the same for before and after 11.3. 9b16b11e310ea51e772647bd05a9e06e2510fca7,Matvey Arye,Thu May 23 18:07:31 2019 -0400,Fix query optimizations in PG11.3,"Postgres changed some behavior in 11.3 (commit 925f46f) so that `apply_scanjoin_target_to_paths` resets all paths in partitioned relations and re-does them (without calling the `set_rel_pathlist` hook on the append relation again). This broke our constraint aware append and ordered append optimizations because they are applied in `set_rel_pathlist`. This hook is not called again by postgres. After some discussion on pgsql-hackers it seems like this will be changed in the future. This fix uses the enable_partitionwise_aggregate PG GUC which is and disabled by default to control whether our partitionwise optimization is enabled. Thus, by default this optimization is off for now. This is a short-term fix until PG fixes the hook. " 47cdbd3656d71117e0f98b0a0c09bac84c5ada11,Sven Klemm,Mon May 27 20:13:19 2019 +0200,Add ordered append support for MergeJoin,When doing a JOIN with a hypertable ordered append required the ORDER BY clause to have the time partitioning column as first expression. This patch loosens that restriction. If the ORDER BY clause of a query does not reference our hypertable but has an equality condition on the time partitioning column of our hypertable we will do an ordered append because it allows the planner to skip the Sort step for MergeJoin. 5e10efdff3d810865f9795ed042afaf78a2ef430,Matvey Arye,Wed May 29 16:38:03 2019 -0400,Run ABI tests on consistent alpine versions,Previously the compile image for the ABI tests was based on a different postgres image than the run image (e.g. 10-alpine vs 10.2-alpine). This created dependency problems between different version of ssl packages that may be available under different alpine versions. We fix this by forcing the run image to be based off of the same image as the compile version. dd247068411df403bc9c12f9f4b71717aaeb2441,Matvey Arye,Wed May 29 14:59:49 2019 -0400,Fix telemetry test,The current telemetry test fails in ApacheOnly mode because it uses community functions. The PR separates out the community from the apache only testing and add a telemetry_community test that only runs if not in ApacheOnly mode. Also adjust the travis file to print telemetry output for the telemetry test run without openssl. 59179385699451c3dc0d096ac85895fbab9be51a,Matvey Arye,Wed May 29 10:59:11 2019 -0400,Fix ABI smoke test,"This fix makes the smoke test return the correct exit code. It also adds libssl1.0, which is needed for some openssl functionality. " f1db6dc84e7a29e8d8c746414cc73af89682ac8b,Sven Klemm,Mon May 27 15:52:43 2019 +0200,Fix compilation error on PG11.2, 0d3a77d8af90b5b20a917717d8808a187974e4c1,Sven Klemm,Tue May 7 00:56:10 2019 +0200,Add ChunkAppend node and use it for ordered append,"This patch adds a new ChunkAppend node. This node combines the functionality of ConstraintAwareAppend and Append and additionally adds support for runtime chunk exclusion. This patch only changes the ordered append plans to the new node. The patch also adds support for space partitioned hypertables to ordered append and for hypertables with indexes not on all chunks. Runtime chunk exclusion will allow chunk exclusion to exclude chunks for JOINs, LATERAL JOINs and correlated subqueries. " 5b7eea4cfea41c77cf54d351df4079d62adfac15,gayyappan,Wed May 15 13:40:16 2019 -0400,Pass int64 using Int64GetDatum when a Datum is required,"int64 should be passed to functions that take a Datum parameter using Int64GetDatum. Depending on the platform, postgres either passes int64 by value or allocs a pointer to hold this value. Without this change, we get SEGV on raspberry pi. " 2b4a38933b71f0bcb915812bb0fe9b45f24e41e0,Erik Nordström,Thu May 23 14:51:57 2019 +0200,Fix partitionwise aggregates on PG 11.3,"This change makes the dummy partitioning info that we use for hooking into the partitionwise planning code conform with the macro `IS_PARTITIONED_REL()`. PG 11.3 relies more heavily on the use of this macro to decide how to plan a relation and, since our dummy partitioning info wasn't compliant with the macro, partitionwise aggregates were therefore never planned. " c72aa15dd3cf9446ce99d96e1e523b501958e9bb,Sven Klemm,Thu May 23 10:47:15 2019 +0200,Fix release and apache only tests,Travis did not actually run the release and apache only tests with the apropriate build flags so they were identical to the normal debug runs. This patch fixes passing the build flags and also adjusts regresscheck to not run edition.sql when compiled as apache only because it tests functionality not available under this license. 7d305990c04743400bcf1d1aa961343326213750,Joshua Lockerman,Mon May 20 16:21:15 2019 -0400,Stop preventing REFRESH in transaction blocks,"We prevent REFRESHing continuous aggregates within a transaction block since we COMMIT and start our own transactions during this operation, making it inherently non-transactional. This commit ensures we only prevent REFRESHing continuous aggregates, and not REFRESH on other materialized views. " 5fcf234d1c2e3d7d6d3ea3a1020db7de0252f57c,Rob Kiefer,Mon May 20 14:33:24 2019 -0400,Add missing header guard,Code checking website lgtm.com flagged this as an alert. As far as I can tell this is the only header file missing the guard. 76277c63cfe3de798ddc97b58a508fe08d18ae05,Joshua Lockerman,Fri May 10 14:26:45 2019 -0400,Fix segfault in VACUUM on PG11,"In PG11 the relation field in a VacuumStmt can be NULL, and is only supposed to be used if the oid field is not valid. This commit changes to prefer the oid field, and adds a check that relation is not NULL. " f63bf57dad9b90d20fd3604a50811f47814e432b,Sven Klemm,Mon May 20 06:37:25 2019 +0200,Add missing ORDER BY to test query,Some of the ORDER BY clauses did not produce a unique order. This patch fixes those queries to make the test results consistent. aeac52aef6535e89fba5ce7a197d57f0957ad730,Brian Rowe,Thu May 2 08:25:25 2019 -0700,Rename telemetry_metadata table to just metadata,"This change renames the _timescale_catalog.telemetry_metadata to _timescale_catalog.metadata. It also adds a new boolean column to this table which is used to flag data which should be included in telemetry. It also renamed the src/telemetry/metadata.{h,c} files to src/telemetry/telemetry_metadata.{h,c} and updated the API to reflect this. Finally it also includes the logic to use the new boolean column when populating the telemetry parse state. " f0f1b47df92c223a3dfb018ea171218d58e48162,Sven Klemm,Fri May 17 19:19:01 2019 +0200,Allow Param as time_bucket_gapfill arguments,When using prepared statements or functions the arguments are turned into Params when the plan is changed into a generic plan. This patch allows Params as time_bucket_gapfill arguments so it can be used in functions and prepared statements with variable time_bucket_gapfill arguments. ae97d8c726951c5acea2be9d0877ea8009e8b71d,Mike Freedman,Sat May 11 12:43:29 2019 -0400,Update to reflect PG11 support and version 1.3, 2cd4766ea114eaad879f7d71967f3e86b72138ba,Sven Klemm,Fri May 10 03:23:43 2019 +0200,Fix detecting JOINs for continuous aggs,When explicit JOIN syntax is used the from list will contain a JoinExpr instead of a list of RangeTblRef which was not detected properly by cagg_validate_query. This patch adds a check for RangeTblRef to the validation code. fac493b83a1c44eb785566dd7f576f3a6081d723,Sven Klemm,Thu May 2 17:23:40 2019 +0200,Add more ordered append tests,This patch adds tests for hypertables with indexes not on all chunks and chunks with removed columns to the ordered append tests. bfabb30be05a145b63024490501ebb660c09fe5c,Sven Klemm,Tue May 7 01:06:23 2019 +0200,Release 1.3.0, 19935e2db335ab445d894a68c5963ea9b9787edc,Matvey Arye,Fri May 3 17:05:14 2019 -0400,Fix nits in release 1.3.0 notes,Improves wording and fixes nits. 1a2dec19cbe7884efdfeaaabbe65d9359ad99263,Matvey Arye,Thu May 2 11:51:17 2019 -0400,Improve 1.3.0 release notes,This commit build out the release notes for 1.3.0. b42ee47130305c7886ece0b307344b97bb249660,erthalion<9erthalion6@gmail.com>,Fri May 3 17:26:17 2019 +0200,Handle missing check constraint,"Check constraints are propagated to child chunks by default, but they're miggins from _timescaledb_catalog.chunk_constraint table. It can lead to a situation, when such a constraint will be added as not valid, and then at validation time validate_hypertable_constraint will construct a chunk_cmd with a NULL name (since it was not there and ts_chunk_constraint_get_name_from_hypertable_constraint returns NULL), what eventually leads to a crash. " 0e842e2d9049815c73a0b720a39bec67040f4062,gayyappan,Thu May 2 12:43:09 2019 -0400,Fix partial view targetlist for continuous aggregates,The partial view should always project the time_bucket expression related column as this is a special column for the materialization table. The partial view failed to project it when the user query's SELECT targetlist did not contain the time_bucket expression. The materialization fails in this scenario. c5dd8b1dbe19d9b63584ea29809f74a579b10fe8,Sven Klemm,Thu May 2 17:37:48 2019 +0200,Add missing ORDER BY clause to views test,Add ORDER BY clause to views test to make result of the test consistent 70323f1e434509f17a797e9863df1bd54997b944,Matvey Arye,Wed May 1 14:51:30 2019 -0400,Add telemetry for policies: drop_chunk & reorder,The commit will add telemetry for the number of drop_chunks and reorder policies in the database. This will allow us to see how much these features are used. 2a76041daef62d9afe61c4efc425aa3340864d23,Matvey Arye,Tue Apr 30 21:43:39 2019 -0400,Make cont aggs group column names more intuitive,"This commit change the name given to group columns in the materialized tables to make them more intuitive for the user. The goal was to make the column names the same as the column names in the view. The main change was to change time_partitioning_col to be the same as the view. ""time_partition_col"" is only used as the default when there is no alias. This commit also changes the assignment of the view aliases to the target entries to occur much earlier in the create process. " b05aeb16ee3aaf5b09304b049afe08476a06f642,Matvey Arye,Wed May 1 09:03:19 2019 -0400,Fix various issues for Coverity,"Compilation issues: - Fix variable declaration inside for - Remove unused sortref variable compilation threw a warning about this variable being unset in some branches. But, actually it's never used. So remove. Fixes for defects found by Coverity: - Dereference before NULL in materialize.c - Big parameter passed by value in insert.c - Result is not floating point in plan_expand_hypertable.c - Dereference after NULL check in planner.c " 00c2b74bee4c9d6c3dd16b295c7722d29dbf495c,Matvey Arye,Tue Apr 30 22:50:52 2019 -0400,Suppress not-null notice when creating cont aggs,Add a not-null constraint to the time column on the materialization table automatically. This suppresses the superfluous notices which happens when creating continuous aggregates (and don't tell you much). b41591bcdb9d8cf03be4c7ffb69270c545663e2b,Joshua Lockerman,Tue Apr 30 16:22:26 2019 -0400,Test continuous aggregates with space partitions,Just a sanity check to make sure they work correctly. cd715565048cadb3a3ed6827a36b10f628a9915a,Sven Klemm,Tue Apr 30 12:58:33 2019 +0200,Fix UPSERT with prepared statements,When doing upsert with prepared statement the code to restore arbiter_indexes would not trigger when an error occurred on the INSERT leading to a segfault on the next EXECUTE of the prepared statement. 899cd0538db29744393da0f428ac9031c21807b3,Joshua Lockerman,Mon Apr 29 19:15:47 2019 -0400,Allow scheduled drop_chunks to cascade to aggs,"This commit adds a cascade_to_materializations flag to the scheduled version of drop_chunks that behaves much like the one from manual drop_chunks: if a hypertable that has a continuous aggregate tries to drop chunks, and this flag is not set, the chunks will not be dropped. " 8d48245c622f37e6c1cea863b765e273c8370bff,Joshua Lockerman,Mon Apr 29 19:12:51 2019 -0400,Fix cascade in scheduled drop chunks,"Before this commit, cascade was unconditionally set to true, as stored if the pointer to the inputted form data was non-null instead of the value stored within it. This commit changes that, so the intended setting for cascade is stored instead. " 297b9ed66aeddd6c3ba4feec43a1f5f1fabbef4e,gayyappan,Fri Apr 26 17:03:57 2019 -0400,Add default index for continuous aggregates,Add indexes for materialization table created by continuous aggregates. This behavior can be turned on/off by using timescaledb.create_group_indexes parameter of the WITH clause when the continuous agg is created. 74f8d204a5abb9c2ef2686e1c0cd312273fa4cea,Matvey Arye,Mon Apr 29 13:50:16 2019 -0400,Optimize getting the chunk_id in continuous aggs,We replace chunk_for_tuple with chunk_id_from_relid for getting chunk id fields when materializing continuous aggs. The old function required passing in the entire row. This was very slow because a lot of data was passed around at execution time. The new function just uses the internal `tableoid` attribute to convert the table relid to a chunk_id. This is much more efficient. We also add memoization to the new function because it is most often called consecutively for the same chunk. d5716fe200f07fd4be10f51d2fc177ed2a22b9f7,Joshua Lockerman,Mon Apr 29 14:19:54 2019 -0400,Fix sanitizer and release tests,"We were not building timescaledb with debug enabled in the sanitizer tests, causing tests which needed debug symbols to fail. This commit changes the sanitizer tests to include such symbols. We still enable optimizations, as that results in more realistic build environment for memory tests. This commit also ensures some tests which should only be run in debug builds are in fact only run in debug builds. " d3e2b69f336ceadef5d82be001087e51c6446a2f,Sven Klemm,Mon Apr 29 15:38:28 2019 +0200,Fix release build warning,The address variable in process_create_trigger_start is only used in an assert in debug builds so compiler will warn about this variable in non-debug builds. This patch marks the variable as being used for asserts only to silence the warning. 580117c48984839c8dfd78f6056e8bf7ed90d01e,Sven Klemm,Sat Apr 27 04:16:27 2019 +0200,Refactor time_bucket chunk exclusion,Change the time_bucket chunk exclusion to not use expressions to construct the upper boundary but instead generate a Const with the upper boundary. The previous approach with using expressions made it harder to check cleanly for the existance of overflows cause even though we could catch the error we would have a catalog cache leak in the error path. 22fda543672980c8e99c62126ca30e5a47d9df0b,Matvey Arye,Fri Apr 26 12:18:15 2019 -0400,Adjust tests for time_bucket optimization warning,"Currently the time_bucket optimization causes a cache leak warning to be printed when there is an overflow. This will be fixed in a future PR, but since this is an edge-case we currently just adjust our tests to not print this warning by adjusting client_min_messages. " d5f1008dd37e9103538719e49573d95932b1ac17,Joshua Lockerman,Thu Apr 25 15:43:30 2019 -0400,Fix cross-platform printing in continuous aggs,To print an int64 you need to use the INT64_FORMAT macro because of differences between windows and linux. 5eea9b9ae0159c54dba5313276a7268669126201,Joshua Lockerman,Thu Apr 25 15:08:31 2019 -0400,Fix linking issues on Windows,"Windows requires all functions to have the same linkage everywhere: if they have TSDLLEXPORT in one header, they need to have it in every C file, and every other header also. " 652c6cec0f1df350d795d671cdd6ea4167b089a3,Joshua Lockerman,Wed Apr 17 15:14:37 2019 -0400,Add additional tests for Continuous Aggregates,Tests include: 1. UPDATEs get logged to the invalidation log. 2. the invalidation log is not written to on ABORT. 3. custom time-partitioning functions are not supported (also updates error message for this case). bc55ca984ef457588cf059e91022702f8e16c880,Joshua Lockerman,Tue Apr 23 12:43:45 2019 -0400,Make continuous agg queries more efficient,"Our Chunk Exclusion and Ordered Append cannot understand the variables passed in to the SPI, and thus were not firing from our materialization worker. To work around this issue, we input the arguments as constants using string formatting. Note that the same issue may occur for prepared statements, so we may need to fix this more generally in the executor. " c463d2634cafb8da9a5a11332c1a44015191eb26,Matvey Arye,Mon Apr 22 15:58:38 2019 -0400,Add telemetry for continuous aggs,"This adds a top-level telemetry entry for `num_continuous_aggs`. 2 designs were considered: 1) add entry to telemetry_metadata that's modified on add/drop continuous aggs. 2) scan the catalogs for number of continuous aggs when making the report. After briefly trying (1), I switched to (2) for 2 reasons: 1) This avoids the dependency from continous_aggs on telemetry and reverses that dependency. I think the other way is cleaner. 2) The code became simpler and more self-contained. " 58601cc3a0742a398ce12f80548a57739c04162c,gayyappan,Mon Apr 22 14:02:03 2019 -0400,Separate the memory allocation context for tsl_finalize_agg,Allocate the internal data structures in different memory contexts based on the lifetime i.e. global to query or global to the group used by the finalize_agg query. This helps minimize lookups for global lifetime objects like Oids. e034978eefa23ab6b1c9605a6788613b86b43dd1,Matvey Arye,Mon Apr 22 16:42:01 2019 -0400,Change default max_interval_per_run,"Changed the default max_interval_per_run to 20*bucket width. The previous default was infinite, which had bad behaviour on backfill because the first run processed all of the existing data. The new default is much more conservative. " 6e882ec1bc942952f70289c2c9fa39d9e1e15157,Matvey Arye,Fri Apr 19 12:38:31 2019 -0400,Adjust retry_period of cont aggs,We lower the retry_period of cont agg jobs from a constant 1 day to the schedule_interval because 1 day was too long. The retry time formula is: retry_period * 2^(consecutive_failures - 1) So this seems reasonable. Also changed the update logic to set retry_period to refresh_interval on WITH clause alters. 15e65bf7ed3a23778ba76678278a0f174d397f07,Matvey Arye,Wed Apr 17 15:28:29 2019 -0400,Add usage test for continuous aggs,This adds a usage test that serves to document the user-facing interface for continuous aggregates. It could serve to educate the user as well as provide examples. It should match the functional spec. 321944928ddcc9ef3d0d06d62681387445ec2d24,Matvey Arye,Fri Apr 19 17:10:14 2019 -0400,Stop BGW in watermark test,Fixes test flakiness because the materializing bgw will sometimes run and change the metadata tables in a way that we anticipate doing explicitly in the test. ae3480c2cb77acafb56276125bf26fab1f3a1cac,Joshua Lockerman,Fri Apr 19 09:43:04 2019 -0400,Fix continuous_aggs info,This commit switches the remaining JOIN in the continuous_aggs_stats view to LEFT JOIN. This way we'll still see info from the other columns even when the background worker has not run yet. This commit also switches the time fields to output text in the correct format for the underlying time type. 445895bfd7cafad40529a02217f10f0781253112,Joshua Lockerman,Mon Apr 15 16:38:30 2019 -0400,Add isolation tests for continuous aggs,Testing that INSERTs SELECTs and mutliple REFRESH don't interfere 2f64197a791cbc55e78025377fb7504efe700636,Matvey Arye,Wed Apr 17 21:58:47 2019 -0400,Fix continuous agg BGWs to work under community,"Previously, continuous aggregate background workers required the enterprise license. This was wrong and we change that to only requre the community license " eec90593fe06c3644b633deb2ac1f03cc686951e,Matvey Arye,Mon Apr 15 21:27:46 2019 -0400,Rename continuous aggs files for consistency,Rename continuous aggs files to be more consistent and follow our conventions. 1f89478e4bfd298134e2588772b93977de967915,Matvey Arye,Wed Apr 10 17:56:53 2019 -0400,Rename continous aggregate internal objects,"Rename the materialized hypertable, partial view and direct view with a hypertable_id suffix. This avoids truncation problems and lets us use a longer prefix. This change also renames the columns in the mat table to have either a _agg or _grp prefix and to include the column number from the original view. " ef50ee2ed527ec5129238f138a886eb91d156b59,Joshua Lockerman,Mon Apr 15 15:37:50 2019 -0400,Fix continuous agg trigger handling,Add the continuous aggregate invalidation trigger to chunks that existed before the continuous aggregate was created. Propagate DROPs of the invalidation trigger to chunks. 8c62f31f10801a2d95071f65f443bcc54973e92b,Matvey Arye,Fri Apr 12 14:29:11 2019 -0400,Add timescale aggregates to continuous agg tests,"Previously, last, first, and histogram aggregates did not work as they required FINALFUNC_EXTRA support. Now that this is added, we start testing these aggregates in our dump/restore tests. Also add the test to the list in CMake so that it is run automatically. This was forgotten before. " 0187c581ab086a9005e4557c4f785ac0cc688361,Joshua Lockerman,Fri Apr 5 17:12:48 2019 -0400,Add a Readme for continuous aggregates,"Like it says, adds a Readme containing the most relevant information needed to understand what continuous aggregates do, and how the do it. Read the committed Readme itself for more details. " 3895e5ce0efb2a01cf0b2d1ee37aa7a5309d3661,Joshua Lockerman,Thu Apr 11 17:07:43 2019 -0400,Add a setting for max an agg materializes per run,"Add a setting max_materialized_per_run which can be set to prevent a continuous aggregate from materializing too much of the table in a single run. This will prevent a single run from locking the hypertable for too long, when running on a large data set. " 567890185f134657c6e4d58e130265e08fb74e67,Joshua Lockerman,Thu Apr 11 13:54:42 2019 -0400,Block multiple and recursive continuous aggs,"We currently disallow more than one aggregate per-hypertable, and creating continuous aggregates based on the materialization table for a continuous aggregate while we resolve practical issues involving scheduling that would result from this. " 6a90f2da94949845bbb264efce322b9fee2255aa,Matvey Arye,Fri Apr 12 09:30:03 2019 -0400,Support input_types parameter to finalize,"To support FINALFUNC_EXTRA aggregates we added a parameter to finalize_agg that takes an array of parameter name input types as array[array[type_schema, type_name]]. This commit change the continuous agg view creation code to pass down the appropriate const array of type to the finalize_agg. " b8f9b91e60e25590a06c50194b209d34e5d8ef33,gayyappan,Wed Apr 10 15:41:20 2019 -0400,Add user view query definition for cont aggs,"Add the query definition to timescaledb_information.continuous_aggregates. The user query (specified in the CREATE VIEW stmt of a continuous aggregate) is transformed in the process of creating a continuous aggregate and this modified query is saved in the pg_rewrite catalog tables. In order to display the original query, we create an internal view which is a replica of the user query. This is used to display the definition in timescaledb_information.continuous_aggregates. As an alternative we could save the original user query in our internal catalogs. But this approach involves replicating a lot of postgres code and causes portability problems. " dc0e250428ac71bf4408b6f36795de0b58713af8,Matvey Arye,Tue Mar 26 15:33:04 2019 -0400,Add pg_dump/restore tests for continuous aggs,The data in caggs needs to survive dump/restore. This test makes sure that caggs that are materialized both before and after restore are correct. Two code changes were necessary to make this work: 1) the valid_job_type constraint on bgw_job needed to be altered to add 'continuous_aggregate' as a valid job type 2) The user_view_query field needed to be changed to a text because dump/restore does not support pg_node_tree. 572a433fe65b8a25361d840141148fd2a16dc4bf,Matvey Arye,Mon Mar 25 22:51:31 2019 -0400,Make first/last aggs not serialize Oids,First/Last aggs previously serialized the types used by their polydatum structures as Oids. This will not work with continous aggs as the partial state must be usable across pg_dump/restore actions. Change the serialized state to store the type schema and name instead. 99fc048e12cc2198f1e17e40349aa329c2491c38,Joshua Lockerman,Wed Apr 10 17:42:29 2019 -0400,Handle negative refresh_lag better,"Negative refresh lag can be useful in order to ensure that an aggregate is always materialized to the most recent value, or to start the materialization when a time_bucket is partially filled. This commit fixes an integer underflow when using negative values, and makes sure that the materialization end calculations don't overflow when the max time is near INT64_MAX. " 28ec87f7b59f06f50193bb6ff375ae79911c6b2b,Joshua Lockerman,Wed Apr 10 14:57:46 2019 -0400,Block most ALTER commands on materializations,"Most ALTER commands are either irrelevant or potentially incorrect on continuous aggregate materialization tables, so block them for now. Only allow CREATE/DROP INDEX, ALTER TABLE RENAME, and ALTER TABLE SET TABLESPACE. " 6a04e304ffc6e57c7c80bc431c42e5ea3d9011f2,Joshua Lockerman,Wed Apr 10 13:55:34 2019 -0400,Block TRUNCATE on tables with or underlying a agg,"We haven't decided on what the semantics of TRUNCATE on a table that has a continuous aggregate, so we block TRUNCATE on such tables. We also block TRUNCATE on the underlying materialization table, for now, the only way to delete from the materialization is to delete the underlying rows from the raw hypertable. " 45fb1fc2c81215fde662e17062b3051280b25bcf,Joshua Lockerman,Tue Apr 9 14:47:42 2019 -0400,Handle drop_chunks on tables that have cont aggs,"For hypetables that have continuous aggregates, calling drop_chunks now drops all of the rows in the materialization table that were based on the dropped chunks. Since we don't know what the correct default behavior for drop_chunks is, we've added a new argument, cascade_to_materializations, which must be set to true in order to call drop_chunks on a hypertable which has a continuous aggregate. drop_chunks is blocked on the materialization tables of continuous aggregates " 18d1607909a7bbf8b557bee5560745946804a7d3,gayyappan,Tue Apr 9 16:25:48 2019 -0400,Add timescaledb_information views for continuous aggregates,Add timescaledb_information.continuous_aggregate_settings and timescaledb_information.continuous_aggregate_job_stats views cc862a3c5a3fc287e77950907f26e3870ebce2c1,Matvey Arye,Mon Apr 8 16:45:42 2019 -0400,Implement WITH options for continuous aggs,"1) Change with clause name to 'timescaledb.continuous' Used to be timescaledb.continuous_agg as a text field, now is a bool. 2) Add more WITH options for continuous aggs - Refresh lag control the amount by which the materialization will lag behind a the maximum current time value. - Refresh interval controls how often the background materializer is run. 3) Handle ALTER VIEW on continuous aggs Handle setting WITH options using continuous views. Block all other ALTER VIEW commands on user and partial views. " 7fc23ee08ce821709daae31656755f6d2197c35d,Joshua Lockerman,Mon Apr 8 11:36:48 2019 -0400,Handle renames for continuous aggregates,This commit adds the ability to RENAME and SET the SCHEMAs of continuous aggregates; whenever one of those commands is executed they must cascade to the continuous aggregate catalog table. 4766758294c27974af5dab0cf7544c94ee06a0c1,Joshua Lockerman,Fri Apr 5 12:14:28 2019 -0400,Handle tables being dropped before materialization,"If the raw hypertable or the materialization table are dropped before the materialization can start, we should error out gracefully. (This can occur since we close the bgw's transaction before we start the actual materialization) " 19d47daf23cb8a6e240a8f5d20e008d7b055f2e2,Matvey Arye,Fri Apr 5 10:43:09 2019 -0400,Delete related catalog rows when continuous aggs are dropped,This PR deletes related rows from the following tables * completed_threshold * invalidation threshold * hypertable invalidation log The latter two tables are only affected if no other continuous aggs exist on the raw hyperatble. This commit also adds locks to prevent concurrent raw table inserts and any access to the materialization table when dropping caggs. It also moves all locks to the beginning of the function so that the lock order is easier to track and reason about. Also added a few formatting fixes. 1cbd8c74f7465b6a21a0aab1fb85271c1687f254,gayyappan,Wed Apr 3 15:40:45 2019 -0400,Add invalidation trigger for continuous aggs,Add invalidation trigger for DML changes to the hypertable used in the continuous aggregate query. Also add user_view_query definition in continuous_agg catalog table. 0737b370a323ba8ef7d4e916f52147582800414e,Joshua Lockerman,Wed Apr 3 17:17:33 2019 -0400,Add the actual bgw job for continuous aggregates,"This commit adds the the actual background worker job that runs the continuous aggregate automatically. This job gets created when the continuous aggregate is created and is deleted when the aggregate is DROPed. By default this job will attempt to run every two bucket widths, and attempts to materialize up to two bucket widths behind the end of the table. " 7a4191bd84a48ca8438a685da009fdbf00f13e18,Matvey Arye,Tue Apr 2 16:39:46 2019 -0400,Handle drops on continuous agg views and tables,"Previously we used postgres dependency tracking to ensure consistent deletes between various continuous agg postgres objects (views and tables). This does not work with dump/restore and thus this PR removes that dependency tracking in favor of handling these deps ourselves in various drop hooks. This PR also adds logic for deleting rows in the continuous_agg metadata table when the view is dropped. It does not yet handle deleting associated threshold metadata, that's left for a future PR. Most of this logic is in the apache-licensed code and not in the TSL since we want people who downgraded from TSL to apache to still be able to drop continuous views. " bf44985ac3848a7d1ca066fb0bcfad604dc49132,Joshua Lockerman,Fri Mar 15 11:59:32 2019 -0400,Add REFRESH MATERIALIZED VIEW for continuous aggs,"This commit also tests end-to-end materialization. This commit finishes the materialization path for continuous aggregates, and adds the ability to use REFRESH MATERIALIZED VIEW to invoke it. Materialization is invoked via continous_agg_materialize, and happens in two transactions: 1. In the first transaction we lock the relevant tables, determine the point below which we will end materialization, and update the invalidation threshold. 2. In the second transaction, we read the actual data and perform the actual deletions and updates to the materialization table. We materialize in this manner because in order to allow mutations to the underlying hypertable concurrently with materialization, the invalidation threshold must be updated strictly before materialization starts; anything else could cause us to lose invalidations. (Simply blocking all mutations to the table while materialization is occurring is a non-starter) More precisely, the operations we perform are as follows: Transaction 1: 1. read the completed threshold for the continuous aggregate 2. find the maximum timestamp in the hypertable that is greater than the old completed threshold, scanning the entire table if this is the first materialization 3. if we found a new maximum value, and said value is sufficiently old (exact definition of sufficiently TBD in a later PR), update the invalidation threshold to point at this new value (said update is performed under an AccessExclusiveLock to ensure there are no concurrent mutations) Transaction 2: 1. drain the invalidation log 2. read the invalidation threshold 3. delete from the materialization table everything below the invalidation threshold that was invalidated 4. scan the raw table and insert new materializationa for everything invalidated, and everything between the completed threshold and the invalidation threshold 5. update the completed threshold to equal the invalidation threshold " 04f79756f69493d9626d1cdc9feec5c58b206a2e,gayyappan,Mon Mar 25 16:22:20 2019 -0400,Validate continuous agg queries,"Verify that the query passed to CREATE VIEW for continuous aggregate queries is valid. The restrictions are as follows: CREATE VIEW WITH ( timescaledb.
WITH (timescaledb.transaction_per_chunk); should cause less contention than a regular CREATE INDEX, in exchange for the possibility that the index will be created on only some, or none, of the chunks, if the command fails partway through. The command holds a lock on the root index used as a template throughout the command, and each of the only additionally locks the chunk being indexed. This means that that chunks which are not currently being indexed can be inserted to, and new chunks can be created while the CREATE INDEX command is in progress. To enable detection of failed transaction_per_chunk CREATE INDEXs, the hypertable's index is marked as invalid while the CREATE INDEX is in progress, if the command fails partway through, the index will remain invalid. If such an invalid index is discovered, it can be dropped an recreated to ensure that all chunks have a copy of the index, in the future, we may add a command to create indexes on only those chunks which are missing them. Note that even though the hypertable's index is marked as invalid, new chunks will have a copy of the index build as normal. As part of the refactoring to make this command work, normal index creation was slightly modified. Instead of getting the column names an index uses one-at-a-time we get them all at once at the beginning of index creation, this allows to close the hypertable's root table once we've determined all of while we create the index info for each chunk. Secondly, it changes our function to lookup a tablespace, ts_hypertable_get_tablespace_at_offset_from, to only take a hypertable id, instead of the hypertable's entire cache entry; this function only ever used the id, so this allows us to release the hypertable cache earlier " 3ef6732b09eefe917af59babdce921c7a3cf1509,Joshua Lockerman,Fri Jan 18 12:26:08 2019 -0500,Add WITH-clause parser,"For a number of DDL statements we want to pass in our own arguments as part of the WITH statement. This commit adds a parser for such statements, and tests to ensure that this parser behaves correctly. Currently, all our WITH options will be namespaced under ""timescaledb"". In other words, if we wish to add an option to CREATE INDEX called ""foo"", it will be used CREATE INDEX ... WITH (timescaledb.foo='bar') This should ensure that all our options are consistent with each other, and that it is obvious which options are ours, and which are not. " 037fbf634d7f8dab62119c9c7f981342e69417e8,Sven Klemm,Fri Feb 22 17:56:25 2019 +0100,Add missing ORDER BY to chunk_utils tests, 207391bc5430532eb814b7292c998b6adf27fde1,Sven Klemm,Tue Jan 22 18:08:11 2019 +0100,Remove parent oid from find_children_oids result,"Since the parent table of a hypertable cannot contain any rows including it in the plan is not necessary. PostgreSQL might also not have good statistics for the main table as autoanalyze is triggered by inserts which never happen to the main table, leading to worse plans. This patch adds a new guc enable_constraint_exclusion to disable planner constraint exclusion. Constraint exclusion will not work with cursors and WHERE CURRENT OF so the new guc will allow you to disable that feature. This patch also fixes constraint exclusion for JOINs and sort optimization for queries with chunks_in. " 72f4692c21a139af543e881b57f7ba36aa74c583,Joshua Lockerman,Wed Feb 20 14:50:36 2019 -0500,Add the ability to set the number of parallel tests from cmake and have sanitizer tests do so,"With the forthcoming new sanitizer image running tests in parallel causes machines to run out of resources, and fail tests spuriously This commit forces test_sanitizers.sh to run tests sequentially to prevent this. " 124e7159160fd69ed66f7f9bae4b369cb729c482,Joshua Lockerman,Wed Feb 20 11:13:57 2019 -0500,Fix use-after-free in gapfill,"Changing paths in-place causes issues when add_paths realizes the old path is no longer optimal, and decides to free it. Instead duplicate the path list, and use add_path ourselves to add our new gapfill paths " 8f3444caaed5f253c13bf1bbcace9f4eb244ca52,Joshua Lockerman,Wed Feb 20 10:10:50 2019 -0500,Fix off by 1 error in gapfill/exec.c:foreach_column,We cannot access the column array in the increment; at that point we do not yet know if the access is in bounds. Fixed by moving it to the condition cdcd8c9ab8cef8579ddafc5295fd565a132006b3,Sven Klemm,Fri Feb 15 23:36:46 2019 +0100,Update CHANGELOG, 33ef1de542e1b5bb69c55668e0082bf9294b94e3,Sven Klemm,Fri Feb 15 16:38:07 2019 +0100,Add treat_null_as_missing option to locf,"When doing a gapfill query with multiple columns that may contain NULLs it is not trivial to remove NULL values from individual columns with a WHERE clause, this new locf option allows those NULL values to be ignored in gapfill queries with locf. We drop the old locf function because we dont want 2 locf functions. Unfortunately this means any views using locf have to be dropped. " 6ee217fed38b3d5601b4b5ccc76c293c02fe8f50,Matvey Arye,Thu Feb 14 13:22:18 2019 -0500,Add .editorconfig for better github display,"Our code uses a tabwidth = 4. By default, github displays everything with tabwidth = 8. But github respects .editorconfig files. So this commit adds the appropriate .editorconfig. This could also be useful for other editors as .editorconfig seems to be widely used. " bf3b3b8a6902798cd83208921c918208d4ad9c4f,Joshua Lockerman,Tue Feb 12 16:29:43 2019 -0500,Add sanitizer based tests,"This commit adds a sanitizer run script, and updates travis to run it in our nightly builds. Valgrind has been very slow and unstable, so much so that we only run it during pre-release tests, and even then it doesn't necissarily provide useful output. The sanitizer tests run about 10x faster, and seem stable. We currently only run ASan and UBSan, as MSan has too many issues within postgres's own initialization functions. As MSan cannot be added to a binary that enables ASan, it would require separate jobs anyway. " 291214199ec25e374b53bf204e1acd8308c9a334,Joshua Lockerman,Tue Feb 5 14:08:55 2019 -0500,Fix issues discovered by sanitizers,"This commit fixes three issues: 1. It is technically UB to pass in NULL as the data argument to appendBinaryStringInfo We fix this by providing a wrapper function which checks if len == 0 before calling appendBinaryStringInfo. 2. It is technically UB to pass in NULL as any argument for memcpy, we use a similar fix as in 1. 3. The Datum array in histogram was misaligned; Datums require 8 byte alignment, while the data in a bytea is only guaranteed to be aligned to 4. We fix this by switching to a better-typed internal histogram type. This will use approximately double the memory for serialization, but is much simpler to read, and more morally correct. " 8e86bda9ddac1ebbfc4cf571697e7640a3ab8421,Sven Klemm,Tue Feb 12 21:07:42 2019 +0100,Fix sort_transform optimization,The sort transform optimization to enable index usage are only safe transformations when the modified PathKey is either the last member of pathkeys or the only member. b6d4202e221b18bbca7fe5856370decb0e45094a,Sven Klemm,Sat Jan 26 16:46:46 2019 +0100,Infer time_bucket_gapfill arguments from WHERE clause,Infer time_bucket_gapfill start and finish arguments from WHERE clause when they are not supplied as arguments. 5a3edfdf6559c9bd8445a1a63584a92f7a414d2f,Narek Galstyan,Fri Feb 8 15:17:51 2019 -0500,Fix chunk exclusion constraint type inference,"The existing constraint exclusion code creates issues on 32bit processors when constraint given in a query does not exactly match the type of the table dimension but could be promoted to it. For example, if hyper_w_space has bigint time type, the following query will pass 10 as Const variable of type int32 (that has Oid INT4OID). SELECT * FROM hyper_w_space WHERE time < 10 If this type info is not propagated correctly, later Const variable type is inferred as bigint (int64) and DatumGetInt64 is used to get its value. In 64 bit systems, DatumGetInt32 and DatumGetInt64 are identical and so there are no issues, however this makes a difference in 32 bit systems as there 32 bit values are passed by value while 64 bit values are passed by reference. " f202ec24a73876484bfedbaa7ba875916cddae74,Matvey Arye,Thu Feb 7 15:48:31 2019 -0500,Only perform chunk insert state switch actions when necessary,This commit changes the code to only perform cis switch actions when the cis changes instead of on every tuple. Several recent bug fix changes have increase the amount of work that needs to be done in switch actions and so this commit is a performance improvement that avoids doing unnecessary work. 73e4adb59822ab454baf979864935121f8618b97,Joshua Lockerman,Mon Feb 11 10:54:24 2019 -0500,Prepare the repo for the next development cycle, f621c01097ab044d8a5a03212eca55bf292bfe89,Sven Klemm,Sat Feb 9 22:59:09 2019 +0100,Improve error handling for unsupported datatypes,Also fixes a typo in bgw error message. 6d9ffe5c7d08994613ca08663c354586e4fbfb63,Joshua Lockerman,Fri Feb 8 12:52:47 2019 -0500,Release 1.2.1, 91ab95581b43ff3faa7aedd92bada818ca667498,Matvey Arye,Thu Feb 7 14:00:35 2019 -0500,Add additional asserts to upserts,These asserts check that the mtstate->mt_conflproj is the same slot as in the result rel info on conflict projection and that the slot descriptor is set correctly. 2f6b58af21d9ee64ddb590a4b7ed057d810ef901,Matvey Arye,Tue Feb 5 10:54:41 2019 -0500,Fix tlist on hypertable inserts inside CTEs,"Previously, hypertable insert tlists were only fixed for the top-level plan. We also need to fix the tlist for hypertable inserts that appear inside ctes " ed30358f7ba403b73a91c5c4729c3e9b694eb73c,Erik Nordström,Thu Feb 7 11:08:00 2019 +0100,Exclude hidden and CMake directories when formatting code,"The script for running clang-format on the source code now properly excludes hidden directories and CMake directories from the source code scan. Including these directories may otherwise break clang-format because hidden directories are often used for IDE/tools caches (e.g., `cquery`) where the cache files have the same ending as the source code. " 7741722e933d5ba9c16498e98b6f1f3bc3366327,Joshua Lockerman,Thu Jan 24 15:28:06 2019 -0500,Add session locks to CLUSTER,This prevents the cluster index from being deleted or altered while in middle of the CLUSTER. We grab locks and mark the main table as CLUSTERed before working on the chunks in order to prevent deadlocks with concurrent DROPs ba32dab0112e9849b7750035dd13b429e75fb52c,Matvey Arye,Tue Feb 5 16:16:57 2019 -0500,Fix displayed error when an arbiter index is missing.,"When doing an upsert, we expect to be able to translate a hypertable index used as an arbiter to a chunk index. But, it is possible that the corresponding chunk index has been dropped. Previously this resulted in a postgres generated error like ""unexpected failure to find arbiter index"". We now handle this more gracefully. " 34edba16a9385a4b0353e8e07a19dba98d7e3fb9,Matvey Arye,Tue Feb 5 16:26:38 2019 -0500,Run clang-format on code, b891a28b80d6ea732730d15c21641cbcee5882cb,Matvey Arye,Wed Dec 5 15:06:04 2018 -0500,Add final commas to designated initializers,Final commas are needed for clang-format to format the code appropriately. cc52a78b9bf3b611993f33f93594a2c503e7aeec,Matvey Arye,Thu Nov 8 15:52:44 2018 -0500,Add scripts for clang-format,"We are switching from pgindent to clang-format for several reasons: 1) pgindent is harder to set up and depends on other tools like gobjdump that differ between OSes making it harder to set up a proper development environment. 2) clang-format is more standard and works better with various IDEs and editors. Our clang-format wrapper script is transactional in that this version of the script copies all files to a temporary directory, runs clang-format in the temp dir, and only then copies the files back. This should prevent mis-formatting when the script is cancled in the middle of running. We changes travis to use clang-format instead of pgindent and thus it uses a different docker image to do the format test. We also changes travis to use the new clang docker image for the license check and prefix checks for consistency. This also paves the way for us to use more clang features for tests in the future. " e6c68f89b90ff9af869efbd6c5c00d2d6fc879dd,Sven Klemm,Mon Feb 4 20:38:43 2019 +0100,Fix group handling in time_bucket_gapfill,When time_bucket_gapfill is used with multiple groups but the query returned no rows we can not generate any rows in the gapfill node. 3a81559453a5a0baa24818b46d31db018253f1fb,Joshua Lockerman,Fri Feb 1 12:31:50 2019 -0500,Add cron a pre-release Release mode testing,"Most of our testing is performed with Debug enabled. This is desired as it enables more assertions, and helps ensure that our code is correct. Still, it's good to ensure we're not accidentally depending on Debug code in Release builds, so run tests in that mode nightly and before each release. " 034a0b0b8e8d102f223a56d740fac138cf67c921,Sven Klemm,Sun Feb 3 05:35:28 2019 +0100,Check for NULL pointer in ts_bookend_finalfunc,"When the sfunc is never called to initialize InternalCmpAggStore state can be NULL even though PG_ARGISNULL is false, so we check for that condition here. Also add a similar check to ts_hist_finalfunc. " 8fec685172e269df0695403b475bf88783b8c5d3,Sven Klemm,Sun Feb 3 14:21:20 2019 +0100,Fix typos in comments, 7973b4a753ee1967b2d83b57a899d407d13375de,Sven Klemm,Sat Feb 2 16:10:26 2019 +0100,Stop background worker on rename database,When background workers are running RENAME DATABASE will not work because there are still active sessions to that database open. This stops background workers on RENAME DATABASE so databases can be renamed without manually stopping background workers. c7803730d8ee1b626217c95418b8c8012f483534,Joshua Lockerman,Fri Feb 1 12:56:36 2019 -0500,Fix typo failing our update 9.6 update tests, 1a54e0695396aa9d9debf6a8531925a03b486d6e,Joshua Lockerman,Fri Feb 1 12:14:47 2019 -0500,Run travis ci with optimizations turned on,"Our users will pretty much always run timescale with optimizations, so we might as well run our CI in a more realistic setup. Plus, this may make CI run faster, since build times are negligible. " 47fcfc07d18a9d7fb689e82b49c2e1bd4614dfbc,Erik Nordström,Fri Feb 1 12:50:35 2019 +0100,Fix release build tests and exclude debug/test code,"This change excludes test and debug code from the release build of the TSL library and also makes tests properly work under release builds. Excluding test/debug code is important since otherwise a significant amount of dead code will inflate the size of the library. Some issues with the TSL isolation tests have also been addressed. Generated spec files are removed from the GitHub repository and are now output to the build/binary directory instead. To be able to run specs from the build directory, the `pg_isolation_regress.sh` script has been updated to be able to run tests from a non-default specs dir. " 68312bd346de7a65cc854b1852a7983a4a361a30,Erik Nordström,Thu Jan 31 21:13:16 2019 +0100,Make code formatting cover TSL test source code,This change makes code formatting (`pgindent`) cover the TSL test sources under `tsl/test/src/`. It also addresses some minor formatting issues with existing sources in other directories. 32cc64596d79e87b7e5212b443c2be06b80e8a57,Joshua Lockerman,Wed Jan 30 21:11:40 2019 -0500,Fix loading the tsl license in parallel workers,"Parallel workers have an unusual load path, which doesn't behave the same as other processes. Since we don't have any obvious hooks we can use to enable loading, and the library load order is already frozen by the main process, fix this by adding a _PG_init which will enable the tsl in background workers when needed. " bf3a69c021dfeee315183b702a0d5d1a3e56c878,Joshua Lockerman,Tue Jan 29 15:22:06 2019 -0500,Add warning about restarting loader for this release,"And a ""thanks"" to the user who found the bug that requires it " 61ccb6ece8ebb4d1b8acfddbade99babcf4c6275,Joshua Lockerman,Tue Jan 29 14:55:36 2019 -0500,Fix loader for old versions of timescale,The loader needs to be backwards compatible. Therefore we handle the case where post_load function doesn't exist gracefully. 57b1d11919d83408463572901a302c402758cf4c,Joshua Lockerman,Tue Jan 29 12:56:51 2019 -0500,Prepare the repo for the next development cycle, 4295c04caf59326806e0f1b220bcf579cffdd4ad,Joshua Lockerman,Fri Jan 25 14:28:37 2019 -0500,Release 1.2.0, a53173395c359a89e42754152cfa360faac1bcd7,gayyappan,Mon Jan 28 16:37:29 2019 -0500,switch cis state when we switch chunks., 5c6b619326425777d6a36c59566a76b027661587,Matvey Arye,Sun Jan 27 22:19:08 2019 -0500,Make a copy of the ri_onConflict object in PG11,"In PG11 the OnConflictProjInfo object is a member of ri_onConflict. Previously, the ri_onConflict object was copied by reference when creating our rri. But since OnConflictProjInfo has to change for each chunk and CIS this was incorrect as a change to OnConflictProjInfo changed the ""global"" ri_onConflict. This commit changes the copy of ri_onConflict to a deep one so that each CIS now has it's own independent OnConflictProjInfo. " 61e524e0c10e8345c6a25e7139ee2c695a6c72cc,Matvey Arye,Sat Jan 26 23:02:28 2019 -0500,Make slot for upserts be update for every chunk switch,"Previously, the slots for upserts were only updated when a ChunkInsertState was created, but not when it was found in the ChunkDispatch cache. This was wrong -- we need to update the slots every time we switch to the chunk. Right now the switch function is called on every tuple. We will optimize that later to only be called when the previous chunk was different. But, that is not part of this bug-fix PR. " 8a7c127f6c88b49be39dcb9bbe6bd647937ac48d,Matvey Arye,Sat Jan 26 18:42:26 2019 -0500,Fix for ExecSlotDescriptor during upserts,"Previously we set the mt_existing and onConflictProjInfo slot descriptors only when the current chunk had a tup_conv_map this was based on a false assumption that these slot descriptors had the hypertable's slot descriptor. In fact, since we are reusing these slots across chunks, the slot descriptor will correspond to the previous chunk's one. This led to an error if the previous chunk had a tup_conv_map but the current one did not. Most of the work done here was by @gayyappan with minor improvements by @cevian. The latter is submitting because of a time-crunch having to do with upcoming release. " fd8a5197c803e267d5198fdfdc12867dfb6ef209,Sven Klemm,Sat Jan 26 16:46:46 2019 +0100,Make time_bucket_gapfill start and finish optional,"Make time_bucket_gapfill start and finish optional, this is in preparation for deducing them from WHERE clause. We make this optional now to not introduce breaking change later. This also only allows simple expressions for bucket_width, start and finish because only those can be evaluated safely in gapfill_begin. " 53d5760535d831fdef3ab0b92e727606a5532fe0,Narek Galstyan,Mon Jan 28 07:50:42 2019 +0000,Fix Datum conversion in gapfill, 858da6dcf74a976928ca58f517da3bf34b587324,Narek Galstyan,Sun Jan 27 16:36:24 2019 +0400,Fix some DirectFunctionCall calls (correctly convert arguments to Datum), 88c7149c2c7bb913602f9f99c02b5e6e089a8c7e,Joshua Lockerman,Fri Jan 25 14:56:33 2019 -0500,Fix issues with non-dev versions when generating the update scripts,"Without this, we generate multiple rules for the latest script " b525db447fe615e7b558d03504f64d421eef0bf9,Narek Galstyan,Sun Jan 27 15:59:25 2019 +0000,Change long to int64,"Depending on host hardware, long can be 32 or 64 bits long. Changing it to int64 or int32 makes the code more portable and helps avoid issues when running in 32bit systems. " cf67ddd9b0faf24bfc0e9dff49b27268d016ee60,David Kohn,Fri Jan 25 12:36:57 2019 -0500,Add informational views for policies,Add views so that users can see what the parameters are for policies they have created and a separate view so that they can see policies that have been created and scheduled on hypertables. 73d3a1466537d546785cf3830633e89be59ef126,David Kohn,Fri Jan 25 12:13:48 2019 -0500,Rename alter_policy_schedule & main_table for better UI,Rename alter_policy_schedule to alter_job_schedule for consistency with the job_id argument passed in. Also rename main_table to hypertable in all of the policy related functions as they must deal with hypertables that have already been created. fa61613440977ed2be39ffad19b61ba1d9aabdd7,Sven Klemm,Fri Jan 25 18:10:54 2019 +0100,Change time_bucket_gapfill argument names,"time_bucket_gapfill used end as argument name which is a sql keyword and has to be quoted when used, this changes the argument names from start/end to start/finish. " fabaa6ea5122e27ba05b307f1c4215cae5a13634,Sven Klemm,Sat Jan 19 12:15:36 2019 +0100,Use hypertable for parallel tests,Generated data has been offset by 1 to align with chunk boundaries. 319b79c8ecf7cbba17d000c0cc03bccad5209a02,niksa,Tue Jan 22 16:59:14 2019 +0100,Making chunks_in function internal,This function needs chunk ids as input. Since chunk ids are TimescaleDB internal metadata it feels more natural to make this function internal. 03f8264f926b1caca77ad01e56ea575af3fddc62,Matvey Arye,Fri Jan 18 13:32:42 2019 -0500,Mark Postgresql includes in cmake as SYSTEM includes.,This is needed for tools like include-what-you-use to decide which files should be included with quotes vs carrots. This is probably more correct anyway as postgres files are included with carrots in our code. Also made the change for OpenSSL because it seems more correct. 6f55c9c657eb5f0e07b1b983a28016fa823257a9,Sven Klemm,Tue Jan 22 20:48:12 2019 +0100,Bail on ordered append when pathkeys don't match,"When an index is not available on all chunks pathkeys of the child might not match pathkeys of the MergeAppendPath, PostgreSQL fixes this when creating the merge append plan by inserting a sort node for the child. Unfortunately this is too late for us so we don't do this optimization for those cases for now. " 77562f24e3d98ea535380a74f82fa616de9841d5,Joshua Lockerman,Fri Jan 18 11:53:13 2019 -0500,Fix formatting for error message, a7bd9ccdaffd9a271df35cdea94082a68dcf7382,Sven Klemm,Sat Jan 19 13:32:35 2019 +0100,Remove variable declaration in for loops,"Since we add PostgreSQL CFLAGS to our own CFLAGS on some platforms using variable declarations in for loops will generate compile warnings, depending on the CFLAGS PostgreSQL is compiled with. " c77f4ab1b30cbc7b70ca6eafa53e8c00df8a4d50,niksa,Wed Jan 9 17:46:11 2019 +0100,Explicit chunk exclusion,"In some cases user might already know what chunks need to be scanned to answer a particular query. Using `chunks_in` function we can skip calculating chunks involved in particular query which should result in better performances as well. A simple example: `SELECT * FROM hypertable WHERE chunks_in(hypertable, ARRAY[1,2])` " 77ee72804cdfd8637b8a575f1511faf5bd10784e,Diana Hsieh,Fri Jan 18 12:48:05 2019 -0500,Update README for small typo,"Small typo -> changing ""build"" to ""built"" when discussing Apache-ONLY" fdaa7173fb42bd62be77a5ecc027a1104203d407,Joshua Lockerman,Thu Jan 17 13:18:47 2019 -0500,Update telemetry with prettier os info,"The info gotten from uname is difficult to work with, so read the os name from /etc/os-release if it's available. " 01be3946f2565d913083629ce03633bfd99b3ae3,Joshua Lockerman,Thu Jan 17 11:59:58 2019 -0500,Fix bgw_launcher restart when failing during launcher setup,"We do cleanup for bgw_message_queue in the before_shmem_exit function, but we were registering said function after initializing the database hashtable. When bgw_launcher would crash in middle of populating the hashtable, this would cause issues as the bgw_message_queue wouldn't be reset, and the launcher would fail to restart as it would think that another launcher was running. This commit moves the hook registering to before we touch any of the state that the hook cleans up, in order to ensure that the state will be cleaned regardless of where the failure occurs. This is already safe for the bgw_message_queue since that function validates that the caller owns the queue before altering it, and is made safe for the hashtable via an additional level of indirection, so we can check if the hashtable exists. This commit also cleans up some of the lock ordering around the bgw_message_queue to prevent potential race conditions; specifically, if we're doing a test-and-set, both the ""test"" and the ""set"" have to be within the same lock-acquisition. " 37ed69dfbd388f7333b46c15025bebf281d521c4,Sven Klemm,Thu Jan 17 12:13:42 2019 +0100,Fix typo in background worker error message, 7b3929e30dd79d2940998afde19fa06b4cceadaf,Sven Klemm,Fri Jan 11 10:57:11 2019 +0100,Add ordered append optimization,This optimization will replace the MergeAppendPath for queries on hypertables ordered by the time partitioning column and with a LIMIT clause with an ordered AppendPath. This optimization will remove the need for last point queries to access every chunk of a hypertable. This commit also adds struct TimescaleDBPrivate which is stored in RelOptInfo->fdw_private to store TimescaleDB-specific plan state between different planner hook invocations in the plan. We needed to add TimescaleDBPrivate to store a flag indicating whether or not to use ordered append between different parts of the planner. 934eab71cf98811e953cbf0fc579a8c0491361a7,Matvey Arye,Wed Jan 16 18:50:15 2019 -0500,Fix bugs in chunk_scan, 7581c50e2d443007f16541ddc5e47f4c22a8c063,Sven Klemm,Wed Jan 16 21:03:18 2019 +0100,Install findutils in build container,"The find syntax used by the codecov script is not compatible with busybox find, installing GNU find makes the codecov script functional again. " a69f84c14d2fa28358cad7bd23382651a42865d7,Joshua Lockerman,Mon Jan 7 12:52:38 2019 -0500,Fix signal processing in background workers,"When receiving a SIGHUP, background workers should reload pg_conf to get any changed guc values. Also fix issue where the test bgw_scheduler was not properly waiting for child processes to die before shutting itself down. The test scheduler did not setup the sigterm handler, so it wouldn't shutdown child processes correctly. This meant that our tests, which would later test the number of jobs running, would depend on the exact timing signals would propagte to bgw processes. This should increase the robustness of bgw_db_scheduler. Change tune telemetry guc to PGC_SIGHUP because it looks like BACKEND gucs interact oddly with ALTER_SYSTEM in our tests. " 6d26388eadc14f55da51282f25b45b534a4c7856,Joshua Lockerman,Mon Jan 14 15:35:19 2019 -0500,Only use seqscans for reorder,"In our workloads we expect that there will never be a case in which an indexscan based reorder will perform better than a seqscan based one. In order to prevent pessimization, disable indexscan based reordering for btrees, and only use it when we absolutely must. " 320e93d2c8e48138af2555784efebf6830cb4c61,Joshua Lockerman,Mon Jan 14 14:52:36 2019 -0500,Add tests of scheduled jobs within actual background workers,To date all our bgw tests run in the normal psql environment. This commit adds tests that the jobs function correctly when run from actual background workers. This should catch cases where the background worker behaves differently than a normal connection. 00f23f82e8b1d244a67e651c3ecfdf94210d56bc,Joshua Lockerman,Mon Jan 14 12:50:58 2019 -0500,Use job_stat last started time on reorder fast restart instead real time,"In reorder bgw fast restart, use the time we declared the reorder job should start in the job_stat table instead of using the transaction's start time as the next start time. This allows us to test that fast restart operates correctly when running using a virtual timer. Use the default schedule interval for integer open dimensions in reorder This makes reorder jobs behave properly on non-time open dimensions " 49df133c723a2d9bd2a771f21441b65339ebf626,Joshua Lockerman,Mon Jan 14 14:16:53 2019 -0500,Disallow scheduled drop_chunks policies on non-time open dimensions,"Scheduled drop_chunks works by dropping chunks older than a certain interval. Since we have no way to calculate what 'now()' is for non-time dimensions, we currently cannot have a scheduled drop_chunks job that works with them. Currently this causes an error when the policy attempts to execute. This commit moves the error, so that it occurs when a user tries to add the policy in the first place. " f75106ebc2e8cf77ffc1463ff27fd56a4e6feaa3,Joshua Lockerman,Mon Jan 14 16:07:40 2019 -0500,"Point the travis build badge at ""history"" instead of ""current""","Right now, clicking the travis build badge directs the user to the currently running travis job. Since this is usually a PR, and we gate PR merges on travis anyway, it's not very useful for determining the health of the project. This commit switches the link to point at the build history, which shows the previously run cron jobs; the jobs that the badge points at anyway. " b4f6b2c8ba7b372ff01aceb38b54d9edf8da33d5,Sven Klemm,Mon Jan 14 21:42:34 2019 +0100,Run all diff tests after normal tests,plan_hashagg_results_x_diff and plan_expand_hypertable_results_diff rely on output of previous tests so run them after other tests 623c7f3dc6fc2052b8b3203f3cf86258f5621c20,Sven Klemm,Sat Jan 12 01:05:54 2019 +0100,Ignore rolenames pgregress test,"The rolenames test seems to be fail quite often in CI, so we ignore the result from that test for now. " f89fd07c5bf742b26993fee6a3f763948f142bc6,Sven Klemm,Sun Jan 13 14:20:23 2019 +0100,Remove year from SQL file license text,This changes the license text for SQL files to be identical with the license text for C files. 65894f08cf8dff08beb48a7d35e14e24fbefba5d,Joshua Lockerman,Thu Jan 10 16:38:09 2019 -0500,Add view displaying info about the current license,"Currently the view displays the current edition, expiry date, and whether the license is expired. We're not displaying the license key itself in the view as it can get rather long, and get be read via SHOW. We also do not display the license's ID since that is for internal use. " 47b5b7d55371de0451c369e83db8100fb0b2ae3c,Joshua Lockerman,Wed Jan 9 14:47:39 2019 -0500,Log which chunks are dropped by background workers,"We don't want to do this silently, so that users are able to debug where their chunks went. " 27cd0fa27dfbd95a13d64ae640c0df82e8c2f396,Joshua Lockerman,Wed Jan 9 16:47:25 2019 -0500,Fix speeling, 8f7f585e1ce304e745a8590f2ae8d2fe39df4e6e,Joshua Lockerman,Wed Jan 9 15:38:54 2019 -0500,Set the reorder job to run immediately if there are more chunks in need of reordering,This will allow the job to catchup if there's a backlog of chunks in need of reordering. 4a7ad778f528965ca8535f3f0884e030d6e08d89,Joshua Lockerman,Wed Jan 9 11:20:53 2019 -0500,Clean missing braces warning,"This shouldn't be needed, but it fixes a spurious warning for those of us that enable more warnings. " 4cfd9ff47a7d9cb55edda7fd39b731cf9871433d,Joshua Lockerman,Tue Jan 8 13:52:38 2019 -0500,Clean error handling for invalid licenses,We shouldn't segfault fafc98d34317f2acdc5c7ea20c6d8ecc5d523857,Joshua Lockerman,Tue Jan 8 12:51:49 2019 -0500,Fix warnings for TSL licenses,"So as to reduce the amount of logspam users receive, restrict printing license info to the following: 1. On CREATE EXTENSION a. in the notice, print the license expiration time, if any b. if the license is expired additionally print that c. else if the license will expire within a week print an addional warning 2. On the first usage of a TSL function, print if the license is expired or will be expired within a week " 057eef1b8ea8bb3b6a69a466a4deb62bd8d363df,David Kohn,Tue Jan 8 17:59:29 2019 -0500,Log to pg log when running reorder jobs,Log before and after we run the reorder job on a chunk so that there is more information in the server logs around when and how reorder jobs occur. ea718e0b8474afd9ecd5a87f1cb500745afd1fee,dianasaur323,Tue Dec 25 09:56:02 2018 -0500,Make older_than / newer_than error message user friendly, 428adfb559805be8776f5d887b0d4a955d052737,Joshua Lockerman,Mon Jan 7 16:29:42 2019 -0500,Disable parallel workers in background workers,We currently do not have a parallel worker context in out BGW and we wouldn't want our BGW to start parallel workers anyway in order to conserve resources. This commit disables parallel workers in BGW jobs. ab317913c7f92a8667dff56f9cbb6ef4e65a2f55,Sven Klemm,Mon Jan 7 12:38:50 2019 +0100,Execute regression tests in parallel,Change the schedule generation for the regression tests to generate a parallel schedule to improve test runtime. Also moving tablespace directory creation to pg_regress.sh to prevent redundant work and race conditions between parallel runs. 5943b98782bbf50f43cecfac1948e0f9dc37b8b2,Sven Klemm,Mon Jan 7 12:32:50 2019 +0100,Replace hardcoded database name in test files,Replace hardcoded databases name with :TEST_DBNAME in test scripts and :DBNAME in include files. This will allow parallel execution of test scripts. 899a8fe13d8dbd25925244579227334ee8cfb123,Sven Klemm,Mon Jan 7 12:31:00 2019 +0100,Fix race condition in drop_rename_hypertable test,The size of the bgw_job_stat table depends on running jobs and might be inconsistent between runs. 155ce1a7cb9811078dff2f26d737bf4073e067d4,Joshua Lockerman,Mon Jan 7 12:51:47 2019 -0500,Reload license manually in BGW,"Currently BGW jobs get guc values but do not necessarily run the assign hook. This is an issue for the TSL because the TSL is loaded in just such a hook; if they are never called, TSL functions cannot be used. Fix by manually reloading the guc in the TSL. " 34c8096831e23df7eaae260f3334f462ff08f5d4,dianasaur323,Tue Dec 25 09:56:02 2018 -0500,Make older_than / newer_than error message user friendly, 81d85ff60dfef4cd8d668535a201012ed6133d6d,Sven Klemm,Mon Jan 7 14:23:22 2019 +0100,Differentiate between required license in error hints,Show different error hint depending on the license required to use a feature. a808d03b8df0d0a52563432e3637f0d2a6669e13,Sven Klemm,Fri Jan 4 18:26:16 2019 +0100,Add time_bucket_gapfill tests for int2/4/8,The int2 and int8 versions of time_bucket_gapfill weren't well tested since almost all the int tests were testing int4. 10b56e9040d4b63a8b1900b10cb570e5ffd6f383,Joshua Lockerman,Mon Jan 7 11:46:07 2019 -0500,Add cron job to test ApacheOnly builds nightly on travis,"It’s good to check if these work, and we definitely need to before every release " 48e13c9fa41764b95007f3fd0d735b3fee5ca899,Joshua Lockerman,Mon Jan 7 14:01:27 2019 -0500,Fix Apache only builds for older cmake versions,add_compile_definitions is to new to run on travis. 73ad5b9dde91ee2869bbd785d70a17649fb07927,Joshua Lockerman,Mon Jan 7 13:57:10 2019 -0500,Allow module shutdown to work in ApacheOnly builds,In cases where we're going from ApacheOnly to ApacheOnly allow a nop shutdown hook. This prevents spurious errors. 2f5a40fc22169303dc7fe4de3a0848ddde72bfeb,Joshua Lockerman,Mon Jan 7 14:05:43 2019 -0500,Ignore the generated plan_gapfill tsl tests,These tests should never be committed to version control as they need to be generated based on the version of postgres installed locally. 28265dcc1f6bad1e8adfe0966b7eddc38c00b08b,Joshua Lockerman,Wed Dec 26 14:16:52 2018 -0500,Use a fixed file for the latest dev version,"When developing a feature across releases, timescaledb updates can get stuck in the wrong update script, breaking the update process. To avoid this, we introduce a new file ""latest-dev.sql"" in which all new updates should go. During a release, this file gets renamed to ""--.sql"" ensuring that all new updates are released and all updates in other branches will automatically get redirected to the next update script. " 290b726bb79b50407fc4b1c77f89d3d550ac6a66,Sven Klemm,Mon Jan 7 13:06:09 2019 +0100,Fix order in timestamp test,The order specified in some of the timestamp tests did not result in consistent ordering when a different plan was chosen. 7dc385f9ccfb8d6f26087368457f88177d26e7c9,Sven Klemm,Sat Jan 5 19:23:52 2019 +0100,Add include for compat.h to gapfill/interpolate.c,"When compiling against Postgres 9.6 before 9.6.5 TupleDescAttr is not defined, compat.h defines the macro for versions before 9.6.5 " 10c189116b922267ea861215b12b8552b4dab0c3,Rob Kiefer,Fri Jan 4 16:35:24 2019 -0500,Remove git merge/rebase lines from TSL,It seems that a merge conflict was not completely cleaned up after in the Timescale License. a5bcefb0ee47cedc0136ec45d190faaa5b694b24,Joshua Lockerman,Fri Jan 4 13:33:30 2019 -0500,Copy tsl into build directory in smoketest,We need to ensure we have all the source in order to build timescaledb. Lacking this breaks the test. f45ce14334468caa377779c6ef7f9d444f224106,Rob Kiefer,Thu Jan 3 14:11:19 2019 -0500,Remove out-dated and unused scripts,"Nothing appears to use scripts/start-test-docker.sh, so it can be safely removed. docker-run.sh was still referenced on our docs page, but should soon be removed. It no longer actually provides anything useful (i.e., any memory settings it used to have have been removed) and is out-of-date with newer Docker setups. To remove the burden of keeping it up to date, we should just remove. " 192409257e329fa826db97d1b1ce5bcf2b48350e,Sven Klemm,Thu Jan 3 13:02:55 2019 +0100,Add update test for PostgreSQL 10,Add an update test for updating the extension on PostgreSQL 10. Change PostgreSQL 9.6 update test to only run triggered by cron. 9b41f24603eadeb7b83329ee7004279abc1d2b9e,Matvey Arye,Thu Jan 3 14:58:30 2019 -0500,Fix pgtest schedule of tests,The way that ignores were used previously failed with parallel tests because it only matched the first test in the parallel list of tests. ff0225f0f7af663f79ce2681ecdaf6c612df0011,Joshua Lockerman,Thu Jan 3 15:30:04 2019 -0500,Add dummy values to test last_tuned telemetry,Adds dummy values so that last_tuned telemetry gets reported in our tests. Also moves the location of the telemetry adding code to prevent a segfault 8b99d918774055d6003a3705312c553df4c04e5b,Joshua Lockerman,Thu Jan 3 12:09:27 2019 -0500,Add telemetry reports for timescaledb.last_tuned and last_tuned_version,"Our tool timescaledb-tune records when it was last run in the postgresql.conf file, which we can use in telemetry to find out how many people are tuning their DB with that tool. It may also record which version of the tool was run, so we're adding the ability to report this now. " acc41a771209d1d6673f5eb58a2cb7203978b1ae,Joshua Lockerman,Wed Jan 2 13:49:52 2019 -0500,Update license header,Only have the copyright in the NOTICE. Hopefully only having to update one place each year will keep it consistent. 5c671d5f17c3f9a1a4e8195670c343cc65519faa,Joshua Lockerman,Wed Jan 2 12:35:39 2019 -0500,Fix casing in telemetry,"Make the license.edition field for an Apache only license ""apache_only"" to be consistent with ""enterprise"" and ""community"". Request from solar. " 6811296db7d3e089f50258a2131a374ede6c63fc,Joshua Lockerman,Wed Jan 2 14:32:51 2019 -0500,Clean windows warnings,"After this, the only remaining warnings on Windows are from postgres itself. " 888dea71b5259725fceb72bf058c3c82e62b33f8,Joshua Lockerman,Wed Dec 26 12:57:51 2018 -0500,Stop using the extra field for now and other Windows bugs,"Something is causing a heap corruption upon setting the license key to default when we try to use the guc extra on windows. For now stop using it and just rerun the validation function, if we get to the assign hook we must have a valid key, so it will never fail. Also Fixes error message on windows; turns out windows does not like to print NULL strings. Don't do that. Fixes other minor windows bugs. " 2a284fc84e0d7b15fa6c137f6c4e5c2ccc26eeea,Joshua Lockerman,Mon Dec 31 12:49:12 2018 -0500,Move 1.2.0 updates to the correct file, 0436d3b4b08160f1cfd72ab0753e94a8efb860db,Joshua Lockerman,Tue Dec 25 14:38:54 2018 -0500,Edit launcher to load TSL strictly after timescale itself,"Reloading shared libraries (which is done to start parallel workers) is done is the order in which load_file completes. If we try to load the TSL from PG_init, this causes postgres to attempt to load the TSL before timescale itself, which causes an error. Also disables the warning on normal TSL startup. " 470cb19d43021eee7c3360acca47268e1a849876,Sven Klemm,Thu Dec 27 15:28:33 2018 +0100,Fix bgw_policy test, b771322d80e7a95227b83a42389b5a715b83a1bb,Joshua Lockerman,Tue Dec 25 14:38:54 2018 -0500,Update telemetry test for new telemetry, 6125111dfa289785824dece5862408a67f825099,Sven Klemm,Fri Dec 21 19:15:54 2018 +0100,Mark gapfill functions parallel safe,Gapfill functions need to be marked parallel safe to not prevent parallelism. The gapfill node itself is still parallel restricted but child nodes can be parallel 4e413635e287355ad5fe775c965aa0e08fce7e0a,Erik Nordström,Sat Dec 22 20:55:06 2018 +0100,Make header format consistent across TSL,This change makes include guards consistently formatted across TSL code and also adds a missing include guard in `planner.h`. The include guards include `TSL` in the prefix so that they don't conflict with the ones in the same-named non-TSL header files. 4e1e15f0790b5839b39e0444379dbd44a7963fec,Joshua Lockerman,Fri Dec 21 15:36:43 2018 -0500,Add reorder command,"New cluster-like command which writes to a new index than swaps, much like is done for the data table, and only acquires exclusive locks for said swap. This trades off disk usage for lower contention: we hold locks for a much lower period of time, allowing reads to work concurrently, but we have both the old and new versions of the table existing at once, approximately doubling storage usage while reorder is running. Currently only works on chunks. " 36c145efb3bbc5759f37e4289ba86bf34c45a717,Amy Tai,Wed Dec 19 16:57:10 2018 -0500,Add enterprise checks to the 5 enterprise-only functions, 9ad73249e1255605b73a3996339f3d4687d89811,Amy Tai,Fri Dec 21 11:45:58 2018 -0500,Move enterprise updates to newest update file, 2e4bb5de2a657cb63d0be455614ebe2c89e3a4e2,Amy Tai,Thu Dec 6 14:13:46 2018 -0500,Recluster and drop chunks scheduling code,"For both recluster and drop_chunks policies, adds the scheduler code to run the appropriate function after reading the policy arguments from the appropriate internal policy table. For the recluster jobs, also picks chunks that are eligible for recluster, based on the current selection: 1) chunks at least 3rd newest chunk, 2) have not been reclustered before. The logic will try to pick the oldest such chunk. " f73d5931aacd48838d5381c76983821838f5f514,Sven Klemm,Sat Dec 15 12:21:36 2018 +0100,Fix ORS in check_file_license.sh, 87b4d959c1bf648d486037011a267dcb39be11dd,Sven Klemm,Sat Dec 15 12:25:58 2018 +0100,Fix return value of gapfill_aggref_mutator, ef43e5210785e769b1b03ed410fe6a8808cea5cb,Amy Tai,Wed Dec 12 13:32:42 2018 -0500,Add alter_policy_schedule API function, 5ba740ed98fb2873fd3add9d322811a598e9887d,Sven Klemm,Fri Dec 14 18:52:47 2018 +0100,Add gapfill query support,"This patch adds first level support for gap fill queries, including support for LOCF (last observation carried forward) and interpolation, without requiring to join against `generate_series`. This makes it easier to join timeseries with different or irregular sampling intervals. " be7c74cdf3bcf745384aa205862483eaa51a3055,Amy Tai,Tue Oct 16 16:06:06 2018 -0400,Add logic for automatic DB maintenance functions,This commit adds logic for manipulating internal metadata tables used for enabling users to schedule automatic drop_chunks and recluster policies. This commit includes: - SQL for creating policy tables and chunk stats table - Catalog code and C code for accessing these three tables programatically - Implement and expose new user API functions: add_*_policy and remove_*_policy - Stub scheduler logic for running the policies 0e5fb3dae492518cff7a80892b4bce2d04cd3795,Amy Tai,Thu Dec 13 09:56:42 2018 -0500,Fix PG11 compatibility for current tsl/ module, 1b41b1499440f775bd0267814d125234c53ec2e6,Amy Tai,Tue Dec 11 20:13:16 2018 -0500,Update licensecheck to look for correct string,"Previously, files under tsl/test/ were expected to have ""Apache"" rather than ""Timescale"" license in their license header. This has been fixed. " 568749c5ac40d4c07f27c3313991a1c99bcb6d78,Joshua Lockerman,Wed Dec 5 17:27:39 2018 -0500,Add more license key fields and telemetry,Add more license fields and useful for tracking which kinds of licenses are in use. 4ff6ac7b917a3dae8796b4a4e36001598db645d0,Joshua Lockerman,Tue Oct 16 16:06:06 2018 -0400,Initial Timescale-Licensed-Module and License-Key Implementation,"This commit adds support for dynamically loaded submodules to timescaledb as well an initial license-key implementation in the tsl subdirectory. Dynamically loaded modules allow our users to determine which licenses they wish to use for their version of timescaledb; if they wish to only use Apache-Licensed code, they do not load the Timescale-Licensed submodule. Calls from the Apache-Licensed code into the Timescale-Licensed submodule are handled via dynamicaly-set function pointers; see tsl/src/Readme.module.md for more details. This commit also adds code for license keys for the ApacheOnly, Community, and Enterprise editions. The license key determines which features are enabled, and controls loading the submodule: when a license key that requires the sub-module is installed, the module is automatically loaded. Currently the ApacheOnly and Community license-keys are hardcoded to be ""ApacheOnly"" and ""Community"" respectively. The first version of the enterprise license-key is described in tsl/src/Readme.module.md " 65a741b7ba04b829dfae1cf0c90582dc26502911,Joshua Lockerman,Mon Dec 31 12:03:13 2018 -0500,Reduce lock timeout on deadlock_dropchunkcs_select test,Detects the deadlock due to the lock timeout instead of the test runner timeout. Speeds up the test. a72fe4b92320d7b85ceefafae869763c04cac318,Sven Klemm,Sat Dec 29 15:17:18 2018 +0100,Create directory for postgres tablespace test,This is will not fix the tablespace test for remote instances but local instances will work 92586d8fc91b9bd3f4496762bbbdba0ece2cd300,Sven Klemm,Mon Dec 31 16:29:46 2018 +0100,Fix typos in comments, 43eb4ffba6c90673cc8aa9cd42ac8746c4b68fc0,Sven Klemm,Sat Dec 29 16:35:07 2018 +0100,Fix update test script,Remove EXIT trap handler because it seems to occasionally capture exit code from something that is not an update test resulting in error state even though all update tests were successful. fc425398f2820b2366e30b03d7768d2e89a5e726,Michael J. Freedman,Thu Dec 27 18:19:03 2018 -1000,Add new top-level licensing information,"Clarify better how source code is split between Apache 2.0 and Timescale License, to now serve as a reference file to which GitHub will point. Also cleanup NOTICE to point to LICENSE files. " 787cc0470e356966667ebd231f03018f9657c261,Sven Klemm,Tue Dec 25 22:31:42 2018 +0100,Replace hardcoded database name from regression tests,Replace hardcoded database name from regression tests with :TEST_DBNAME Remove creation of database single_2 from test runner and add it to bgw_launcher and loader test since no other tests used those use SQL comments in test scripts 31e9c5b5ccb8e0cc9b32004754e834cbbfeb46ef,Sven Klemm,Fri Dec 28 13:27:47 2018 +0100,Fix time column handling in get_create_command,Fixes get_create_command to produce a valid create_hypertable call when the column name is a keyword Add test.execute_sql helper function to test support functions 910d08f2841346f8be307791f1f2a007e39cebf6,Sven Klemm,Fri Dec 28 13:51:27 2018 +0100,Fix typo in pginstallcheck output filename, f043aa68717499173e7d4401be596a539f36b192,Sven Klemm,Thu Dec 27 11:20:00 2018 +0100,Execute codecov for successful builds only,Execute codecov for successful builds only Don't run codecov for pgindent and license check 8c6ab3b0398fca106f87044abd46c0ce5f7a538b,gayyappan,Thu Dec 20 18:55:34 2018 -0500,Fix for deadlock between select and drop_chunks,Acquire foreign key related table locks prior to dropping chunks. This helps avoid deadlocks in scenarios where select and drop chunks are trying to acquire locks on the same set of tables. Changes to isolation test suite to prevent diffs related to displaying hypertable id differences across test suite runs. Fixes #865 5e4bcffa1186320f3ad328571470ea55c5eb8ba4,Joshua Lockerman,Mon Dec 24 13:08:30 2018 -0500,Add .codecov.yml and ignore test/src,We don't care about the test coverage of our test infrastrucutre ac73406973abe56aac95561b0f615f4d6dbd2d09,Joshua Lockerman,Mon Dec 24 12:53:09 2018 -0500,Add telemetry report test,"We should really test that this function works, and we're outputting all the fields we expect. Not testing the actual values as that changes across installs and runs. Should also improve our code coverage. " 8a5607688f85a615ef1b880d0b360a9ae4aa74f1,Sven Klemm,Sat Dec 22 11:18:41 2018 +0100,Improve CI settings,run pginstallcheck with parallel schedule don't stop on dependency error for installcheck don't build postgres with --enable-coverage 44362b9227614d2ead8687379c9e474981a77828,Joshua Lockerman,Thu Dec 20 11:18:08 2018 -0500,Update testrunner to only start if there are tests to run,"Exactly like it says, for instance it you run with TESTS=append the isolation test runner won't even start. " 73b81bb89292c9beb7ab85550b861f06cf96a3c2,Erik Nordström,Thu Dec 20 12:56:32 2018 +0100,Do not exit immediately on failure in update test script,"This fixes a number of issues with the update test script that prohibited it from displaying a log of any failing tests. In particular, the script needs to be run with `set +e` to ensure a test error doesn't immediately cause an exit without recording the failed log. " 53f1578dd8afb6d3b2831e6c0304e9b00f3d504f,Mike Freedman,Thu Dec 20 13:00:10 2018 -0500,Update README to clarify relationship of Apache vs. TSL code,"Clarify that code isn't dual licensed, but source code is either Apache-2 or Timescale-Licensed, and you can build an Apache-2 or Timescale-Licensed binary. Addresses question raised in https://github.com/timescale/timescaledb/issues/922 " a4e79d43c68508cad24d7f632882b80fe0643960,Joshua Lockerman,Thu Dec 20 12:16:17 2018 -0500,"Release 1.1.1 on master, prepare for 1.2.0-dev", bb01a73f5176f1663cfdb48af986f5a1dcd84432,Rob Kiefer,Wed Dec 19 12:08:27 2018 -0500,Update README with pointers to tools and fixes,"The installation links were a bit out-of-date given our new page structure. Also, our tools were not mentioned anywhere, so now the README has pointers to them at the bottom, and in particular a brief section to point to timescaledb-tune. " 1b8cecade3eb9ce821b594ae722201c098eb57c3,David Kohn,Tue Dec 18 11:33:13 2018 -0500,Avoid loading twice in parallel workers and load only from $libdir,"We removed a parallel workers check a while back because there were problems with the macro on Windows, we define a new macro that works on Windows as we received reports of some instances in which the Timescale library appeared to be loading twice in parallel workers. Parallel workers instead restore the library state of their parent process via their normal pathway and the loader will take no action. Additionally, prepend ""$libdir/"" to all calls to .so's so that we don't end up accidentally loading from a place different from where we've loaded when we create functions, which all refer to $libdir. This would only affect users who have multiple separate .so's in different directories, but could be another case where multiple .so's were loaded. " 76d78753fdfeab455133b91dd4ec14a8663b6ee3,David Kohn,Wed Dec 19 19:54:14 2018 -0500,Don't throw errors when extension is loaded but not installed yet,"Pg_upgrade was failing in extension_version() as it tries to check that libraries that have functions associated with them are available before copying data from the old cluster over to the new. (Therefore the extension does not yet exist as the data has not been copied and the check for extension_version fails, but it should not be called if the extension does not yet exist). " ff0ccfcef399db24df3da3648628e1207423e81b,didier,Mon Nov 26 18:00:31 2018 +0100,Ensure deterministic results in bw scheduler test, 0e1849d65ba2820a7a8379c28c2b7490c36e824b,dianasaur323,Wed Dec 19 14:01:20 2018 -0500,Update README with new license, eecd845781fcc4163d1f9e81d18f116b77a561f8,Michael J. Freedman,Wed Dec 19 12:38:22 2018 -0500,Add Timescale License (TSL), d2edf3a1f592367e5c4e86dceea402bb8f39c83f,Erik Nordström,Thu Dec 13 11:08:05 2018 +0100,Use create_upper_paths_hook for hypertable insert plan modifications,"This change refactors how we do planner modifications to insert data into a hypertable. Instead of altering the finished plan tree produced by the standard planner, we now use the create_upper_paths_hook to do these modifications at the path stage during planning. Apart from being a cleaner, and arguably more principled, way of doing plan modifications, it also gives us better ability to modify plan state during planning since the hook happens during the normal planner run and not after. For instance, our CustomScan insert plan is now created using a callback that has access to the PlannerInfo data, which will be useful when planning, e.g., foreign queries using the foreign data wrapper API. " b1378449bcf7f6ff97e28c18506781ff4f2604bb,Sven Klemm,Sat Dec 15 11:56:35 2018 +0100,Remove unused functions,Remove the following unused functions: ts_cache_switch_to_memory_context ts_chunk_free ts_chunk_exists ts_chunk_index_delete_children_of ts_chunk_index_delete_by_hypertable_id ts_hypertable_scan_relid ts_tablespaces_clear ts_tablespaces_delete 3de7d4bfbf0cf1c3306d28b6f359623997467b1e,Michael J. Freedman,Fri Dec 14 08:01:48 2018 -0500,Add PostgreSQL License information to NOTICE and selected files, a0cc8c062f2024fc0bcb3e73353a027f2c99f3b3,Narek Galstyan,Sun Dec 16 18:49:09 2018 +0000,Fix string conversion format in timer_mock, 4b42b301a7125573807131542c91ce89838851fe,Joshua Lockerman,Fri Dec 14 11:07:15 2018 -0500,Free ChunkInsertStates when the es_per_tuple_exprcontext is freed,"Instead of freeing ChunkInsertStates immediately when flushed from the SubspaceStore, defer the freeing until the es_per_tuple_exprcontext is being cleaned up. This ensures that pointers into the CIS don't become invalidated without us freeing any of postgres's memory, like we were doing in the prior fix; this fixes a segfault in COPY. In the normal INSERT case these frees happen at approximately the same time so there should be no regression in memory usage, while if there was a delay in freeing the es_per_tuple_exprcontext our old strategy of clearing the context ourselves was invalid. " 9cff0495b564694bd1ed3f2240c822e1c5c1f6b3,Joshua Lockerman,Thu Dec 13 15:58:03 2018 -0500,Fix accidental usage of c99 for loop, 43e20213002a648928bfdf82d4e6b006318fcb97,Sven Klemm,Tue Dec 11 11:36:10 2018 +0100,Set C standard to C11, d4f0cbd0d3879afe01bab9a627a6270e0c01d94d,Joshua Lockerman,Thu Dec 13 15:41:15 2018 -0500,Prepare the repo for the 1.2.0 development cycle, 1295ac320015b761aa4481c0703405b5694cbf94,Joshua Lockerman,Thu Dec 13 15:09:50 2018 -0500,Release 1.1.0,"PG11! Otherwise, mainly performance improvements and bugfixes. " a7b54b159bc06352d67f6244fc2b7a69b3032d85,Joshua Lockerman,Wed Dec 12 14:41:14 2018 -0500,Have prefix check verify all functions in headers and clean up missings,"This commit makes four changes: 1. Adds the ts_ prefix to some functions that were missed in the last pass 2. Adds static to some variables that were confined to a single file but missing it 3. Adds the ability to disable using ""hidden"" visibility by default by setting the USE_DEFAULT_VISIBILITY variable 4. Switches the prefix-check in travis to use the flag defined in 3 so that the checker now checks all non-static symbols we define " c59a30feedfd336a9b3b09e83a2d3e262825751d,Sven Klemm,Wed Dec 12 12:28:24 2018 +0100,Remove unused functions from utils.c,Remove the following unused functions: _timescaledb_internal.to_microseconds(TIMESTAMPTZ) _timescaledb_internal.to_timestamp_pg(BIGINT) _timescaledb_internal.time_to_internal(anyelement) 98615f4ae06a383fabf8ea1753fb279aafdd14a9,Sven Klemm,Wed Dec 12 19:11:57 2018 +0100,Remove compat.c,The only function in compat.c was oid_cmp which had no users. 981147fd8bfd5d76b70da0d79a10713f4df5605c,Sven Klemm,Wed Dec 12 19:14:59 2018 +0100,Fall back to commit hash in update tests,"Travis CI occasionally has trouble figuring out the branch, so this change makes sure it falls back to using at least a commit hash " 5339531c26ebc86600169cac1a8fa834513f6fcb,Sven Klemm,Wed Dec 12 10:21:29 2018 +0100,Remove unused functions from trigger.c, cb0fcf15a0e0a001ca9c07ac216b04c2f12227bb,Narek Galstyan,Fri Dec 7 19:46:16 2018 +0000,Run custom_type tests only on 64-bit architectures, 71f3a0cadcef55cbc1779dfd61962f837a3b56ae,Narek Galstyan,Sat Nov 24 20:49:19 2018 +0000,Fix Datum conversion issues, 5aa1edac157221196ef6ebfa332bee1edb42b7ed,David Kohn,Fri Sep 28 13:30:04 2018 -0400,Refactor compatibility functions and code to support PG11,"Introduce PG11 support by introducing compatibility functions for any whose signatures have changed in PG11. Additionally, refactor the structure of the compatibility functions found in compat.h by breaking them out by function (or small set of similar functions) so that it is easier to see what changed between versions and maintain changes as more versions are supported. In general, the philosophy has been to try for forward compatibility wherever possible, so that we use the latest versions of function interfaces where we can or where reasonably convenient and mimic the behavior in older versions as much as possible. " e4a4f8e2f8e342843b72f838f0981b42dc577f24,Erik Nordström,Thu Aug 2 10:40:01 2018 +0200,Add support for functions on open (time) dimensions,"TimescaleDB has always supported functions on closed (space) dimension, i.e., for hash partitioning. However, functions have not been supported on open (time) dimensions, instead requiring columns to have a supported time type (e.g, integer or timestamp). This restricts the tables that can be time partitioned. Tables with custom ""time"" types, which can be transformed by a function expression into a supported time type, are not supported. This change generalizes partitioning so that both open and closed dimensions can have an associated partitioning function that calculates a dimensional value. Fortunately, since we already support functions on closed dimensions, the changes necessary to support this on any dimension are minimal. Thus, open dimensions now support an (optional) partitioning function that transforms the input type to a supported time type (e.g., integer or timestamp type). Any indexes on such dimensional columns become expression indexes. Tests have been added for chunk expansion and the hashagg and sort transform optimizations on tables that are using a time partitioning function. Currently, not all of these optimizations are well supported, but this could potentially be fixed in the future. " bb6efbef0d54bf09cd34de4290f7be40be09ae08,Joshua Lockerman,Fri Dec 7 12:59:26 2018 -0500,Check that functions start with the ts_ prefix in travis, 116beff610302fb6ce8b1d7081e0dc184d453d4b,Mike Freedman,Sat Dec 1 00:40:52 2018 -0500,Fix header formatting in markdown subtitle, ed5067c356422f74062b079716a7078092aa78ac,Sven Klemm,Mon Dec 10 17:59:01 2018 +0100,Fix interval_from_now_to_internal timestamptz handling,fix interval_from_now_to_internal to handle timezone properly for timestamptz and simplify code 423d1c691e459daa0dff51c7cc59d197d70e73f3,Joshua Lockerman,Mon Dec 10 16:13:04 2018 -0500,Have AppVeyor ignore the bgw_db_scheduler test for now, 9b52909b173c926c2aa1b57a60e062a419370db4,Joshua Lockerman,Mon Dec 10 16:10:56 2018 -0500,Add the ability to ignore tests from the command line using IGNORES, 8da8ac3d431ab74e9a1855985927f40c6a5d2fb8,Joshua Lockerman,Tue Dec 4 14:38:14 2018 -0500,"Release 1.0.1 on master, prepare for 1.1.0-dev", 019971c402b8947d687024e6393df53c5bbfd8a1,niksa,Thu Nov 8 16:48:19 2018 +0100,Optimize FIRST/LAST aggregate functions,"If possible replace aggregate functions FIRST/LAST with subqueries of the form (SELECT value FROM table WHERE sort IS NOT NULL AND existing-quals ORDER BY sort ASC/DESC LIMIT 1). Given a suitable index on sort column, this plan can be much faster then scanning all the rows and running an aggregate function. The optimization can't be performed if: - query uses GROUP BY or WINDOW function - query contains CTEs - query contains other aggregate functions (eg. Combining MIN/MAX with FIRST/LAST. We can't optimize accross different aggregate functions) - query uses JOIN - FIRST/LAST used in ORDER BY Optimization also works with subqueries, or if FIRST/LAST is used in CTE subquery. In order to standardize existing FIRST/LAST aggregate function with PostgreSQL and FIRST/LAST optimization, we exclude NULL values in sort by column. " 74eb04596b815379d33db62ef15ff289589b01b1,niksa,Thu Dec 6 17:49:05 2018 +0100,Exclude paths where folder/name starts with '.'. Previous regex would exclude everything if any of the folders in your path contains a dot (eg. /github.com/, 83014ee2b0f39d76d6ca3cdcfaabecbbb86f1521,Amy Tai,Wed Nov 28 15:10:40 2018 -0500,Implement drop_chunks in C,"Remove the existing PLPGSQL function that implements drop_chunks, replacing it with a direct call to the C function, which also implements the old PLPGSQL checks in C. Refactor out much of the code shared between the C implementations of show_chunks and drop_chunks. " 62571e2a56643e12350f97964b8f778c0bef5d63,didier,Wed Dec 5 12:04:02 2018 +0100,Plug a trivial memory leak, 701e921f05339fd0d61907cbbd00a4025ec65b56,didier,Wed Dec 5 09:20:14 2018 +0100,Only set soname if it's used,inval_cache_callback call recursively do_load but only one has loaded == false after calling extension_version(). 9de504f958ac2a9b45db9f897c9fe02dd4310bc8,Joshua Lockerman,Tue Dec 4 15:32:22 2018 -0500,Add ts_ prefix to everything in headers,"Future proofing: if we ever want to make our functions available to others they’d need to be prefixed to prevent name collisions. In order to avoid having some functions with the ts_ prefix and others without, we’re adding the prefix to all non-static functions now. " 54b189a7e40d28ee388830319c03baaaba238a11,Amy Tai,Fri Nov 30 15:36:29 2018 -0500,Remove unused function from hypertable.c,hypertable_delete_by_id has long since been replaced by hypertable_delete_name and isn't used anywhere in the codebase. b7f32f2a7962bac1ce9b99abc9f60643528e1a40,Amy Tai,Sun Dec 2 12:31:57 2018 -0500,Make views test more deterministic, bf15cfec24243763edbf67ef406264f0cd3e4474,Amy Tai,Sat Dec 1 11:10:02 2018 -0500,Move BGW job tester functions to test module,"Previously, delete/insert_job were exposed, even though they were only used for testing. Moved both the C implementations and the SQL functions to the test module. " b9b439fde4b232220d4cf37bf5083844d617ca76,Sven Klemm,Fri Nov 30 12:04:40 2018 +0100,Remove unused functions from utils.c,"Remove int_cmp, create_fmgr and makeRangeVarFromRelid from utils.c since they were not used and had no test coverage. " 80fffc94527addb9270c87679ca69be7992c2e5f,Sven Klemm,Fri Nov 30 09:08:49 2018 +0100,Refactor ChunkDispatch to remove ChunkDispatchInfo, 36039c855788aadf7ace8d64d7f10c9ad5495ebe,Sven Klemm,Fri Nov 30 14:45:29 2018 +0100,Remove EXEC_BACKEND tests from travis, 9a3402809f05e5ac9830797106005dcd127d998e,Narek Galstyan,Thu Jul 26 17:20:02 2018 -0400,Implement show_chunks in C and have drop_chunks use it,"Timescale provides an efficient and easy to use api to drop individual chunks from timescale database through drop_chunks. This PR builds on that functionality and through a new show_chunks function gives the opportunity to see the chunks that would be dropped if drop_chunks was run. Additionally, it adds a newer_than option to drop_chunks (also supported by show_chunks) that allows to see/drop chunks in an interval or newer than a point in time. This commit includes: - Implementation of show_chunks in C - Additional helper functions to work with chunks - New version of drop_chunks in sql that uses show_chunks. This also adds a newer_than option to drop_chunks - More enhanced tests of drop_chunks and new tests for show_chunks Among other reasons, show_chunks was implemented in C in order to be able to have both older_than and newer_than arguments be null. This was not possible in SQL because the arguments had to have polymorphic types and whether they are used in function body or not, PL/pgSQL requires these arguments to typecheck. " d4619598a9aa065c6fb932a778742cbd97305b2c,Sven Klemm,Tue Nov 20 21:30:06 2018 +0100,Add view to show hypertable information,"Add timescaledb_information.hypertable view that lists hypertables, their owner, number of chunks and storage size " 0267f46b5650085cd9688ce1d0eab1317aa1413e,Amy Tai,Mon Nov 19 12:38:18 2018 -0500,Clean up catalog code,"Organize catalog.c so that functions that access the Catalog struct are physically separate from the functions that modify the actual catalog tables on disk. Also added macros and made static some functions that aren't used outside the catalog files. Also refactored out the CatalogDatabaseInfo struct, which is independent of the Catalog struct and will be reused in the future. " 35dee48e9fe1242c6d1c69ea3d89dd221948a79a,Amy Tai,Sat Nov 17 10:40:50 2018 -0500,Remove version-checking from client-side,"Because the server now sends a boolean indicating whether the local version is up-to-date, we simply validate the server's response and remove any version-checking code from the client-side. The client now only checks for well-formed versions before printing, if the local version is not up-to-date. " 3686ff5b813fa10538e77bede3e74b2105730fb6,Sven Klemm,Mon Nov 26 14:00:16 2018 +0100,Set ORS to empty string instead of NUL in license check script,This fixes multiple warnings in bash 4.4: scripts/check_file_license_apache.sh: line 22: warning: command substitution: ignored null byte in input and is also more correct since we dont want to inject 0-Bytes into the string 46d3479757340693b79f72107d06e7319ec5985d,Sven Klemm,Fri Nov 23 14:25:31 2018 +0100,Add tests for calling functions with NULL arguments, 5b6a5f45118472128c111eb6e56f0845aa1beb07,Sven Klemm,Fri Nov 23 12:10:51 2018 +0100,Change size utility and job functions to STRICT,"Change hypertable_relation_size, hypertable_relation_size_pretty, chunk_relation_size, chunk_relation_size_pretty, indexes_relation_size, indexes_relation_size_pretty, partitioning_column_to_pretty, insert_job and delete_job to STRICT " 7e55d910eb0bcd90a4756d5e33cd7b6c97fc0d07,Sven Klemm,Fri Nov 23 14:22:05 2018 +0100,Add checks for NULL arguments to DDL functions,"Add checks for NULL arguments to ts_chunk_adaptive_set, ts_dimension_set_num_slices, ts_dimension_set_interval, ts_dimension_add and ts_hypertable_create " 80e0b05348517ae534cabdc73e76835471195554,Amy Tai,Wed Nov 7 12:35:17 2018 -0500,Provide helper function creating struct from tuple,Refactored the boilerplate that allocates and copies over data from a tuple to a struct. This is typically used in the scanner context in order to read rows from a SQL table in C. d82cc2f49a25595a9f34765377481159b0cb7efa,didier,Fri Nov 9 16:22:15 2018 +0100,set client_min_messages to 'error' not 'fatal',after postgres commit c09daa9104099422ea998e0398934ca82eb37898 'fatal' is either not valid or replaced by 'error'. 13c68e0bb8b5e3ed65e72b48d27d21b313dae40b,Matvey Arye,Tue Nov 20 15:55:28 2018 -0500,Backpatch TupleDescAttr for 9.6.3 and 9.6.4, c1db6086641cf9367a1763378cd00fc614bb5c49,Matvey Arye,Fri Nov 16 14:36:10 2018 -0500,Fix upsert TLE translation when mapping variable numbers,"Previously, there was a bug when chunks contained dropped columns in the mapping of variable numbers. This PR fixes the bug and adds a test for this case. " f3c0aa718aab18468ce152ebb849b3cdd0082e34,Joshua Lockerman,Wed Nov 14 13:50:35 2018 -0500,Add appveyor.yml,After this commit AppVeyor will run all TimescaleDB tests on windows d1c9cfc163f8335f7b8913ef5faff1d682e96608,Joshua Lockerman,Mon Nov 19 13:43:10 2018 -0500,allow loader tests to run on windows, 1987b8f0666063dc40905a200b545f6ab8045483,Joshua Lockerman,Wed Nov 14 12:13:52 2018 -0500,Get the hostname of the server from env instead of hardcoding localhost,fixes regresschecklocal for remote server 9a2c6a50a96e84a30be5b351edeec1deb1e13462,Sven Klemm,Mon Nov 19 09:42:36 2018 +0100,Use PG_INTXX_MAX macros instead of INTXX_MAX, 327fa906f0c9672c12a025096b8171d31f1e51a6,Joshua Lockerman,Tue Nov 13 12:27:39 2018 -0500,Fix version_parse on windows,"snprint has different behavior on some windows versions than on unix. This commit replaces the snprintf version with one that uses sscanf, which behaves the same on all platforms. (This is a higher priority commit than it would appear, since we're tracking down a stack-overflow, and this is one of the candidates f " e06733acf012baeff0f2f4b2183121c1ea28de12,Joshua Lockerman,Thu Nov 15 14:42:06 2018 -0500,Fix casing in SQL license header to be consistent with elsewhere, 55a378ef7a932237b6d9174e3b04c30fb37066ae,Sven Klemm,Mon Nov 5 14:21:44 2018 +0100,Check extension exists for DROP OWNED and DROP EXTENSION,In a database without timescaledb installed drop owned failed with an error while trying to lookup the extension owner. 0c8c085cda7cdecb3e26c637195c8b8f95eca236,niksa,Fri Nov 9 11:59:18 2018 +0100,Exclude unneeded chunks for IN/ANY/ALL operators,"If IN/ANY/ALL operator is used with explicit values we can effectively restrict chunks that need to be scanned. Here are some examples of supported queries: - SELECT * FROM hyper_with_space_dim WHERE time < 10 AND device_id IN ('dev5','dev6','dev7','dev8'); - SELECT * FROM hyper_with_space_dim WHERE device_id = ANY(ARRAY['dev5','dev6']) AND device_id = ANY(ARRAY['dev6','dev7']); There are som cases that are not optimized: - subqueries within IN/ANY/ALL - open dimension (eg. time) when using IN/ANY with multiple args - NOT operator " f27c0a3f28302743dd9c92dd402deccb9dc8f4c7,Sven Klemm,Sat Nov 3 13:27:26 2018 +0100,Move int time_bucket functions with offset to C, 92d4dbde9ce8b5374f56986416a60d2e78745de9,Amy Tai,Thu Nov 8 16:00:26 2018 -0500,Change to heap allocation for runtime array size,"Surprisingly, the compiler didn't complain about allocating a stack array with a size only known at runtime. Modified memory allocation to be heap-allocation to remedy this error. " 8b793f3433af7b3cd2e0cb35d86217a4edad8e71,Amy Tai,Thu Nov 8 14:47:59 2018 -0500,Ensure deterministic results in cluster test,Adding ORDER BY to some select statements in cluster.sql test to ensure determinism. 28e2e6a2f747c545e17e718e3e23946f0c4911f1,Erik Nordström,Fri Sep 28 10:42:36 2018 +0200,Refactor scanner callback interface,"This change adds proper result types for the scanner's filter and tuple handling callbacks. Previously, these callbacks were supposed to return bool, which was hard to interpret. For instance, for the tuple handler callback, true meant continue processing the next tuple while false meant finish the scan. However, this wasn't always clear. Having proper return types also makes it easier to see from a function's signature that it is a scanner callback handler, rather than some other function that can be called directly. " b77b47ada88bacddac3ae4ec5620fd23a9c75025,Amy Tai,Sun Oct 21 22:03:36 2018 -0400,Enable scheduler to update its jobs list,"Previously, the scheduler only populated its jobs list once at start time. This commit enables the scheduler to receive notifications for updates (insert, update, delete) to the bgw_job table. Notifications are sent via the cache invalidation framework. Whenever the scheduler receives a notification, it re-reads the bgw_job table. For each job currently in the bgw_job table, it either instantiates new scheduler state for the job or copies over any existing scheduler state, for persisting jobs. For jobs that have disappeared from the bgw_job table, the scheduler deletes any local state it has. Note that any updates to the bgw_job table must now go through the C, so that the cache invalidation framework in catalog.c can run. In particular, this commit includes a rudimentary API for interacting with the bgw_job table, for testing purposes. This API will be rewritten in the future. " e5e8e2aac42afb64a6c4ccd012bb3084d6fff4e7,Joshua Lockerman,Wed Oct 31 16:20:43 2018 -0400,Add documentation to loader/loader.c, cac0791a5cbe5e39034204a3751c3209f012c3b8,Rob Kiefer,Tue Oct 30 18:40:23 2018 -0400,Prepare the repo for next development cycle, 1dc3aa9161f15f254934583674ac917c55292d7a,Rob Kiefer,Mon Oct 29 17:36:35 2018 -0400,Release 1.0.0, c0d9afb7fae9fcb5c9ea767b9975d598be603a10,RobAtticus,Mon Oct 29 17:04:57 2018 -0400,Add bug report issue template, 319933d14e5b7ced49d10adb7b3250453e248136,Matvey Arye,Mon Oct 29 18:01:52 2018 -0400,Fix incorrect function call,Using DatumGetTimestampTz is requires on some 32-bit platforms under PG 9.6. 939acc29111ceada97de1fd3f37ab10c919d8b8b,Joshua Lockerman,Mon Oct 29 12:42:49 2018 -0400,"Perform license/formatting tests first, because they're fastest", d4f2a37e4b57183ab0796535569f0536bb1186f8,Joshua Lockerman,Mon Oct 15 17:18:19 2018 -0400,Add automated license check to travis, eaa796eeba23b9cb15f201e77d714f23b54f5917,Joshua Lockerman,Mon Oct 22 17:08:35 2018 -0400,Add license headers to the more interesting scripts, 20ec6914c073a51d6e866d2c2142b7ca898b518d,Joshua Lockerman,Mon Oct 22 13:16:04 2018 -0400,Add license headers to SQL files and test code, d8e41ddabadd9b46b2b50d91eb3db442d31bfb61,Joshua Lockerman,Mon Oct 15 15:53:50 2018 -0400,Add Apache License header to all C files, 546152d806ca729547ffd35b014eb81673567d34,Joshua Lockerman,Mon Oct 15 15:46:37 2018 -0400,Rename LICENSE to LICENSE-APACHE to be more specific, 08bac400210598601c42f043225e59bc32dc1a53,Joshua Lockerman,Fri Oct 26 14:15:31 2018 -0400,Add beta NOTICE when using adaptive chunking,"Adaptive chunking is in beta, and we don't want users enabling it in production by accident. This commit adds a warning to that effect. " b7a4851cd0d029edc85da12959089bb81f6ee86a,Erik Nordström,Tue Oct 2 20:38:38 2018 +0200,Run update tests in parallel,This change makes the update tests from each prior version of TimescaleDB to run in parallel instead of in serial order. This speeds up the testing significantly. 7fb6fbacdd758588a085f110ceeb4bd4818cb0e5,Erik Nordström,Tue Oct 2 11:41:43 2018 +0200,Refactor docker-build.sh to reuse images,"The docker-build.sh script, which is used to build a Docker image from the currently checked-out source code, has been refactored to make better use of cached images. This speeds up the build process and allows faster Travis builds. " 5e51735e4fcb90824c8790b89f474eb621fd08ca,Sven Klemm,Tue Oct 23 13:24:11 2018 +0200,"Test get_git_commit, get_os_info and get_version returned types", b2130f8039222571fc3267192d16fff3e4456ac8,Erik Nordström,Mon Oct 22 16:32:45 2018 +0200,Move all time_bucket funtions to same source file,This change moves all time_bucket-related functions to the same source file (time_bucket.c) for consistency. There are no changes to code logic. 2a8cdafb3289be30f1592654bea925dcea9b25d3,Rob Kiefer,Thu Oct 18 16:03:20 2018 -0400,Prepare the repo for next development cycle, e9f69edce07a38a77971ed4d573522744c9a3c85,Rob Kiefer,Thu Oct 18 14:48:34 2018 -0400,Release 1.0.0-rc3, f7ba13d698608554b75d53e18ace647f9ab2171a,Narek Galstyan,Sat Oct 13 10:46:25 2018 -0400,Handle and test tablespace changes to and from the default tablespace,"This commit tests ALTER TABLE ... SET TABLESPACE, attach_tablespace, detach_tablespace behaviour when pg_default is the argument tablespace. It also makes sure that a table can always be moved to the default tablespace of its database. Previously, this was blocked whenever The user owning the table did not own the tablespace. " 9ccda0df0074d1d7cb2b7a18afa4996e14efc9a9,David Kohn,Wed Oct 17 13:39:45 2018 -0400,Start stopped workers on restart message,"Modify the restart action to start schedulers if they do not exist, this fixes a potential race condition where a scheduler could be started for a given database, but before it has shut down (because the extension does not exist) a create extension command is run, the start action then would not change the state of the worker but it would be waiting on the wrong vxid, so not see that the extension exists. This also makes it so the start action can be truly idempotent and not set the vxid on its startup, thereby respecting any restart action that has taken place before and better defining how each interacts with the system. Additionally, we decided that the previous behavior in which launchers were not started up on, say, alter extension update actions was not all that desirable as it worked if the stop action had happened, but the database had not restarted, if the database restarted, then the stop action would have no effect. We decided that if we desire the ability to disable schedulers for a particular database, we will implement it in the future as a standalone feature that takes effect across server restarts rather than having somewhat ill-defined behavior with an implicit feature of the stop action. " 3e3bb0c796c08cedf034e880c5d362ac41cb2c58,Sven Klemm,Wed Oct 17 16:46:01 2018 +0200,Add bool created to create_hypertable and add_dimension return value,"Add bool created to return value of create_hypertable and add_dimension. When if_not_exists is true and creation is skipped because the object already exists created will be false, otherwise it will be true. This modifies the functions to return meta data even when no object was created. " 45278ff2d2ae7e5e134d446ce4e280288f5999b2,Sven Klemm,Tue Oct 16 09:58:46 2018 +0200,Change case of timescaledb restoring to be consistent with docs, 42e56646e3844ad32601d60ffcd0f13c5e06a1ec,David Kohn,Mon Oct 15 15:24:39 2018 -0400,Fix pg_dump test so that it doesn't rely on number of functions,"The pg_dump test breaks if someone forgets to update the test when adding a new function. Given that that is not what we are really testing for in this test, it makes sense to instead save the number of dependent objects in a psql variable and test against it later in the script as needed. " 53ff6567eff880a842958e34d5769a821bd2e686,Matvey Arye,Wed Oct 10 13:06:43 2018 -0400,Add state machine and polling to launcher,"This PR changes the launcher to use a state machine to keep track of the state of each database scheduler. Further, it add polling to go through the list of databases and check their states. This solves several issues 1) A CREATE DATABASE call using a template that already has TimescaleDB installed previously did not start a scheduler until the next database restart. A test for this case has been added. 2) A lack of available slots or background workers when a new database was added meant that the scheduler would not be stared until the next database restart. Now this will be retried on every polling event. This PR also simplifies logic since database entries are never removed from the hash table and thus never added more than once. State transitions are now easier to read and reason about. Documentations for the state transitions has been added. " d9b2dfed6b1ffc2e85afce9ac30b1b7865a9153f,Sven Klemm,Sat Oct 13 10:29:55 2018 +0200,Change return value of add_dimension to TABLE,"Change the return value of add_dimension to return a record consisting of dimension_id, schema_name, table_name, column_name. This improves user feedback about success of the operation but also gives the function an API returning useful information for non-human consumption. " 19299cf34953a33d091722681761e512a15a3a76,Matvey Arye,Tue Oct 9 16:43:40 2018 -0400,Make all time_bucket function STRICT,All time bucket function should return NULL on any NULL parameters. debd91478aac69a35dc91ae36fa2981e3d51821e,Matvey Arye,Tue Oct 9 15:11:40 2018 -0400,Move to using macro for time_bucket_ts,Macro is used for 2 reasons: 1) It's more correct in that it doesn't mix Timestamp and TimestampTz types. There is no implicit conversion of the two beneath the hood. 2) It is slightly faster as it avoid an extra function call. This is a very performance sensitive function for OLAP queries. 297d88551b8b2244d104a3c4aa4aac4050a1df98,Matvey Arye,Mon Oct 8 17:34:33 2018 -0400,Add a version of time_bucket that takes an origin,This allows people to explicitly specify the origin point. e74be3092556ef5115d89f7d701700f89f40a998,Matvey Arye,Fri Sep 28 09:24:15 2018 -0400,Move time_bucket epoch to a Monday,"Since Monday is the ISO start of the week, it makes sense to move the time_bucket epoch to start on a Monday. Before the epoch was the same as the Postgres epoch (2000-01-01, a Saturday). " ba88f481cf3041945ed9b0d68f25da60a88b30a2,jesperpedersen,Wed Oct 10 12:14:38 2018 -0400,Highlight that PG_CONFIG can be used to specify the path to the PostgreSQL installation., 46564c1e06ea66fa8a2169edba3ef10caf5a266b,Amy Tai,Wed Oct 10 15:29:10 2018 -0400,Handle ALTER SCHEMA RENAME properly,"Previously, we did not intercept the ALTER SCHEMA [name] RENAME command, which meant all Timescale catalog tables that store a schema name were not getting updated properly. This caused problems when users tried to drop a hypertable in a renamed schema, but also would have also caused problems with other commands (which we now add tests for in this PR). " 4f815ddc6af5df11951fc2bdad026f1f12133b50,didier,Fri Oct 12 18:39:06 2018 +0200,Search for pg_isolation_regress in pg_config bin directory, f18e0405d3641ab56e6682afc04b499ec63fb177,jesperpedersen,Wed Oct 10 11:34:32 2018 -0400,Remove invalid assert in process_altertable_set_tablespace_end, a83e2838c9b0f8345387fc1bdd8dd05d354a191e,Sven Klemm,Wed Oct 10 23:37:19 2018 +0200,Change return value of create_hypertable to TABLE,"Change create_hypertable to return a record consisting of (hypertable_id, schema_name, table_name). This improves user feedback about success of the operation but also gives the function an API returning useful information for non-human consumption. " b46ec2ef1ff625e67a660c073fb082cf13a69184,Rob Kiefer,Thu Oct 11 14:22:05 2018 -0400,Update code comments in adaptive chunking; mark as beta,"We want it to be clear that the adaptive chunking feature is currently in beta, so a notice at the top of the C file should suffice even if the user does not see the docs. Also while I was there I commented some of the functions without comments. " 6efaffd21d488eef126cd7040011e49aedfaafd3,jesperpedersen,Wed Oct 10 12:34:02 2018 -0400,Only install the mock functions for debug builds, 5883d0d8333e9191ac206c548657574350d3b678,Sven Klemm,Tue Oct 9 20:46:36 2018 +0200,Check GRANTs in update tests, aea7c7edaca7f5443e497dfac2fddf25d973dd07,Sven Klemm,Tue Oct 9 21:26:41 2018 +0200,Add GRANTs to update script for pg_dump to work, 119963a33408641869f29641776e066f81346693,Sven Klemm,Wed Oct 10 17:29:48 2018 +0200,Replace hardcoded bash path in shell scripts, 4464b3674acac7f206006a45498dccbb6ef076b8,Amy Tai,Tue Oct 2 13:42:23 2018 -0400,Turn off telemetry in Docker images used in CI,"Using the new flag available in Docker images, turn off telemetry in Timescale-built Docker images used by Travis. We do not pass the flag to Postgres Docker images. " e11608a18402fa6373832e13fa133c1d850d060e,Sven Klemm,Sun Oct 7 11:15:10 2018 +0200,Fix typos in comments, cd51aa170cb043dab6d6127d1505a62990cb570e,Erik Nordström,Tue Oct 2 09:23:02 2018 +0200,Add Travis test for telemetry without OpenSSL,"This change makes Travis compile the extension without OpenSSL support (in addition to compiling with OpenSSL support). We also run only the telemetry test, which is the only test so far that is affected by SSL. This ensures that we can compile without OpenSSL and that tests pass. " 65817c8cc53d0f1834cf86db507959c681db52bf,Erik Nordström,Mon Oct 1 15:24:07 2018 +0200,Fix telemetry test when compiled without SSL support,The telemetry test failed with a missing SSL support error when there was no SSL support compiled into the extension. This change transparently uses plain connections in case of no SSL support so that the test doesn't fail with an SSL-specific error. A minor bug is also fixed in `plain_connect()`. This bug caused the function to return success in case of name resolution errors. This was in most cases harmless since the subsequent connect would fail in any case. bff0d4e405b26f029c521497efa51eb3606b9974,Rob Kiefer,Fri Sep 28 13:46:27 2018 -0400,Prepare the repo for the next development cycle, d19f86901612c22559b46e67b024b39899bd6fe0,Rob Kiefer,Thu Sep 27 12:49:48 2018 -0400,Release 1.0.0-rc2, 83eb99f90f859813979e7634f8e86e376319486a,Joshua Lockerman,Thu Sep 27 13:13:51 2018 -0400,Disable isolation tests on the ABI smoke test,We have no need to run the pg_isolation_regress tests with the ABI smoke test. This commit disables them. b43574f82e1be9f492059d41a6aed2e8f7de455e,Joshua Lockerman,Thu Sep 27 12:44:12 2018 -0400,Switch 'IO' error prefix to 'TS',Our errorcodes have a 'IO' prefix from when we were Iobeam. This commit switches that prefix to 'TS' for consistency. 974788516aff57effd69d8b9970edc7f97bcecb2,Joshua Lockerman,Fri Sep 21 17:04:30 2018 -0400,Prefix public C functions with ts_,We've decided to adopt the ts_ prefix on all exported C functions in order to avoid having symbol conflicts with future postgres functions. We've already started using this prefix on new functions and this commit adds the prefix to to the old functions. 39510c3c5a7beb1b0415a9cb1d7662bca8f3070b,Matvey Arye,Wed Sep 26 17:49:30 2018 -0400,Block unknown alter table commands on hypertables,"This PR causes errors to be generated when alter table commands that are not explicitly listed/handled are encountered on hypertables. Previously the behavior was to let postgres do it's usual actions, This new behavior is safer and more appropriate for 1.0. " 2408a831508b0e6b2a2b68435fa8955c6ae854b3,Narek Galstyan,Wed Sep 26 12:15:38 2018 -0400,Add support for ALTER TABLE SET TABLESPACE on hypertables, Previously setting tablespace was blocked for hypertables. This commit enables them and adds proper testing. Setting tablespaces of indexes was handled in a previous commit here: 097db3d58928ab4784c679b8308ab0ebf3bf1fe0 41d984696ce2c2cede00c0e4dcfcabdb339cac0a,didier,Fri Sep 21 17:19:06 2018 +0200,Enclose macro replacement list and arguments in parentheses, cc59d5151f228afc677d2b0de8c33d4863b143fc,didier,Fri Sep 21 09:36:58 2018 +0200,Replace macro LEAST_TIMESTAMP by a static function,"Its arguments were substituted twice. As a result, they might be called two times when the statement is executed. If they have side effects or if they take a long time to compute it's an unsafe macro. " 281f363f352f8195ca92e8734663909c4fb2bc21,Amy Tai,Mon Sep 24 14:21:53 2018 -0400,Modify telemetry BGW to run every hour the first 12 hours,"To distinguish short-running instances from long-running instances, we have the telemetry BGW run hourly for the first 12 hours. After this initial period, the BGW will run the default once every 24 hours. Also added a scheduler test that tests background jobs are able to set next_start time within the job itself. Also modified telemetry function to return a boolean so that the scheduler can properly account for failures. " a09b3ec3ca764d5182b44d694a778079b3b91534,Josh Lockerman,Wed Sep 26 14:19:50 2018 -0400,Add pg_isolation_regress support to the timescale build system,"Postgres has a tool pg_isolation_regress which allows testing the behavior of concurrent commands. Despite not being installed by default, the tool is still build with the rest of postgres, and can run outside of the postgres src directory. This commit adds hooks within the timescale build system to call pg_isolation_regress from the postgres source tree, and run timescale tests using it. This commit also adds some basic tests for the behavior of concurrent INSERTS that create chunks. " 086e0c71a77b9508d4420862292880d57d4add37,Matvey Arye,Tue Sep 25 17:22:08 2018 -0400,Add generated_as_identity back to list of tests to run,It was previously removed by mistake in a merge/rebase. 223ee37974d3a7894ad95ead7ce5997a5d512760,Matvey Arye,Tue Sep 25 17:13:29 2018 -0400,Add rowsecurity test for PG9.6,"Unfortunately, the RLS feature is significantly different between PG 9.6 and 10 because of the addition of PERMISSIVE | RESTRICTIVE clauses. Thus we had to manually backport the rowsecurity-10 modified test to rowsecurity-9.6 test. " f40516950e3e67e723f540a5dd52dac967960a31,Matvey Arye,Tue Jul 31 17:02:35 2018 -0400,Change rowsecurity test to operate on hypertables.,"We want to test row-level-security on hypertables. So, we take the original rowsecurity test from core postgres and systematically change it so that it operates on hypertables instead just plain tables. This was done by looking for all CREATE TABLE statements and adding a create_hypertable call for each new tables (except when the table was a referenced ""metadata"" table). This commit also allows enabling and disabling RLS rules on chunks. This is more in-line with how Postgres itself handles RLS on inherited children. The rowsecurity tests relating to partitioned tables were converted to work on hypertables/chunks instead and the rules for handling permissions on chunks were tested to be akin to the rules for child partitions. " 4023abb21c2089998c6336d7e8c9d24658d1a42b,Matvey Arye,Mon Jul 30 12:44:18 2018 -0400,Import the rowsecurity test from core postgres,This adds the rowsecurity test from core postgres to our repo. We plan to modify this test to operate on hypertables but keeping a separate commit just for the import of the original test so that we can track changes to this test to convert it to operate on hypertables. 2c267ba323f3e8c5d0329761dc316c1648d1c74a,David Kohn,Thu Sep 20 15:50:18 2018 -0400,Handle SIGTERM/SIGINT asynchronously,"SIGTERM/SIGINT shouldn't cause an exit in the handler functions as you might interrupt something important (like having a spinlock etc) Instead, we use the default handler for SIGINT and a small wrapper around the Postgres `die()` function, the default for most backends, which set a variable that is processed during the next `CHECK_FOR_INTERRUPTS()` call. This also changes the behavior on a SIGTERM for the launcher. Previously we exited 0 so that the postmaster would not restart the launcher, this will no longer be the case, which we decided was proper as it allows us to unify behavior with Postgres and our other code, and if users wish to disable the launcher we proved a means of doing so by setting `timescaledb.max_background_workers = 0`. " 5377e2db3d3643f4edcef7c20a46447d06e400ff,Amy Tai,Fri Sep 21 11:47:55 2018 -0400,Fix use-after-free bug for connections in the telemetry BGW,"Connection creation function could return a NULL connection, and this wasn't handled properly, causing a SEGFAULT. Also updated telemetry error handling code to goto a cleanup section so that any started txns could be committed properly, in case an error happened somewhere inside the telemetry BGW. Also, added some tests to test that bad connections do not cause the telemetry BGW to crash. " 248f6621e4211b53fafee696e4f72231b84707d2,Sven Klemm,Tue Sep 25 14:42:34 2018 +0200,Fix pg_dump for unprivileged users,"When timescaledb is installed in template1 and a user with only createdb privileges creates a database, the user won't be able to dump the database because of lacking permissions. This patch grants the missing permissions to PUBLIC for pg_dump to succeed. We need to grant SELECT to PUBLIC for all tables even those not marked as being dumped because pg_dump will try to access all tables initially to detect inheritance chains and then decide which objects actually need to be dumped. " e6adfc1af0745a73c27cb9c1519e0c1f328ecfd9,Erik Nordström,Tue Sep 25 12:40:46 2018 +0200,Refactor source structure and build system,"The source tree under `test/` should mirror the directory structure under `src/`. However, the loader mock was currently in `test/loader-mock/src` instead of `test/src/loader`, which has now been fixed. The loader mock was also built from CMake files in `src/loader` since that made it inherit a number of required build definitions used by the non-mocked loader. To avoid having to redefine these definitions for the build in `test/`, they have been split out into a `build-defs.cmake` file that can be included in both the regular loader and the mock build. The build system has also been cleaned up in other ways. For instance, header files are no longer added as dependencies unless they are, e.g., generated headers. Test-specific code is built separately from the main extension and only linked in case of debug builds. " 193fa4a3ac8afb0070cb4485f6c4294856d082d6,Matvey Arye,Tue Sep 25 11:43:26 2018 -0400,Stop background workers when extension is DROP OWNED,An extension can be dropped indirectly via DROP OWNED. This PR makes sure that such cases are correctly handled by stopping the appropriate background workers. 1adccd7a9d682335f9cdd403e29d7eb86ba17864,Matvey Arye,Mon Sep 24 16:12:23 2018 -0400,Delete extra spaces at line endings in bgw_launcher test, 625e3fa1290a3339bd8185171642a92660b5b902,Sven Klemm,Tue Sep 25 13:52:05 2018 +0200,Fix negative value handling in int time_bucket,The integer variants of time_bucket don't handle negative time values properly and return the upper boundary of the interval instead of the lower boundary for negative time values. a33224bba3c69f2e3af9a64824571fbd63a94690,Matvey Arye,Fri Aug 31 08:42:42 2018 -0400,Add loader API version function,"A loader API version is added for future-proofing. It will allow versioned-extension to figure out the version of the loader API that is installed. For example, if a future versioned extension uses new functions made available by a new loader version it could use the loader API version number to figure out if this new capability is actually available in the installed loader (note a loader could be older than the versioned-extension if the database was not restarted). " 9b3f4d7d9e5cd8bc0a0c9eb4929edf187925294b,Amy Tai,Thu Sep 20 12:26:28 2018 -0400,Change detection of USE_OPENSSL in Postgres to use pg_config,"Instead of grepping around in the pg_config.h file, which might not contain proper defines/undefs for binary installations, we use pg_config --configure to determine if the underlying Postgres installation is compiled with OpenSSL. This is more robust. In particular, the old way of grepping in pg_config.h does not work because USE_OPENSSL is not explicitly defined in pg_config.h in the rpm installation. " 18b8068ad769bc5e782c7878b1a197f9fdc16b34,Erik Nordström,Fri Sep 21 11:06:01 2018 +0200,Remove unnecessary index on dimension metadata table,"The `dimension` metadata table had both a `(hypertable_id)` and a `UNIQUE(hypertable_id, column_name)` index. Having only the latter index should suffice. This change removes the unnecessary index, which will save some space, and make the schema more clear. " 8e84566a0ea9140cb39f52fc5068eaf38e8c5d91,Rob Kiefer,Fri Sep 21 13:58:54 2018 -0400,Fix ABI smoketests in Travis to not run unncessary steps, d09405d2b38ba095938c90c00c81831eb0c21f4f,Rob Kiefer,Thu Sep 20 18:19:55 2018 -0400,Fix adaptive chunking when hypertables have multiple dimensions,"The use of the incorrect index to find the hypertable that a dimension belongs to caused adaptive chunking to break whenever dimension_id != hypertable_id, which happened with hypertables that have multiple dimensions where time is not the first one. " 10333000d59205d661bccd1f67c6bb4effd3d8f6,Rob Kiefer,Thu Sep 20 13:41:08 2018 -0400,Add release 0.12.1 changes to master, a1615e844ebe7350d24ab0e4173fb84316d16140,Rob Kiefer,Thu Sep 20 11:26:28 2018 -0400,Reorganize Travis to disable some jobs on master,"Currently we force PRs to be up-to-date with master so most checks on master, except cron, are superfluous/redundant. " a81dc18920fb4028824450283b81c1d576bb7f15,Matvey Arye,Wed Sep 19 13:42:00 2018 -0400,Block signals when writing to the log table in tests,This PR adds a signal block while writing to the log table in our tests. This improves test reliability since messages that get generated during signal handling are no longer lost if the signal is received while another log message is being commited. e6355f48c420acf2e7e4d7ae797c89258d5f8e37,Joshua Lockerman,Thu Aug 30 17:24:39 2018 +0200,Update travis to run tests with EXEC_BACKEND also,"Postgres has a compile flag EXEC_BACKEND which makes linux behave more like windows in order to ease testing of windows behavior. By default the postgres docker images are built without this, so we've built our own version with the flag enabled for our CI. This commit enables testing on those versions. Unfortunately, not all of postgres's tests pass in this mode, so we don't run the full postgres regression suite. Even so, this should be a reasonable stopgap as we improve our Windows testing. " 958b5c733f0a73158f80221d356090fab8afbef4,Joshua Lockerman,Mon Sep 17 17:35:13 2018 -0400,Fix and clean vacuum test,"The original fix used the wrong array-to-string function getting the wrong output order. This commit fixes that and also removes the irrelevant ORDER BY schemaname, since we always constrain schemaname to be a constant and ORDERs BY attrname to ensure that we see similar orders in all tests regardless of the values of histogram bounds. " 5838ca1920809c67d64355e8bf7c1d84023d15c8,Matvey Arye,Wed Sep 19 14:28:19 2018 -0400,Add const qualifier to version_delimiter definition,This prevents a possible memory overflow bug. Also adds Assert checking for overflows. 64bba29366442e13a12414cacd8ae33fa365721e,Amy Tai,Wed Sep 19 12:12:13 2018 -0400,Give a helpful build warning when Postgres does not have OpenSSL enabled,"Previously, whenever the underlying Postgres installation did not have OpenSSL enabled, but Timescale was trying to compile WITH OpenSSL, users would get a compile error because the Postgres header file include/pg_config.h tries to redefine an OpenSSL function. To prevent this compilation error, we have modified CMake to simply fail the build if Postgres is not OpenSSL-enabled, but the user is trying to compile WITH OpenSSL. CMake will now print a message with 2 actions the user can take to fix the error. " d5a639263ee10ba57f90691c12a17d36c8517eb4,Rob Kiefer,Tue Sep 18 15:57:43 2018 -0400,Fix adaptive chunking so it chooses correct index,"Previously our check that the index matched the type of the index was incorrect, using attno instead of atttypid. However, this also had a problem in that if there was another column with the same type as the time column, e.g., two BIGINT columns, we could pick the wrong index to use to find min and max. The check now makes sure the type is correct AND the name is the same as the table column name. " 3489ccaf82bd4a1ab77aa86091e96ce012ef28f1,Matvey Arye,Wed Sep 19 05:24:40 2018 -0400,Fix sigterm handling in background jobs,"Previously TimescaleDB background jobs used Postgres's default `bgworker_die` sigterm handler. However this signal handler had a few problems: 1) It does not respect critical sections and causes PANIC when the signal is received inside a critical section 2) It can get called while the system has an acquired spinlock. Both of these problems are solved by using the CHECK_FOR_INTERRUPTS system inside native Postgres but this requires using the `die` signal handler (note this is what the parallel infrastructure uses). So, we switch to using a handler that calls `die` but also supplements with better log messages. We changed both the db scheduler and jobs to use this method of signal handling. We also added a safety check to the launcher to handle critical sections in a safer way. " 2369ae94500a1ea5e45260159469ad89e3a66055,Kevin Bowling,Wed Sep 12 10:42:57 2018 -0700,"Remove !WIN32 for sys/time.h and sys/socket.h, pg provides fills", ebbb4ae03f41bbc3ad389367d277bc0dd3a07b29,Kevin Bowling,Tue Sep 11 23:04:10 2018 -0700,Also add sys/time.h for NetBSD. Fixes #700, 1a9ae17f992d7ec61fcda5987b536ea95a3b166a,Kevin Bowling,Tue Sep 11 21:45:51 2018 -0700,Fix build on FreeBSD wrt sockets, 8225cd2defa76fa896210786bd84d0c552693ca2,Rob Kiefer,Mon Sep 17 19:54:01 2018 -0400,Remove (redefined) macro PG_VERSION and replace with PACKAGE_VERSION,The previous macro collided with one already defined by PostgreSQL and on some platforms it is not always in the same form causing lots of build warnings. c872e7e5f9dd4474c66bb6c064d891cca78410b9,Amy Tai,Thu Sep 13 15:33:39 2018 -0400,Fix custom error handling callbacks in scheduler test code,"There was a Valgrind stack overflow bug that occurred occassionally inside the bgw_db_scheduler test. This led us to identify several places in the scheduler test code in which infinite recursion could occur. The biggest source of a potential stack overflow is inside the emit_log_hook_callback. In particular, if the callback encounters an error, the code that throws the error will be trying to write a log message, which will trigger a call to the log_hook_callback. If this error persists (ie is encountered on every subsequent call to the callback), then the log_hook_callback will infinitely loop on itself, thereby causing a stack overflow. Another potential source of infinite recursion is the static variable prev_signal_func. If the setter function for this variable is called more than once (which appears possible in the test code), it will be set to our custom callback. Because these static variables are functions that are called within our callback functions, then our callback functions will end up calling themselves ad infinitem. To eliminate this possibility, I have made sure prev_signal_func is only set if it is NULL (the first time). " 2a07cf9e4385fe5ba86974503d7e1394933dcc2f,David Kohn,Thu Sep 13 12:33:18 2018 -0400,Release SpinLock even when we're about to Error due to over-decrementing, b2a15b8b8161a7e0ff9ed2b9361bc93f08625e49,Amy Tai,Sun Sep 16 12:02:47 2018 -0400,Make sure DB schedulers are not decremented if they were never incremented,"Previously, accounting for BGWs assigned to DB schedulers was incorrect, because NULL scheduler handlers were translating to BGW_STOPPED. This caused the accounter to decrement BGW_counter despite the counter never being incremented. We fix this by imposing the invariant that entries for schedulers should be not be created in the hash table until they have grabbed a slot via the bgw_counter. Also did some cleanup with respect to the message-handling functions. In particular, message_restart_action is now a true-restart rather than a force-restart. " 6731c86fae7311ed7f9340c885d80fcdd87a48dc,Erik Nordström,Fri Sep 14 11:44:18 2018 +0200,Add support for pre-release version checks,"Version checks fail on pre-release versions (e.g., `1.0.0-rc1`) since the version parsing didn't expect a pre-release tag at the end. This change adds support for pre-release tags, using lexicographical comparison in case of pre-release tags. Note that having a pre-release tag vs not having one makes the version strictly ""smaller"", given otherwise identical versions. " f435e8f1a98729bf46c5fb1e98d381ccc325a3ec,Joshua Lockerman,Fri Sep 14 12:35:16 2018 -0400,Update vacuum test with more consistent output ordering,"EXEC_BACKEND ordered rows for this test differently than other platforms This commit adds an ORDER BY for histogram_bounds also, sot that all platforms must be the same " ae0320f9a4de17dc467b5a273f34f3e04689eb3b,Joshua Lockerman,Thu Aug 30 17:24:39 2018 +0200,Add formatting tests to travis,"We currently check code formatting and remind people to obey style by hand. This is tedious and error-prone, made worse by github’s unusual rendering, and badly-formatted code has made it into our repository. This commit adds a travis job which checks PRs for correct formatting, so we no longer need humans to check for this in the common case. " 7c105af02cd9129f5424761836fcef573b23485c,Rob Kiefer,Thu Sep 13 18:30:44 2018 -0400,"Fix ABI test, correctly this time","The previous fix did not actually include openssl-dev when trying to build TimescaleDB. Also, this fixes broken tests due to lack of permissions. " 6c8a9c1b3392763dd5ed4c16c6d764c9ab5fbde5,Rob Kiefer,Thu Sep 13 10:43:24 2018 -0400,Update ABI test Docker image to have OpenSSL dev headers, 0e4fce5dad1c307669fbe68bcb720c6f34421f98,Rob Kiefer,Tue Sep 11 20:04:21 2018 -0400,Use AttrNumberGetAttrOffset in missed bgw code,"This cleanup was run on master before background worker code was merged, so this PR captures the code that was missed. " c0e68b0d951edf19654950f6873f636c3e79f414,Rob Kiefer,Wed Sep 12 08:16:41 2018 -0400,Prepare the repo for the next development cycle, b1710f16bb84c5b50babd307b2506cac651665e3,Rob Kiefer,Tue Sep 11 17:24:28 2018 -0400,Release 1.0.0-rc1, 346191d15c8bdd6bed1a3b15d95387afcfb15ecd,Amy Tai,Tue Sep 11 11:53:37 2018 -0400,Suppress NOTICE welcome message in tests,"Suppress NOTICE in tests, either by setting client_min_messages=error or by redirecting pg_restore output (because pg_restore will clobber any client_min_messages value). Also added installation_metadata test back into CMakeList. " acebaea40c5feb0d5584b55d13067cdfed03cc94,David Kohn,Tue Sep 11 12:47:51 2018 -0400,Don't start schedulers for template databases., bdb0f095baf67db8ae43830091504afe51bb3c53,Rob Kiefer,Tue Sep 11 14:34:14 2018 -0400,Change Travis to use remote images for Valgrind,"These pre-built images should give us more robust Valgrind tests due to the fact that each job does not have to build the image each time, which caused a lot of flakes. " f221a12a9edf254fdece6d02066a8f4ab6c92e85,Rob Kiefer,Tue Sep 11 13:54:10 2018 -0400,Fix use-after-free bug in telemetry test, 4b91e21a302f38096573696f56c134378bb659bd,Rob Kiefer,Tue Sep 11 12:23:15 2018 -0400,Update bootstrap output to include build type and install method,"Setting the default build type was done after the build type was outputted in bootstrap which caused a confusing output saying the build type was blank. Also, this adds the install method to verify binary builds are correct. " 2092b2a9de24ae39552328f8b056d09eac9f3419,Rob Kiefer,Mon Sep 10 23:11:44 2018 -0400,Fix unused variable warning in Release build, 0dc5bbb447869b67fe9403f6419b69e1c2c8ccb0,didier,Mon Sep 10 19:52:49 2018 +0200,Use pg_config bindir directory for pg executables,"In case pg_dump, createdb, dropdb and pg_restore aren't in system default path. " 04318f93e9f8c73417fa93dc1b2511ace4c1b0dc,Rob Kiefer,Thu Aug 30 17:04:15 2018 +0200,Update Contributing.md with updated guidelines and extras, 24eafba7dbdc5eeb97a8e3b1156ff7d566f2c219,Rob Kiefer,Wed Sep 5 16:10:44 2018 -0400,Update travis file for quicker prerelease builds,Valgrind and memory test jobs previously spent a minute building a Docker image that was never used (needed for other jobs). Also the coverity job was building every build or PR even though most of the time it never did anything useful. 2b5bc309372803af12b11166b5c10a837a84fa57,Lee Hampton,Mon Sep 10 16:04:05 2018 -0400,Prepare the repo for the next development cycle, 86633c29f5cdcdf841701794115006de39293c09,Lee Hampton,Mon Sep 10 13:40:18 2018 -0400,Release 0.12.0, f662ae11914787bb9f3a5f4910b2594227c0292c,Matvey Arye,Wed Aug 29 09:35:53 2018 +0200,Add telemetry job and turn off default jobs in tests,This adds the telemetry job to the job scheduler. Telemetry is scheduled to run every 24 hours with a 1 hour exponential backoff retry period. Additional fixes related to the telemetry job: - Add separate ID space to the bgw_job table for default jobs. We do not dump this ID space inside pg_dump to prevent job insertion conflicts. - Add check to update scripts for default jobs. - Change shmem_callback so that it doesn't modify state since state transitions are not atomic with respect to interrupts and interrupt callbacks. - Disable default telemetry job in regression and docker tests. efab2aa48ed8451dbeab862a880c80406a7e88f1,Erik Nordström,Thu Sep 6 21:22:26 2018 +0200,Fix net lib functionality on Windows and improve error handling,"The net library now uses Windows-specific APIs for socket operations, ensuring more consistent functionality with UNIX systems. PostgreSQL itself implements its own sockets compatibility layer, but it is built around non-blocking sockets, while we currently use blocking sockets. Therefore, to avoid unexpected behavior with the PG functions, we bypass them by calling the winsock API directly. Error handling is also refactored to return connection-specific errors instead of throwing PostgreSQL exceptions deep in the net lib. Instead, applications/calling functions can now handle errors themselves. " 02d2576393188564970b668fd3899ba4254c25fd,Erik Nordström,Fri Sep 7 20:42:44 2018 +0200,Fix symbol_conflict test on Windows,The symbol_conflict test failed on Windows because the CMake build script accidentally did not include the required code in the loader during Debug builds. This change fixes the CMake script so that the code is included in the laoder. 71589c487f038f6c7baf31ff0a6cc8c98c1662d0,Erik Nordström,Wed Sep 5 18:32:24 2018 +0200,Fix issues when OpenSSL is not available,"This fixes a number of compilation and test issues when OpenSSL is not available. While we still default to OpenSSL enabled, we allow explicitly setting -DUSE_OPENSSL=false to compile and run tests without OpenSSL installed. But if not specified, CMake will fail if OpenSSL is not available on the system. " f3ac7e0f0b890b67f7153e896ed9ea04bd3e3005,Rob Kiefer,Wed Sep 5 13:36:14 2018 -0400,Fix compilation warnings/errors and static analysis issues,A deadcode issue is fixed by changing an AND to OR so that it is actually possible to hit the condition. One change of palloc to palloc0 to potentially prevent a Valgrind issue. a43cd042ddfb1a6e97e2d3f6910a846602233eed,Amy Tai,Tue Sep 4 20:58:26 2018 -0400,Call the main telemetry function inside BGW executor,"Call the main telemetry function whenever the job type is the telemetry type. A separate PR will add a row with this job type to the jobs table, so that the scheduler actually runs this executor function. " ebe091566982c3ea14addb5187de10ce73ec12ff,Erik Nordström,Wed Aug 29 22:42:46 2018 +0200,Refactor telemetry and fixes,"The installation metadata has been refactored: - The installation metadata store now takes Datums of any type as input and output - Move metadata functions from uuid.c -> metadata.c - Make metadata functions return native types rather than text, including for tests Telemetry tests for ssl and nossl have been combined. Note that PG 9.6 does not have pg_backend_random() that gives us a secure random numbers for UUIDs that we send in telemetry. Therefore, we fall back to the generating the UUID from the timestamp if we are on PG 9.6. This change also fixes a number of test issues. For instance, in the telemetry test the escape char `E` was passed on as part of the response string when set as a variable with `\set`. This was not detected before because the response parser didn't parse the start of the response properly. A number of fixes have been made to the formatting of log messages for telemetry to conform to the PostgreSQL standard as well as being consistent with other messages. Numerous build issues on Windows have been resolved. There is also new functionality to get OS version info on Windows (for telemetry), including a SQL function get_os_info() to retrieve this information. The net library will now allow connecting to a servicename, e.g., http or https. A port is resolved from this service name via getaddrinfo(). An explicit port can still be given, and it that case it will not resolve the service name. Databases the are updated to the new version of the extension will have an install_timestamp in their installation metadata that does not reflect the actual original install date of the extension. To be able to distinguish these updated installations from those that are freshly installed, we add a bogus ""epoch"" install_timestamp in the update script. Parsing of the version string in the telemetry response has been refactored to be more amenable to testing. Tests have been added. " faf481b06186374dac940dc319177e8f8a964ca2,Amy Tai,Fri Aug 3 12:28:13 2018 -0400,Add telemetry functionality,"Adding the telemetry BGW and all auxiliary functions, such as generating a UUID, creating the internal metadata table for storing UUIDs, and parsing the server-side response with the latest version of TimescaleDB. " 4f6b92ab37dd93112d12a9c6f794de40b6ec951d,Erik Nordström,Wed Aug 29 12:55:44 2018 +0200,Refactor net lib functionality and fix build issues,"This makes the following changes related to the net lib: - A CMake run now fails in case OpenSSL is not found when the local PostgreSQL installation has been compiled with SSL enabled. The reason why it is best to fail the CMake run in this case is because USE_OPENSSL will be defined and 1 in `pg_config.h` and thus we will compile with SSL support. - Use palloc/pfree in connection library. - Split net library into separate source files. The net library is refactored so that the code for different connection types live in their separate source files. In particular, the source code for mock connections is now moved to `test/src/net`. - The generate_typedefs.sh script now runs in subdirectories so that source files in those subdirectories are properly pgindented. - The pgindent target previously did not cover files under `test/src`. This is now fixed. An exclude file has also been added that avoids running pgindent in hidden directories. This fixes issues with, e.g., trying to indent files in cquery caches. - Fix formatting with pgindent Fix parameter types in HTTP lib. Add `const` to parameter types, such as strings. Use `size_t` for length parameters - Parse and validate the status line of HTTP responses. The beginning of HTTP responses weren't properly parsed and validated, allowing invalid character sequences at the start of requests. " 45a2b76d46793df9191fa819fa9e3119121e30e4,Amy Tai,Fri Jun 29 18:31:14 2018 -0400,Add new Connection and HTTP libraries,Adding a library for making socket connections (both SSL and non) and HTTP-handling library that parses HTTP responses and constructs HTTP requests. b6fe6570edad20457dd96834e072ea528b848953,David Kohn,Sun Aug 26 22:23:38 2018 +0200,"Fix max_background_workers guc, errors on EXEC_BACKEND and formatting",Fix timescaledb.max_background_workers GUC configuration so that there aren't assert errors when higher than max_worker_processes. Don't error when unable to register background workers due to requesting more workers than max_worker_processes. Add functions to increment and decrement the bgw_counter multiple times simultaneously. Error on decrementing background workers below 1 (One worker always reserved for the launcher). Deal with EXEC_BACKEND/Windows forking differently and causing problems with Shared memory by a) not mapping multiple times and checking for found on mapping of shared memory segments. b) dealing separately with reinit and init cases esp in use of GetNamedLWLockTranche Change behavior of queue reinit and counter reinit to be less intrusive and to process messages might otherwise have been lost. Remove bgw_launcher test for terminate so the bgw_launcher continues running in the background for further tests. Slight formatting fixes and unification of error messages. 5d8c7cc6f6324ab496a932946c82bbf4938f8383,Matvey Arye,Fri Aug 3 20:02:43 2018 -0400,Add a scheduler for background jobs,"TimescaleDB will want to run multiple background jobs. This PR adds a simple scheduler so that jobs inserted into a jobs table could be run on a schedule. This first implementation has two limitations: 1) The list of jobs to be run is read from the database when the scheduler is first started. We do not update this list if the jobs table changes. 2) There is no prioritization for when to run jobs. There design of the scheduler is as follows: The scheduler itself is a background job that continuously runs and waits for a time when jobs need to be scheduled. It then launches jobs as new background workers that it controls through the background worker handle. Aggregate statistics about a job are kept in the job_stat catalog table. These statistics include the start and finish times of the last run of the job as well as whether or not the job succeeded. The next_start is used to figure out when next to run a job after a scheduler is restarted. The statistics table also tracks consecutive failures and crashes for the job which is used for calculating the exponential backoff after a crash or failure (which is used to set the next_start after the crash/failure). Note also that there is a minimum time after the db scheduler starts up and a crashed job is restarted. This is to allow the operator enough time to disable the job if needed. Note that the number of crashes is an overestimate of the actual number of crashes for a job. This is so that we are conservative and never miss a crash and fail to use the appropriate backoff logic. Note that there is some complexity in ensuring that all crashes are counted since a crash in Postgres causes /all/ processes to SIGQUIT: we must commit changes to the stats table /before/ a job starts so that we can then deduce after a job has crashed and the scheduler comes back up that a job was started, and not finished before the crash (meaning that it could have been the crashing process). " 55a71419531f77829c4b499bf06e39faad2af8a2,David Kohn,Mon Aug 27 17:30:35 2018 +0200,Implement a cluster-wide launcher for background workers,"The launcher controls how Timescale DB schedulers for each database are stopped/started both at server start time and if they are started or stopped while the server is running which can happen when, say, an update of the extension is performed. Includes tests for multiple types of behavior within the launcher, but only a mock for the db schedulers which will be dealt with in future commits. This launcher code is mostly in the loader, as such it must remain backwards compatible for the foreseeable future, so significant thought and design has gone into making interactions with this code well defined and consistent so that maintaining backwards compatibility is relatively easy. " 5bc705f56d79a0a8dd6980ffb1a051ed97a7c9ee,Evan Carroll,Sun Sep 9 13:34:44 2018 -0500,Update bootstrap to check for cmake and exit if not found, 61c56fdde1bdb6f5b75007c874cac6f4b9f342fa,Rob Kiefer,Fri Sep 7 16:59:35 2018 -0400,Fix pg_dump test to work correctly on remote DBs,"Using shell scripts in utils/ along with environmental vars we know to be setup for the test, we can make the dump/restore test work on remote platforms. " 98e56ddb8171826dfc05978a247546ae86e9f50f,Rob Kiefer,Fri Sep 7 16:58:18 2018 -0400,Improve show_indexes test func to be more platform agnostic,Windows and Unix platforms had different orderings for some tests in pg_dump. Now we order by a name that orders consistently on both platforms. c67a359cf6507cdeed4f3c798978856fe223ac95,Rob Kiefer,Thu Sep 6 14:26:56 2018 -0400,Update Windows badge to point to corrected account name, c71960c048b238d8f4418e59eafb65d544243c70,Amy Tai,Mon Aug 20 12:55:55 2018 -0400,Fix pg_dump test and make insert block trigger a non-internal trigger,"Previously, the pg_dump test was broken because it is not possible to reference psql variables from inside bash commands run through psql. This is fixed by hardcoding the username passed to the bash commands inside the test. Also, we changed the insert block trigger preventing inserts into hypertable to a non-internal trigger, because internal triggers are not dumped by pg_dump. We need to dump the trigger so that it is already in place after a pg_restore, to prevent users from accidentally inserting rows into a hypertable while timescaledb_restoring=on. " b928caae003257b2d4ce249906fd82a7a543d1ec,Joshua Lockerman,Tue Aug 28 10:02:00 2018 +0200,Note how to recreate templated files,"Our files generated from templates do not get recreated by `make` when they get deleted, and have to be recreated manually or by rerunning cmake. This commit adds a note to the CMakeLists.txt to that effect. " 8571e412975a641ac4c2e4be720714c216ab49e5,Joshua Lockerman,Mon Aug 27 11:22:24 2018 +0200,Use AttrNumberGetAttrOffset instead of Anum_name - 1 for array indexing,"Use the postgres macro instead of manual subtraction everywhere to be more readable, and gain the Assert. Done with `find ./src -type f -exec sed -i -e 's/Anum_\(.*\) - 1/AttrNumberGetAttrOffset(Anum_\1)/' {} \;` " d1710efde5bf53524a615b54201f71a192e639bf,Rob Kiefer,Mon Aug 27 17:30:35 2018 +0200,Improve regression test script to cleanup more thoroughly,Using the regression test script on a remote machine would leave behind a secondary database (by default called `single_2`) that would cause the next run to fail when trying to remove the role that owned the secondary database. This change has the script remove the secondary database so that no longer happens. fc3677f7fa95955e1e494a7ed00c5c2982b8ba9b,Joshua Lockerman,Tue Aug 7 15:58:38 2018 -0400,Reduce number of open chunks per insert,Our default `max_open_chunks_per_insert` was based on an assumption that chunks used less memory than they actually do. Upping that constant to reduce memory usage. 3e42150e3b0402ae865d9a827d8d178568a0d27e,Erik Nordström,Fri Aug 24 09:45:11 2018 +0200,Fix formatting issues,This fixes a number of formatting issues that were captured by pgindent. 027b7b29420a742d7615c70d9f19b2b99c488c2c,Erik Nordström,Thu Aug 16 23:18:33 2018 +0200,Hide extension symbols by default on Unix platforms,"On platforms using libraries and executables in ELF format (e.g., Linux), exported symbols can collide. If a symbol (e.g., a function) is already available, loading a module with a function that has the exact same name will cause the newly loaded library to use the already existing function instead of the one in the new module. Unfortunately, GCC and Clang exports all symbols by default. To avoid symbol collisions, this change compiles the extension with `-fvisibility=hidden` on Unix platforms, which means a symbol isn't exported unless explicitly specified. Note that symbols are not exported by default on Windows. For GCC and Clang, a symbol can be exported by annotating a function or variable with `__attribute__ ((visibility (""default"")))`. This is done transparently using our custom `TS_FUNCTION_INFO_V1` macro. Note that functions exported with that macro might still collide with existing symbols so they should have proper name prefixes that make collisions unlikely. A test for conflicting symbols runs in Debug configuration. When compiled in Debug configuration, code with an identically named function is added to both the loader and then versioned extension, although the functions return different outputs. The functions are called from both the loader and the versioned extension and the output should contain different module names if no conflict. The test fails on Linux if not compiled with -fvisibility=hidden. " 6a3abe5c92adf5313e3ca73e0395504145621298,Joshua Lockerman,Tue Aug 7 15:58:38 2018 -0400,Fix SubspaceStore to ensure max_open_chunks_per_insert is obeyed,"SubspaceStore keeps a running count of the number of objects added to it called `descendants`. This patch fixes that count, so that it always keeps track of the number of objects sitting at the leaves of the SubspaceStore. (The current version treats `descendants` as keeping track of the number of leaves at some places, and the number of objects sitting at the next level at others, resulting in the counter containing neither. Also fixes UB in dimension vector: memcpy cannot be used on overlapping memory. " 0c95a694911ed6e04c5521690d47d3e74a87da2b,Lee Hampton,Mon Aug 13 15:39:02 2018 -0400,Remove valgrind tests from Travis cron,The valgrind tests currently fail intermittently and take a lot of time for the amount of CI workers we have available. We can still trigger them by pushing to the prelease_test branch. b376fc3b259095d71673e2813e3102000038fed9,Lee Hampton,Thu Aug 9 00:56:11 2018 -0400,Prepare the repo for next development cycle, f67c10772fdb05d90d68daf36407b42040fc8856,Lee Hampton,Wed Aug 8 12:15:18 2018 -0400,Release 0.11.0, e0872978617b14cd0bb187684eb22f242983c87c,Rob Kiefer,Wed Aug 8 15:06:43 2018 -0400,Update base image for out-of-order memory test, f0075340f25c3443a051f220c5844a111be05330,Matvey Arye,Wed Aug 8 15:27:47 2018 -0400,Fix memory initialization error,"This error incorrectly intialized nulls to true, false. Now changed both to be true. Error found by valgrind. " e71422e6e663455436224a1d1a076bf92bec7108,Matvey Arye,Wed Aug 8 12:44:45 2018 -0400,Add prelease_test branch testing in travis,Force memory tests to run when pushing to prelease_test branch in addition to cron. This allows us to run a these tests explicitly. 9ba2e819356ccecd9af40c7dda04d99bfda7a5d5,Joshua Lockerman,Tue Aug 7 12:54:17 2018 -0400,Fix segfault with custom partition types,"Postgres assumes that exprs will live for the entire duration of the SQL expression that they are created from and stores pointers into them based on that assumption. As a memory usage optimization our INSERT dynamically creats chunk constraints which live for only an individual chunk insert, causing segfaults when postgres tries to read through the cleaned up memory. To fix this we now free the per_tuple_exprcontext _before_ deleting a chunk insert state memory context, ensuring the segfaulting callback is called on a valid state. " 7e9bf25ce61269e3c3412a87c951d91928927c63,Rob Kiefer,Wed Aug 8 11:19:49 2018 -0400,Change default chunk size to one week,"The previous default of 30 days was typically too long for most starting use cases, which would lead users to have one large chunk that lacked some of the benefits of using TimescaleDB, at least for the initial chunk. This chunk size should be more reasonable, especially for initial workloads. " 506fa18624f6dba6989075356b8dc0e082bf7307,Matvey Arye,Tue Aug 7 15:52:06 2018 -0400,Add tests for custom types, 1d9ade714551333eebe113ed9e2ff34404d4436f,Floris van Nee,Sun Apr 22 20:02:33 2018 +0200,add support for other types as timescale column, 570f2f8b9652611f0208db3481b7426f79af4a02,Narek Galstyan,Tue Aug 7 16:27:03 2018 -0400,Validate parameters when creating partition info,Partitioning function information for closed (space) dimensions is passed from SQL as REGPROC. This leaves no possibility for this info to be NULL when internal partitioning function data structure is populated. 148f2da9248289ed396337e32b31998b5fcdd266,Erik Nordström,Wed Aug 8 16:42:56 2018 +0200,Use shared_buffers as the available cache memory,"When configuring adaptive chunking and estimating the chunk_target_size, we should use shared_buffers as an indication of cache memory instead of trying to estimate based on effective_cache_memory, system memory or other settings. This is because shared_buffers should already have been set by the user based on system memory, and also accurately reflects the cache memory available to PostgreSQL. We use a fraction (currently 0.9) of this cache memory as the chunk_target_size. This assumes no concurrency with other hypertables. " e0a15c18765074d526f3a5eb3c055fbe35a8bbe8,Rob Kiefer,Tue Aug 7 14:29:31 2018 -0400,Add additional comments to explain algorithm,"In particular, more comments on how the algorithm adjusts when all chunks are undersized. Also some slight cleanups of log meassages and refactoring. " d81dccbb6360c0d3f8349f0783ac1bc25dc901c0,Erik Nordström,Tue Aug 7 18:08:15 2018 +0200,Set the default chunk_time_interval to 1 day with adaptive chunking enabled,"When adaptive chunking is enabled, and no `chunk_time_interval` is set, it is better to start with a small chunk size rather than a too big, since it will adapt faster. This change sets the `chunk_time_interval` to 1 day if adaptive chunking is enabled on a hypertable. Note that this only happens if adaptive chunking is enabled when `create_hypertable()` is called. Otherwise, the existing `chunk_time_interval` will be used. " 2e7b32cd9156a43741ec39983171a77c70aa7f21,Erik Nordström,Mon Aug 6 21:09:29 2018 +0200,Add WARNING when doing min-max heap scan for adaptive chunking,"Adaptive chunking uses the min and max value of previous chunks to estimate their ""fill factor"". Ideally, min and max should be retreived using an index, but if no index exists we fall back to a heap scan. A heap scan can be very expensive, so we now raise a WARNING if no index exists. This change also renames set_adaptive_chunk_sizing() to simply set_adaptive_chunking(). " 6b452a8b9eba3d2d5678a9fe68a8aaa8eac78aaf,Rob Kiefer,Tue Jul 17 19:50:30 2018 +0000,Update adaptive chunk algorithm to handle very small chunks.,"Originally the adaptive chunk algorithm would ignore all chunks under a particular threshold, which prevented the algorithm from every correcting a poorly set initial chunk size if it was too small. Now the algorithm will attempt to do an intermediate resizing of chunk intervals to make future chunks at least sufficiently large enough to use in regular chunk interval estimation. This allows poorly chosen chunk intervals of a too small size to correct, while also not overreacting to very small chunks. Additional changes include a fix to correctly use the most recent chunks to a time rather than the first three chunks before a time; adjusting and adding new debug messages to help understand how new chunk intervals are reached; and tweaking thresholds to strike a balance between insert performance, number of chunks, and number of truncated chunks due to unaligned chunk intervals. " 9c9cdca6d34d4bf7866ec72c6bf6af8960e990b4,Erik Nordström,Mon Sep 4 17:24:30 2017 +0200,Add support for adaptive chunk sizing,"Users can now (optionally) set a target chunk size and TimescaleDB will try to adapt the interval length of the first open (""time"") dimension in order to reach that target chunk size. If a hypertable has more than one open dimension, only the first one will have a dynamically adapting interval. Users can optionally specify their own function that calculates the new dimension interval. They can also set a target size of 0 in order to estimate a suitable target size for a chunk based on available memory. " 25520304db2b3402d0f9f80f5a44a7c56cc1b66d,Hyeong Joon Suh,Mon Jul 16 12:57:02 2018 -0400,Add valgrind tests to Travis,Automates valgrind memory tests by adding cron jobs on Travis that run a test script cloned from a private repository. The test script runs the regression test suite under valgrind and crashes if there are any problems. 24296d21b19bd25fba43f3d5714fcdc93c26743d,Hyeong Joon Suh,Tue Jul 17 11:06:30 2018 -0400,Add memory spike test for out of order inserts,"Out of order, random inserts into TimescaleDB in the past sometimes caused continuously increasing memory usage. These checks are being added to ensure that there are no memory spike regressions via a cron Travis job. " 1f0897fc3de460aba6cf8e52d60c7b9f2fbf7cfe,Amy Tai,Fri Aug 3 10:03:54 2018 -0400,"Add some system information during build time, including install method","Set variables describing system-specific information during build time, so that they can be collected and reported by a diagnostic thread. " 7f8d17db247f62812e893031c2f8301b527e133d,Matvey Arye,Wed Jul 25 13:06:33 2018 -0400,Handle DEFERRED and VALID options for constraints,Also add tests for deferred constraints in general. 05609cedae11f91019064c04917aee7d21a569c2,Matvey Arye,Tue Jul 24 17:26:54 2018 -0400,Add constraint options to test output, 0c5c21b2e468e57e4f862762853f143d2352afbe,Matvey Arye,Tue Jul 24 14:19:19 2018 -0400,Block using rules with hypertables, 37142e9094dd25e0b3818cb1197900bd16e02185,Erik Nordström,Fri Jun 29 15:28:19 2018 +0200,Block INSERTs on a hypertable's root table,"A hypertable's root table should never have any tuples, but it can acquire tuples by accident if the TimescaleDB extension is not preloaded or `timescaledb.restoring` is set to ON. To avoid the above issue, a hypertable's root table now has a (internal) trigger that generates an error when tuples are inserted. This preserves the integrity of the hypertable even when restoring or the extension is not preloaded. An internal trigger has the advantage of being mostly transparent to users (e.g., it doesn't show up with \d) and it is not inherited by chunks, so it needs no special handling to avoid adding it to chunks. The blocking trigger is added in the update script and if it is detected that the hypertable's root table has data in the process, the update will fail with an error and instructions on how to fix. " 4daf087e115bfb9a1003eb6570f1b597c021c2b4,Narek Galstyan,Wed Jul 25 17:47:42 2018 -0400,Fix some ALTER TABLE corner case bugs on hypertables,Previously the following commands would not throw an error on hypertables and this PR makes sure that they do in order to avoid corrupting hypertable internals: * ALTER TABLE hypertable ADD COLUMN new_device_id int UNIQUE; * ALTER TABLE hypertable DROP COLUMN partitioning_column; * ALTER TABLE hypertable DROP COLUMN time_open_dim; * ALTER TABLE hypertable ALTER COLUMN time_open_dim TYPE TEXT; Fixes #536. 122f5f15a4e0429f3af328e74660f7479b9cb1d5,Matvey Arye,Tue Jul 24 15:08:49 2018 -0400,Block replica identity usage with hypertables, 657a12f6f254146cdfbe029fef5bcd5ec58e2fb1,Matvey Arye,Fri Jul 27 16:28:43 2018 -0400,Add tests for table OF type,This makes sure that tables created OF type work as hypertables. It also makes sure that ALTER table command related to OF work correctly. 8bf552e2adc14ee78f7d1f2d0c15408d9524dd16,Matvey Arye,Wed Jul 25 17:42:27 2018 -0400,Block unlogged tables from being used as hypertables, a8c637e112d4f60fb8ca08cfb4acf3a47cfed23f,Narek Galstyan,Fri Aug 3 12:05:57 2018 -0400,Create aggregate functions only once to avoid dependency issues,There is no CREATE OR REPLACE AGGREGATE which means that the only way to replace an aggregate is to DROP then CREATE which is problematic as it will fail if the previous version of the aggregate has dependencies. This commit makes sure aggregates are not dropped and recreated every time. NOTE that WHEN CREATING NEW FUNCTIONS in sql/aggregates.sql you should also make sure they are created in an update script so that both new users and people updating from a previous version get the new function. sql/aggregates.sql is run only once when TimescaleDB is installed and is not rerun during updates that is why everything created there should also be in an update file. Fixes #612 a97f2affef23d55868cc790d12a22f501dd33ede,Narek Galstyan,Fri Aug 3 11:00:04 2018 -0400,Add support for custom hypertable dimension types,"Previously, if a hypertable dimension type did not have a default hash function, create_hypertable would throw an error. However, this should not be the case if a custom partitioning function is provided. This commit addresses the issue making sure that arbitrary custom types can be used as partitioning dimensions as long as a valid partitioning function is provided. Fixes #470. " dfe026c3b6c3c9e4ba7e32450056fddf12bcf402,Matvey Arye,Thu Jul 26 17:20:02 2018 -0400,Refactor create_hypertable rel access.,"This refactor does three things: 1) Upgrades the lock taken to AccessExclusive. This is to prevent upgrading locks during data migration. 2) Explicitly release lock in the IF NOT EXISTS case. This is more inline with what PG itself does. Also, optimize the easy IF NOT EXISTS case. 3) Exposes a rel inside create_hypertable itself so that checks can use one rel instead of opening and closing a bunch of them. " ed379c3dd81071c7f571c2f047878a3024e78966,Narek Galstyan,Mon Jul 30 15:28:43 2018 -0400,Validate existing indexes before adding a new dimension,"If the argument column of add_dimension is not in all of hypertable indexes that have UNIQUE, PRIMARY KEY or EXCLUSION constraints, then add_dimension call should fail. This commit enforces the above. " 005813da215ff0d63da528cf1636084ccc73bee6,Rob Kiefer,Fri Jul 20 14:30:52 2018 -0400,Make update tests more composable.,"This removes a lot of duplicated code across the two versions of update tests (one with testing of constraints vs one without) and potentially allows for further additions more easily. Also, this splits Travis testing of updates into two jobs so they can run concurrently and reduce the amount of turnaround time for PRs. Finally, added versions since 0.8.0 that were not previously being tested. " 1f2d27671b67dcb82cfd58a85ad6fb506d35d994,Erik Nordström,Thu Aug 2 17:24:47 2018 +0200,Fix and improve show_indexes test support function,This fixes the show_indexes test support function to properly show the columns of the indexes instead of the table. The function now also shows the expressions of expression indexes. 77b00356c402824381d78566a2b00b8e1361eb47,Erik Nordström,Thu Aug 2 19:18:54 2018 +0200,Enforce IMMUTABLE partitioning functions,This adds a simple check to enforce that partitioning functions are IMMUTABLE. This is a requirement since a partitioning function must always return the same value given the same input. cbc5e60abeb92b6b5f3dbf91bd298dffacda5ab3,Erik Nordström,Fri Jun 29 15:28:19 2018 +0200,Block NO INHERIT constraints on hypertables,"Constraints with the NO INHERIT option does not make sense on a hypertable's root table since these will not be enforced. Previously, NO INHERIT constraints were blocked on chunks, and were thus not enforced until chunk creation time, allowing creation of NO INHERIT constraints on empty hypertables, but then causing failure at chunk-creation time. Instead, NO INHERIT constraints are now properly blocked at the hypertable level. " e362e9cf18a0c708dc196d2a0997b0b490bf2446,Matvey Arye,Wed Jul 25 17:04:02 2018 -0400,Block mixing hypertables with postgres inheritance, 011f12be7178c9ac48230120c2ecc35e245143bf,Matvey Arye,Mon Jul 23 17:50:42 2018 -0400,Add support for CLUSTER ON and SET WITHOUT CLUSTER,Add support for: * ALTER TABLE ... CLUSTER ON * ALTER TABLE ... SET WITHOUT CLUSTER on both hypertables and chunks. Commands on hypertables get passed down to chunks. e947c6be3a4d0758352535ea542d2fa158d2e6af,Matvey Arye,Sun Jul 22 13:43:37 2018 -0400,Improve handling of column settings,This PR adds better handling for the following commands: * ALTER TABLE ... ALTER COLUMN ... SET (attribute_name = value) * ALTER TABLE ... ALTER COLUMN ... RESET (attribute_name) * ALTER TABLE ... ALTER COLUMN ... SET STATISTICS * ALTER TABLE ... ALTER COLUMN ... SET STORAGE For each of the above commands the associated settings are not properly propagated to existing chunks and new chunks are created with the same settings as the hypertable. We also now allow these commands to be run on chunks. 6d1c4bc444943f2de331b2012bd96fe0a704e8c5,Matvey Arye,Thu Jul 19 15:15:17 2018 -0400,Test GENERATED AS IDENTITY Postgres commands.,PG 10 added the GENERATED AS IDENTITY feature. This PR adds tests to make sure it works with TimescaleDB. fc4957b90c841bcad2c43bf6ab662123fec59218,Matvey Arye,Thu Jul 19 12:09:26 2018 -0400,Update statistics on parent table when doing ANALYZE,"Previously, when running a VACUUM ANALYZE or ANALYZE on a hypertable the statics on the parent table (hypertable) were not correctly updated. This fixes that problem and adds tests. " 82942bf04ef9a8b9043f458fdd0df2ddc54ef914,Matvey Arye,Mon Jul 23 11:31:32 2018 -0400,Enable backwards compatibility for loader for 0.9.0 and 0.9.1,Old versions used a GUC to determine whether the loader was present. This PR sets that GUC in the loader so that those versions once again work with the new loader. c810be99efc47b06e0bf0946687b4accb8af3324,Rob Kiefer,Fri Jul 20 15:35:50 2018 -0400,Remove unused files from scripts/,"Both setup-db.sh and sql/setup_sample_hypertable.psql don't appear to be used anywhere in our repo or mentioned in the docs. We have code snippets and guides to replace them, so they are no longer needed. " 14967a03f64b317b4891c1e15f9ac36b44cad415,Matvey Arye,Tue Jul 17 18:01:38 2018 -0400,Fix PG version in ABI tests, eb2e262619f951aa3700c5fc0e0f11aa501886c8,Matvey Arye,Tue Jul 10 16:29:18 2018 -0400,Improve version checking,This PR adds a load-time check to the versioned extension for the expected Postgres versions. This better handles the case where the extension is distributed as a binary and compiled on a different Postgres version than the one it is running on. We also change the versioning to require: PG >= 9.6.3 to avoid issues with missing functions in previous versions OR PG >= 10.2 to avoid issues with ABI incompatibility at PG 10.0 and 10.1 d967bce27553820b3bfec2bd3aa7e5c42c4af351,Rob Kiefer,Fri Jul 13 15:02:32 2018 -0400,Prepare the repo for next development cycle, f06f853f891cc3bf4f58f273f8a0d2a6a54a9f0d,Rob Kiefer,Wed Jul 11 14:23:02 2018 -0400,Release 0.10.1, 8b58500d766709853e411393f54c575252b8543f,David Kohn,Tue Jul 10 17:00:41 2018 -0400,"Fix bug where dropping triggers caused dangling references in pg_depend, disallow disabling triggers on hypertables", 745b8ab95df22f98ac342bd21f7329ef86483bd5,Amy Tai,Tue Jul 10 16:55:31 2018 -0400,"Fixing CustomScan pruning whenever the subplan is NOT of a Scan type. This bug manifested in LATERAL joins that have a time restriction and a constraint referencing the upper-level plan. As an optimization, also added a special case when the subplan is of type T_Result, which enables chunk pruning in this special case.", 472bf6a298b3fbddc169a55bc48e367a3b0653a1,Amy Tai,Thu Jul 5 16:35:14 2018 -0400,Add runtime error for pgindent if objdump is not installed, 67a8a41e228a7417fb3438e85a542c231170e0e8,Rob Kiefer,Tue Jul 10 20:55:31 2018 -0400,Make chunk identifiers formatting safe using format, 41af6ff3ae0412fd768f32309581c9e509eb7987,Rob Kiefer,Mon Jul 9 14:07:59 2018 -0400,Fix misreported toast_size in chunk_relation_size funcs,A bug in the SQL for getting the size of chunks would use the TOAST size of the main/dummy table as the toast size for the chunks rather than each chunks' own toast size. 2ec065b53823e50dd1ac1d7cf925ae5f90e293ea,Matvey Arye,Tue Jul 10 17:31:59 2018 -0400,Fix formatting to comply with pgindent,"This PR fixes all the formatting to be inline with the latest version of pgindent. Since pgindent does not like variables named `type`, those have been appropriately renamed. " 4f2f1a6eb7ae5689883987103e3662edb41c49c5,Mike Futerko,Sat Jun 30 14:54:04 2018 +0300,Update the error messages to conform with the style guide; Fix tests,An attempt to unify the error messages to conform with the PostgreSQL error messages style guide. See the link below: https://www.postgresql.org/docs/current/static/error-style-guide.html 3c28f65f44d8ca7b58509ee8ad5699786dc02b79,Erik Nordström,Thu Jun 28 13:21:42 2018 +0200,Release cache pin memory,"Cache pins are allocated on the CacheMemoryContext in order to survive subtransactions, and, optionally, main transactions (in case of, e.g., clustering or vaccuming). However, cache pins that are released also needs to free memory on the CacheMemoryContext in order to avoid leaking memory in a session. This change ensures cache pin memory is freed when a cache pin is released. It also allocates cache pins on a separate memory context, which is a child of the CacheMemoryContext. This separate memory context makes it easier to track the memory used by cache pins and also release it when necessary. " abe76fc70a8a1f28e9084d26acc757e6f49b9e5f,Erik Nordström,Fri Jun 29 16:07:35 2018 +0200,Add support for changing chunk schema and name,"Previously, chunks could be renamed and have their schema changed, but the metadata in the TimescaleDB catalog was not updated in a corresponding way. Further, renaming a chunk column was possible, which could break functionality on the hypertable. Then catalog metadata is now properly updated on name and schema changes applied to chunks. Renaming chunk columns have been blocked with an informational error message. " 6e193c6ec99d8cb8e0ad406df3f56615807eb4f0,Michael J. Freedman,Thu Jun 28 11:45:03 2018 -0400,"Add contributions section in README, error messages to style guide", 106adfb73530082bde8258be106f07d6962f46d1,Rob Kiefer,Thu Jun 28 15:00:30 2018 -0400,Prepare the repo for next development cycle, 5be8fb5301e8d7f81958c2a32c39e9fe842e6868,Rob Kiefer,Mon Jun 25 16:53:51 2018 -0400,Release 0.10.0, 12bc1175f84f474dd0ea5f2df1dd082a41734773,Erik Nordström,Tue Jun 26 11:06:39 2018 +0200,Fix static analyzer warning when checking for index attributes,"This fixes a static analyzer warning about using an unititalized pointer. The analyzer doesn't realize that elog() will generate an exception, so that the unitialized NULL check will never occur. This change will clarify the code and silence the static analyzer. " 7d9f49b4c08af7bf8623d3f6f1e18c1984e4c0cf,Erik Nordström,Tue Jun 26 10:58:05 2018 +0200,Fix missing NULL check when creating default indexes,This fixes a missing NULL check when creating default indexes that was caught by a static analyzer. ff1ab457e747ce936044f401c47812ff715fe021,Matvey Arye,Sat Jun 23 11:21:57 2018 -0400,Bump minimum version for ABI test to 9.6.2,We need the function `convert_tuples_by_name_map` which only appears in 9.6.2. 2e1f3b9fd0db4efad42ca04f4b9e28f34d70c344,Erik Nordström,Fri Apr 20 11:24:06 2018 +0200,Improve memory allocation during cache lookups,"Previously, cache lookups were run on the cache's memory context. While simple, this risked allocating transient (work) data on that memory context, e.g., when scanning for new cache data during cache misses. This change makes scan functions take a memory context, which the found data should be allocated on. All other data is allocated on the current memory (typically the transaction's memory context). With this functionality, a cache can pass its memory context to the scan, thus avoiding taking on unnecessary memory allocations. " ca6e5ef5a8e3099611e9db388e85239d63754e6a,Matvey Arye,Mon Jun 11 16:57:23 2018 -0400,Fix upserts on altered tables.,"Previously, upserts (ON CONFLICT) clauses did not work well on tables where the hypertable attribute numbers did not match chunk attribute numbers. This is common if the hypertable has dropped columns or there were other alter commands run on the hypertable before chunks were created. This PR fixes the projection of the returning clause as well as the update clauses. It also fixes the where clause for ON CONFLICT UPDATE. These fixes are mostly about mapping the attribute numbers from the hypertable attnos->chunk attnos. Some slot tupleDesc also needed to be changed to the tupleDesc of the chunk. Note that because of the limitations in PG 9.6 we had to copy over some expressions from the ModifyTable plan node inside the chunk dispatch. These original expressions are irrecoverable from the ModifyTableState node or the ProjectionInfo structs in 9.6. " 2de6b02c16592d465dc0db6a7274ba144cdf4519,Matvey Arye,Thu Mar 29 13:56:11 2018 -0400,Add optimization to use HashAggregate more often,"This optimization adds a HashAggregate plan to many group by queries. In plain postgres, many time-series queries will not use the hash aggregate because the planner will incorrectly assume that the number of rows is much larger than it actually is and will use the less efficient GroupAggregate instead of a HashAggregate to prevent running out of memory. The planner will assume a large number of rows because the statistics planner for grouping assumes that the number of distinct items produced by a function is the same as the number of distinct items going in. This is not true for functions like time_bucket and date_trunc. This optimization fixes the statistics and add the HashAggregate plan if appropriate. The statistics now rely on evaluating the spread of a variable and dividing it by the interval in the time_bucket or date_trunc. This is still an overestimate of the total number of groups but is better than before. A further improvement on this will be to evaluate the quals (WHERE clauses) on the query to try to derive a tighter spread on the variable. This is left to a future optimization. " d1a05baac54125a0aede3a541a9229f38452a96d,Narek Galstyan,Fri Jun 15 22:28:17 2018 +0000,integrate code coverage through codecov.io, cb9004c5466c91b03774e3aef5cf40ca9e959afb,Rob Kiefer,Tue Jun 19 15:02:09 2018 -0400,Remove include/ from non-Windows/FreeBSD builds,Windows and FreeBSD seem to need include/ to compile correctly but on Ubuntu systems the header files in include/server/ and include/ frequently get out of sync causing compile errors. This change therefore only keeps that library included for Windows and FreeBSD. f765530e4b4a399f082414e86f74d8681d46cf9b,Matvey Arye,Tue Jun 19 15:58:24 2018 -0400,Remove superfluous parallel check,This check should not be necessary and causes problems under Windows builds. So it is removed. 4b4211fe94edc16da99120403dcaf6ea01c46a09,Narek Galstyan,Tue Jun 12 14:15:21 2018 -0400,Fix some external functions when setting a custom schema,Make sure internal references to timescale functions use the correct schema to refer to these functions. Fixes #554 3778bd733bc5e5acf52a1f72bab1995f442b4ebd,Michael J. Freedman,Thu Jun 7 14:14:55 2018 -0400,Updated NOTICE from 440 Labs to Timescale, ad34d6f8f7d8979545a4321ec7089525891053f7,Matvey Arye,Thu May 17 14:01:38 2018 -0400,Bump pg version to test to 9.6.6,We hit a bug in 9.6.5 fixed in 9.6.6 by commit 77cd0dc. Also changed extension is transitioning check to not palloc anything. This is more efficient and probably has slightly less side-effects on bugs like this. b7257fc8f483475382019cadcc7a75fae0b72f0a,Matvey Arye,Tue Apr 17 12:12:28 2018 -0400,Optimize planning times when hypertables have many chunks,"This planner optimization reduces planning times when a hypertable has many chunks. It does this by expanding hypertable chunks manually, eliding the `expand_inherited_tables` logic used by PG. Slow planning time were previously seen because `expand_inherited_tables` expands all chunks of a hypertable, without regard to constraints present in the query. Then, `get_relation_info` is the called on all chunks before constraint exclusion. Getting the statistics an many chunks ends up being expensive because RelationGetNumberOfBlocks has to open the file for each relation. This gets even worse under high concurrency. This logic solves this by expanding only the chunks needed to fulfil the query instead of all chunks. In effect, it moves chunk exclusion up in the planning process. But, we actually don't use constraint exclusion here, but rather a variant of range exclusion implemented by HypertableRestrictInfo. " c660fcd8ff4eafae90b0b2735996c1fdb6bf1391,Rob Kiefer,Wed May 16 11:46:05 2018 -0400,Add hypertable_approximate_row_count convenience function,Getting an approximate row count for a hypertable involves getting estimates for all of its chunks rather than just looking up a single value in the catalog tables. This PR provides a convenience function for doing the JOINs/summing. 9ce1576982ca2d96a4b936a58117e78d0a758bfe,Erik Nordström,Mon May 7 16:12:15 2018 +0200,Fix a compilation issue on pre 9.6.3 versions,"Macros that provide type assertion, like castNode() and lfirst_node() were introduced in PG 9.6.3 and cannot be used if we want to support the entire 9.6 series of releases. This change fixes usage of such macros that was introduced as part of the 0.9.2 release of TimescaleDB. " 02a8fc7e1adfe3aff66a85d603adccf65aca5986,Rob Kiefer,Sun May 6 21:34:11 2018 -0400,Prepare the repo for next development cycle, ff3ab04bd972979e89b99bb4878ceb7c1ff075d1,Rob Kiefer,Thu May 3 19:00:20 2018 -0400,Release 0.9.2, 256b39491c78e9c356a96f797f88eca44ce7abbd,Erik Nordström,Fri May 4 08:17:23 2018 +0200,Fix parsing of GrantRoleStmt,This corrects the way grantees are read from a GrantRoleStmt. b78953b18109e8de5553f1d92848831af1c89b88,Erik Nordström,Fri May 4 08:15:42 2018 +0200,Fix datum conversion typo,"Fixes a DatumGetInt64 that should really be a Int64GetDatum. Caused segfaults on 32-bit (e.g., ARM) platforms. " c7283ef00fd02769971ea4399c59c0385ed36b03,Matvey Arye,Thu May 3 13:21:08 2018 -0400,Fix bug with extension loader when DISCARD ALL is executed,"Previously, the GUC to let the versioned-extension know that the loader was present was incorrectly reset during DISCARD ALL. This caused newly minted parallel workers to throw an error about the extension not being preloaded since it did not know that a loader was, in fact, preloaded. This PR fixes this issue by changing the GUC to a rendezvous variable. It also disables the loader in parallel workers since the loading should have been handled in the parallel leader and the worker start-up logic. " fe20e489a989a07fe0ba689fdbe2021bef308989,Erik Nordström,Wed May 2 11:37:46 2018 +0200,Fix chunk creation with user that lacks TRIGGER permission,"Previously, automatic chunk creation on INSERT failed due to lack of permissions when the hypertable had triggers that needed to be replicated to the new chunk. This happened because creating triggers on tables requires TRIGGER permission, and the internal code used CreateTrigger() that performs permissions checks. Thus, if the user inserting tuples into the hypertable only had INSERT permissions, the insert would fail whenever a new chunk was created. This change fixes the issue by temporarily assuming the role of the hypertable owner when executing CreateTrigger() on the new chunk. " 43f812c158d1cee1af25dd2708e5c4e2d90b750a,Rob Kiefer,Mon Mar 26 17:03:25 2018 -0400,Prepare the repo for next development cycle, 1fbb7e7466684fd64a50ed93a2bb5334817abbf3,Rob Kiefer,Mon Mar 26 10:58:35 2018 -0400,Release 0.9.1, 40ce03760a0ad2d9136887945d995309a809ba5c,David Kohn,Wed Mar 21 14:52:50 2018 -0400,Fix crash on explain analyze with insert cte,"Discovered a crash when doing an explain analyze on a cte containing an insert that did not reference the cte in the select statement. This fixes by copying the eref from the parent hypertable to the chunk so that columns can be described. It will only copy the eref from the hypertable if there is a range table entry available. Also modify copy so it actually sets a dummy variable as the RTE index rather than 1, which is a valid RTE index, even if there is no RTE for the hypertable. " 8378beb236d67c69e5ae1642b1a87715a02fe08e,Matvey Arye,Wed Mar 21 09:50:29 2018 -0400,Enable hypertable inserts within CTEs,"Previously, the chunk dispatch node used arbiter index and other information from the top-level Query node. This did not work with CTEs (and probably subqueries) because the info for this insert was not at the top-level. This commit changes the logic to use that same info from the parent ModifyTable node (and ModifyTableState). This commit also changes the logic to translate the arbiter indexes from the hypertable index to the chunk index directly using our catalog tables instead of re-inferring the index on the chunk. " bdfda75d0f79ef4459b3ae5893d7485d3e4bab86,Matvey Arye,Thu Mar 22 18:29:58 2018 -0400,Fix double-loading of extension,"Previously, there was a case involving cache invalidation when the extension could be preloaded. Namely the following happened: 1) extension_check (outer) was called and found loaded = true. 2) While the extension_check was checking the state of the extension a cache invalidation callback happened. 3) The cache invalidation callback called extension_check (inner). 4) The inner extension_check loads the extension and sets loaded = true. 5) The outer extension_check also loads the extension after the cache invalidation callback returns. This happens because it never re-checks the value of loaded. This commit adds another check of loaded just before it is changed to prevent this situation. " 212bc6081987dbc410fd2ad27e872ae161ed8b5d,Rob Kiefer,Mon Mar 12 17:16:46 2018 -0400,Remove irrelevant 'fatal not a .git repository' error,"Building can still succeed even if not in a git context so this ""fatal"" error should be hidden. " 01ea77e6ef20ee95b8886982bbd856cdf50c7d73,Matvey Arye,Mon Mar 19 07:24:20 2018 -0400,Fix EXPLAIN output for ConstraintAwareAppend inside CTE,The explain output previously relied on the fact that the estate->rtable was indexed the same way as the simple_rte_array during planning. This is not always true when using CTEs. This commit instead passes the oid of the hypertable to the EXPLAIN directly. 9f13b8de65cbb0cad31e6273d048ea1ce7e68ee7,Matvey Arye,Thu Mar 15 12:16:37 2018 -0400,Reword error msgs in loader, fc056377923c3e827472e64abbf17245733b4a7c,Matvey Arye,Wed Mar 14 17:01:23 2018 -0400,Add no preload error to versioned library.,"Previously, only the loader would complain if it was dynamically loaded after preload. This PR adds a FATAL error message if the versioned .so is not preloaded AND the loader has not been previously loaded. This is a common case if the user forgets to edit shared_preload_libraries to add the loader. This PR also fixes the error in cases when it's encountered by a non-superuser. " 38f8e0c9d0c4adf9ac160c27588e8cef40da1c67,Matvey Arye,Wed Feb 14 18:08:54 2018 -0500,Add ABI compatibility tests,"Test the ABI compatibility of postgres across minor version updates wrt timescale. Namely, compile the extension on an early minor version of postgres and test it on the latest snapshot of the major branch. " 744ca0920c4806ea1175950dbad40273b3691e36,Matvey Arye,Tue Mar 13 17:45:26 2018 -0400,Fix Cache Pinning for Subtxns,"Previously, all Subtxn aborts released all cache pins. This is wrong because that can release pins that are still in use by higher-level subtxns and top-level txns. Now, we release only pins corresponding to the appropriate subtxn. That means that we now track the subtxn a pin was created in. " 39010db9f34639c699f6101b018c078905f87741,Matvey Arye,Wed Mar 7 15:40:39 2018 -0500,Move more drops into event trigger system,"This PR moves table, schema, and trigger drop handling into the event trigger system. The event trigger system is a more reliable method of intercepting object drops especially as they can CASCADE via other object drops. This PR also adds a test for DROP OWNED which was previously broken. " 088e71e844d00920e68af3e7eca0e1e20d8302b0,Rob Kiefer,Wed Mar 7 15:55:25 2018 -0500,Update all the docs links in README, e12558530e4d2eab764fcbac6901382c8199d13c,Matvey Arye,Wed Feb 28 13:32:56 2018 +0530,Add second FK to update test,This commit adds a secondary FK to the update test. This FK points to the same metadata table as the previous FK. This case is added because it has caused problems in the past. fc36699a43a2c7b61907b82c1cd820a1380dd03a,Erik Nordström,Wed Mar 7 09:51:15 2018 +0100,Do not fail add_dimension() on non-empty table with 'if_not_exists',"This fixes an issue with the `if_not_exists` option to add_dimension() that caused the command to fail on a non-empty table. When this option is set, the command should only fail if the dimension does not exists and the table is non-empty. " 63230c4c428d2b07241f966cf0f6d4aef0acc757,Rob Kiefer,Mon Mar 5 23:02:16 2018 -0500,Prepare the repo for next development cycle, a4caacbc82635aa94bbaa2647fac31ff0565e545,Rob Kiefer,Sun Mar 4 12:40:15 2018 -0500,Re-release 0.9.0, b873a970f73acfd40ebe6d2f795620c07bda8c30,Rob Kiefer,Thu Mar 1 16:46:23 2018 -0500,Make pg_dump regression test more robust,"Prior to PostgreSQL 10.3, the output for a trigger function did not include the implied 'public' schema in its output when outputting the function. In 10.3 this was changed causing the regression test to fail. To make this more robust, the table is now put into an explicit schema so the test does not rely on the inconsistent behavior across versions as to printing out 'public'. " 4672719e380c8610138044cfa0aa13ba292f2cd9,Matvey Arye,Sat Mar 3 10:13:02 2018 -0500,Fix error in handling of RESET ALL, 42811e32c13f0f8fcfb342a56af9d78324ac0745,Jukka Raimovaara,Thu Mar 1 19:42:53 2018 +0200,Fix typo in CMakeLists.txt, ddd61bc86d38b84092243974d63bacf8a6264c60,Erik Nordström,Wed Feb 28 16:22:54 2018 +0100,Fix update to 0.9.0,This fixes a bug in the upgrade script to 0.9.0 that was caused by a bad query to delete orphaned dimension slices. 7883890a2d659dabf4b5c7c71b81965a0bc06994,Erik Nordström,Tue Feb 27 22:36:10 2018 +0100,Fix update scripts for version 0.6.1,"A fix for updates of version 0.6.1 was lost in the previous PR that refactored the update process. This change adds back that fix. The build process for update scripts now also supports versioned ""origin"" modfiles, which are included only in the update scripts that origins in a particular version given by the origin modfile's name. Origin modfiles make it possible to add fixes that should be included only for the version upgrading from. " 765574270544386686898c3768556bf5ca2e9981,Rob Kiefer,Mon Feb 26 11:50:36 2018 -0500,Update README to emphasize checking out tagged commit,"Upgrades from non-tagged commits are not supported, so building from source should emphasize that for those who want to use TimescaleDB instead of develop for TimescaleDB. " 939930881f5e7df7a0c41170f2eb4dd7dfb659bc,Erik Nordström,Fri Feb 23 12:57:07 2018 +0100,Refactor/simplify update scripts and build process,"Update scripts have so far been built by concatenating all the historical changes and code from previous versions, leading to bloated update scripts, complicated script build process, and the need to keep old C-functions in compat.c since those functions are referenced during updates. This change greatly simplifies the way update scripts are built, significantly reducing the size of update scripts (to basically only the changeset + current code), and removing the need for compat.c. A few principles of building scripts must be followed going forward, as discussed in sql/updates/README.md. " 7bd9437aba0069f26b086f9092fdc8a5d884fb7f,Erik Nordström,Thu Feb 22 21:36:15 2018 +0100,Do not generate versioned copies of SQL files,"This removes the version suffix from SQL files that are copied from the source directory to the build directory during the build process. Versioning the files in this step serves no real purpose and only tends to clutter up the build dir with extra files every time the version is bumped, requiring manual cleanup. " db39eddf2e3b4835d4d052631a2ec4afad230026,Erik Nordström,Wed Feb 21 17:53:43 2018 +0100,Remove unused cache invalidation triggers,"The cache invalidation triggers on our catalog tables aren't used anymore as all modifications to catalog tables happen using the C API, which doesn't invoke triggers and has its own cache invalidation functionality. " 905e1c58e21bc833ddf84e9ea0853fd427b8090e,Rob Kiefer,Wed Feb 21 15:26:25 2018 -0500,Update test suite to check upgradability for all versions,Previously we just checked that we could upgrade from 0.1.0 until latest but this missed some issues upgrading from intermediate versions to latest. Now we check that all released versions can successfully update. 42c2f7b950acb12a3ea2cd9f019e2de08ebe7672,Matvey Arye,Thu Feb 22 02:17:51 2018 +0530,Add support for generating update scripts for skipped versions, 93c76e75e17a0e8a84e365c4560d8e19b9c02b2e,Matvey Arye,Wed Feb 21 02:41:12 2018 +0530,Change trigger behavior on upgrades,"For multi-version upgrades it is necessary to change the location of trigger functions before doing anything else in upgrade scripts. Otherwise, it is possible to trigger an even before you change the location of the functions, which would load the old shared library and break the system. This commit also fixes `/sql/timescaledb--0.8.0--0.9.0.sql` to come from the release build. " 0e79df4ac3bc831a73f71d6e072e2446494f1e1d,Erik Nordström,Mon Feb 19 23:01:30 2018 +0100,Fix handling of custom SQL-based partitioning functions,"Previously, when the FmgrInfo was initialized for partitioning functions, the type information was cached in the fn_extra field. However, the contract for fn_extra is that it is to be set by the called function and not prior to its invokation. Setting fn_extra prior to function invokation is an issue for custom partitioning functions implemented in SQL, as the SQL call manager expects to use fn_extra for its own cache. This change avoids setting fn_extra prior to function invokation, and instead information is cached in fn_extra within our provided partitioning functions. The native partitioning functions now also support nested functions, i.e., the input to the function is the output of another function. " 22112e313716173b4c537d176a849f418b8cd725,Rob Kiefer,Mon Feb 19 14:16:48 2018 -0500,"Revert 0.9.0, revision needed", 1c03784ceb6e19f6251407f9060a2944d74aa927,Rob Kiefer,Thu Feb 15 00:59:00 2018 -0600,Release 0.9.0, b31c8dbd1e01cf152e6a03163ce6a5d6eb49afbd,Matvey Arye,Fri Feb 16 09:05:48 2018 -0500,Address coverity complaints, f13969e3a2fde65784a5d6970cc5b2cc46b465de,Matvey Arye,Thu Feb 15 19:03:29 2018 -0500,Fix possible memory safety issue and squash valgrind error., ef744916a8573b3345bf465ee966a8ffc0669f5c,Erik Nordström,Mon Feb 5 21:44:56 2018 +0100,Migrate table data when creating a hypertable,"Tables can now hold existing data, which is optionally migrated from the main table to chunks when create_hypertable() is called. The data migration is similar to the COPY path, with the single difference that the inserted/copied tuples come from an existing table instead of being read from a file. After the data has been migrated, the main table is truncated. One potential downside of this approach is that all of this happens in a single transaction, which means that the table is blocked while migration is ongoing, preventing inserts by other transactions. " 1d265abe16da125a4e8c0981ae0d2f3c2d377224,Matvey Arye,Thu Feb 15 09:17:43 2018 -0500,Fix regex for version reported by pg_config,Sometimes pg_config reports versions like `PostgreSQL 10.2 (Ubuntu 10.2-1.pgdg16.04+1)`. Thus the regex is adjusted to allow for suffixes after the postgres version number. 26965826f425486579d6b9e714d64c7662403f65,Matvey Arye,Thu Feb 1 19:25:46 2018 -0500,Move index and constraints drop handling to event trigger,This Fixes at least two bugs: 1) A drop of a referenced table used to drop the associated FK constraint but not the metadata associated with the constraint. Fixes #43. 2) A drop of a column removed any indexes associated with the column but not the metadata associated with the index. d6baccb9d736f7d6b89f27a8b338819567f4e8ed,Erik Nordström,Sat Jan 27 00:47:24 2018 +0100,"Improve tablespace handling, including blocks for DROP and REVOKE","This change improves the handling of tablespaces as follows: - Add if_not_attached / if_attached options to attach_tablespace() and detach_tablespace(), respectively - Block DROP tablespace if it is still attached to a table - Block REVOKE if it means the table owner no longer has CREATE permissions on an attached tablespace - Make error messages follow the PostgreSQL style guide " b9a6f890a1e30b8bffd1f6b52afb5f93bef30b66,Erik Nordström,Tue Jan 30 12:41:27 2018 +0100,Handle DROP SCHEMA for hypertable and chunk schemas,"Dropping a schema that a hypertable depends on should clean up dependent metadata. There are two schemas that matter for hypertables: the hypertable's schema and the associated schema where chunks are stored. This change deals with the above as follows: - If the hypertable schema is dropped, the hypertable and all chunks should be deleted as well, including metadata. - If an associated schema is dropped, the hypertables that use that associated schema will have their associated schemas reset to the internal schema. - Even if no hypertable currently uses the dropped schema as their associated schema, there might be chunks that reside in the dropped schema (e.g., if the associated schema was changed for their hypertables), so those chunks should have the metadata deleted. " b534a5a7d9d21aafd0f7a905fd364e460445b834,Matvey Arye,Tue Jan 2 13:51:36 2018 -0500,Add test case for adding metadata entries automatically,"This test adds a case where we use a trigger to automatically populate a metadata table. Such uses are common in IOT where, for example, you want to keep metadata associated with devices and you want new devices to be auto-created. " f48bbe0020148cb28b31c02cecd89d94977d7da0,Rob Kiefer,Thu Feb 1 15:45:06 2018 -0500,Add AppVeyor badge for Windows build status, 425b17b9bb3feaea92afd4a5407ce9befcfbd1b2,Rob Kiefer,Thu Feb 1 14:54:17 2018 -0500,Fix an unexported GUC for Windows,"The compiler does not seem to like when I use the msvc enter/exit guards for utils/guc.h, so the alternative is to grab the value via GetConfigOptionByName. " 6adce4cbd8bb3aaab8f1f8be2b47d2e474083492,Erik Nordström,Wed Jan 31 00:25:59 2018 +0100,Handle TRUNCATE without upcall and handle ONLY modifier,"This change refactors the handling of TRUNCATE so that it is performed directly in process utility without doing an upcall to PL/pgSQL. It also adds handling for the ONLY modifier to TRUNCATE, which shouldn't work on a hypertable. TRUNCATE now generates an error if TRUNCATE ONLY is used on a hypertable. " b7ebe06f2ea14b30aae12bb70cbb35b8fcd71d1a,Erik Nordström,Wed Jan 31 00:47:28 2018 +0100,Handle change owner without upcall,Changing the owner of a hypertable is now handled entirely in the process utility hook without doing an upcall to PL/pgSQL. 7fa6a0416eaf2fcc2895505df50abd593c60af94,Rob Kiefer,Mon Jan 29 14:17:42 2018 -0500,Fix several Windows compile errors and warnings,"Previously stdint.h was not included on Windows so INT16_MAX and friends were not defined. Additionally, having tablespace_attach with PG_FUNCTION_ARGS in the header file caused issues during linking, so a direct call version of the function is now exported for others to use instead of the PG_FUNCTION_ARGS version. Two minor warnings regarding not having a return in all cases are also addressed. " 71b11240a71115b647fb150ad42c298b17fc1866,Erik Nordström,Fri Jan 26 16:03:45 2018 +0100,Delete orphaned dimension slices,"When chunks are deleted, dimension slices can be orphaned, i.e., there are no chunks or chunk constraints that reference such slices. This change ensures that, when chunks are deleted, orphaned slices are also deleted. " fa19a54a886e62662c5ff1a006bf497a257a3dac,Erik Nordström,Thu Jan 25 13:04:11 2018 +0100,Handle deletes on metadata objects via native catalog API,"Deletes on metadata in the TimescaleDB catalog has so far been a mix of native deletes using the C-based catalog API and SQL-based DELETE statements that CASCADEs. This mixed environment is confusing, and SQL-based DELETEs do not consistently clean up objects that are related to the deleted metadata. This change moves towards A C-based API for deletes that consistently deletes also the dependent objects (such as indexes, tables and constraints). Ideally, we should prohobit direct manipulation of catalog tables using SQL statements to avoid ending up in a bad state. Once all catalog manipulations happend via the native API, we can also remove the cache invalidation triggers on the catalog tables. " 6e011d12fbf919d218bdb8c21b419c9ab77ec13e,Erik Nordström,Fri Jan 19 10:32:46 2018 +0100,Refactor hypertable-related API functions,"This is a continuation of prior efforts to refactor API functions in C to: - improve usage of proper error codes - use error messages that better conform with the PostgreSQL standard. - improve security by avoiding that lots of code run under SECURITY DEFINER - move towards doing all metadata updates using a consistent catalog API Most importantly, `create_hypertable()` has been refactored in C, which simplifies a lot of code that previously required upcalls/downcalls between C code and plpgsql code, or duplicated functionality between the two environments. " 5afd39a6ba741fef17eb20e7fa940c360473cfad,Erik Nordström,Thu Jan 25 21:47:20 2018 +0100,Fix locking for serializing chunk creation,"Chunk creation needs to be serialized in order to avoid having multiple processes trying to create the same chunk and causing conflicts. This serialization didn't work as expected, because a lock on the chunk catalog table that implemented this serialization was prematurely released. This change fixes that issue and also changes the serialization to happen around a lock on the chunk's parent table (the main table) instead. This change should allow multiple processes to simultaneously create chunks for different hypertables. " 6dd2c46c94f93f162328ef95cb8fb74a7a9ec099,Floris van Nee,Thu Jan 25 16:41:57 2018 +0100,Add check for null in ca_append_rescan to prevent segfault, 71962b86ec24c1a3ae0c257b1be5a473e22658b1,Erik Nordström,Fri Jan 19 10:32:46 2018 +0100,Refactor dimension-related API functions,The functions for adding and updating dimensions have been refactored in C to: - improve usage of proper error codes - make messages that better conform with the PostgreSQL standard. - improve security by avoiding that lots of code run under SECURITY DEFINER A new if_not_exists option has also been added to add_dimension() and a the number of partitions can now be set using the new set_number_partitions() function. A bug in the validation of smallint time intervals has been fixed. The previous code didn't check for intervals > 0 and smallint intervals accepted values up to UINT16_MAX instead of INT16_MAX. cc254a917ab84c5224b41c025b56bad7472b66b5,Erik Nordström,Mon Jan 15 22:58:24 2018 +0100,Fix CREATE EXTENSION IF NOT EXISTS and error messages,"Issuing ""CREATE EXTENSION IF NOT EXISTS timescaledb;"" generates an error if the extension is already loaded. This change makes this command work as expected, with a NOTICE. Error messages related to CREATE EXTENSION have also been updated to better adhere to the official PostgreSQL style guide and also returns the same or similar error codes and messages as regular PostgreSQL. " d135256ed72d88f731aebacc035a6191c707bedf,Erik Nordström,Mon Jan 15 11:48:41 2018 +0100,Spread chunk indexes across tablespaces like chunks,"Currently, chunk indexes are always created in the tablespace of the index on the main table (which could be none/default one), even if the chunks themselves are created in different tablespaces. This is problematic in a multi-disk setting where each disk is a separate tablespace where chunks are placed. The chunk indexes might exhaust the space on the common (often default tablespace) which might not have a lot of disk space. This also prohibits the database, including index storage to grow by adding new tablespaces. Instead, chunk indexes are now created in the ""next"" tablespace after that of their chunks to both spread indexes across tablespaces and avoid colocating indexes with their chunks (for I/O throughput reasons). To optionally avoid this spreading, one can pin chunk indexes to a specific tablespace by setting an explicit tablespace on a main table index. " e85721a8edb49b8c104364f13bbc92a3367f38c3,Matvey Arye,Thu Jan 11 13:22:32 2018 -0500,Block ONLY hypertable on all ALTER TABLE commands.,ONLY does not make semantic sense for hypertables. This commit blocks the use of this modifier. b6e2780460cfd9d9c6159505110c5513182b56a0,Erik Nordström,Thu Jan 18 12:13:46 2018 +0100,Apply new indentation (pgindent) used in PostgreSQL 10,"Source code indentation has been updated in PostgreSQL 10 to fix a number of issues. This update applies this new indentation to the entire code base. The new indentation requires a new version of pg_bsd_indent, which can be found here: https://git.postgresql.org/git/pg_bsd_indent.git " 78d36b52d4bae379e5736d79bed77ec38b6047c7,Matvey Arye,Tue Jan 2 11:15:56 2018 -0500,Handle subtxn for cache pinning,"This commit adds logic for cache pinning to handle subtxn. It also makes it easier to find cache pinning leaks. Finally, it fixes handling of cross-commit operations like VACUUM and CLUSTER. Previously, such operations incorrectly released that cache pin on the first commit even though the object was used after that. " 26ef77fd25d3db8962707a48c7743085240259f9,Matvey Arye,Sat Dec 30 22:41:37 2017 -0500,Add subtxn abort logic to process_utility.c, 25f32844668ad27095c0ab240959f28ffaa090da,Matvey Arye,Sat Dec 30 15:42:16 2017 -0500,Handle cache invalidation during subtxn rollback,Add logic to invalidate the cache during subtxn rollback. Fixes #376 264956f0c1f8c99ff23a5507a3a6b7347d767e86,Matvey Arye,Wed Jan 10 09:59:11 2018 -0500,Block DROP NOT NULL on time-partitioned columns.,Time-partitioning cannot handle NULL values. ad7d361418aec9aeaae839b6ed40341afe0b0732,Matvey Arye,Thu Jan 4 18:10:04 2018 -0500,Better accounting for number of items stored in a subspace,We add better accounting for number of items stored in a subspace to allow better pruning. Instead of pruning based on the number of dimension_slices in subsequent dimensions we now track number of total items in the subspace store and prune based on that. We add two GUC variables: 1) max_open_chunks_per_insert (default work_mem in bytes / 512. This assumes an entry is 512 bytes) 2) max_cached_chunks_per_hypertable (default 100). Maximum cached chunks per hypertable. 12f92ea1fa9d3eb7768c5316084c2139c4fe5d40,Matvey Arye,Wed Dec 20 12:52:01 2017 -0500,Improve speed of out-of-order inserts,"Previously, the cache in chunk_dispatch was limited to only hold the chunk_insert_state for the last time dimension as a consequence of logic in subspace_store. This has now been relaxed so that a chunk_dispatch holds the cache for any chunk_insert_states that it encounters. Logic for the hypertable chunk cache has not been changed. The rule that we should follow is to limit the subspace store size for caches that survive across commands. But caches within commands can be allowed to grow. " 87f055dd2c5b1b30dc3d9768c30abc260e10cb82,Matvey Arye,Mon Jan 8 18:08:53 2018 -0500,Add support for ALTER TABLE RENAME CONSTRAINT.,"Renaming constraints on hypertables is now supported. Using the ONLY option with RENAME CONSTRAINT is blocked for hypertables. Renaming constraints on chunks is also blocked. This commit also fixes a possible bug in chunk_constraint code. Previously, `chunk_constraint_add_from_tuple` used GETSTRUCT on the tuple to convert the tuple to `FormData_chunk_constraint`. This was incorrect since some fields can be NULL. Using GETSTRUCT for tables with NULLABLE fields is unsafe and indeed was incorrect· in this case. " 0b79156213133a117cbba4182bf6ed5cfc77667a,Rob Kiefer,Thu Dec 7 19:34:57 2017 -0500,Support developer function for git commit in Docker, a0e99baef36a223ff08e912f31bf6931eca3f6fe,Rob Kiefer,Mon Jan 8 15:05:07 2018 -0500,Add README instructions for installing files on Windows, b2f80a332cb2cd86e6cd7fa5ef6f8e2425957cc2,Rob Kiefer,Thu Jan 4 10:59:33 2018 -0500,Bring Windows bootstrap in line with Unix version.,"Flags are now passed on to CMake, the source directory is correctly resolved, and support for removing the old build directory added. Additionally this will limit the VS solution file to only having Release targets unless overriden with a flag. " c373d6b3593a8cdaea34bc3d76b42cc6811ac5f0,Matvey Arye,Tue Jan 2 12:45:13 2018 -0500,Remove executor which had logic for adjusting es_processed,"Previously, the executor code adjusted es_processed to set the number of tuples that have been processed in the completion tag. That was necessary since we had triggers that re-routed tuples to chunks and cancelled the inserts on the base table (thus those tuples were not counted in es_processed). We no longer use triggers and thus this logic was no longer used. In fact, this was mostly dead code because `additional_tuples` was always 0. " a9e18ae4f829d74da5a793e9e813c6410573672d,Matvey Arye,Sun Dec 10 18:23:52 2017 -0500,fix docker-run-tests, da8cc797a48da99003f08281839b813bec517dcf,Matvey Arye,Mon Sep 25 22:41:34 2017 -0400,Add support for multiple extension version in one pg instance,"This PR adds the ability to have multiple different versions of the timescaledb extension be used by different databases in the same PostgreSQL instance (server). This is accomplished by splitting this extension into two .so files. 1) timescaledb.so -- stuff under loader/. Really not a lot of code. This code MUST be backwards compatible in the future. 2) timescaledb-version.so (most of our code). Need not be backwards compatible. Timescaledb.so becomes a small stub which is preloaded and whose main reason for existing is to dynamically load the right timescaledb-version.so when the time comes. This change allows either of the above .so to be loaded in shared_preload_libraries. But timescaledb.so allows for multiple versions used on different databases in the same instance along with smoother upgrades. Using timescaledb-version.so allows for finer-grained control and lock-in and is appropriate in only a few production environments. This PR also adds version checking so that a clear failure message will be displayed if the .so version does not match the SQL extension version. To support multi-version functionality we changed the way SQL update scripts are generated. Previously, the system used a bunch of intermediate upgrade scripts. So with 3 versions, you would have an update script of 1--2, 2--3. But, this PR changes things so that we produce direct ""shortcut"" update files: 1--3, 2--3. This is done for 2 reasons: 1) Each of the update files should point to $libdir/timescaledb-current_version. Since you cannot guarantee that Previous .so for each intermediate version has been installed. 2) You don't want intermediate version updates installed without the .so. For example, if you have versions 1,2,3 and you are installing version 3, you want the upgrade files 1--3, 2--3 but not 1--2 because if you have 1--2 then a user could do ALTER EXTENSION timescaledb UPDATE TO 2. But the .so for version 2 may not be installed. In order to test this functionality, we add a mock extension version .so that we can test extension loading inside the regression framework. " e8eabf4278ece616aab9e168e6b40f3e1b6b84c6,Rob Kiefer,Wed Dec 20 12:10:25 2017 -0500,Add support for passing flags to cmake,"Changing things like the location of pg_config and other macros, additional cmake flags, etc require modifying bootstrap file. This PR instead passes any flags given to bootstrap onto cmake itself. " ca85ec4865892554705c314866cf77a937071f81,Matvey Arye,Tue Dec 19 11:22:18 2017 -0500,Improve updated test,"Adds a FK constraint, check constraint, and exclusion constraint to the update test. " 68faddca249e6299402f5981a5337362175e10ea,Erik Nordström,Thu Dec 14 11:40:24 2017 +0100,Make chunks inherit reloptions set on the hypertable,"When tables are created with storage parameters (reloptions), these should be inherited by chunks so that one can set options such as fillfactor and autovacuum settings on the hypertable. This change makes chunks inherit the reloptions set on a hypertable and allows altering options via the ALTER TABLE command. " 4df8f287a6bd21bc03165853acccb091999a3811,Erik Nordström,Thu Dec 7 12:42:42 2017 +0100,Add proper permissions handling for associated (chunk) schemas,"A hypertable's associated schema is used to create and store internal data tables (chunks). A hypertable creates tables in that schema, typically with full superuser permissions, regardless of whether the hypertable's owner or the current user have permissions for the schema. If the schema doesn't exist, the hypertable will create it when creating the first chunk, even though the user or table owner does not have permissions to create schemas in the database. This change adds proper permissions checks to create_hypertable() so that users cannot create hypertables with a custom associated schema unless they have the proper permissions on the schema or the database. Chunks are also no longer created with internal schema permissions if the associated schema is something different from the internal schema. " 21efcce95c22ff8ca4ac41d2eab0213a61b3be8c,Erik Nordström,Thu Nov 23 10:02:52 2017 +0100,Refactor chunk table creation and unify constraint handling,"This change is part of an effort to create a consistent way of dealing with metadata catalog updates, which is currently a mix of C API and INSERT/UPDATE/DELETE statements from SQL code. This mix makes catalog handling unnecessarily complex as there are multiple ways to update metadata, increasing the risk of security issues with publically exposed SQL functions. It also complicates things like cache invalidation, requiring different mechanisms for C and SQL code. Catalog updates from SQL code require triggers on metadata tables for cache invalidation that do not work with native catalog updates. The creation of chunks has been particularly messy in this regard, making the code hard to follow. Especially the handling of a chunk's constraints, where dimensional and other constraints were handled differently. With this change, constraint handling is now consistent across constraint types with a single API for updating metadata. Reduce memory usage for out-of-order inserts The chunk_result_relation_info should be put on the chunk memory context. This will cause the rri constraint expr to also go onto that context and be correctly freed when the chunk insert state is destroyed. " 513bae35db3af798453b61dc346bd3d2516da491,Rob Kiefer,Tue Dec 19 23:05:07 2017 -0500,Prepare the repo for next development cycle, 446b82e5e98d3610af64cd57b73fab2e297afcdc,Rob Kiefer,Tue Dec 19 12:46:59 2017 -0500,Release 0.8.0, 26971d2d1d2709f34f0bbf955a6dc2a12b63a577,Matvey Arye,Mon Dec 18 15:20:38 2017 -0500,Make `tablespace_show` function return Name instead of CString,Fixes a memory bug found by valgrind. 2fe447ba148de35ed2321e3fe56327908dcf405c,Matvey Arye,Mon Dec 18 11:22:03 2017 -0500,Make TimescaleDB work with pg_upgrade,"Compatibility with pg_upgrade required 2 changes: 1) search_path on functions cannot be blank for pg_upgrade. 2) The timescaledb.restoring GUC had to apply to more code (now moved to higher-level check) `pg_upgrade` must be passed the following option: `-O ""-c timescaledb.restoring='on'""` " 764536a69591f16ed959ecc4bfaf7f6482e46719,Matvey Arye,Mon Dec 18 15:37:16 2017 -0500,Add coverity badge to README, 90c7a6f54664ed5381a0b43ab10c98cb76c34ced,Matvey Arye,Thu Dec 14 11:52:16 2017 -0500,Fix logic for one space partition,This PR fixes handling of one space partition. It avoids creating a CHECK constraint and also fixes the logic in default dimension range creation. b037e06db0cce6aafd98fd99c5e6e84373656cd8,Erik Nordström,Mon Dec 18 12:54:57 2017 +0100,Fix copy test sort order,"A change to the COPY test made the sort order ambiguous for certain tuples, which breaks the test on some machines where the sort order might differ from the expected one. This fix makes the sort order more predictible. " 6cfdd7904be1232ba043b8cd58489f1c1e2d20c9,jwdeitch,Thu Dec 7 22:07:50 2017 -0500,Prevent native partitioning attachment of hypertables,- Raise an exception upon an attach partition event on a hypertable to a native postgres partition 438d79d1ed152d7b76543d668b3a9cb105161a0b,Matvey Arye,Tue Dec 12 17:48:49 2017 -0500,Fix trigger relcache handling for COPY,Previously trigger relcache entries were not always freed during COPY commands in PG10. This PR fixes that. Fixes #351. cc1ad95694aa53a44691e37b610c9f7d48c3e320,Matvey Arye,Mon Dec 11 20:08:40 2017 -0500,Reduce memory usage for out-of-order inserts,The chunk_result_relation_info should be put on the chunk memory context. This will cause the rri constraint expr to also go onto that context and be correctly freed when the chunk insert state is destroyed. 9ef6dd1fa3b875efbee9c84fe3f4200dd79b3690,Rob Kiefer,Sun Dec 10 11:57:52 2017 -0500,Fix bootstrap script to correctly name SRC_DIR, 3e194cd48fab7c7221ed3f385a8b47359cb7db81,Rob Kiefer,Sat Dec 9 16:14:02 2017 -0500,Update prerequisites in README, a0f62c5156490a2591402a9f7578e961ccd36344,Rob Kiefer,Thu Dec 7 14:25:20 2017 -0500,Improve bootstrap script's robustness,"Users can now either force the removal of an existing BUILD_DIR by passing BUILD_FORCE_REMOVE to ./bootstrap, or they will be prompted to remove the dir if it already exists. Additionally, the bootstrap script will now exit if there is an error in mkdir or cmake. " 00a096f2f60933f6d2ffdc85dd99bf39e48d3007,Rob Kiefer,Wed Dec 6 17:21:02 2017 -0500,Modify tests to make more platform agnostic,When testing on Windows these tests were returning differences because of floating point (sql_query_results_*) differences and because the cost was trivially higher (insert_single). Here we remove the costs since it was not relevant to the regressions we want to detect. Further we truncate the floating point averages to 8 decimal places; enough to see significant differences. 0e76b5fa05859f6f76092f971c545e1b9ff32141,Erik Nordström,Sat Dec 9 11:08:35 2017 +0100,Do not add tablespaces to hypertable objects,A hypertable's tablespaces are now always retrieved from the tablespace metadata table instead of being cached with the hypertable. This avoids having to do cache invalidation when updating the tablespace table. 176b75e43d0edccfd5e07fa28a10c790e6af285a,Erik Nordström,Mon Dec 4 17:43:35 2017 +0100,Add command to show tablespaces attached to a hypertable,Users can now call `show_tablespaces()` to list the tablespaces attached to a particular hypertable. 6e923835927ec841572fb2b25e498538484ac9ca,Erik Nordström,Tue Nov 21 16:42:51 2017 +0100,Add function to detach tablespaces from hypertables,"Tablespaces can now be detached from hypertables using `tablespace_detach()`. This function can either detach a tablespace from all tables or only a specific table. Having the ability to detach tablespace allows more advanced storage management, for instance, one can detach tablespaces that are running low on diskspace while attaching new ones to replace the old ones. " e593876cb003361bc9bf92a146407a281ed98b07,Erik Nordström,Mon Nov 20 13:40:27 2017 +0100,Refactor tablespace handling,"Attaching tablespaces to hypertables is now handled in native code, with improved permissions checking and caching of tablespaces in the Hypertable data object. " c4a46ac8a1a7291450508ea139f757527abd1ee1,Erik Nordström,Thu Nov 23 17:46:38 2017 +0100,Add hypertable cache lookup on ID/pkey,Hypertables can now be looked up through the cache on ID/pkey in addition to OID. f38a5785788368871cdbdaa64a74c55de436d9aa,Matvey Arye,Thu Dec 7 12:03:47 2017 -0500,Fix handling of long constraint names, 20c9b28fa73292cd67c14cfedffc0b43871bbd41,Kevin Bowling,Wed Nov 22 01:02:33 2017 -0700,Unconditionally add pg_config --includedir to src build, 12dff61c20ae10f14f6b95520026d56d3aca1829,Matvey Arye,Sun Dec 3 13:12:06 2017 -0500,Fixes insert for 32bit architecture,Fixes problem with insert on 32bit architecture. Fixes #336. e44e47ed880d0b66e070ca2bd06f36ec60deb0db,Rob Kiefer,Sun Dec 3 18:52:42 2017 -0500,Update add_dimension to take INTERVAL times,"The user should be able to add time dimensions using INTERVAL when the column type is TIMESTAMP/TIMESTAMPTZ/DATE, so this change adds that support. Additionally it adds some additional tests and checks for add_dimension, e.g., a nice error when the table is not a hypertable. " 0763e62f8f533b839f9e6604511244166a00597a,Rob Kiefer,Mon Nov 27 14:37:51 2017 -0500,Update set_chunk_time_interval to take INTERVAL times,"For convenience, the user should be able to specify the new chunk time intervals using INTERVAL datatype if the hypertable is using a TIMESTAMP/TIMESTAMPTZ/DATE datatype for its time column. " 87c4b4f7f0a4ad65189ca9ed409954f915139789,Rob Kiefer,Mon Nov 27 14:15:22 2017 -0500,Fix test generator to work for PG 10.1,Previously the test generator would create `parallel` tests with a suffix consisting of the PG major and minor version numbers. This is unnecessary for PostgreSQL 10 currently since we don't expect major breakages between minor versions. So now instead the file is suffixed with just the major version when it is PG10. 51854ac5c94f6396ea2d36bd91b1df346332e941,Michael J. Freedman,Sat Dec 2 11:35:27 2017 -0500,Fix error message to reflect that drop_chunks can take a DATE interval, 30d1483adbe20c77da98517d4c604c54d284d077,Rob Kiefer,Wed Nov 29 15:27:12 2017 -0500,Prepare the repo for next development cycle, 00c979112d5bbd34d1377e1948c257976440f755,Rob Kiefer,Tue Nov 28 14:15:53 2017 -0500,Release 0.7.1 (on master), 72b612cb7a5abaea034ed7f41df8d54da2fc52ad,Matvey Arye,Mon Nov 27 12:35:47 2017 -0500,Fix migration script, 2c2aba8fadd56dd61999042dd475b7c8af123111,Matvey Arye,Tue Nov 21 11:25:18 2017 -0500,Fix installchecklocal,Remove user from regression tests so that the local test will work. The tests itself will still be run with the appropriate user because of the logic in runner.sh. 8b772be99430ed65b326908ce1b9d54a3c05c979,Matvey Arye,Fri Nov 24 21:37:06 2017 -0500,Change time handling in drop_chunks for TIMESTAMP times,"This PR fixes the handling of drop_chunks when the hypertable's time field is a TIMESTAMP or DATE field. Previously, such hypertables needed drop_chunks to be given a timestamptz in UTC. Now, drop_chunks can take a DATE or TIMESTAMP. Also, the INTERVAL version of drop_chunks correctly handles these cases. A consequence of this change is that drop_chunks cannot be called on multiple tables (with table_name = NULL or schema_name = NULL) if the tables have different time column types. " 66396fb81e297a0d9268620c9a650a7277b54ac9,Rob Kiefer,Thu Nov 2 13:52:45 2017 -0400,Add build support for Windows,Windows 64-bit binaries should now be buildable using the cmake build system either from the command line or from Visual Studio. Previous issues regarding unresolved symbols have been resolved with compatibility header files to properly export symbols or getting GUCs via normal APIs. c7cc9114b4b155bf7598df796e81761794a043f1,Erik Nordström,Wed Nov 22 15:47:24 2017 +0100,Do not append-optimize plans with result relations (DELETE/UPDATE),"Plans that have a result relation set (DELETE or UPDATE) should not optimize subplans that contain append nodes. This is because the PostgreSQL must turn such plans on an inheritance table into a set of similar plans on each subtable (i.e., it needs to apply UPDATE and DELETE to each subtable and not just the parent). Optimizing such plans with, e.g., ConstraintAwareAppend makes the planner believe it only needs to do the DELETE or UPDATE on the root table, which means rows in subtables won't be affected. " e1a0e819cf6f71be966ccd155f04862d15a7d886,Erik Nordström,Tue Nov 21 22:44:48 2017 +0100,Refactor and fix cache invalidation,"TimescaleDB cache invalidation happens as a side effect of doing a full SQL statement (INSERT/UPDATE/DELETE) on a catalog table (via table triggers). However, triggers aren't invoked when using PostgreSQL's internal catalog API for updates, since PostgreSQL's catalog tables don't have triggers that require full statement parsing, planning, and execution. Since we are now using the regular PostgreSQL catalog update API for some TimescaleDB catalog operations, we need to do cache invalidation also on such operations. This change adds cache invalidation when updating catalogs using the internal (C) API and also makes the cache invalidation more fine grained. For instance, caches are no longer invalidated on some INSERTS that do not affect the validity of objects already in the cache, such as adding a new chunk. " 7b299fa05fcdf6fa91405713f20b5845f07c7a8a,Erik Nordström,Fri Nov 24 13:02:35 2017 +0100,Handle empty append plans in ConstraintAwareAppend,"Append plans that are empty (no children) are replaced at planning- time with a Result plan node that returns nothing. Such Result nodes occur typically when querying outside the time range covered by chunks. However, the ConstraintAwareAppend plan node did not handle the case of a Result child node, instead expecting either an Append or MergeAppend node. This is now fixed so that encountering a Result node means doing nothing. " 1e947da456d6e6b18df2bc3f07b6a8474c1141e3,Erik Nordström,Thu Nov 23 21:47:42 2017 +0100,Permission fixes and allow SET ROLE,"This change reduces the usage of SECURITY DEFINER on SQL functions and fixes related permissions issues. It also properly checks hypertable permissions relative the current_user instead of the session_user, which otherwise breaks SET ROLE, among other things. " a2b78aea1b7926ff1dff52e504769f8a25994e49,Erik Nordström,Wed Nov 22 12:03:39 2017 +0100,Fix reindex test,"The reindex test outputs the OID of a cloned index. This OID might change with the state of the database, added tests, etc., causing frequent test failures. The test is now updated to output the name of the index instead of the OID. " 33b260225e5d9d9de374631cae7e28d76cb8b062,Erik Nordström,Wed Nov 22 12:55:22 2017 +0100,Fix pgindent on first CMake run,"This changes the order in which subdirectories are processed in the top-level CMakeLists.txt file, so that dependencies from the scripts/ dir are set when processing the src/ dir. Otherwise, the `pgindent` target won't be enabled on the first CMake run. " 32b90e5e03b8cc500f5af61a558900979244dc90,Rob Kiefer,Tue Nov 21 23:41:34 2017 -0500,Prepare the repo for next development cycle, f0d69f60848200ae843a95581c835780b271826f,Rob Kiefer,Mon Nov 20 22:05:55 2017 -0500,Release 0.7.0, 13e1cb53436a940e5ea952a98fc8b8d591f2093c,Matvey Arye,Tue Oct 10 12:41:57 2017 -0400,Add reindex function,"reindex allows you to reindex the indexes of only certain chunks, filtering by time. This is a common use case because a user may want to reindex chunks after they are no longer getting new data once. reindex also has a recreate option which will not use REINDEX but will rather CREATE INDEX a new index and then DROP INDEX / RENAME new_index to old_name. This approach has advantages in terms of blocking reads for a much shorter period of time. However, it does more work and will use more disk space during the operation. " 6594018e32cb19d1bb2f6b4835630a5828f3039b,jwdeitch,Tue Nov 21 08:51:10 2017 -0500,Handle when create_hypertable is invoked on partitioned table,- create_hypertable will raise exception on invocation 818bdbc31b0d24f69d7c37a244166c06b185b211,Matvey Arye,Fri Nov 3 13:48:33 2017 -0400,Add coverity testing, 5d0cbc12fdcddb893267139f075e8742f87ef52e,Erik Nordström,Tue Nov 14 12:14:02 2017 +0100,Recurse CLUSTER command to chunks,"Clustering a table means reordering it according to an index. This operation requires an exclusive lock and extensive processing time. On large hypertables with many chunks, CLUSTER risks blocking a lot of operations by holding locks for a long time. This is alleviated by processing each chunk in a new transaction, ensuring locks are only held on one chunk at a time. " 9c7191e89882e159c6355e1be4878ddd1560693e,Matvey Arye,Thu Nov 16 14:48:25 2017 -0500,Change TIMESTAMP partitioning to be completely tz-independent,"Previously, for timezones w/o tz. The range_end and range_start were defined as UTC, but the constraints on the table were written as as the local time at the time of chunk creation. This does not work well if timezones change over the life of the hypertable. This change removes the dependency on local time for all timestamp partitioning. Namely, the range_start and range_end remain as UTC but the constraints are now always written in UTC too. Since old constraints correctly describe the data currently in the chunks, the update script to handle this change changes range_start and range_end instead of the constraints. Fixes #300. " 741b25662e48fcb4ddb298162a35ddf30c0a7815,Erik Nordström,Fri Nov 17 13:28:36 2017 +0100,Mark IMMUTABLE functions as PARALLEL SAFE,"Functions marked IMMUTABLE should also be parallel safe, but aren't by default. This change marks all immutable functions as parallel safe and removes the IMMUTABLE definitions on some functions that have been wrongly labeled as IMMUTABLE. If functions that are IMMUTABLE does not have the PARALLEL SAFE label, then some standard PostgreSQL regression tests will fail (this is true for PostgreSQL >= 10). " 2ffb30d99007d1b8387851ce0697316d6e739e91,Erik Nordström,Fri Nov 17 12:13:28 2017 +0100,Make aggregate serialize and deserialize functions STRICT,"Aggregate functions that have serialize and deserialize support functions (histogram, last, first, etc.) should have these support functions marked STRICT. PostgreSQL's regular test suite will fail when the timescaledb module is loaded without these functions being marked STRICT. " c552410f69e8e6a6f783b3686dc3a7c7b8970bd8,Olof Rensfelt,Wed Oct 11 09:49:59 2017 +0200,Add build target to run the standard PostgreSQL regression tests,"To verify that PostgreSQL operations on regular tables are unaffected by the TimescaleDB extension, the regular PostgreSQL regression tests can be run with the TimescaleDB extension loaded. To enable the build target (`pginstallcheck`), one needs to set the CMake `PG_SOURCE_DIR` variable to point to the PostgreSQL source code, which needs to have at least the regression test suite compiled. The PostgreSQL tests suite will run on every Travis build. " 291050b39569087cc436b0e71d034bf88c373572,Matvey Arye,Thu Nov 16 16:23:05 2017 -0500,Change DATE partitioning to be completely tz-independent,"The insert logic in time_value_to_internal was not tz independent, causing problems with date in some timezones. This is now fixed. Fixes #306. " ca0968aaa073b1fb51ff2714f988a871f754d1d4,Erik Nordström,Fri Nov 17 15:27:38 2017 +0100,Make all partitioning functions take anyelement argument,All partitioning functions now has the signature `int func(anyelement)`. This cleans up some special handling that was necessary to support the legacy partitioning function that expected text input. 0497744803bcfc8b34c6028fbfb42dad9d47bd01,Erik Nordström,Wed Nov 15 10:50:29 2017 +0100,Fix compilation warning in copy code,This change fixes a compilation warning in the copy code that was introduced with an earlier refactoring PR. e6936cbfa54d4b32e23f013cd89a1885bc32642a,Erik Nordström,Wed Nov 15 10:46:50 2017 +0100,Make warnings into errors in Travis build,Builds will now fail in Travis if a warning is raised. 69d7d5800c6842e430d7b791ec93d9b1fc392bed,Erik Nordström,Wed Nov 15 13:46:23 2017 +0100,Add pgindent target to format C source,The C source files can now be formatted with pgindent using a new CMake build target. The location of pgindent and dependencies are auto-discovered by CMake or can be overridden on the command line. a4e1e32d025569a8d736010b44c43e39f08c4a8c,Matvey Arye,Sun Nov 12 16:52:03 2017 -0800,Change range_start and range_end semantics,"We now use INT64_MAX and INT64_MIN as the max and min values for dimension_slice ranges. If a dimension_slice has a range_start of INT64_MIN or the range_end is INT64_MAX, we remove the corresponding check constraint on the chunk since it signifies that this end of the range is infinite. Closed ranges now always have INT64_MIN as range_end of first slice and range_end of INT64_MAX for the last slice. Also, points corresponding to INT64_MAX are always put in the same slice as INT64_MAX-1 to avoid problems with the semantics that coordinate < range_end. " 2dfbc829b05fc863c9a1ed481548195738137b10,Matvey Arye,Mon Oct 23 09:27:47 2017 -0400,Fix off-by-one error on range-end,"A hash function can have the value 2147483647, so range_end has to be 2147483648. " 5be7521a0b9cc884746a6a442945fa4879d72b2d,Matvey Arye,Mon Nov 6 17:49:09 2017 -0500,Implement changes suggested by coverity analysis, 959034642ec9889a6c6d3b514ae5533a972f09a9,Matvey Arye,Mon Nov 6 19:09:40 2017 -0500,Remove code related to COPY TO statements,Refactor the copy code to get rid of the NOT is_from path. This removes a lot of confusing dead code. 500563ffe5aa10df967a6d4d04102c97d7c65f0b,Erik Nordström,Sun Sep 24 00:54:19 2017 +0200,Add support for PostgreSQL 10,"The extension now works with PostgreSQL 10, while retaining compatibility with version 9.6. PostgreSQL 10 has numerous internal changes to functions and APIs, which necessitates various glue code and compatibility wrappers to seamlessly retain backwards compatiblity with older versions. Test output might also differ between versions. In particular, the psql client generates version-specific output with `\d` and EXPLAINs might differ due to new query optimizations. The test suite has been modified as follows to handle these issues. First, tests now use version-independent functions to query system catalogs instead of using `\d`. Second, changes have been made to the test suite to be able to verify some test outputs against version-dependent reference files. " bd36e6d02036ddc670621061b528abe918d9fb3d,Erik Nordström,Thu Nov 9 17:27:53 2017 +0100,Move SQL source file lists to CMakeLists.txt,"SQL source files that are joined together were previously listed in the `load_order.txt` and `extra_extension_files.txt` files. These SQL file list were then read by CMake. However, since no dependencies existed on these list files in Make targets, the project was not automatically rebuilt if the lists were updated. This change moves the lists directly to the CMakeLists.txt file, ensuring that the project is properly rebuilt every time the lists are updated. " 10fa207a121db37b4ef49eddc946222d416ffc28,Rob Kiefer,Tue Nov 7 14:50:53 2017 -0500,Prepare the repo for next development cycle, 325556e6ac9a321329f1aee38a5686d7526d89da,Rob Kiefer,Mon Nov 6 18:39:49 2017 -0500,Release 0.6.1, 73f0d75882f4806a6591cbc1b07398100b04a516,Matvey Arye,Mon Nov 6 14:39:32 2017 -0500,Fix EXPLAIN for ConstraintAware and MergeAppend,This fixes a bug with an explain for ConstraintAware plans when all chunks were excluded and a MergeAppend was used. The problem was that the EXPLAIN output for MergeAppend nodes expected at least one child to print the sorting column info. When all chunks were excluded there are no children. This PR removes all child execution states from the ConstraintAware node when all chunks are excluded. bc595c1826a26555238dfe9a49ef4dbb86365243,Erik Nordström,Mon Nov 6 13:35:02 2017 +0100,Use per-chunk memory context for cached chunks,"The chunk cache needs to free chunk memory as it evicts chunks from the cache. This was previously done by pfree:ing the chunk memory, but this didn't account for sub-allocated objects, like the chunk's hypercube. This lead to some chunk objects remaining in the cache's memory context, thus inflating memory usage, although the objects were no longer associated with a chunk. This change adds a per-chunk memory context in the cache that allows all chunk memory to be easily freed when the cache entry is evicted or when the chunk cache is destroyed. " 201a94845292bf69d2daf686b3ffa5b879792512,Olof Rensfelt,Mon Oct 16 14:32:21 2017 +0200,Check that time dimensions are set as NOT NULL.,"Add check that time dimensions are set as NOT NULL in the main table that a hypertable is created from. If it is not set, the constraint will be added. " 1dd1e4f0018e0e20ed117bcca7b6c798a9eafcea,Erik Nordström,Mon Oct 30 14:07:11 2017 +0100,Do not convert tuples on insert unless needed,"When inserting tuples into a chunk, the tuples are converted to chunk table format if the chunk's attributes differ from the parent table's. This can happen if, e.g., the parent table has had one of its columns removed, leading to a garbage attribute that is not inherited by new chunks. Unfortunately, PostgreSQL compares the typeid of tuples to decide if tuple conversion is needed, while also creating a new typeid for every table (including child tables). This results in tuple conversion occurring between a parent and a child table, even if not strictly needed. This optimization adds an extra check that avoids tuple conversion when not needed. Note that this updated check is also in recent PostgreSQL versions (>= PG10), so it won't strictly be needed when running on those versions. " 5de6abfa8512c216e8a0e5f65408fe6d5a086b45,Erik Nordström,Mon Oct 30 13:49:56 2017 +0100,Fix assertion assignment bug,This fixes two assertion statements in the subspace store. The first fix is for an assigment that occurred in the assertion instead of an equality comparison. The other is just a cleanup to remove an unnecessary if-statement. dd64b2a8942d681f9222a2babd0c48b6ca291571,Erik Nordström,Mon Oct 30 13:37:46 2017 +0100,Limit growth of range table during chunk inserts,"Everytime a new chunk insert state is created, a new range table entry is also created in the executor's range table. This change ensures that a range table entry is reused in case one already exists for a chunk that has been closed and reopened again in the same transaction. This can happen if the chunk insert state is evicted from its associated cache due to out-of-order inserts. Thus, this will limit the growth of the range table to at most the number of chunks in the hypertable. " 4532650411380b9bd638ef1a8876119ce2d4f259,Erik Nordström,Fri Oct 27 15:06:31 2017 +0200,Allow setting partitioning function,Users might want to implement their own partitioning function or use the legacy one included with TimescaleDB. This change adds support for setting the partitioning function in create_hypertable() and add_dimension(). 4a0a0d8253f36563c192e6138b003a5a1d87de0d,Erik Nordström,Mon Oct 30 08:27:11 2017 +0100,Fix column type change on plain tables,"A recent change blocked changing types of space-partitioned hypertable columns. However, this blocking should not apply to regular tables, which otherwise causes a crash. This change fixes this issue by properly checking that the the table is a hypertable. " cf009cc584ba7368b0f1a9d205e40937958b52da,Erik Nordström,Mon Oct 16 20:54:02 2017 +0200,Avoid string conversion in hash partitioning,"Hash partitioning previously relied on coercing (casting) values to strings before calculating a hash value, including creating CHECK constraints with casts. This approach is fairly suboptimal from a performance perspective and might have issues related to different character encodings depending on system. Hash partitioning now instead uses a partitioning function that takes an anyelement type that calls type-dependent hash functions internal to PostgreSQL. This should provide more efficient hashing both by avoiding unnecessary string conversions and by using more optimal type-specific hash functions. Support for the previous hash partitioning function is preserved for backwards compatibility. Hypertables created with the previous function will continue to use to old hashing strategy, while new tables will default to the updated hash partitioning. For safety, this change also blocks changing types on hash-partitioned columns, since it seems hard to guarantee the same hash result between different types. " 2ba1a40a9ffecd89cd7ab8153793174945d1d369,Erik Nordström,Thu Oct 19 18:27:32 2017 +0200,Fix issue with creating expression indexes,"When creating an index on a chunk based on a index on a hypertable, it is necessary to adjust the attribute numbers of referenced columns in case the hypertable and the chunk have different number of attributes (this can happen, e.g., due to removing columns on the hypertable that aren't inherited by a newly created chunk). This adjustment was handled for regular indexes, but not expression indexes, causing an error to be raised. This change adds the proper handling of expression indexes. " d8e384bff527866c72f5606f9f988ea010184673,Matvey Arye,Mon Oct 23 17:06:01 2017 -0400,Fix memory leak due to constraint expressions.,"By default, Postgres places constraint expressions on the query memory context. However, these expressions only apply to individual chunks and cannot be re-used across chunks. Placing them on the chunk_insert_state context (created as part of this PR) allow these expressions to be freed along with the insert state of individual chunks. " 81510987d059b276c3e3b43f7a68cc693b7712fd,Matvey Arye,Thu Oct 19 14:59:44 2017 -0400,Improve update testing by adding a rerun test,The rerun test recreates the hypertable in the newest version of the extension. This allows us to test correctness of the data in the catalog. c420c11e4450f0da44d72c4f83b1aea1031606ca,Matvey Arye,Thu Oct 19 14:57:20 2017 -0400,Create a catalog entry for constraint-backed indexes,This change makes our catalog more in-line with the pg catalog. It also will simplify a lot of other code. ec746d1ef80244b99962da64508afff7a4c2e88a,Matvey Arye,Wed Oct 18 15:07:57 2017 -0400,Add ability to run regression test locally, 44f9fec2fa84a761c127c40a0f6085adc0486c39,Matvey Arye,Wed Oct 18 15:06:29 2017 -0400,Add analyze to parallel test for stability, 728481ee2995d8b356c9d7da52b5daf219c55db7,Matvey Arye,Tue Oct 17 21:14:33 2017 -0400,Fix bugs found by valgrind, b6afa0221c23d91ae37175bc98e82869421fa139,Jordan Deitch,Mon Oct 23 12:37:50 2017 -0400,Consider precvious CIS state in copy FROM file to rel,- Will consider previous_cis when releasing bulk insert state in copying from a file to a relation fbd43492341d315cc9118ac67a5742e4bd62af9f,Rob Kiefer,Tue Oct 17 12:34:53 2017 -0400,Change integral drop_chunks() to use BIGINT,"Previously drop_chunks() only took INTEGER, which prevented it from being called with BIGINT values, e.g. for nanoseconds. " 9e0422ab7ee59a3168788217c6eb943a531336c2,Matvey Arye,Thu Oct 12 18:30:26 2017 -0400,Fix bug with pointer assignment after realloc, 114fa8d537927c761c18aa6d1613b9de4f09bbf4,Erik Nordström,Wed Oct 11 11:47:07 2017 +0200,Refactor functions used to recurse DDL commands to chunks,"When recursing DDL operations to chunks, you often want access to the hypertable in the chunk handler function. This change gives the handler access to the hypertable data structure rather then just the hypertable OID, obviating the need to do a separate lookup that has likely already been done previously. " b1ec4fa17985183f4b987d606019b572e91b6c52,Erik Nordström,Thu Oct 5 11:38:56 2017 +0200,Refactor build system to use CMake,"Moving the build system to CMake allows easy cross-platform compiles, dependency checks, and more. In particular, CMake allows us to easily build on Windows, and Visual Studio now has native CMake support. " aceb2e9835efc979a6efa1557f5c4eeb8f3db047,Rob Kiefer,Fri Oct 13 10:34:42 2017 -0400,Prepare the repo for next development cycle, f97d6243595bf15c3d87ab238c9d02ab15756734,Rob Kiefer,Fri Oct 13 10:33:45 2017 -0400,Make event trigger creation idempotent, 9a432c834137e81d984c0da3a3c7316ad041b50b,Rob Kiefer,Mon Oct 9 13:16:35 2017 -0400,Release 0.6.0, f39d42b182d2dfb4498502cd04a4380521ad46d7,Olof Rensfelt,Wed Oct 11 14:45:15 2017 +0200,Add pg version to dump_meta_data script., 1d73fb81c1a2f3e74cfb9af2294e272451fe003b,Matvey Arye,Tue Oct 10 17:12:01 2017 -0400,Fix bug with extension starting too early.,Extension_new_state() was sometimes using the pg cache before it was fully initialized. This commit adds a check to make sure that does not happen. fd390ec31155f35ac918f1c0e67b0e0b95dc4946,Erik Nordström,Tue Oct 10 10:29:03 2017 +0200,Fix chunk index attribute mismatch and locking issue,"This change fixes two things that were overlooked in a prior refactoring of chunk index handling. First, column attribute numbers of a hypertable might not match a chunk if, e.g., a column on the hypertable has been removed. In such circumstances, indexes created on chunks based on a corresponding hypertable index need to account for differences in column attribute numbers. This change ensures that column attributes are always translated to match the chunk an index is created on. Second, ShareLock was acquired by mistake on each hypertable index when recursing these indexes to chunks, potentially causing deadlocks. ShareLock should only be taken on the heap relation that an index is created on. This is now fixed. Further, locking during index creation has been cleaned up so that it is easier to overview the locks taken on various relations. " 742acf5dc24c7d79b43704ebd63ffb46e83efcae,Erik Nordström,Tue Oct 10 12:24:53 2017 +0200,Fix mixed declaration and code warning and indentation,"A previous commit mixed declarations and code, which causes warnings on some compilers. This change quenches the warning and fixes some indentation/formatting issues. " 430ed8a113e95df3ce5d23abc3a5f835e698f16b,Olof Rensfelt,Fri Oct 6 21:56:36 2017 +0200,Fix bug with collected commands in index statement.,We previously blocked valid plain postgres statements. The blocking code is now moved to only affect hypertables. a3d8f1062f57813829e65a4c96be97b2494a1295,enordstr,Sun Oct 8 11:16:41 2017 +0200,Quench a number of type mismatch warnings,These changes will fix a number of warnings generated by some compilers and platforms (most notably on Windows) that mostly relate to assigning values to variables with mismatching types. A number of datum functions were also called without actually passing datums or assigning return values to datums. Those are now passed datums instead of raw data types. 543a75f86595636c04dd6f081935435f731b34c3,Matvey Arye,Wed Oct 4 18:07:43 2017 -0400,Refactor regression tests,This commit moves a lot of test setup logic to runner.sh. Also passes the right commands to the regression infrastructure to create appropriate users and run tests as a regular user. 614c2b782107fca191f34b46b619ac147be2711d,Matvey Arye,Tue Oct 3 18:07:05 2017 -0400,Fix permissions bugs and run tests as normal user,All regression tests will now use a non-superuser unless superuser is necessary. This PR is a meant to prevent things like issue #226. This PR also fixes some more permission bugs found during this testing. ce1210456914bee47c55cec26b0076232000e841,Erik Nordström,Thu Oct 5 13:50:30 2017 +0200,"Fix ""ON CONFLICT ON CONSTRAINT"" on plain PostgreSQL tables","In the planner hook, we erroneously blocked ""ON CONFLICT ON CONSTRAINT"" statements on plain tables. This fixes the issue so that this error only occurs on hypertables. " 4c451e0f0f2f2dab798e9124577113e82c1c66a8,Erik Nordström,Thu Oct 5 13:32:42 2017 +0200,Fix rename and reindex bugs when objects are not relations,"When issuing rename or reindex commands that do not involve relations, the relation rangevar in the utility statement will be NULL. This change adds extra checks to avoid calling, e.g., RangeVarGetRelid() when a relation's rangevar is NULL, as that function does not accept NULL input values. " c3ebc676e3b4fc2884feb1e86047e37db4ae2a14,Matvey Arye,Sun Oct 1 21:43:45 2017 -0400,Fix permission problems with dropping hypertables and chunks,This change fixes permissions with dropping hypertables and chunks. Fixes #226. e1d01b9fff8bae29b4079356440f863aae617cf2,Erik Nordström,Thu Oct 5 13:39:06 2017 +0200,Fix pgindent issues due to windows-related changes,Files that have been modified with the PGDLLEXPORT macro are no longer pgindent formatted. This change fixes these pgindent issues. 040e815dbaef2e62bc66dde24b9acbea58b541b2,Erik Nordström,Wed Oct 4 14:33:51 2017 +0200,Remove truncate and hypertable metadata triggers,"This is part of the ongoing effort to simplify the metadata tables and removing any triggers on them that cause side effects. This change includes the following: - Remove the on_change_hypertable() trigger on the hypertable catalog table. - Remove the TRUNCATE blocking triggers on all metadata tables. If we think such blocking is important, we should do this in an event trigger or the processUtility hook. - Put all SQL files in a single load_order.txt instead of splitting across three distinct files. Now all SQL files are included in update scripts as well for simplicity and consistency. - As a result of removing triggers and related functions, the setup_main() and restore_timescaledb() functions are no longer needed. This also further simplifies the database restore process as calling restore_timescaledb() is no longer needed (or possible). - Refactor create_hypertable_row() to do more validation before allocating a new hypertable ID. This avoids incrementing the serial ID unnecessarily in case some validations fail. " 5c263286a31cc4f2d3c805c16bbcea064c7f0c33,Erik Nordström,Tue Oct 3 17:59:25 2017 +0200,Fix INSERT on hypertables using sub-selects with aggregates,"When inserting into a hypertable using a sub-select clause that involves an aggregate, the insert fails with the error ""Aggref found in non-Agg plan node"". This happens because the target list from the aggregate sub-select expects an aggregate parent node and we simply reuse the target list when modifying the insert plan with new CustomScan plan nodes. This change creates a modified target list to use with the CustomScan node that avoids this issue. " b57e2bf1f4fb0f8d0dcc4a493218be518f3729a3,Rob Kiefer,Fri Sep 22 14:18:34 2017 +0000,Prepare C code for compiling on Windows,"For all exported functions the macro PGDLLEXPORT needs to be pre- pended. Additionally, on Windows `open` is a macro that needed to be renamed. A few other small changes are done to make Visual Studio's compiler happy / get rid of warnings (e.g. adding return statements after elog). " 097db3d58928ab4784c679b8308ab0ebf3bf1fe0,Erik Nordström,Mon Sep 11 17:19:48 2017 +0200,Refactor chunk index handling,"This change refactors the chunk index handling to make better use of standard PostgreSQL catalog information, while removing the hypertable_index metadata table and associated triggers, including those on the chunk_index table. The chunk_index table itself is also simplified. A benefit of this refactoring is that indexes are no longer created using string mangling to construct the CREATE INDEX command for a chunk, based on the string definition of the hypertable index. Instead, indexes are created in C using proper index-related internal data structures. Chunk indexes can now also be renamed and are added in the parent index tablespace. Changing tablespace on a hypertable index also recurses to chunks, as expected. Default indexes that are added when creating a hypertable use the hypertable's tablespace. Creating Hypertable indexes with the CONCURRENTLY modifier is currently blocked, due to unclear semantics regarding concurrent creation over many tables, including how to deal with snapshots. " a2bad2b5f76b9ec2b7a126216c1a518707575b4e,Erik Nordström,Fri Sep 29 14:49:27 2017 +0200,Fix constraint validation on regular tables,"Constraints on regular PostgreSQL tables need to be validatated so that they don't have a foreign key constraint that references a hypertable, as this is not yet supported. This change fixes a bug in this validation caused by not checking for the proper node types when parsing a CreateStmt node. The CreateStmt is now also parsed in the DDL end hook, instead of the processUtility hook, which ensures that the CreateStmt node has been run through parse analysis. The parse analysis changes the contents of the node to be more consistent across different CREATE statements. For instance, a CREATE LIKE statement will look more similar to a regular CREATE statement after parse analysis. " bf6fd3158f53aa65fe6cad1ed22cc9a348e391f4,Erik Nordström,Fri Sep 29 13:46:19 2017 +0200,Tweak the docker image build script,- Save an intermediate build image to avoid having to download APK packages for every build - Add debug symbols and assertions to extension build - Install git and gdb by default for easier debugging fb5717fd2239c5d238135f6ab12cff76b4e88593,Rob Kiefer,Wed Sep 20 16:10:43 2017 -0400,Remove explicit schema for time_bucket,"`public` schema may not exist or the user may be installing the extension to a different schema other than public, so like the rest of our public API functions it should be left implicit rather than assume `public`. " 04d01ce6cab239c0edf130743fb9b10a67fe7223,Erik Nordström,Mon Sep 18 16:50:38 2017 +0200,Split DDL processing into start and end hooks,"The ProcessUtility hook doesn't give any information on applied DDL commands, which makes it hard to implement DDL processing that requires the result of a DDL command on a hypertable (for instance, adding a constraint or index without an explicit name). This change splits the DDL processing over start and end hooks, handling DDL commands before and after regular PostgreSQL processing, respectively. The start DDL hook is still based on the ProcessUtility hook, while the end DDL hook is based on an event trigger that allows getting information on the created/dropped/altered objects. " 48fbfaf99a9ff5ac59153fd072a2c60e07b73552,Erik Nordström,Sat Sep 16 15:15:41 2017 +0200,Make update test script restore database instead of reinserting data,"The update test script compares a TimescaleDB instance, which is updated to the latest version from an older version, to a cleanly installed one at the latest version. It does this by inserting the same data on both and comparing the data and metadata tables. This comparison breaks if the new version has changed some behavior (e.g., chunk indexes might use a different naming scheme), because the updated version will retain the metadata related to the old behavior while the latest version will create metadata according to the new behavior. To fix this issue, the update script now performs a database backup on the old version and restores it on the cleanly installed version of TimescaleDB. This retains the metadata in accordance with the old behavior, although the new version will create new metadata according to the new behavior. This has the further benefit of testing the restore process during upgrades, which is probably the way people will move to a new version of TimescaleDB if they do not update the extension with the data in place. The capture errors related to broken update scripts, the update test now includes an integrity test for some metadata tables. " 6496b020492dba1d64af2e6e37c9f36247ed92a7,Rob Kiefer,Wed Sep 20 12:14:20 2017 -0400,Prepare the repo for next development cycle, e7d51df5f9a58d2a3bfc83781b06207e2a68bda5,Rob Kiefer,Tue Sep 19 12:26:00 2017 -0400,Release 0.5.0, 4d3e2c05234cdc6d9dc82e41a3abd20eea319b2f,Erik Nordström,Wed Sep 20 17:39:32 2017 +0200,Fix error message about creating constraints using exiting index, afcb0b11daa93dbea6c002798c1aa0d4897b1ed7,Matvey Arye,Mon Sep 18 11:59:06 2017 -0400,Fix NULL handling in first/last functions.,There are 2 separate fixes: 1) Fix for NULL value handling when values are numeric types. This fixes #209. Thanks to @raycheung for reporting this bug. 2) Correct handling for NULLs in compare/ordering elements. The functions will now always return NULL if any comparison element in group was NULL. 2cafb2446c7a0450965d7d2a27692bb2b083f691,Erik Nordström,Tue Sep 19 16:43:01 2017 +0200,Add another missing break statement, d53c70579679843e2c4d9115fcb6d2c405776a8f,Olof Rensfelt,Mon Sep 18 15:20:31 2017 +0200,Add script to dump meta data that can be useful for debugging.,This also adds support for a tag that is used to allow changing outputs for parts of a test without causing an error. aa904fa5d0a19d99f2e67c2fce5d483ec60ce2f4,Erik Nordström,Mon Sep 18 16:58:49 2017 +0200,Block adding constraints without a constraint name,"The current approach of handling alter table commands does not allow getting the result of the command; for instance, the object address of a created constraint on a hypertable. Thus there is no way to get the auto-generated name of a constraint, which is needed when the corresponding constraints are created on the hypertable's chunks. Therefore, this change blocks the ability to create constraints without an explicit name. When a better approach to handling alter table is deviced, it is possible to remove this restriction. " 1d54ccea45ff3e56fee98692dcc18558a2621877,Erik Nordström,Mon Sep 18 18:26:00 2017 +0200,Add missing break statement in process utility hook, a13039fd5821d1d8f1afe323e116625ee4039fa3,Erik Nordström,Sat Sep 16 16:37:56 2017 +0200,Fix dump and restore for tables with triggers and constraints,"During a database restore, the creation of triggers and constraints on hypertables should not recurse to chunks since the chunks will be restored with existing triggers and constraints. Failing to block recursing during restore risks duplicating these objects on chunks. Further, operations on chunks (e.g., ALTER TABLE) should be allowed during the restore process since those operations are needed to recreate the constraints and triggers. To fix this, the ""restoring"" GUC is used to bypass custom handling of events in the ProcessUtility hook. " 8cf8d3c3775b028eedea7873711f36ce195855ce,Olof Rensfelt,Tue Sep 5 15:27:29 2017 +0200,Improve the size utils functions.,"The hypertable, chunk, and index size functions are now split into main function and a corresponding ´pretty´ function. In chunk_relation_size_pretty() the ranges are now converted into a human readable form when they are time types. " 6660348ab6087a99569c10bb6df238d8c9cb0684,Erik Nordström,Fri Sep 15 10:28:10 2017 +0200,Cleanup process utility for consistency,- Move DDL upcalls to function macros for clarity and readability - Rename some functions for consistency and remove unused parameters - Refactor some code that did unecessary work 27675488f653eaba73315e1bdb83713b68d84be2,Erik Nordström,Fri Sep 15 13:38:31 2017 +0200,Block adding constraints using an existing index,"Adding a constraint using an existing index changes the existing index's name as a side effect (PostgreSQL wants the index name to match the constraint name). This creates a mismatch between the index's name and our metadata in the chunk_index table. Further, since the name change happens as a result of an internal command (and not via DDL command that we can capture) there is currently no way to sync up this information. " 5cee104d57a753f5ccea2b93cb81e4de3921a90a,Matvey Arye,Thu Aug 24 14:55:44 2017 -0400,Allow chunk_time_interval to be specified as an INTERVAL type, 6232f98edc35b35d493e6adc2112965005f3d39b,Sarah Pan,Wed Sep 13 15:41:52 2017 +0200,Add histogram function.,A rebase with only minor modifications from commit e80720dd9b412a31d1079533323d0b01231926e3 51821b370947ccbca6d385baae885327f7c633a6,Matvey Arye,Wed Sep 13 15:18:37 2017 -0400,Move trigger handling from PLPGSQL to C,"Applying triggers to chunks requires taking the definition of a trigger on a hypertable and executing it on a chunk. Previously this was done with string replacement in the trigger definition. This was not especially safe, and thus we moved the logic to C where we can do proper parsing/deparsing and replacement of the table name. Another positive aspect is that we got rid of some DDL triggers. " 238003316e91fb56cc44ba73e2c47753eb2eb91e,Matvey Arye,Mon Sep 11 17:50:29 2017 -0400,Block ALTER TABLE and handle DROP TABLE on chunks,This PR adds logic to block alter table and handle drop operations on chunks. 72d668150ebcc3854fb20ddfb2383c312de000fd,Matvey Arye,Mon Sep 11 12:22:55 2017 -0400,Move security checks for ALTER TABLE ALTER COLUMN to C, 19d3d8981be7e709a893565ba08a1dd22723841e,Matvey Arye,Thu Sep 7 17:58:30 2017 -0400,Handle changing the type of dimension columns correctly.,Update the type in the dimension metadata table and recreate the check constraints. 17c4ba9ec3c850f04ada369f9132faca9954a950,Matvey Arye,Thu Sep 7 16:05:56 2017 -0400,Handle ALTER TABLE rename column,Update the column names stored in the dimension metadata table. 1d95dfbe4480fa2b66659c19907df729bc03a04b,Erik Nordström,Mon Sep 4 11:45:27 2017 +0200,Add limit option to scanner,This adds an option to set a limit on how many tuples to return in a relation scan using the scanner implemention. This avoids a common pattern of manually implementing limits in the tuple handling function. 66932cf8d5a63e9b340ecb2c8c905931ad07b1a2,Matvey Arye,Mon Sep 11 11:40:44 2017 -0400,Forbid relocating extension after install.,"Unfortunately, Postgres forbids relocating the ""public api"" schema of extensions that use multiple schemas after initial installation. This PR marks the extension as not relocatable and tests relocation during initial install. " 98cc8a84f626e7cc3937525cb3ae8d01e2ddb189,Matvey Arye,Wed Sep 6 16:58:26 2017 -0400,Apply changes suggested by scan-build static analyzer and pgindent, d2561cc4fd55b8820c684d4bed060c2b8a96b211,Matvey Arye,Wed Aug 23 11:09:53 2017 -0400,Add ability to partition by a date type, 48e0a611316fb696e6c6c18a4b134575966ba930,Matvey Arye,Fri Sep 1 13:19:44 2017 -0400,Remove triggers from chunk and chunk_constraint,Streamline code and remove triggers from chunk and chunk_constraint. Lots of additional cleanup. Also removes need to CASCADE hypertable drops (fixes #88). 4dcbe6114dda6d4a59224033be03552d8338a4b9,Matvey Arye,Wed Aug 16 14:47:32 2017 -0400,Add support for hypertable constraints,"This PR add support for primary-key, foreign-key, unique, and exclusion constraints. Previously supported are CHECK and NOT NULL constraints. Now, foreign key constraints where a hypertable references a plain table is support (while vice versa, with a plain table references a hypertable, is still not). " 3b2afb6e9abb2e43bedaeefea9fa5968b75ab562,Rob Kiefer,Wed Sep 6 14:30:00 2017 -0400,Prepare the repo for next development cycle, fce6b0630c06787224ef3cde73e33f585ce6fdf5,Rob Kiefer,Wed Sep 6 14:28:37 2017 -0400,Release 0.4.2 and update for master, 997556af40ff2e5c266043efb377bcd20e5a085c,Rob Kiefer,Tue Sep 5 16:19:48 2017 -0400,Add backup/restore scripts for single hypertables,"Using pg_dump/pg_restore for a database with hypertables will back up all internal tables too, which is sometimes not desired. These scripts allow one to backup regular PostgreSQL tables separately and then pick and choose which hypertables to backup and restore. This also provides a forward-compatible way of backing up and restoring hypertables, since data is dumped to CSV and restored by re-creating and re-inserting the data. " c2f686dbba2bd0a5f8d8a4b32a395cc2208b2899,Erik Nordström,Tue Aug 15 17:28:11 2017 +0200,Refactor chunk creation to handle chunk collisions and alignment,"When new chunks are created, the calculated chunk hypercube might collide or not align with existing chunks when partitioning has changed in one or more dimensions. In such cases, the chunk should be cut to fit the alignment criteria and any collisions should be resolved. Unfortunately, alignment and collision detection wasn't properly handled. This refactoring adds proper axis-aligned bounding box collision detection generalized to N dimensions. It also correctly handles dimension alignment. " bb8ea379655769b90a9c6f276765be355c86dcd0,Rob Kiefer,Tue Sep 5 11:58:02 2017 -0400,Remove out-dated migrate_data.sh script,This script no longer is needed since chunks close appropriately even when the insert spans multiple chunks. 4e033ef318644efd449074dfa1331c84bfbe80e4,Rob Kiefer,Mon Sep 4 15:50:47 2017 -0400,Prepare the repo for next development cycle, 55c58017a89dd9b6c6ba3fc44cdb9915e71f3067,Rob Kiefer,Sun Sep 3 22:05:26 2017 -0400,Release 0.4.1, 50c8c4c2f57396c72702857993c6b65d58295560,Matvey Arye,Fri Sep 1 10:19:28 2017 -0400,Fix possible segfault in planner,"Previously, the planner had a global context that kept a reference to the hypertable cache. The reference counting system in that global context was unsafe and could lead to a segfault. This PR gets rid of that context and makes the whole system a lot safer. Fixes #184 " e49e45c74885d6acc2c0db7e8a248ca1550969a4,Erik Nordström,Tue Aug 29 14:15:27 2017 -0400,Fix failure when constraint-aware append excludes all chunks,"This change fixes a bug that occurs when a constraint-aware append plan excludes all chunks at execution time. Append nodes do not handle the case when all subplans are pruned and the append node's subplan list is empty. To avoid a failure in that case, the constraint-aware plan node will shortcut the execution of the query and never call the append node. " c3b6fb90d9dcd73d47adf2a986bce981b4f8c5fa,Matvey Arye,Tue Aug 22 12:26:49 2017 -0400,Fix bug with negative dimension values,"Previously, negative dimension values had an off-by-one bug where the wrong chunk was created for points on chunk borders. This PR fixes that. " 3c69e4fc9e9002970a03e084ca61d946cb196731,Matvey Arye,Wed Aug 23 16:17:32 2017 -0400,Fix semantics of time_bucket on DATE input,"Previously, date was auto-cast to timestamptz when time_bucket was called. This led to weird behavior with regards to timezones and the return value was a timestamptz. This PR makes time_bucket return a DATE on DATE input and avoids all timezone conversions. " 0137c92cdbb48153fbe1a4a7fdb539e565b2ca22,Olof Rensfelt,Tue Aug 22 10:11:42 2017 +0200,Fix output order of chunk dimensions and ranges in chunk_relation_size., 2085116ba63bba010ee9cfc7de2cad6e95615dfd,Erik Nordström,Tue Aug 22 22:18:25 2017 +0200,Fix append tests,One of the append tests accidentally used the PostgreSQL now() function instead of a mocked now_s() function. This caused tests to eventually fail. This change corrects that mistake. 645b530ed2ad1176912ee991743437c2fd7cb674,Erik Nordström,Mon Aug 21 22:30:39 2017 +0200,Convert inserted tuples to the chunk's rowtype,"When routing inserted tuples to chunks, the rowtype of the chunk is expected to match that of the root table. However, this is not always the case if the root table has been modified, e.g., it had a column removed. Newly created chunks will in that case have a different rowtype and therefore tuples need to be converted to the chunk's rowtype. " fcb7114c26b5bc2f432eabd76d11f07739f9552b,Rob Kiefer,Mon Aug 21 14:00:30 2017 -0400,Prepare the repo for next development cycle, c54dd6fd0866285ec9d0a1d28c080f19ac4bebe4,Rob Kiefer,Mon Aug 21 11:23:46 2017 -0400,Release 0.4.0, 139fe3402235f61a3620790ef6c0a8fac1498757,Erik Nordström,Wed Jul 26 23:03:20 2017 +0200,Implement constraint-aware appends to exclude chunks at execution time,"With standard PostgreSQL append plans, planning-time constraint exclusion does not work if a query expression contains restrictions that are non-constants (e.g., mutable functions like now()). This leads to full scans of all chunks (child tables), significantly increasing query time. To fix this, we wrap append plans (Append and MergeAppend) in a ConstraintAwareAppend plan, which excludes child relations at execution time by pruning the child relations from the regular append plan. This happens after first constifying the original restriction clause every time the plan is executed. This optimization only happens if there are non-constant restrictions in the original query and the entire optimization can be disbled with a GUC variable. " 2a51cf0701721c5af06516f4c589511cf36dc4cd,Rob Kiefer,Tue Aug 15 18:32:30 2017 -0400,Add support for negative values in dimension range,"Previously dimension_slice would only allow ranges to start and end with values >= 0. However, this did not allow for timestamps prior to 1970 because those times would be represented by negative integers. With this PR we now support dates prior to 1970. Addresses issue #153. " 2f5e43676f05519f3f68b1b98c2afb432f925541,Rob Kiefer,Thu Aug 17 12:25:20 2017 -0400,Add -t flag to test/runner.sh's mktemp call,On my machine (OSX 10.10) `mktemp -d` prints out the usage message and doesn't create a temporary directory. f2d5c3fc5db01dfb35e63fe5fb33290f677d5a66,Michael J. Freedman,Thu Aug 17 13:07:12 2017 -0400,Error if add_dimension given both partition number and interval length, 382b712b8b57e02f274d682932540cc84e3c441e,Michael J. Freedman,Thu Aug 17 12:36:56 2017 -0400,Fix link to update info in docs, f3df02d50c3f3d24eb16db053b72fc6bb85c30f5,Rob Kiefer,Tue Aug 15 21:34:14 2017 -0400,Improve handling of non-TIMESTAMP/TZ timestamps,"Previously the default chunk time in microseconds was too large for a SMALLINT or INTEGER field. Now, we only assign a default value if the type is TIMESTAMP or TIMESTAMPTZ. Integer timestamps, such as SMALLINT, INTEGER, and BIGINT, need to be explicitly set since only the user knows what units the numbers represent. Further, we check to make sure the chunk time interval is not too large for SMALLINT and INTEGER so as to avoid confusing problems later when the user goes to insert. " 6a5a7eb398dcd4c470f115cf1e6945a00cffa9cc,Erik Nordström,Thu Aug 17 11:03:30 2017 +0200,Reduce memory usage on long-running COPY operations,"This change ensures that the per-tuple exprcontext (on which per-tuple state is allocated), is reset for every new tuple processed in a long-running COPY transaction. " 953346c18b5b66add593def9970377b815c31d97,Erik Nordström,Wed Aug 9 15:23:36 2017 +0200,Make VACUUM and REINDEX recurse to chunks,"Previously, when issued on hypertable, database maintenance commands, like VACUUM and REINDEX, only affected the main table and did not recurse to chunks. This change fixes that issue, allowing database maintainers to issue single commands on hypertables that affect all the data stored in the hypertable. These commands (VACUUM, REINDEX) only work at the table level for hypertables. If issued at other levels, e.g., schema, or database, the behavior is the same as in standard PostgreSQL as all tables are covered by default. REINDEX commands that specify a hypertable index do not recurse as that requires mapping the hypertable index to the corresponding index on the chunk. This might be fixed in a future update. " f103fc7de320890ec1da622c3931e4130a9a6408,Erik Nordström,Thu Aug 10 11:00:38 2017 +0200,Make generate-typedefs.sh work on Linux, 55bfdf753a5c557720cbf538026e09db70ddd58e,Erik Nordström,Tue Aug 8 11:35:42 2017 +0200,Release all cache pins when a transaction ends,"Currently, if a transaction ends (normally or abnormally) while a cache is pinned, that cache will never be released, leading to a memory leak. This change ensures that all taken cache pins are released when a transaction ends by registering all pins in a list and cleaning it up in a transaction end callback. This makes the use of cache_release() optional. However, timely calls to cache_release() is still encouraged since it is better to release memory as soon as possible during long-running transactions. " a1a28ecf6f8bdc3a2448129e67e04bd29ddef970,Erik Nordström,Mon Aug 7 13:35:21 2017 +0200,Refactor process utility,This breaks up the handling of process utility statements into a clean and efficient switch statement with separate functions for each statement. 69ac1a37d85317c5ceec87179b58bb9f21b21275,Rob Kiefer,Mon Jul 31 13:29:34 2017 -0400,Prepare the repo for next development cycle, d1f66c5af44a43de2fd694ad73ae143cfc5195bb,Rob Kiefer,Mon Jul 31 11:06:36 2017 -0400,Release 0.3.0, 2200010ddb904685139b3333b50d15b126659af8,Erik Nordström,Fri Jul 28 17:41:57 2017 +0200,Fix EXPLAIN and PREPAREd statements on INSERTs,"This fixes some issues that caused EXPLAIN and PREPAREd statements to fail on INSERTs. The plan modification on INSERTs passed on a plan node to the execution stage. However, that plan node pointer becomes invalid when the plan is copied, as happens with prepared statements. The fix required another modification to the plan by wrapping also the ModifyTable plan node. This wrapping is necessary to access the plan state node (ModifyTableState) on execution. Having access to the ModifyTableState node is necessary to set the correct arbiter index list for ON CONFLICT statements. This change also fixes the output of EXPLAIN to properly show the plan tree, and performs some cleanups. " 486da7fd5f1df8efdcf29c3ebd2e00f69465407e,Matvey Arye,Thu Jul 13 06:25:19 2017 -0400,Reduce local non-docker dependencies for update tests,Get rid of dependencies on a local psql and stop mounting data volumes. 23f9d3c0b0399c7bd2c365a2d51601b0f3b96ba4,Erik Nordström,Wed Jul 26 15:57:40 2017 +0200,Add support for upserts (ON CONFLICT DO UPDATE),"Upserts are now supported as long as there is a unique index that includes all hypertable dimensions. Upserts using ON CONFLICT ON CONSTRAINT statements are currently unsupported, although this should be possible in the future. " 1f3dcd814fa2960b1d8df9d665734dc988364b35,Erik Nordström,Fri Jul 7 14:46:33 2017 +0200,Make INSERTs use a custom plan instead of triggers,"With this change, hypertables no longer rely on an INSERT trigger to dispatch tuples to chunks. While an INSERT trigger worked well for both INSERTs and COPYs, it caused issues with supporting some regular triggers on hypertables, and didn't support RETURNING statements and upserts (ON CONFLICT DO UPDATE). INSERTs are now handled by modifying the plan for INSERT statements. A custom plan node is inserted as a subplan to a ModifyTable plan node, taking care of dispatching tuples to chunks by setting the result table for every tuple scanned. COPYs are handled by modifying the regular copy code. Unfortunately, this required copying a significant amount of regular PostgreSQL source code since there are no hooks to add modifications. However, since the modifications are small it should be fairly easy to keep the code in sync with upstream changes. " f23bf58285dad6824014359f3cd710d8f4c54254,Olof Rensfelt,Fri Jul 14 14:09:58 2017 +0200,Remove empty chunks on TRUNCATE hypertable., 68a6cbac271bb5d87a83b146dfbb041d5f0df95a,Rob Kiefer,Mon Jul 24 11:30:29 2017 -0400,Fix sed syntax for getting extension version.,Previous syntax was not working on Ubuntu but fine on OSX. I've tested this one on both. 7d8ea917c59ef45287edc3b25f1d5da6fb7ab209,Thomas Desrosiers,Wed Jul 19 22:35:30 2017 -0700,Fix README example; 3 hours ago to last 3 hours,"Currently it only select data from exactly 3 hours ago, but I think this probably meant in the last 3 hours :D" 6cc5aa19bb7d9d58b6349c39d3fdc8bea5b7631e,Olof Rensfelt,Fri Jul 14 14:45:04 2017 +0200,Add update scripts from earlier versions to the extension.,"To allow upgrading versions older than the most recent one, update scripts from older versions need to be added to the extension. This is now done by including all files in the file extra_extension_files.txt to the extension. Currently that file needs to be manually updated when a new release is built. " 6d075af80211047771325348147e1d47f8d2a14d,Rob Kiefer,Wed Jul 12 16:56:51 2017 -0400,Release 0.2.0, 3c1f7522bb29fa38d02e286b71c88de1fc52f9d8,Matvey Arye,Wed Jul 12 17:50:47 2017 -0400,Make travis run update tests, a0db5004f4fa409daa83257f00b5ca3091685c3f,Matvey Arye,Wed Jul 12 16:35:54 2017 -0400,Add update script for PR #121,Also includes some update infrastructure changes. Main difference is that sql/updates/ scripts are now split into pre- and post- files. This is needed because table updates need to happen before reloading functions (some functions checked for accessing right schema at definition time). Meanwhile trigger updates need to happen after since triggers must refer to valid functions at definition time. 83c75fd154fa4432319711f9f3d9b5c6587fee3a,Matvey Arye,Mon Jul 10 23:52:25 2017 -0400,Add support for most triggers on hypertables,"This adds support for all types of triggers on a hypertable except INSERT AFTER. UPDATE and DELETE ROW triggers are automatically copied from a hypertable onto the chunks. Therefore, any trigger defined on the parent hypertable will apply to any row in any of the chunks as well. STATEMENT level triggers and iNSERT triggers need not be copied in this way. " e0eeeb9bdb09c2304149f337cc304f88b59c0bfe,Olof Rensfelt,Thu Jul 13 10:31:10 2017 +0200,"Add hypertable, chunk, and indexes size utils functions.", 4d2a65daa66b97ba2d85115c9ab2abaa939068a0,Olof Rensfelt,Tue Jul 11 15:15:37 2017 +0200,Add infrastructure to build update script files.,"SQL code is now split into setup, functions, and init files to allow a subset to be run when the extension is updated. During build, an update script is now also generated. " a5725d987558ec674bb7eba89dfcf47f23928ad5,Olof Rensfelt,Thu Jul 6 13:15:48 2017 +0200,Add support to rename and change schema on hypertable., 4888bd863551b57d183d4f1252c23cd584d5b2a6,Rob Kiefer,Sun Jul 9 19:47:19 2017 -0400,Remove out-dated docs/API in favor of docs.timescale.com, 142f58c2cc897823ab622c7e0ec827094e35620a,Erik Nordström,Mon Jul 3 13:56:07 2017 +0200,Cleanup planner and process utility hooks,"Make sure parameter names are in lowercase-underscore format. Also, rename the query tree walker change_table_name_walker() to hypertable_query_walker() to reflect that it is no long renaming anything (only setting up state in case of a query on a hypertable). " 84b0338498a697fa1c0187c68a1e60a59cf2e352,Olof Rensfelt,Wed Jun 28 14:26:42 2017 +0200,Add function to get git version of code base., 766c65b5633a59657c354f56544d59e0679bc8ba,Michael J. Freedman,Fri Jul 7 01:01:14 2017 -0400,Fix grammar nits on contributing page, 59204fa9ec0db6b833c7bb9e592d7e540c272aed,Rob Kiefer,Wed Jun 28 11:37:13 2017 -0400,Release 0.1.0 with normal extension upgrade support, 70ede45083453369efa64b97904a9dc935b773cc,Matvey Arye,Tue Jun 27 16:45:08 2017 -0400,Fix off-by-one error when creating aligned chunks,It is not an error for range_end of a new chunk to be equal to range_start of an existing chunk because range_end is exclusive. Fix logic not to throw an error in that case. 37d47b59359a516b01ca35a8938cff31fe78be79,Rob Kiefer,Tue Jun 27 13:02:22 2017 -0400,Fix warnings from gcc compiler, ad444f898267c206f8ebe451be8e76e7955120e7,Erik Nordström,Tue Jun 27 19:34:59 2017 +0300,Remove unused functions,Remove the following unused functions in order to quench compiler warnings: - chunk_constraint_from_form_data() - chunk_constraint_from_tuple() - hypercube_free() 696cc4c4f12deaecdc4a8f9c60a6d5841eaf0d0b,Erik Nordström,Thu Jun 22 15:46:39 2017 +0200,Provide API for adding hypertable dimensions,"A new public-facing API `add_dimension(table, column, ...)` makes it possible to add additional dimensions (partitioning columns) to a hypertable. Currently, new dimension can only be added to empty tables. The code has also been refactored with a corresponding internal function that is called by both `add_dimension()` and `create_hypertable()`. " 97681c2328349c449edb32a472cdb6ff2ed39f57,Matvey Arye,Thu Jun 22 19:07:32 2017 -0400,Fixes permission handling,"Previously, catalog tables were not fully protected from malicious non-superusers. This PR fixes permission handling be severely restricting permissions to the catalog and instead using SECURITY DEFINER functions to alter the catalog when needed without giving users permission to do those same operations outside of these functions. In addition, these functions check for proper permissions themselves so are safe to use. This PR also makes sure that chunk tables have the same owner as the hypertable and correctly handles `ALTER TABLE...OWNER TO` commands to keep this info in sync. " aca7f326b3d92982d175a9439d538a4e1b9760d7,Matvey Arye,Tue Jun 6 17:25:59 2017 -0400,Fix extension drop handling,"Previously, the extension could end up in a bad state if it was dropped as part of a cascade. This PR fixes that by checking explicitly for the presence of the proxy table to make sure we are not in the middle of an extension drop. Fixes #73. " 9b8a4471b84c8c8cf620fcf1283f8b68b3f9b347,Matvey Arye,Thu Jun 22 11:50:54 2017 -0400,Limit the SubspaceStore size; Add documentation.,This PR imposes a limit on the growth of the SubspaceStore size by allowing only one slice to be present for the first dimension at a time. 14ac892c88fc76cf2b4ec4f5a4f0ee9209c8f011,Matvey Arye,Mon Jun 26 18:30:32 2017 -0400,Fix possible segfault,Previously cache_destroy NULLed out memory in a MemoryContext that it just deleted. This could cause a segfault and is fixed by this PR. 44da2c0be6038651e920e56289cd3b078c17e200,Matvey Arye,Thu Jun 22 19:22:48 2017 -0400,Run pgindent on code, 7c0074434a68d40a9aad1ca506b38421d4a9bbb1,Erik Nordström,Thu Jun 22 12:21:58 2017 +0200,Remove .dockerignore file,"Since Docker builds are no longer in the main repo, there is no need for the .dockerignore file. " 6e57e3662b9d063b9dac88e46d7b6ff48300c89f,Michael J. Freedman,Thu Jun 22 14:01:32 2017 -0400,"Expand README for basic use and installation, more pointers to docs", 0f4169c1718fa7a1279d7e9b6f3e2524f0ab8637,Erik Nordström,Thu Jun 22 16:20:02 2017 +0200,Fix check constraint on dimension table,Also: - Fix indentation - Fix comments a6309dac480017c8e0313adfd45927990cf531c5,Erik Nordström,Thu Jun 22 12:02:49 2017 +0200,Fix a number of comments and cleanup unused code, ce3d630b6d5138790d2b31e28c2ec983d275e8e7,Matvey Arye,Wed Jun 21 14:00:00 2017 -0400,Run pgindent on code, 9489d065958f2292077deb29aa9603cb2d369719,Matvey Arye,Wed Jun 21 12:06:39 2017 -0400,Some minor code cleanup, 71c5e7801f2962e266ecaf5927787d362fd26d12,Erik Nordström,Wed Jun 21 20:34:12 2017 +0200,Fix and refactor tablespace support,"Tablespaces can now be associated with a hypertable using the new user-facing API attach_tablespace(). Alternatively, if the main table, which is to be converted into a hypertable, was created with an associated tablespace, that tablespace will automatically be associated also with the hypertable. Whenever a chunk is created, a tablespace will be chosen from the ones associated with the chunk's hypertable (if any). This is done in a way that ensures consistency in one dimension. I.e., if a hypertable has a closed (space) dimension with a fixed number of slices (ranges), it will ensure that chunks that fall in the same slice will alsp be stored in the same tablespace. If a hypertable has more than one closed dimension, the first one will be used to assign tablespaces to chunks. If the table has no closed dimensions, but one or more open (time) dimensions, then the first time dimension will be used. However, since open dimensions do not have a fixed number of slices, tablespaces will be assigned in a round-robbin fashion as new slices are created. Still, chunks in the same time slice will be stored in the same tablespace. " 15657b5435be73179e454af4f83e7227586c304b,Erik Nordström,Wed Jun 21 20:34:11 2017 +0200,Remove unused code and add more code documentation,- Remove get_or_create_chunk() - Fix warnings in metadata_queries + cleanups - Documentation nits - Fix mixed declaration and code warning 5452dc56d9219c93a20d48f646df5982a6fca5ab,Matvey Arye,Tue Jun 20 13:33:47 2017 -0400,Fix partiton functions; bug fixes (including memory), e75cd7e66b4ae7df8c5d31b52dc16ea4c0421169,Erik Nordström,Wed Jun 21 20:33:36 2017 +0200,Finer grained memory management,Also fix a number of memory allocation bugs and properly initialize chunks that are allocated during a scan for chunks. 3c460f02b48a450077f8f7fc6bb80ef7ea616b44,Matvey Arye,Mon Jun 19 15:39:24 2017 -0400,"Fix partitioning, memory, and tests", fe51d8d7fcdc375b68de110c5eccb9967d48896c,Erik Nordström,Wed Jun 21 20:33:34 2017 +0200,Add native scan for the chunk table,- The chunk table can now be scanned in the C code - Rename DimensionAxis to DimensionVec fc68baa8cc3221e8469dfeeb163f8ecb07f41320,Matvey Arye,Sat Jun 17 09:17:09 2017 -0400,Separate out subspace_store and add it to the hypertable object as well, c8124b8b951b2bec19370b0879434af49f18fd3d,Erik Nordström,Wed Jun 21 20:33:33 2017 +0200,Use hypercube instead of dimension slice list,Also a bunch of cleanups f5d7786eed65c056116c606a012bef76ff1979da,Matvey Arye,Fri Jun 16 10:08:11 2017 -0400,Change the semantics of range_end to be exclusive, 700c9c8a79a561b917065190235117140dbc1cdc,Erik Nordström,Fri Jun 16 11:42:03 2017 +0200,Refactor insert path in C.,Also in this commit: - Rename time/space to open/closed for more generality. - Create a Point data type for mapping a tuple to an N-dimensional space. - Numerous fixes and cleanups. 0584c47c7a60a783842c5a2a2dda68c47d653b16,Matvey Arye,Thu Jun 15 17:39:27 2017 -0400,Created chunk_get_or_create in sql with an SPI connector in C, 7b8de0c592ab5a6932ae242c825b5664583ce055,Erik Nordström,Thu Jun 15 13:04:04 2017 +0200,Refactor catalog for new schema and add native data types,This is the first stab at updating the table and data type definitions in the catalog module in the C code. This also adds functions for natively scanning the dimension and dimension_slice tables. d3bdcbaf1b614c76cda80d0344dc4177b41aa35d,Matvey Arye,Wed Jun 14 18:57:38 2017 -0400,Start refactoring to support any number of partitioning dimensions,"The current schema and code support only one ""time"" and one ""space"" dimension for partitioning. While this is probably good enough for many applications, it won't allow partitioning along two ""space"" dimensions, like longitude and latitude, for instance. This commit is a first stab at refactoring the metadata schema and internal functionality to support any number of partitioning dimensions. The idea is to think of a hypertable as partitioned in N dimensions, where a partition (chunk) is a hypercube in the N-dimensional hyperspace. Each dimension is divided into a number of ""slices"" (dimensional partitions) that each occupies a range along the dimension's keyspace axis. A dimension can either be ""closed"" or ""open"", indicating a space-like or time-like dimension, respectively. Closed dimensions have a limited number of partitions that cover the entire domain. Open dimensions have an unlimited number of partitions and thus must be created on-demand as needed for the inserted data. Note that the open/closed notation is preferred over time/space, since an open dimension could be used for something other than time (for instance, a sequence number). Conversely, closed dimensions need not be space-like as it is entirely possible to use time as a closed dimension. Another advantage of this refactoring is that it now unifies a lot of the logic for time and space that used to be separate. On a schema-level, this gets rid of partition/partitition_epoch tables and replace them with dimension, dimension_slice, and chunk_constraint. It also removes the time columns associated with a chunk and instead makes this a separate dimension stored as a chunk_constraint. " 216afe6ad72e6715a1f52f2a4dbf8b36044a3082,Rob Kiefer,Wed Jun 21 11:40:33 2017 -0400,Release 0.0.12-beta, 3192c8a6bf03c470883d91c482e053425dfccffb,Erik Nordström,Mon May 22 10:09:54 2017 +0200,Remove Dockerfile and docker.mk,"Release Docker builds are now provided in a separate repository, https://github.com/timescale/timescaledb-docker. Tests and development builds for Docker are now provided by two new scripts in the scripts directory: - `docker-build.sh` to build a development image from current sources. - `docker-run-tests.sh` to run tests for current sources through a Docker container. " 2a01ebca617a2880fabb2f312518047b427c1913,Olof Rensfelt,Fri Jun 9 14:12:19 2017 +0200,Ensure that chunks are aligned.,"Previously, chunks could end up unaligned betweeen partitions if the chunk_time_interval was updated between chunk creations. Now there is a check when a chunk is created whether there is a chunk matching the time point in another partition and if there is, the new chunk is created with the same interval. " 73622bf1ebcb97adf0882982b017c4f3bf1b9995,Matvey Arye,Tue Jun 6 18:13:33 2017 -0400,Fix default index creation duplication of indexes,"Previously there was a bug causing indexes for partition, time to be created twice. This fixes #77. " c8872fec4774cea01eb20a14af50a5d8235366d6,Matvey Arye,Mon Jun 5 21:45:52 2017 -0400,Fix command-tag for COPY and INSERT,"Previously the count returned by insert and copy was wrong because the count was reset on every execute. But, often there is an execute in the middle of an insert (i.e. to create a chunk). This fixes the logic to reset the count only at the start of the top-level statement. Fixes #64 " bfe58b61f7d69e5ae93cee9d32bebcd978ba6759,Matvey Arye,Tue May 23 17:25:50 2017 -0400,Refactor towards supporting version upgrades,Clean up the table schema to get rid of legacy tables and functionality that makes it more difficult to provide an upgrade path. Notable changes: * Get rid of legacy tables and code * Simplify directory structure for SQL code * Simplify table hierarchy: remove root table and make chunk tables * inherit directly from main table * Change chunk table suffix from _data to _chunk * Simplify schema usage: _timescaledb_internal for internal functions. * _timescaledb_catalog for metadata tables. * Remove postgres_fdw dependency * Improve code comments in sql code 28569f7bf537ddd30b4882e5c0effdd77c1b393e,Michael J. Freedman,Tue Jun 6 18:16:25 2017 -0400,Simplify README and point to new docs, db01c84c2d3134da72d19318dc48a9b8afd1dfe3,Matvey Arye,Fri Jun 2 15:55:17 2017 -0400,Make time-bucket function parallel safe,There was no reason for it not to be parallel safe before. 18db11c7cff5b1016ad016b5f664b6a41ff5f7f8,Erik Nordström,Wed May 24 14:35:30 2017 +0200,Fix timestamp test,"The timestamp test broke with Postgres 9.6.3, because that version contained an accuracy fix for the `to_timestamp()` function, giving slightly different results for some timestamp values. This fixes the timestamp test to expect the correct `to_timestamp()` output as of version 9.6.3 of Postgres. " 97bbb5979ee2ef020ecb711c261e3d391106cf94,Matvey Arye,Tue May 30 16:06:32 2017 -0400,Make constraint exclusion work with non-text partition keys,Fixes #66 f2b42ebe0e78ec197fa8b79147569eec4b2bcd7a,Matvey Arye,Wed May 31 19:21:54 2017 -0400,Fix problems with partitioning logic for padded fields,Padded fields such as character(20) did not work correctly before because of differences between type output and coercion/casting. 10f486fa5a321ae1c69fee5bdccf89adcea29f74,Erik Nordström,Thu Jun 1 13:53:15 2017 +0200,Run Travis tests in Docker without building image,"Tests are now run on Travis using a plain PostgreSQL Docker image, requiring no Docker build step or other dependencies on the host. The source directory is mounted into the running image and the build and test steps are run in the container via `docker exec`. This simplifies the build process, obviates the need fo building a new test Docker image, and uses Postgres tools (such as psql and pgdump) from the image for consistency. " d17976538bd61f7a9244cf6d436237017cb90a7d,Erik Nordström,Thu Jun 1 11:51:20 2017 +0200,Run tests on temp Postgres instance,"Tests are now run on a temporary Postgres instance, which is launched by the `pg_regress` test runner. This allows running tests without having an existing running instance with a matching configuration and also obviates the need for preloading the TimescaleDB extension. As a result, tests are simpler to setup and run, and are more reliable and consistent. " 997029a86d158cd728e6e791a695195f9c48cf61,Olof Rensfelt,Tue May 30 13:13:11 2017 +0200,if_not_exist flag to create_hypertable now works on hypertables with data as well, 347a8bdacf6fbb11342f70f0a621f197d218949f,Erik Nordström,Wed May 31 13:48:18 2017 +0200,Reference the correct column when scanning partition epochs,This fixes a bug that caused the wrong column to be referenced when scanning the partition_epoch table. The bug did not usually manifest itself because the referenced (hypertable) ID was typically incremented in unison with the desired (partition epoch) ID. ac25c72f3d62876ec21827a2c0262843f9530148,Rob Kiefer,Wed May 24 14:59:49 2017 -0400,Add installation instructions for apt (Ubuntu),This commit updates the install section to include instructions for apt on Ubuntu. It also re-arranges the structure so the configuration step is not repeated three times. 88a9849adb5172f2e2cdf4b8e8d31317c4ba2b0b,Matvey Arye,Tue May 30 10:07:19 2017 -0400,Fix bug with timescaledb.allow_install_without_preload GUC not working,Can't use a GUC variable before extension is loaded. 275f88c7057d4c0002dd7c1c32a5424c45f2f139,Rob Kiefer,Tue May 23 16:49:54 2017 -0400,Release 0.0.11-beta, 8ccc8cc156cf3a6fefd9bf4fed46c06484be81ed,Olof Rensfelt,Wed May 24 15:08:36 2017 +0200,Add if_not_exists flag to create_hypertable(),Previously create_hypertable() would throw an error when called on an already existing hypertable. This can now be skipped by setting if_not_exists argument to true. a3502b286c2ff209a1b741bd8b0c80eb080ee655,Erik Nordström,Wed May 24 14:54:20 2017 +0200,"Cleanup ""Restoring a database from backup"" in README", 2bc60c79e35ecde76ec7dff014e99547ed37c55b,Erik Nordström,Tue May 23 15:16:00 2017 +0200,Fix time interval field name in hypertable cache entry,This change makes the naming of the time interval field in the hypertable cache entry consistent with the table schema. 4638688e5b477e42a6ddbf61bf811d38f871b2d6,Matvey Arye,Sat May 20 17:15:19 2017 -0400,Improve GUC handling,This PR improves GUC handling to be more inline with the Postgres GUC system. c7bb9c04a884338897a10e5303e51c8031492ccd,Matvey Arye,Fri May 19 14:32:56 2017 -0400,Fix versions of pg_dump and pg_restore used by travis, d225b2405d3f87db3df4c7b3d59516c13e769823,Matvey Arye,Fri May 19 10:12:45 2017 -0400,Make travis print regression diff on failure, cedcafc959366b7d9fb8ea1896b2d3bc4cf2c86e,Matvey Arye,Fri May 19 09:34:10 2017 -0400,Remove setup_timescaledb() and fix pg_dump/pg_restore.,"This PR removes the need to run setup_timescaledb. It also fixes pg_dump and pg_restore. Previously, the database would restore in a broken state because trigger functions were never attached to meta tables (since setup_timescaledb() was not run). However, attaching those triggers at extension creation also causes problems since the data copy happens after extension creation but we don't want triggers fired on the data restored (that could cause duplicate rows, for example). The solution to this chicken-and-egg problem in this PR is to have a special configuration (GUC) variable `timescaledb.restoring` that, if 'on', would prevent the extension from attaching triggers at extension creation. Then, after restoration, you'd call restore_timescaledb() to attach the triggers and unset the GUC above. This procedure is documented in the README as part of this PR. " 34ad9a00a9c3e49afcb013ee3d9bb86185c56a6a,Matvey Arye,Thu May 18 09:51:42 2017 -0400,Add error when timescaledb library is not preloaded.,"Because this extension uses hooks, it should be preloaded into postgres. This PR adds an error message if it is loaded dynamically (i.e. when CREATE EXTENSION is run before without the library being preloaded). " fc4ddd65b51a3cde9f5655013369095ff9ad6d77,Matvey Arye,Wed May 17 16:54:38 2017 -0400,Fix bug with dropping chunks on tables with indexes, 32215ff6c5a66faab98c47370a8af7b0e92cf323,Matvey Arye,Wed May 17 16:30:34 2017 -0400,Add default indexes for hypertables,"Changed create_hypertable to add time DESC and partitioning_column, time DESC indexes by default (the latter only if partitioning column not null). Indexes only created if there is no index on time and partitioning_column, time respectively. Index creation can be turned off with the create_default_indexes parameter to create_hypertable (default true). " b2900f9f853ecaf05c86052e2d08d1e1f918c79f,Matvey Arye,Tue May 16 16:18:49 2017 -0400,Disable query optimization on regular tables (non-hypertables),This PR disables query optimizations on regular tables by default. The option timescaledb.optimize_plain_tables = 'on' enables them again. timescaledb.disable_optimizations = 'on' disables all optimizations (note the change from 'true' to 'on'). 75fc21ef9f3433137f29140e427e9b14a4c5a13d,Rob Kiefer,Wed May 17 18:16:46 2017 -0400,Cleanup sql/timescaledb--*.sql files on `make clean`,Old versions of the extension sql files could hang around bloating package sizes if not caught. f227db4b52a7090dcda9d18e263e7305aa65f273,Matvey Arye,Sat May 13 07:38:43 2017 -0400,Fixes command tag return for COPY on hypertables.,"Previously, COPY on hypertables always returned 'COPY 0' in the command tag. Now, returns the correct count. Fixes #40. " 1ab3c634cc319d8f236dc3f6a6ec40818dbfd59c,Solar Olugebefola,Tue May 16 10:05:35 2017 -0400,Fix typos, eb320810859f86803da3a279d9c1ea4f76acc920,Matvey Arye,Sat May 13 10:20:01 2017 -0400,Fix Invalid database ID error,This fixes a superfluous error that happened during cache invalidation. 662be94d694dd68f48d4fb4a695a5521066c9ece,Matvey Arye,Sat May 6 16:24:32 2017 -0400,"Add the first(value, time),last(value, time) aggregates","Add the first and last aggregate functions, which allows for getting the first and last value, by time, per group. Addresses issue #30. " 384a8fb0d18b9403d388cef33e517cd5db336a48,Matvey Arye,Thu May 4 16:37:15 2017 -0400,Add regression tests for deleted unit tests, 31ee92a6eee6384c154b2d392e1a226751c8c85d,Matvey Arye,Wed May 3 12:38:28 2017 -0400,Remove unit tests and sql/setup,Remove unit tests because most of our testing is in regression tests anyway and regression tests are better integrated with the Postgres extension workflow. sql/setup scripts were old and not used except for unit testing. 13d3acbcfa63d07f3754f36058df5b029004bb40,Olof Rensfelt,Wed May 10 09:56:08 2017 +0200,Fix bug with alter table add/drop column if exists,"Previously, an ALTER TABLE ADD COLUMN IF NOT EXISTS on a hypertable caused an error if the column already did exist. The reversed problem with DROP COLUMN is also fixed. Resolves issue #42. " f960c24cd4596ba69793526287523bd52e328601,Matvey Arye,Tue May 9 19:53:32 2017 -0400,Fix bug with querying a row as a composite type,"Previously a query like `SELECT ""1dim"" FROM ""1dim""` on a hypertable would fail since the change_table_walker did not properly change the types on whole row vars. That is now fixed. " 236d5fed5366a87103e3a74282018b50724de126,Rob Kiefer,Thu May 4 12:30:54 2017 -0400,Release 0.0.10-beta and add CHANGELOG, 1c4868d754a91c35e0db0ed3059f03f932114874,Olof Rensfelt,Thu May 4 13:11:41 2017 +0200,Add documentation for chunk_time_interval argument, 55fd2f2f0096c5cd296efb91ff2c393902454611,Matvey Arye,Tue May 2 17:39:49 2017 -0400,Fixes command tag return for INSERTS on hypertables.,"Previously, an INSERT on a hypertable would always return INSERT 0 0 as the command tag. This makes it return the proper number of items inserted for the second number. Fixes #28. " 48ea4330d565811f82c10ec66a0071f80f4a0f36,Matvey Arye,Wed May 3 18:11:21 2017 -0400,Run pgindent, 48eb6fdaee6ddd67deadd921c4d88a9524e4922e,Matvey Arye,Wed May 3 19:47:37 2017 +0000,Merged in add-time-bucket (pull request #148),Add time_bucket functions Approved-by: Olof Rensfelt Approved-by: Erik Nordström Approved-by: ci-timescale c3f930f6c87d2c5342ee931eaa0c683d9d22dd2c,Matvey Arye,Tue Apr 18 15:54:19 2017 -0400,Add time_bucket functions,"Adds functions to bucket timestamps. These functions are more powerful versions of date_trunc, since they can take arbitrary intervals. " b128ac2a8792f8ed1923d05571f3be268e651943,Matvey Arye,Tue May 2 13:22:53 2017 -0400,Fix bug with INSERT INTO...SELECT,"Previously, INSERT INTO...SELECT FROM hypertable was broken. This was caused by incorrectly tracking the commandType in change_name_walker. Fixes #24. " e20edf890c881a4ae7637b7891991ca9ceaa0a17,Matvey Arye,Mon May 1 18:05:08 2017 -0400,Add better error checking for index creation.,"This PR adds more regression tests for index creation and tests for more user-errors. Significantly, it checks for the presence of both the time and spaced-partition columns in unique indexes. This is needed because Timescale cannot guarantee uniqueness if colliding rows don't land in the same chunk. Fixes #29. " 7b2097a975f4a3bf4b60d2612de076e6dd18e39c,Robin Thomas,Fri Apr 28 10:56:18 2017 -0400,ran pgindent successfully with local typedefs; ended up correcting several code style discrepancies from master., 72f754a57650a9d5e92e27366656b4f514c227bc,Robin Thomas,Mon Apr 24 14:55:46 2017 -0400,"use PostgreSQL's own hash_any function as default partfunc; remove all murmur3-related source code. Alter regression tests to reflect new hash values for inputs, and a slightly different set of input data to ensure that sufficient chunks and partitions are tested. Some changes to .sh scripts in sql/setup that seem to be used only to power the ""unit tests"", which I cannot yet run successfully.", 035cfc9f72b60361530172eb0313aba5b5ccab41,RobAtticus NA,Fri Apr 28 15:16:26 2017 +0000,Merged in remove-chunk-limitation (pull request #152),Remove limitation regarding big COPY. Approved-by: ci-timescale Approved-by: Matvey Arye Approved-by: Olof Rensfelt 79946ca2ecc3507adf1c112029a3784cba2f61ac,Rob Kiefer,Wed Apr 26 17:40:24 2017 -0400,Remove limitation regarding big COPY.,"With the introduction of fixed-time chunks, this limitation no longer applies. " fd93b68656a20ae4ee2d98214fb0b829d490ed7e,Erik Nordström,Fri Apr 28 14:24:19 2017 +0000,Merged in enordstr/backend-database/enordstr/add-emacs-dir-locals-el (pull request #146),Add PostgreSQL's .dir-locals.el style file Approved-by: Olof Rensfelt Approved-by: ci-timescale Approved-by: Matvey Arye 88be5e8e06615fa2067e35070bc78998ada05dd5,Erik Nordström,Thu Apr 13 10:16:51 2017 +0200,Add PostgreSQL's .dir-locals.el style file,This adds the .dir-locals.el Emacs style settings file from the PostgreSQL source. This will make it easier for Emacs users to conform to the official PostgreSQL coding style. a037418aafe4cf77e6404cae5c6a4a53c7c95a6f,RobAtticus NA,Wed Apr 26 20:39:35 2017 +0000,Merged in update-sample (pull request #151),Remove sample data instructions and point to docs site Approved-by: ci-timescale Approved-by: Solar Olugebefola 39f4c0fd6ecdc76f91985398c6901ebee340b380,Rob Kiefer,Wed Apr 26 13:22:07 2017 -0400,Remove sample data instructions and point to docs site,This commit directs users to our docs site for using sample data so we don't have to keep both places in sync. 00d3325d6f255acdd906151504401b93079591f7,Michael Randers-Pehrson,Tue Apr 25 09:20:42 2017 -0400,Added spaces,Adding a few spaces to make it consistent with the rest of the SQL. 9015314987a8e4f776d66b693d0e7804429d54ec,banderson,Wed Apr 19 12:21:30 2017 -0700,Revised the get_general_index_definition function to handle cases where indexes have definitions other than just 'CREATE INDEX', 1c5b2f96a52c7aa182883dfa383271d6651e618a,Rob Kiefer,Mon Apr 24 19:47:14 2017 -0400,Release 0.0.9-beta, 90ef6c171bf6ab8e799613488f68924a42fb5240,Mike Freedman,Mon Apr 24 14:32:16 2017 +0000,Merged in link-cla (pull request #150),Add link to CLA Approved-by: ci-timescale Approved-by: RobAtticus NA a7cbff70502b08d32d9ff6b494554510c1af701a,Michael J. Freedman,Fri Apr 21 21:21:26 2017 -0400,Add link to CLA, d070514f2a3506117d4685b40f44b85ae1e0bb7d,Olof Rensfelt,Thu Mar 23 13:57:12 2017 +0100,Fixed time chunk ranges.,"To allow better caching and reduce the need for locking, chunks now have fixed start and stop times set when the chunk is created. " 305562dea05140ed0e76f97b01e15edc4d5f1e91,Lacey Powers,Thu Apr 6 18:41:12 2017 -0700,Adding #elif defined(__FreeBSD__) to allow building of timescaledb under FreeBSD 11., ecb5d617dace74cf6c2724294e6f9e6409e20e35,Rob Kiefer,Thu Apr 20 12:10:34 2017 -0400,Release 0.0.8-beta, a92802e5998ec986e17ceab36485033c6dbca120,Erik Nordström,Thu Apr 20 14:39:55 2017 +0000,Merged in enordstr/backend-database/enordstr/style-guide (pull request #147),Add code style guide Approved-by: Mike Freedman Approved-by: RobAtticus NA Approved-by: ci-timescale d27462e23f49d9ad85f290bc869b7390eab39bea,Erik Nordström,Thu Apr 13 12:32:11 2017 +0200,Add code style guide,This code style doc adds information about code style and formatting as well as information on tools and editors. a79a6caffdaab94a158dcb80ac8b00d7dee20105,Rob Kiefer,Thu Apr 20 09:02:50 2017 -0400,Fix a mistake in the docker run command, ed06f0a36e7e667acc1ba2bfc81753d691afc577,RobAtticus NA,Thu Apr 20 02:40:16 2017 +0000,Merged in docker-instructions (pull request #149),Add instructions to run Docker image. Approved-by: Matvey Arye Approved-by: Ajay Kulkarni Approved-by: ci-timescale 9e3bafc2f4b2bf036f627989ba82e2dc15626243,Rob Kiefer,Wed Apr 19 15:31:43 2017 -0400,Add instructions to run Docker image.,This commit adds an example how to run the Docker image so the data is persisted between restarts based on our docker-run.sh script. It also fixes a typo in the docker-run.sh script where the default DATA_DIR was not correctly set. 515f2016fc2b1dff58e1cbf8960027508c5563be,Mike Freedman,Wed Apr 19 14:20:40 2017 +0000,Merged in contributors (pull request #145),Add contribution instructions Approved-by: Erik Nordström Approved-by: ci-timescale Approved-by: RobAtticus NA 367ada624ed1ed8956a7407a324203f6c541caeb,Michael J. Freedman,Wed Apr 12 23:25:05 2017 -0400,Add contribution instructions, 7debe76ec3b6106e564e762ed8a8660ffce0f76d,Matvey Arye,Wed Apr 12 16:57:17 2017 +0000,Merged in add-copy-from-test (pull request #143),Add test for COPY TO Approved-by: Erik Nordström Approved-by: ci-timescale b134cdffd8be971845e8287d40ee98ef0c8be5c6,Matvey Arye,Tue Apr 11 18:07:15 2017 -0400,Add test for COPY TO, f975acd99187c64453a34362c5ac417c52f3f122,Matvey Arye,Wed Apr 12 16:43:19 2017 +0000,Merged in remove-dblink (pull request #142),Remove dblink dependency. Approved-by: Erik Nordström Approved-by: ci-timescale a78f1570bad0003b96e8e198b113621b278017dd,Matvey Arye,Tue Apr 11 16:43:04 2017 -0400,Remove dblink dependency.,The dblink extension is blacklisted by some cloud-hosting providers and is an unnecessary dependency for single-node operation. Since we don't plan to use dblink to implement clustering this PR removes the dependency. 6142d94d418b9756897692675b5f3c93f850210c,Rob Kiefer,Thu Apr 6 12:15:53 2017 -0400,Release 0.0.7-beta (fix bugs in tests), d59b5a4d5354cf1e52e59c54a6c81aa254d43bbb,Rob Kiefer,Thu Apr 6 12:14:47 2017 -0400,Remove all usage of hstore, 7d6c91b2e671caf32b2e9dbf09d0b7499b8a1b9f,Rob Kiefer,Thu Apr 6 11:32:21 2017 -0400,Release 0.0.6-beta, 851a3d2c23fc7d0cd1beda3b87330765e7ac2ea2,Matvey Arye,Wed Apr 5 20:54:28 2017 +0000,Merged in mat/groupby-mergeappend (pull request #137),Optimize queries with date_trunc in ORDER BY and a LIMIT. Approved-by: Erik Nordström Approved-by: ci-vast 83346079d76c5e62fb9f7f2ebccbe0b9c2b40a00,Matvey Arye,Fri Mar 31 12:30:30 2017 -0400,Make optimization of date_trunc more general.,"Handle case where there is an additional constraint on time. This makes the pathkeys for the IndexPath incompatible with upper level ordering. Since we know the appropriate transform for those pathkeys already, apply it to those paths too. " ea0c97d590cc08d50ae66825882d2152e040dc1c,Matvey Arye,Wed Mar 22 09:26:43 2017 -0400,Optimize queries with date_trunc in ORDER BY and a LIMIT.,"Common time-series rollups are of the form SELECT date_trunc(const, time) as t, aggregates... GROUP BY t ORDER BY t DESC LIMIT const2. Previously, Postgres would not optimize such queries because it could not match date_trunc(const, time) to any indexes on time, since it has no way to know that a sort on time is always a valid sort on date_trunc(const, time). This commit implements this optimization for date_trunc but it could apply to a wider range of functions, to be implemented later. " 5723f332af42768b7c036c4ec0e6fb70628c83f7,Rob Kiefer,Mon Apr 3 17:17:55 2017 -0400,Change Docker Hub organization to 'timescale, 6a2ee3320b1eb3f2067539bba6a2fc78b96e2c88,Matvey Arye,Mon Apr 3 16:01:04 2017 +0000,Merged in fix-whole-cache-invalidate (pull request #141),Fix logic for when entire cache is invalidated Approved-by: ci-vast Approved-by: RobAtticus NA Approved-by: Olof Rensfelt Approved-by: Erik Nordström e87bc774d7f3ece9e11a49d502c1e04cb2ffe2c9,Matvey Arye,Fri Mar 31 07:41:34 2017 -0400,Fix logic for when entire cache is invalidated,"Sometimes postgres wants to invalidate the entire cache. In this case it send a cache invalidation message with relid == InvalidOid. This should invalidate all the caches. Previously, it did not effect the cache of the extension state. This fixes that problem and creates one cache reset code path for both extension dropping and whole-cache reset, cleaning up some of the logic in the process. " d1dd911a62a19afec8a23cbd0494fd1d6a0113ec,Rob Kiefer,Wed Mar 29 14:40:50 2017 -0400,Fix alignment issue in README, 537d8d227c64447f1c0a29ff42c25bdbb4cd51ab,Rob Kiefer,Mon Mar 27 22:35:45 2017 -0400,Update README build status image, 003a71adca5b9cb0976a93fdf1bf22cf5d01d9df,Solar Olugebefola,Mon Mar 27 19:43:47 2017 +0000,Merged in readme-prereq (pull request #140), 44490c59289cbdad03d81e701fc07b0624c0b88d,Solar Olugebefola,Mon Mar 27 15:31:10 2017 -0400,Add psql prerequisite, cbec8399fb78cb9310317f8acebb5f2ec1831b12,Rob Kiefer,Mon Mar 27 14:15:48 2017 -0400,Update Homebrew tap name, b290c18b1b3aea31d6f00e790da8ad5d5291a4bd,Rob Kiefer,Fri Mar 24 16:47:36 2017 -0400,Make minor consistency tweaks to README, 41a9d53a5a0827a66ecdb119bd8cf175a46d8647,RobAtticus NA,Fri Mar 24 20:42:23 2017 +0000,Merged in install-options (pull request #139),Add new methods of installation to README Approved-by: ci-vast Approved-by: Solar Olugebefola 065d9ed16f852d7c34036d43eed18892d59fbc6a,Rob Kiefer,Fri Mar 24 14:04:04 2017 -0400,Add new methods of installation to README, e031dfc7955281d24f9c73c1a30aa12ec10cbb86,Rob Kiefer,Fri Mar 24 12:48:58 2017 -0400,Release 0.0.5-beta, d13f1e86dbeba6ce604389937c95b97eacca4984,Matvey Arye,Fri Mar 24 16:31:11 2017 +0000,Merged in mat/fix-simultaneous-chunk-creation-bug (pull request #138),Fix bug with simultaneous chunk creation. Approved-by: Olof Rensfelt Approved-by: RobAtticus NA Approved-by: ci-vast f908671e0402d12b7670c1d0f2630c21db4a9c67,Matvey Arye,Fri Mar 24 12:12:07 2017 -0400,Fix bug with simultaneous chunk creation.,Previously chunks could be simultaneously created for the same partition_id and a start_time and end_time of both NULL. This prevents such bugs bug adding additional unique constraints and locking the partition for chunk creation (as originally intended). ee06ac5523e4606bdce6a003415ee8989e45b030,Erik Nordström,Fri Mar 24 07:32:26 2017 +0000,Merged in enordstr/backend-database/enordstr/use-cache-memory-context-on-updates (pull request #135),Use a cache's own memory context on cache updates Approved-by: Matvey Arye Approved-by: ci-vast 5c2bef47f1e7fbc3cf6a4860c2b98e7c0a4e988b,Erik Nordström,Thu Mar 23 10:32:22 2017 +0100,Use a cache's own memory context on cache updates,"In case new allocations are made during a cache entry update, we should use the cache's internal memory context to ensure that new data is allocated and freed in a way consistent with the rest of the cache. " 9c8e40fbcfa06c09d11960d60f0d214f6a6c201c,Erik Nordström,Fri Mar 24 07:14:14 2017 +0000,Merged in enordstr/backend-database/enordstr/cleanup-tests (pull request #136),Cleanup tests and make errors less verbose Approved-by: RobAtticus NA Approved-by: ci-vast 9023de61db03c0a2609045e4728392e484e4ae5f,Erik Nordström,Thu Mar 23 12:30:54 2017 +0100,Cleanup tests and make errors less verbose,"Previously, each test set their own (although mostly the same) configuration for log output and error verbosity. This is now set globally in the test runner so that tests only need to set these configuration parameters if they need to override the defaults. The log verbosity is also reduced so that errors aren't generated with the line number of the source file that output the error. Line numbers in the output can break tests when upgrading to a new PostgreSQL version that outputs a different line number. " 3fd8bdbd7109aa426ad9426a0bd2e7dd28cebe2c,Rob Kiefer,Wed Mar 22 19:38:39 2017 -0400,Update Docker to use PostgreSQL 9.6.2, 5f5a49dffb571736180883bea369248f62659416,Rob Kiefer,Wed Mar 22 18:57:11 2017 -0400,Release 0.0.4-beta, 4c050ad815857d09d38e57ca1f8aa74d28ae5afc,Rob Kiefer,Wed Mar 22 18:56:00 2017 -0400,Update sample queries' results for datasets, 2fb71768a336a024df975707ff248aec0cb8fba0,Erik Nordström,Wed Mar 22 18:49:46 2017 +0000,Merged in enordstr/backend-database/enordstr/fix-drop-extension (pull request #131),Fix DROP EXTENSION Approved-by: Matvey Arye Approved-by: ci-vast c60b08e83a20375cc237a32b50ab6208cd0ccdda,Erik Nordström,Mon Mar 20 15:22:06 2017 +0100,Fix DROP EXTENSION,"DROP EXTENSION didn't properly reset caches and other saved state causing various errors related to bad state when the extension was dropped and/or recreated later. This patch adds functionality to track the state of the extension and also signals DROP EXTENSION to other backends that might be running, allowing them to reset their internal extension state. " 643cba1dd22c87f3f506063f2bd9f3ba4e6eb073,RobAtticus NA,Wed Mar 22 17:04:47 2017 +0000,Merged in check-postgres (pull request #134),Update Makefile to check for proper PostgreSQL version Approved-by: Matvey Arye Approved-by: ci-vast c45e1e1f2ce34ca4835630bc811238eafc36ae77,Rob Kiefer,Wed Mar 22 10:09:03 2017 -0400,Update Makefile to check for proper PostgreSQL version, e4928c083501b43fe11224090eb3b59e523e657a,enordstr NA,Wed Mar 22 14:32:37 2017 +0000,Merged in enordstr/backend-database/enordstr/cleanup-insert-state-on-error (pull request #133),Cleanup insert state on error Approved-by: RobAtticus NA Approved-by: Matvey Arye Approved-by: ci-vast 473d743927944d763616b9bd98ae0d3fc9278ea3,Erik Nordström,Wed Mar 22 13:41:51 2017 +0100,Cleanup insert state on error,"If an error is generated in any of the insert triggers, the insert state kept during a batch insert might be left in an undefined state, breaking the next insert. This patch makes sure errors are captured in the insert triggers so that the state can be cleaned up. " cdfa8bd9466cc1abab09d9d191baf8172122ab3d,RobAtticus NA,Wed Mar 22 14:11:26 2017 +0000,Merged in default-port-fix (pull request #132),Rename setup_db() and fix port for local connections Approved-by: Matvey Arye Approved-by: Olof Rensfelt Approved-by: ci-vast cb90eef350ca9a7a24dc9e9b0b9a54c5737f2a0f,Rob Kiefer,Tue Mar 21 16:27:54 2017 -0400,Rename setup_db() and fix port for local connections,"If a user attempts to setup a database while not connecting using the network, port is NULL and thus fails constraint checks. Instead, we now use the default Postgres port of 5432 when this happens. Also, setup_db() is renamed to setup_timescaledb() for clarity in the presence of other extensions. " b2682a7b5731322f6f8bac6d1162134c65b55336,enordstr NA,Wed Mar 22 09:42:17 2017 +0000,Merged in enordstr/backend-database/enordstr/cache-stats-and-cleanup (pull request #130),Add cache statistics and do minor cleanup Approved-by: Matvey Arye Approved-by: ci-vast Approved-by: Olof Rensfelt 89692c97617564e84f2a9b051d474d1ebee09ddf,Erik Nordström,Fri Mar 17 13:53:25 2017 +0100,Add cache statistics and do minor cleanup,"Track statistics for cache hits and misses in the cache module. Currently not exposed to SQL, but might be useful for internal debugging. " 9ef0e6c900944b5e6e2c80066770cb7b19ad7f0d,Matvey Arye,Tue Mar 21 15:37:36 2017 +0000,Merged in add-sql-query-test (pull request #129),Add query tests for ORDER BY time DESC queries LIMIT queries Approved-by: ci-vast Approved-by: enordstr NA Approved-by: Olof Rensfelt e0d63e915c3684ec30a43718a1b990ccfee03b31,Matvey Arye,Sun Mar 19 20:13:32 2017 -0400,Add query tests for ORDER BY time DESC queries LIMIT queries,These tests show that non-aggregated queries that have an ORDER BY time DESC LIMIT x structure perform well. Postgres processes such queries by using a time DESC index and then performs a MergeAppend. Some further optimization to avoid touching unnecessary tables due to constraints can be added but they are not a priority. Much worse is that these optimizations do not work when grouping by time (either through integer division or using date_trunc). In order to use the non-aggregated query optimizations we had to create an index with time as the leading field and no WHERE clause on the index. b32c3789cf537873a0246e6805b45f2e8043e014,Solar Olugebefola,Tue Mar 21 15:29:37 2017 +0000,Merged in device-to-location (pull request #128), 03e4618bf048f9bd58357732f6c98561735f7ed9,Solar Olugebefola,Fri Mar 17 12:40:54 2017 -0400,Change 'device_id' to 'location_id', f66ae0a8a7c92b5bb2cbe1793bd6c7461867aacd,Rob Kiefer,Thu Mar 16 18:39:31 2017 -0400,Add build status image to top of README, 5e01e150a16dda8168c946aed55369b6dc4e5d31,Rob Kiefer,Thu Mar 16 16:55:39 2017 -0400,Add travis build file, 20d1cc46de5674571a860739f6510760572e05d0,Rob Kiefer,Thu Mar 16 16:47:21 2017 -0400,Clarify limitation for user triggers on hypertables, 1b92de0f47c68eb019f4e166d132f9853276b459,enordstr NA,Thu Mar 16 09:58:47 2017 +0000,Merged in enordstr/backend-database/enordstr/move-typedefs-script (pull request #125),Move generate_typedef.sh to scripts directory Approved-by: Olof Rensfelt Approved-by: ci-vast b4c9055184ccf99381b59a6a5c45f614371265ac,Erik Nordström,Thu Mar 16 10:15:09 2017 +0100,Move generate_typedef.sh to scripts directory, 267a38238b744087fdf0201dd97b9f4382abb668,enordstr NA,Thu Mar 16 09:47:46 2017 +0000,Merged in enordstr/backend-database/enordstr/add-depfiles-to-gitignore (pull request #126),Add dependency files to .gitignore Approved-by: Olof Rensfelt Approved-by: ci-vast f6adb5da658639d7727cd584424056d09613eea4,Erik Nordström,Thu Mar 16 10:17:16 2017 +0100,Add dependency files to .gitignore, b2b33875d08a6d3c702c2606e7f2fa7a07344d8c,enordstr NA,Thu Mar 16 08:53:41 2017 +0000,Merged in enordstr/backend-database/enordstr/header-dependencies (pull request #121),Generate object dependencies to force 'make' to recompile on changes to headers Approved-by: Matvey Arye Approved-by: Olof Rensfelt Approved-by: ci-vast e5b489b1bac16bed50ec64cd0deede4a3c07d87b,Erik Nordström,Wed Mar 15 09:06:56 2017 +0100,Generate object dependencies to force 'make' to recompile on changes to headers,"Previously, running 'make' did not recompile source files when changes were made to header files. Now the compiler generates dependency files that will pick up changes to header files and recompile any source files that include the updated header. Also, remove insert.h, which is not needed. " b4d62637096d6e0d044c07d6ad9197a52b250a23,RobAtticus NA,Wed Mar 15 21:46:31 2017 +0000,Merged in docs-link (pull request #124),Add link to docs site in README Approved-by: ci-vast Approved-by: Ajay Kulkarni cd05cba75c0cffeb7aa05a5b3ae1630fe682e0bb,Rob Kiefer,Wed Mar 15 15:42:25 2017 -0400,Add link to docs site in README, 25947958f68c6e1bc7e89d653173efc218c4a6bf,Ajay Kulkarni,Wed Mar 15 17:14:31 2017 +0000,Merged in readme-edits-ajay (pull request #123),Fix bullets 8ae80b1bbe7687e6cf12f4a96f2686ca29dd4ca4,Ajay Kulkarni,Wed Mar 15 10:13:17 2017 -0700,Fix bullets, 5259a0cc4c8f4df99ba310b629b82c7510f7792f,Ajay Kulkarni,Wed Mar 15 17:10:29 2017 +0000,Merged in readme-edits-ajay (pull request #122),Clarify README Approved-by: ci-vast Approved-by: Matvey Arye Approved-by: Solar Olugebefola cb5a5d3bd141fc610bffae2c1ab4c85ae69d6379,Ajay Kulkarni,Wed Mar 15 09:28:24 2017 -0700,Clarify README, 940e0049bc73f618292013f1a30995f0475dfd6b,Rob Kiefer,Wed Mar 15 10:08:13 2017 -0400,Update README with PostgreSQL version, 69a2cf23c8157200b045aa2aaa45e3daaef5a647,Matvey Arye,Tue Mar 14 19:35:42 2017 +0000,Merged in insert-main (pull request #118),Make insert operate on main table instead of on root table Approved-by: enordstr NA Approved-by: ci-vast Approved-by: RobAtticus NA e9c554a901d0791593bd714d00b6c2e748c518e6,Matvey Arye,Mon Mar 13 16:26:58 2017 -0400,Make insert operate on main table instead of on root table, 012fe8749994728524b666d8f8e567015e680633,Matvey Arye,Mon Mar 13 15:17:47 2017 -0400,Forbid users from creating custom triggers on hypertables, 01c4cdc0dd53ffd9a104a719cbba9e9e575fa229,enordstr NA,Tue Mar 14 13:19:30 2017 +0000,Merged in enordstr/backend-database/enordstr/break-up-timescaledb-c (pull request #120),Break up timescaledb.c Approved-by: Olof Rensfelt Approved-by: ci-vast 7236b9723b784897c7085c43a8c2a6b01290b19e,Erik Nordström,Tue Mar 14 12:51:18 2017 +0100,Rename internal planner data types for consistency, 001c41f84a392addcff2601cade6d9c3de9dccf5,Erik Nordström,Tue Mar 14 12:31:34 2017 +0100,Rename IobeamLoaded() to extension_is_loaded(), aa51e221f397e6c9224b5aaca126edae62e0ee67,Erik Nordström,Tue Mar 14 12:22:47 2017 +0100,Break up timescaledb.c into separate source files.,"This patch refactors the source code so that a bunch of unrelated code for the planner, process utilities and transaction management, which was previously located in the common file timescaledb.c, is now broken up into separate source files. " e641ae4f5c1e81e1da773b81a5933fa51e54673a,Erik Nordström,Tue Mar 14 11:19:40 2017 +0100,Fix broken comment., 5f4b158c8369e85bedb8981e8266647492bff712,enordstr NA,Tue Mar 14 12:37:29 2017 +0000,Merged in enordstr/backend-database/enordstr/no-storage-for-negative-cache-entries (pull request #119),Avoid allocating hypertable cache storage for negative entries. Approved-by: Olof Rensfelt Approved-by: ci-vast 852ba7ee971a6e18463b54c7decd1949a0deee84,Erik Nordström,Tue Mar 14 11:20:23 2017 +0100,Avoid allocating hypertable cache storage for negative entries.,"The hypertable cache stores negative cache entries to speed up checks for tables that aren't hypertables. However, a full hypertable cache entry was allocated for these negative entries, thus wasting cache storage. With this update, the full entry is only allocated for positive entries that actually represent hypertables. " a1da9bf5c51f6cec96d7ebfa6d93b0a4b58991c2,Erik Nordström,Tue Mar 14 10:08:28 2017 +0100,Fix misleading comment, bffc594d85747246256c98044cef8b735abda4c8,Rob Kiefer,Mon Mar 13 20:34:25 2017 -0400,Release 0.0.3-beta & fix install script, 1e9c215535fb244b4cf493f9e69ed4e15229a46c,Rob Kiefer,Mon Mar 13 15:22:37 2017 -0400,Update README with note about upgrades, 3e37745dc0e9c0de9944721eabde2dbd8c14ab4c,Rob Kiefer,Mon Mar 13 15:17:42 2017 -0400,Release 0.0.2-beta (and update build scripts), 84d32e2fbe901df6c4ea1c1d4fe2bfdbf92e5772,Matvey Arye,Mon Mar 13 18:30:39 2017 +0000,Merged in planner-use-cache (pull request #117),Make the planner use metadata cache Approved-by: enordstr NA Approved-by: ci-vast 1f13354bf946b6960b8b5af911dc147306a491b1,Matvey Arye,Thu Mar 9 18:18:15 2017 -0500,Make the planner use metadata cache,"Previously, the planner used a direct query via the SPI interface to retrieve metadata info needed for query planner functions like query rewriting. This commit updates the planner to use our caching system. This is a performance improvement for pretty much all operations, both data modifications and queries. For hypertables, this added a cache keyed by the main table OID and added negative entries (because the planner often needs to know if a table is /not/ a hypertable). " 089c4275675a525e71a5d5754cebb274c4ebf0b3,Matvey Arye,Mon Mar 13 14:12:49 2017 +0000,Merged in fix-permissions (pull request #106),Allow non-superusers to work with the db (but also mess up the catalog) Approved-by: RobAtticus NA Approved-by: Olof Rensfelt Approved-by: ci-vast Approved-by: enordstr NA fbdcab186fdeef1f93369f40b0378812e3e5233a,Matvey Arye,Wed Mar 8 12:40:49 2017 -0500,Allow non-superusers to work with the db (but also mess up the catalog),"Previous to this commit non-superusers could not do anything inside a database with the timescale extension loaded. Now, non-superuser can create their own hypertables and work inside the db. There are two big caveats: 1) All users have read/write permissions to the timescaledb catalog. 2) Permission changes applied to the main tables are not propagated to the associated tables. " 21d4ebb23bd726547960829338b2ad64b35d440a,enordstr NA,Sun Mar 12 09:33:50 2017 +0000,Merged in enordstr/backend-database/enordstr/move-table-defs-to-catalog (pull request #116),Move catalog table definitions to catalog.h Approved-by: ci-vast Approved-by: Matvey Arye 3a033483563f9c2a9112484333351b09be503e48,Erik Nordström,Fri Mar 10 17:00:27 2017 +0100,Move catalog table definitions to catalog.h,"This patch continues work to consolidate catalog information and definitions to the catalog module. It also refactors the naming of some data types to adhere to camelcase naming convention (Hypertable, PartitionEpoch). " 98d6d6a6fa2f11b06e917aa8175e9bc0d86b7055,enordstr NA,Fri Mar 10 19:21:04 2017 +0000,Merged in enordstr/backend-database/enordstr/insert-trigger-sorting (pull request #113),Refactor insert path to use triggers instead of temp copy table. Approved-by: Matvey Arye Approved-by: ci-vast 3c1e9cc4cefbf22b9a7d3fd90bc24a2b8fee6c05,Erik Nordström,Wed Mar 8 14:25:34 2017 +0100,Refactor insert path to use triggers instead of temp copy table.,"This patch refactors the insert path to use insert triggers instead of a temporary copy table. The copy table previously served as an optimization for big batches where the cost of inserting tuple-by-tuple into chunks was amortized by inserting all tuples for a specific chunk in one insert. However, to avoid deadlocks the tuples also had to inserted in a specific chunk order, requiring adding an index to the copy table. With trigger insertion, tuples are instead collected over a batch into a sorting state, which is sorted in an ""after"" trigger. This removes the overhead of the copy table and index. It also provides a fast-path for single-tuple batches that avoids doing sorting altogether. " a089baf9b79c6c13ad0da927283141c684989d96,Rob Kiefer,Fri Mar 10 12:52:23 2017 -0500,Clarify on UsingSamples.md that dataset names are clickable, fdfe87f2dbeaa2ea40e9c61453dab6930086bd8a,Andrew Staller,Fri Mar 10 17:50:23 2017 +0000,Merged in andrew-readme-edit (pull request #115),config file name Approved-by: RobAtticus NA 2be3f7caf1d7afa791c0c08ef6b8c6f0344811a8,Andrew Staller,Fri Mar 10 17:47:11 2017 +0000,Update README with correct .conf filename, 7e7e71baed6013be10e29ffc5be02ef7358752ea,Ajay Kulkarni,Fri Mar 10 17:21:16 2017 +0000,Merged in ajay-readme-edits (pull request #114),Add mention/link to technical paper. Approved-by: RobAtticus NA Approved-by: ci-vast b6640ca34f61f94c8372c91cb7a4e14fddde7a81,Ajay Kulkarni,Fri Mar 10 12:14:11 2017 -0500,Add mention/link to technical paper., d719ed8976acf8bea437314557f68fd0de506033,enordstr NA,Fri Mar 10 17:04:50 2017 +0000,Merged in enordstr/backend-database/enordstr/chunk-replica-native-scans-master (pull request #111),Perform native scans for chunks (replicas) Approved-by: Olof Rensfelt Approved-by: ci-vast Approved-by: Matvey Arye 4cd4df3add835589d8b23f266140dd27b9ccc936,RobAtticus NA,Fri Mar 10 16:49:09 2017 +0000,Merged in limitations (pull request #110),Update README with additional limitations Approved-by: ci-vast Approved-by: Ajay Kulkarni c7b9dbbc101d067d8b3b2789cd6a8c4d6b43ebf4,Rob Kiefer,Thu Mar 9 16:26:52 2017 -0500,Update README with additional limitations, dd7ad1664474b3b06c14e573bc14e118fe508d30,RobAtticus NA,Fri Mar 10 16:43:29 2017 +0000,Merged in migrate-script (pull request #109),Add a migrate data script for large datasets Approved-by: ci-vast Approved-by: enordstr NA 55736b926211558968c8fcc3bd3f1ff0b9fe07c1,Rob Kiefer,Thu Mar 9 16:21:10 2017 -0500,Add a migrate data script for large datasets,"Currently TimescaleDB does not close chunks mid-insert, so large batches will overfill a chunk. This commit adds a script to split large CSV files into smaller batches to allow reasonable closing of chunks. " d1ad3afd49ce40867783d648507ecafeb6f99d4a,Erik Nordström,Thu Mar 9 15:52:05 2017 +0100,Perform native scans for chunks (replicas),"Previously, chunk replicas were retreived with an SPI query. Now, all catalog items are retrieved with native scans, with the exception of newly created chunks. This commit also refactors the chunk (replica) cache, removing some data structures that were duplicating information. Now chunks are cached by their ID (including their replicas) instead of just the set of replicas. This removes the need for additional data structures, such as the replica set which looked like a chunk minus time info, and the cache entry wrapper struct. Another upside is that chunks can be retrieved from the cache directly by ID. " 88eed92733a76b55c880da4f8fae12911a1b27c8,Olof Rensfelt,Fri Mar 10 13:24:49 2017 +0000,Merged in olof_rensfelt/backend-database/ore/add_rename_test (pull request #107),Add tests for rename and drop hypertables Approved-by: ci-vast Approved-by: RobAtticus NA Approved-by: Matvey Arye Approved-by: enordstr NA de3370deb8172ced876f338cff6011df104c6a08,Olof Rensfelt,Thu Mar 9 16:15:51 2017 +0100,Add tests for rename and drop hypertables, 79b7ba0f6315df6583bfdb325f2f6b32a3ef6afd,enordstr NA,Fri Mar 10 08:47:32 2017 +0000,Merged in enordstr/backend-database/enordstr/fix-compilation-warnings (pull request #112),Fix a number of compilation warnings Approved-by: Olof Rensfelt Approved-by: ci-vast 8ef6d415456ab091e95b383ad903d958b676c274,Erik Nordström,Fri Mar 10 09:31:35 2017 +0100,Fix a number of compilation warnings,"Quench compilation warnings about ""mixed declaration and code"" and comparing different type of enums. " 329c309ecd1d9cadb4ef36894cadc11e91bbc2e2,Matvey Arye,Thu Mar 9 17:07:54 2017 +0000,Merged in catalog-multiple-indexes (pull request #108),Enable catalog to support multiple indexes per table Approved-by: ci-vast Approved-by: enordstr NA eb09ccc980f019ac185982f8de831441cf72d8cd,Matvey Arye,Thu Mar 9 10:17:33 2017 -0500,Enable catalog to support multiple indexes per table, b1410271ebc6aaae8d489d858996e4adea12f49d,Rob Kiefer,Tue Mar 7 15:16:30 2017 -0500,Update README to clarify using postgres as user, fbd079f242295f5fad481393bf268938d5c350e5,Rob Kiefer,Tue Mar 7 14:57:20 2017 -0500,Update sample query output on UsingSamples.md,"With update to TimescaleDB, we needed to create new datasets. This commit updates the sample output to reflect the new datasets. " 4a2f89c7e66eaa4137020ecd6d89b39e6ed212ee,RobAtticus NA,Tue Mar 7 19:44:24 2017 +0000,Merged in api-docs (pull request #102),Add an API reference in Markdown Approved-by: Matvey Arye Approved-by: ci-vast Approved-by: Ajay Kulkarni aefb6a2d19945170515908ae3f1ac23cdf338b5c,Rob Kiefer,Mon Mar 6 15:57:50 2017 -0500,Add an API reference in Markdown and update README, 57769ef5321244ace82704cb39b787f528927a79,Matvey Arye,Tue Mar 7 14:28:56 2017 -0500,Fix bug with planner not excluding chunks by space partition., 3b113c163fa3739ebc585f5063d1eb7af4fa6884,Matvey Arye,Tue Mar 7 17:06:54 2017 +0000,Merged in fast-path-close (pull request #103),Implementing a c-based fast path for checking whether to close a chunk Approved-by: ci-vast 00b69ac01040811f229a5a87da1578363c91092c,Matvey Arye,Mon Mar 6 16:32:22 2017 -0500,Change close-chunk logic to use a c-based fastpath,This change is a performance improvement. Previously each insert called a plpgsql function to check if there is a need to close the chunk. This patch implements a c-only fastpath for the case when the table size is less than the configured chunk size. 2379a4483a30fd02603fa505f9ee82e83475fda9,Mike Freedman,Tue Mar 7 17:03:38 2017 +0000,Merged in add-license (pull request #104),Add Apache 2.0 LICENSE and NOTICE Approved-by: ci-vast Approved-by: RobAtticus NA 4bf76d830c72367c03b1f3492ad58840b53c26a4,Michael J. Freedman,Mon Mar 6 12:09:53 2017 -0500,Add Apache 2.0 LICENSE and NOTICE, 4c558a9466aef71069c71f7c6d006c7f6613d795,Ajay Kulkarni,Tue Mar 7 16:43:08 2017 +0000,Merged in readme-edits (pull request #105),Add instructions on including shared libraries in postgres.conf to README Approved-by: Matvey Arye Approved-by: ci-vast d35d069bdda3dc37a72f9a71b0327e775c33c33f,Ajay Kulkarni,Tue Mar 7 11:24:15 2017 -0500,Add instructions on including shared libraries in postgres.conf to README, 37609935196e1c923fe6a9318d0b9ffe37d26c0a,Rob Kiefer,Mon Mar 6 15:36:29 2017 -0500,Renames setup_single_node() to setup_db(), 9aa86e1549e169e609b3f60630ab0ba7e8540841,Rob Kiefer,Mon Mar 6 15:31:54 2017 -0500,Sets initial version and extension description., 969d452ec1e8f4cbacfee8f270994f15dbb77eac,Matvey Arye,Mon Mar 6 20:21:42 2017 +0000,Merged in pg_indent_run (pull request #99),pgindent format Approved-by: ci-vast Approved-by: enordstr NA Approved-by: RobAtticus NA 32c45b75b27e3f690236a9d1c8d13a025316ad2f,Matvey Arye,Mon Mar 6 15:20:00 2017 -0500,formatting with pgindent, 73f4dcaaf0d0c51cbdefb56547d1348e51f06643,enordstr NA,Mon Mar 6 20:12:14 2017 +0000,Merged in enordstr/backend-database/enordstr/at-least-two-partitions-with-partitioning-column (pull request #101),Default to asking for number of partitions when partitioning column set. Approved-by: RobAtticus NA Approved-by: ci-vast Approved-by: Matvey Arye 11938ebb0822e0bf99380ad94193626a288a4057,Erik Nordström,Mon Mar 6 13:28:09 2017 +0100,Default to asking for number of partitions when partitioning column set.,"Since create_hypertable() allows you to optionally specify a partitioning column, it makes sense to default to one partition when no column is specified and asking for the number of partitions when a column is specified and the number of partitions is not (instead of defaulting to one). This patch also changes the order and type of partitioning-related input arguments to create_hypertable() so that the number of partitions can easily be specified alongside the partitioning column and without type casting. " 57ef40531656754275ea54123d8e2edbaca53f31,RobAtticus NA,Mon Mar 6 16:19:00 2017 +0000,Merged in rename-timescale (pull request #93),Rename iobeamdb to TimescaleDB Approved-by: ci-vast ee3ad4678d5c7355194f9762ed20d4b45d721aa5,Rob Kiefer,Tue Feb 28 12:54:07 2017 -0500,Rename iobeamdb to TimescaleDB, e17457261f477cca068c90bb93fd0cc6e2321202,enordstr NA,Mon Mar 6 12:47:18 2017 +0000,Merged in enordstr/backend-database/enordstr/no-partitioning-column-segfault-fix (pull request #100),Fix segfault due to uninitialized partitioning info. Approved-by: Olof Rensfelt Approved-by: ci-vast Approved-by: Matvey Arye 675aecc61468ad08a70b33dc55e5cfd1cfb02db8,Erik Nordström,Mon Mar 6 13:10:29 2017 +0100,Fix segfault due to uninitialized partitioning info.,Hypertables that have no partitioning column set will experience a segmentation fault at query time when a partitioning info struct is read without being properly initialized. This change will zero- initialize the struct so that the partitioning column name will be a NULL pointer instead of garbage if not set. 0e36e3287fe14d0f158576b3023f2c325c5a26e6,Matvey Arye,Sun Mar 5 20:24:43 2017 +0000,Merged in cevian/ordered-insert (pull request #96),Cevian/ordered insert Approved-by: ci-vast Approved-by: enordstr NA 4d4ac78ef510aa25f795ef69dfe99a2395959ebb,Matvey Arye,Thu Mar 2 22:07:55 2017 -0500,cleanup, 2404940315f66d10222c5d3cd2704955ecbaa617,Matvey Arye,Thu Mar 2 19:31:58 2017 -0500,getting rid of move_plan in cache. creating the plan caused a lock to be taken on the chunk table -- a lock which lead to a deadlock, 64e8ec1877f23de828d7c19ce4af1f4f27fe79e0,Matvey Arye,Tue Feb 28 22:54:19 2017 -0500,Ordering inserts to avoid deadlocks, 2093b0ac6f9cf3ee30613a1ecd46ac0a988c9cb1,Rob Kiefer,Thu Mar 2 13:48:52 2017 -0500,Fix verbosity level from normal to default, 19a780526c184de78bd3d6f9a69c3fcf2b4c6fd3,RobAtticus NA,Thu Mar 2 18:33:22 2017 +0000,Merged in test-verbosity (pull request #98),Reduce verbosity to remove line nums from output. Approved-by: ci-vast Approved-by: enordstr NA Approved-by: Olof Rensfelt 5478231f3e4981cef90c8dfad9f69247f926b572,Rob Kiefer,Thu Mar 2 13:09:33 2017 -0500,Reduce verbosity to remove line nums from output., d1af4cfab08cb10e3e991f66c36f2eadac7f3bc4,RobAtticus NA,Thu Mar 2 16:41:41 2017 +0000,Merged in update-readme-testing (pull request #97),Approved-by: Ajay Kulkarni Approved-by: Solar Olugebefola Approved-by: ci-vast e7169dfcb55ffc61612cdcdf81b7dc36ea63b5e9,Rob Kiefer,Wed Mar 1 16:01:43 2017 -0500,Update README to move testing instructions.,Having the testing instructions in the installation instructions was confusing and error-prone due to having to set up your postgresql.conf file. It is not a necessary step for installation. eae34e1bbb8cf34a252ffaf1b0f0ae950700b09c,enordstr NA,Thu Mar 2 14:38:55 2017 +0000,Merged in enordstr/backend-database/enordstr/cached-inserts-with-native-scans (pull request #95),Cached inserts with native scans Approved-by: Matvey Arye Approved-by: ci-vast eebd9bbbc13972b065297b180366ee18cf5868f0,Erik Nordström,Wed Mar 1 19:46:31 2017 +0100,Use native scan for chunks.,"The chunk catalog table is now scanned with a native scan rather than SPI call. The scanner module is also updated with the option of of taking locks on found tuples. In the case of chunk scanning, chunks are typically returned with a share lock on the tuple. " 61668c837a88f5ae4424a8d653fc270fb8104ed3,Erik Nordström,Wed Mar 1 13:20:01 2017 +0100,Allow heap/index scans to abort early if condition is met.,"Aborting scans can be useful when scanning large tables or indexes and the scan function knows that a condition has been met (e.g., all tuples needed have been found). For instance, when scanning for hypertable partitions, one can abort the scan when the found tuple count equals the number of partitions in the partition epoch. " 1daec06ce668fd9700e8f3dbf13c895b2e567c53,Erik Nordström,Wed Mar 1 00:53:37 2017 +0100,Use native scans for partition epochs and partitions.,"This patch refactors the code to use native heap/index scans for finding partition epochs and partitions. It also moves the partitioning-related code and data structures to partitioning.{c,h}. " c92fa28ea8c35a3fff6c6479297bd4d1dc2cecf2,Erik Nordström,Tue Feb 28 17:51:40 2017 +0100,Add scanner module implementing both heap and index scans., f99669c8801c934e2e29187e8cd3f42fb876bf5e,Erik Nordström,Sun Feb 26 20:41:20 2017 +0100,Use direct index scan for hypertable lookups.,"This is a first stab at moving from SPI queries to direct heap/index scans for cleaner and more efficient code. By doing direct scans, there is no need to prepare and cache a bunch of query plans that make the code both slower and more complex. This patch also adds a catalog module that keeps cached OIDs and other things for catalog tables. The cached information is updated every time the backend switches to a new database. A permission check is also implemented when accessing the catalog information, but should probably be extended to tables and schemas in the future. " 67ad21ee36c0944aab1c6d26861eb0ce15419901,Matvey Arye,Wed Feb 15 12:12:49 2017 -0500,Cache hypertable metadata information for faster inserts.,Add caches for hypertable metadata to make it faster to map INSERT rows to the chunk they should go into. 86d2e7889484851b5f76fe5842e06956a1503e14,RobAtticus NA,Wed Mar 1 20:33:50 2017 +0000,Make sure UsingSamples steps are in ordered list, 8a24110886fda99ec43b0f0d54a3529eb0e129e4,enordstr NA,Tue Feb 28 19:59:18 2017 +0000,Merged in enordstr/backend-database/enordstr/add-num-partitions-to-epoch (pull request #92),Add number of partitions to partition_epoch table. Approved-by: Matvey Arye Approved-by: RobAtticus NA Approved-by: Olof Rensfelt c09f6013fa8a62606e5b71a5d8518f287a56aa17,Erik Nordström,Tue Feb 28 20:01:20 2017 +0100,Add number of partitions to partition_epoch table.,"There are two reasons for adding the partition count to the partition_epoch table: * It makes the partition_epoch more self-describing as it makes it easy to see how many partitions are in the current epoch as well as past ones. * It simplifies native code that can read the partition epoch, allocate memory for the right number of partitions, and finally scan the partition table filling in each entry. " 6dfe84036e365be6bc60d26f6f2fff732df694ab,RobAtticus NA,Fri Feb 24 01:30:37 2017 +0000,Merged in sample-instructions (pull request #83),Add examples to Sample readme Approved-by: Matvey Arye Approved-by: ci-vast Approved-by: Ajay Kulkarni 1787793b64b58c45e4c26a5552fd7e14965b341f,enordstr NA,Thu Feb 23 20:52:44 2017 +0000,Merged in enordstr/backend-database/enordstr/close-chunks-at-end-of-insert (pull request #89),Fix chunk-related deadlocks. Approved-by: ci-vast Approved-by: Matvey Arye Approved-by: RobAtticus NA bc0567616634c7b8d9e054e1c5f757c33dcfb320,Rob Kiefer,Thu Feb 16 16:00:16 2017 -0500,Add examples to Sample readme, 4f31531babf10d656775ed64104652e543d4d0c5,Olof Rensfelt,Thu Feb 23 19:48:52 2017 +0000,Merged in olof_rensfelt/backend-database/ore/block_alter_table (pull request #88),Block renaming of hypertables Approved-by: ci-vast Approved-by: RobAtticus NA Approved-by: Matvey Arye e3fabf993a5c32a3fd35c85978b48546daeac4d9,Erik Nordström,Thu Feb 16 10:08:16 2017 +0100,Fix chunk-related deadlocks.,"This patch fixes two deadlock cases. The first case occurred as a result of taking partition and chunk locks in inconsistent orders. When creating the first chunk C1 in a table, concurrent INSERT workers would race to create that chunk. The result would be that the transactions queue up on the partition lock P, effectively serializing these transactions. This would lead to these concurrent transactions to insert at very different offsets in time, one at a time. At some point in the future, some n'th transaction Tn queued up on P would get that lock as the preceeding inserters T1-(n-1) finish their inserts and move on to their next batches. When Tn finally holds P, one of the preceeding workers starts a new transaction that finds that it needs to close C1, grabbing a lock on C1 and then on P. However, it will block on P since Tn already holds P. Tn will also believe it needs to close C1, thus trying to grab a lock on C1, but will block, causing a deadlock. The second case can occur on multi-partition hypertables. With multiple partitions there are more than one open-ended chunk at a time (one for each partition). This leads to a deadlock case when two processes try to close (and thus lock) the chunks in different order. For instance process P1 closes chunk C1 and then C2, while process P2 locks in order C2 and C1. The fix for the first case is to remove the partition lock altogether. As it turns out, this lock is not needed. Instead, transactions can race to create new chunks, thus causing conflicts. A conflict in creating a new chunk can safely be ignored and it also avoids taking unecessary locks. Removing the partition lock also avoids the transaction serialization that happens around this lock, which is especially bad for long-running transactions (e.g., big INSERT batches). The fix for the second multi-partition deadlock case is to always close chunks in chunk ID order. This requires closing chunks at the end of a transaction, once a transaction knows all the chunks it needs to close. This also has the added benefit of reducing the time a transaction holds exclusive locks on chunks, potentially improving insert performance. " 549b69ea810d9ac02ff2f7a89d3663991d74e6b8,Olof Rensfelt,Thu Feb 23 12:55:39 2017 +0100,Block renaming of hypertables, 5e44e996f45588652035eabb986323d458a96875,Olof Rensfelt,Thu Feb 23 08:05:20 2017 +0000,Merged in olof_rensfelt/backend-database/ore/remove_clustering (pull request #87),Make all tests use create_single_db.sql Approved-by: ci-vast Approved-by: RobAtticus NA Approved-by: enordstr NA 220e938930afd663004ec906af3bdf77c44de5b8,Olof Rensfelt,Wed Feb 22 15:18:56 2017 +0100,Make all tests use create_single_db.sql, ffc146c27f2b964b4eba9b74647d276015dd314c,Olof Rensfelt,Wed Feb 22 13:46:17 2017 +0000,Merged in olof_rensfelt/backend-database/ore/remove_clustering (pull request #86),Remove clustered tests. Approved-by: ci-vast Approved-by: enordstr NA 406dbb7bd6175fcb199af5ae182f7b578ee00192,Olof Rensfelt,Tue Feb 21 10:14:56 2017 +0100,Remove clustered tests., 1d62f206cc093c173c6f3d9140f2d331882707d0,Olof Rensfelt,Tue Feb 21 11:00:10 2017 +0000,Merged in olof_rensfelt/backend-database/ore/fix_docker_build (pull request #84),Add .dockerignore Approved-by: RobAtticus NA Approved-by: Matvey Arye Approved-by: ci-vast 4e247e0248c9fd3fdaca34f26ce36e931024f691,Olof Rensfelt,Mon Feb 20 10:34:59 2017 +0100,Add .dockerignore,Remove data/ adn docs/ from the docker build process. e8473b7ce84e99dff4219af57f5f2698c5432035,Ajay Kulkarni,Tue Feb 14 16:38:13 2017 +0000,Merged in readme-edits (pull request #81),"Readme edits for usability, clarity, consistency, and general awesomeness. Approved-by: ci-vast Approved-by: RobAtticus NA " 7a87b0f18b4df36a1693bc14deb34af94811d14c,enordstr NA,Tue Feb 14 13:19:56 2017 +0000,Merged in enordstr/backend-database/enordstr/custom-time-conversion (pull request #80),Fix conversion between TIMESTAMP and internal BIGINT UNIX time representation. Approved-by: Matvey Arye Approved-by: ci-vast 4683d8e03edec29f096a4291200a2418a42f4dc4,Erik Nordström,Fri Feb 10 16:08:49 2017 +0100,Fix conversion between TIMESTAMP and internal BIGINT UNIX time representation.,"Currently, the internal metadata tables for hypertables track time as a BIGINT integer. Converting hypertable time columns in TIMESTAMP format to this internal representation requires using Postgres' conversion functions that are imprecise due to floating-point arithmetic. This patch adds C-based conversion functions that offer the following conversions using accurate integer arithmetic: - TIMESTAMP to UNIX epoch BIGINT in microseconds - UNIX epoch BIGINT in microseconds to TIMESTAMP - TIMESTAMP to Postgres epoch BIGINT in microseconds - Postgres epoch BIGINT in microseconds to TIMESTAMP The downside of the UNIX epoch functions are that they don't offer the full date range as offered by the Postgres to_timestamp() function. This is because of the required epoch shift might otherwise overflow the BIGINT. All functions should, however, offer appropriate range checks and will throw errors if outside the range. " c332f32dbfc852a09db894fc1917e10e3ce1f8d8,Ajay Kulkarni,Fri Feb 10 08:28:04 2017 -0700,"Readme edits for usability, clarity, consistency, and general awesomeness.", a910d100aafea5ab0c18adb5615028f2125d9ccf,enordstr NA,Mon Feb 13 19:09:27 2017 +0000,Merged in enordstr/backend-database/enordstr/fix-compiler-warnings (pull request #82),Fix C-style and compiler warnings Approved-by: ci-vast Approved-by: RobAtticus NA ab3070aa893eb5d3b7e67c17005899657983bdb3,Erik Nordström,Mon Feb 13 11:45:04 2017 +0100,Fix C-style and compiler warnings,- Add emacs-style indentation settings headers - Fix compiler warnings related to C style - Indentation fixes 03c95659151f0edfea6cfe7fd2d5ea9c6f8cbd3e,Rob Kiefer,Thu Feb 9 15:28:38 2017 -0500,README tweaks, 027dc5608564dae4cd1354fd842f0c3225709186,Rob Kiefer,Thu Feb 9 14:36:42 2017 -0500,Update sample doc instructions and add links, 5c4fdb4a9877bff08e1866033eddaf9b4b27de30,enordstr NA,Thu Feb 9 14:12:48 2017 +0000,Merged in enordstr/backend-database/enordstr/readme-and-retention (pull request #76),Update README and add user-facing API to drop chunks Approved-by: Ajay Kulkarni Approved-by: Matvey Arye Approved-by: ci-vast 81d966f90849e01bee6243535684976374c913ab,Erik Nordström,Tue Feb 7 16:13:05 2017 +0100,Update README and add user-facing API to drop chunks,- Add complete database setup instructions to the README. - Add drop_chunks() user-facing API - Add retention policy section to README 87035d9916d205dbc31bbe3d97257c86eb094353,Olof Rensfelt,Wed Feb 8 09:09:18 2017 +0000,Merged in perf-improvements (pull request #78),Performance improvements Approved-by: ci-vast 58b2d2b066d2800142a12ab6b50a552d68957ed7,enordstr NA,Wed Feb 8 08:25:07 2017 +0000,Merged in enordstr/backend-database/enordstr/time-conversion-fix (pull request #77),Fix time conversion bug Approved-by: RobAtticus NA Approved-by: ci-vast Approved-by: Matvey Arye 321babee811a1cf6d50d1168d2b9ad6e24a23029,Matvey Arye,Tue Feb 7 17:22:00 2017 -0500,cleanup, f12a880d8db6f4e036d62fc22b749ba3b3ddb783,Matvey Arye,Tue Feb 7 16:31:43 2017 -0500,prepare hypertable_info plan, f1fcf5147666df223e4440c858b8b455c4f5f4d2,Matvey Arye,Tue Feb 7 13:27:48 2017 -0500,Make inserts use temporary tables for performance., 68a060e8a7ea74237919feb109e736cf811d3cf6,Matvey Arye,Fri Feb 3 18:02:35 2017 -0500,various performance fixes, 839710d567ed95927ccf9dc6f45ac2ff39678ef1,Erik Nordström,Tue Feb 7 18:23:43 2017 +0100,Fix time conversion bug, 25ad65b47c96cba69509abbd925bfc736a5eb0b7,enordstr NA,Tue Feb 7 15:15:46 2017 +0000,Merged in enordstr/backend-database/enordstr/simplify-cluster-setup (pull request #75),Simplify cluster setup Approved-by: Olof Rensfelt Approved-by: ci-vast 036c40886c8727520dd298adc943a501a5af7d4b,Erik Nordström,Fri Feb 3 15:49:25 2017 +0100,Simplify cluster setup,"Setting up single node is now: ``` CREATE EXTENSION IF NOT EXISTS iobeamdb CASCADE; select setup_single_node(); ``` To setup a cluster do (on meta node): ``` CREATE EXTENSION IF NOT EXISTS iobeamdb CASCADE; select set_meta(); ``` on data node: ``` CREATE EXTENSION IF NOT EXISTS iobeamdb CASCADE; select join_cluster('metadb', 'metahost'); ``` This assumes that the commands are issued by the same user on both the meta node and the data node. Otherwise the data node also has to specify the user name to use when connecting to the meta node. " d4374ce1f3820c742bb4b57a03fa2e760fc7a941,RobAtticus NA,Thu Feb 2 19:32:23 2017 +0000,Merged in sample-docs (pull request #74),Initial draft of sample docs. Update README Approved-by: Ajay Kulkarni Approved-by: ci-vast a2198b7c72b1714b34f3bd7aab03400ba9bc295d,Rob Kiefer,Thu Feb 2 12:31:13 2017 -0500,Initial draft of sample docs. Update README, acd387d29e79f8a86f411ee9fae901ac15b3b593,RobAtticus NA,Thu Feb 2 15:34:44 2017 +0000,Merged in bash-setup (pull request #72),Add bash script for setting up single-node cluster with user-defined DB name Approved-by: ci-vast Approved-by: Matvey Arye Approved-by: enordstr NA 574d6202cdd98bf7ce4920fec42db394027c5241,Rob Kiefer,Wed Feb 1 16:00:08 2017 -0500,Add bash script for setting up single-node cluster with user-defined DB name, d3a3670aa6d4f1fe458f9cc2ec4098672cfda942,Matvey Arye,Tue Jan 31 22:03:14 2017 +0000,Merged in install-nits (pull request #71),nit to installation/docker ce2879a2c8ee9e15444452f40b0119bc52f7a5ba,enordstr NA,Tue Jan 31 19:17:56 2017 +0000,Merged in enordstr/backend-database/enordstr/refactor-dir-structure (pull request #70),Refactor directory structure and tests 7b94c573ba0df56687522cfdbfa0b32bd1d177b7,Erik Nordström,Tue Jan 31 14:56:41 2017 +0100,Refactor directory structure and tests,- Directory structure now matches common practices - Regression tests now run with pg_regress via the PGXS infrastructure. - Unit tests do not integrate well with pg_regress and have to be run separately. - Docker functionality is separate from main Makefile. Run with `make -f docker.mk` to build and `make -f docker.mk run` to run the database in a container. 4fd1d9d299698df98e7e039e724018ad25f89cf6,Olof Rensfelt,Tue Jan 31 15:41:02 2017 +0000,Merged in olof_rensfelt/backend-database/ore/add_server_port (pull request #66),Add port to server configuration 95f618bc0675961283372e42d118bd4a00f7fd2b,Olof Rensfelt,Tue Jan 31 12:18:17 2017 +0000,Merged in olof_rensfelt/backend-database/ore/build-fixes (pull request #62),runs test docker om different port b919df869f6bc1a7e1130dd668bb6a931c5b80ab,Olof Rensfelt,Fri Jan 27 10:32:00 2017 +0100,runs test docker om different port, 17d6d0cff668c4d8059f577e37b233b26e3f0549,Olof Rensfelt,Fri Jan 27 18:34:41 2017 +0100,Add port to server configuration Change the remote node schema_name, 1523ad89c6b55779db8968eb66c2c73fa74df5de,enordstr NA,Tue Jan 31 10:07:09 2017 +0000,Merged in enordstr/backend-database/enordstr/tablespaces (pull request #63),Add support for tablespaces 8abc301e51b59b0cf904faabd5b67eb346c44701,Erik Nordström,Tue Jan 24 17:48:29 2017 +0100,Add support for tablespaces,Identically named tablespaces need to exist on each data node before a hypertable is created from a table that references a tablespace. b52dc630ee01da2924b6c48bab866764b77d6f0b,enordstr NA,Tue Jan 31 09:12:06 2017 +0000,Merged in remove-distinct (pull request #67),Remove distinct tables. d4747af795da5998eed5f0f3b532fd1d3c8b732e,Matvey Arye,Mon Jan 30 14:25:48 2017 -0500,Remove distinct tables.,Distinct tables will need to be re-implemented on a per-chunk basis in the future. 7dfad229275b995aa1890fc55b83b07066b6dd8e,RobAtticus NA,Mon Jan 30 19:59:44 2017 +0000,Merged in truncate (pull request #64), 6c3ee77d23abc044c20ef5dbf788103276bbcd72,Matvey Arye,Mon Jan 30 14:56:50 2017 -0500,Adding API doc link, bd364ae023b24c53fa32574d442a083de8be8dcc,Rob Kiefer,Fri Jan 27 12:36:15 2017 -0500,Block TRUNCATEs on meta tables, b3ae7208b5d8b1fed6ee54998391a90ed96c3dad,RobAtticus NA,Mon Jan 30 17:28:05 2017 +0000,Merged in create-hyper-meta (pull request #65), 69f30abf7ce95fd5a8df9613afc3dce4efa70e44,Rob Kiefer,Fri Jan 27 15:27:51 2017 -0500,Throw error when creating hypertable on non-data node, 9c9ef0981f047db598dbc51d0ab1101e6e397764,Matvey Arye,Mon Jan 30 17:07:13 2017 +0000,Merged in schema-cleanup (pull request #61),Allow extension to be installed in a different schema 95afe4f2a06e7bfe4a1768a5b0e301ed2cd3087e,Matvey Arye,Thu Jan 26 17:08:37 2017 -0500,Allow extension to be installed in a different schema,This changes the extension to allow it be installed in a different schema. Part of this change also adds a partioning_func_schema option so that the partitioning_func does not need to live in search_path. 7f7e9eff687e71175a0d79d140f96037242dfa69,enordstr NA,Mon Jan 30 15:50:45 2017 +0000,Merged in enordstr/backend-database/enordstr/allow-0-time-epoch (pull request #57),Allow partition epochs to start and end at time 0. 334e302c8cc0429f086d1f681154b690c506da79,Erik Nordström,Wed Jan 25 12:00:43 2017 +0100,Allow partition epochs to start and end at time 0.,"This aligns the partition epoch table with the chunk table, which allows data in chunks to start and end at time 0. " 16e97be00d745eadfb409393f84b5105022946c3,RobAtticus NA,Fri Jan 27 16:51:33 2017 +0000,Merged in trigger-rename (pull request #60), 22cbee18fe4c3c0e9243e8dd059b4221f73b0fed,Rob Kiefer,Thu Jan 26 15:52:11 2017 -0500,Common function for trigger errors, 32b91eeded15d49a623529c4afc71461b0e61502,Rob Kiefer,Thu Jan 26 15:12:41 2017 -0500,Rename triggers from on_create_X to on_change_X, ee2d914d34ab8ab901ff9ba0e7f187eba8633d57,RobAtticus NA,Thu Jan 26 20:19:49 2017 +0000,Merged in update-delete (pull request #58), 3cc1940c577fed8003f4a856f698421500574aa9,Matvey Arye,Wed Jan 25 14:21:50 2017 -0500,Add support for UPDATE and DELETE commands, 459f9fd3d54d19c7b5f07464d087313ed7cd9394,Matvey Arye,Thu Jan 26 19:56:58 2017 +0000,Merged in enordstr/backend-database/enordstr/only-time-partitioning (pull request #52),Allow partitioning in only time dimension d87a6ee5a06b7f6f51ec02d52d4ae07b3be1a190,Erik Nordström,Tue Jan 24 12:53:26 2017 +0100,Allow partitioning in only time dimension,"This allows creating hypertables as follows: ``` CREATE TABLE only_1dim(time timestamp, temp float); SELECT create_hypertable('only_1dim', 'time'); INSERT INTO only_1dim VALUES('2017-01-12T08:11:03', 23.4); ``` It is implemented by making the specification of a partitioning column optional, and when NULL the number of partitions will be set to 1. " 4a6e707bdd936bd9bdac46cc677568c82d6864bb,enordstr NA,Thu Jan 26 09:05:06 2017 +0000,Merged in name-to-serial (pull request #59),Change hypertable.name to hypertable.id with type SERIAL 2646480657f91484cd154081804127a290ce8cec,Rob Kiefer,Wed Jan 25 16:49:30 2017 -0500,Change hypertable.name to hypertable.id with type SERIAL, 5dde9d4ea2fab80d9d35922e5db5d43a83f6b72d,RobAtticus NA,Wed Jan 25 17:55:19 2017 +0000,Merged in cleanup (pull request #55), 657c929aa8c437b81e02885a90311777e52800c8,Rob Kiefer,Tue Jan 24 15:34:36 2017 -0500,Cleanup some identation and alignment issues, 115179476eb78643f06c1b2a3b210f81532c4968,Olof Rensfelt,Wed Jan 25 16:32:10 2017 +0000,Merged in olof_rensfelt/backend-database/ore/check_conflict (pull request #51),Removed ON CONFLICT DO NOTHING statements where it works. 2b08c4a78a204804fbe6d783b28024351ad3eece,Olof Rensfelt,Tue Jan 24 14:02:18 2017 +0100,Removed ON CONFLICT DO NOTHING statements where it works., 974632119078e39668114278caf9060458faef21,Matvey Arye,Wed Jan 25 16:09:30 2017 +0000,Merged in usability-fixes (pull request #56),Misc usability fixes 97205a7cec32216526ce40a9dc637061a5cfc5ee,Matvey Arye,Tue Jan 24 17:26:39 2017 -0500,Misc usability fixes, a86bb7e9d016ee25fc8ebeab95bd0b9c0948fa14,Matvey Arye,Tue Jan 24 20:29:34 2017 +0000,Merged in docker-fixes (pull request #54),Changes to Makefile and associated scripts to make docker commands act as expected + to mount a data volume ce5e1aecd7c1cf269496ab854bc0cc538928a5a7,Matvey Arye,Tue Jan 24 14:42:20 2017 -0500,Changes to Makefile and associated scripts to make docker commands act as expected + to mount a data volume, 20489d2a6ce6cb7be491f176aeebf2ab93150b78,Rob Kiefer,Tue Jan 24 13:07:21 2017 -0500,Cleanup indentation in schema_info.sql, a1d25906b57a544b7df4cffee7fd6aa70730d6de,RobAtticus NA,Tue Jan 24 17:40:01 2017 +0000,Merged in move-internal (pull request #53), 7f548f6f917a6361b272c85a4b4f5335da4ef451,Rob Kiefer,Tue Jan 24 12:25:44 2017 -0500,Remove ioql helper functions, b25e71aa4eaa31c406f7ee31e995d18b55fe227e,Rob Kiefer,Mon Jan 23 16:42:20 2017 -0500,Move more functions into _iobeamdb_internal., 3f0917746758b6920bb136d3316c50f5f9083727,enordstr NA,Tue Jan 24 17:19:41 2017 +0000,Merged in enordstr/backend-database/enordstr/remove-ioql (pull request #50),Remove IOQL 35276966c480b49d3d1a45b4d019917560c1fe52,Erik Nordström,Tue Jan 24 09:12:06 2017 +0100,Remove IOQL, c40fe596c4c2b563210701be37fd1baf270f7700,Rob Kiefer,Tue Jan 24 11:46:53 2017 -0500,Replace all _sysinternal instances in tests, 2b8855543bb7a8ae71037fdedfbb303155da4a1c,Rob Kiefer,Tue Jan 24 11:30:29 2017 -0500,Fix missed references to _meta, 5c68b739e2f02009501cfb8227c976f9cc21708f,RobAtticus NA,Tue Jan 24 16:10:10 2017 +0000,Merged in rename-sysinternal (pull request #47), 2594d0b6ceed243c90688e795be825d733ec69e6,Rob Kiefer,Mon Jan 23 14:15:29 2017 -0500,Rename _sysinternal to _iobeamdb_internal and _meta to _iobeamdb_meta., d2ba389c83463ddf53a09b5980434a72be254f7a,Matvey Arye,Tue Jan 24 15:18:59 2017 +0000,Merged in fix-pg_dump (pull request #48),Fix (and test) pg_dump of extension tables 3efccceae0709077ad36ab6ab9ffc2b73685f9db,Matvey Arye,Mon Jan 23 16:22:06 2017 -0500,Fix (and test) pg_dump of extension tables, c49a51a4f0c3474a5d135160c35cf2c419a24515,enordstr NA,Tue Jan 24 08:59:12 2017 +0000,Merged in enordstr/backend-database/enordstr/delete-retention (pull request #41),Allow dropping hypertable chunks older than some timestamp 3976919e76990f70520cd98d57ced4a713000777,Erik Nordström,Tue Jan 17 10:32:24 2017 +0100,Allow dropping hypertable chunks older than some timestamp,- Can only be done on metanode currently - Does not implement a per-table retention 53221884735e0b8c7caa8a15893a337f17d1c2d3,Ajay Kulkarni,Mon Jan 23 23:14:03 2017 +0000,Merged in readme-edits (pull request #49),Readme changes for clarity (and my first repo commit!) 386d7bb5aa75c1bb2a785ee887d6bd661ea16445,Ajay Kulkarni,Mon Jan 23 17:58:54 2017 -0500,Readme changes for clarity (and my first repo commit!), 5e7e9efc08ed7d7b04a276844505f4c536b0d4af,Matvey Arye,Mon Jan 23 21:15:18 2017 +0000,Merged in enordstr/backend-database/enordstr/remove-kafka (pull request #45),Remove kafka-related code 4a6b409fc9bf9bd01e33734910be6fe2c5817287,Olof Rensfelt,Mon Jan 23 20:33:10 2017 +0000,Merged in olof_rensfelt/backend-database/ore/add_regression_test_output (pull request #44),Add output from regression test to stdout. a77599c20dee93538736517e55f1076e44962134,Olof Rensfelt,Mon Jan 23 11:34:52 2017 +0100,Add output from regression test to stdout. Take regression test files as command line args to run a subset of tests., bb474c9c8e07306648ffec4f8f8b73c8c3d949ec,Olof Rensfelt,Mon Jan 23 18:34:42 2017 +0000,Merged in olof_rensfelt/backend-database/ore/fix_create_hypertable (pull request #46),Create schema for hypertable if it does not exists. 41abf6ba4c48695de365d84adb4b383b7aad8cc3,Matvey Arye,Mon Jan 23 16:43:37 2017 +0000,Merged in user-input-checks (pull request #40),User input checks a3b9881b6ed18302ad30f7bf7bf2f141491f85e3,Matvey Arye,Thu Jan 19 18:09:57 2017 -0500,Adding checks for user input to ddl functions, 2f28a9d183e897e898718e8ec106c1c8f5dce452,Matvey Arye,Thu Jan 19 16:25:40 2017 -0500,added some more testing to create_hypertable twice case, f0dbb392d430051fd55b73cff0ab4ccb7884c8cd,Olof Rensfelt,Mon Jan 23 11:34:52 2017 +0100,Create schema for hypertable if it does not exists., 661b7cdc94c96faaed381fef9b5923a551e06df7,Erik Nordström,Mon Jan 23 14:04:33 2017 +0100,Remove kafka-related code, 188b1b39dd81d2b311b0ac59308c295a844d1fdd,enordstr NA,Mon Jan 23 08:26:22 2017 +0000,Merged in fix-placement-code (pull request #43),Fix chunk placement code ce2af03ea0cf036acb2ec4db1e06f84309fd131a,Matvey Arye,Sat Jan 21 15:35:52 2017 -0500,Fix chunk placement code,Also fix getting chunk_sizes of tables that are remote. Also fix timestamp conversion logic if timezone setting different across nodes. 436739da342c8a88f50826e48224c1880bb4e50a,RobAtticus NA,Fri Jan 20 23:30:40 2017 +0000,Merged in catalog (pull request #42), 2e13805f19864bbfd1b942f2dc6c6600bd18ad00,Rob Kiefer,Fri Jan 20 14:53:55 2017 -0500,Move to _iobeamdb_catalog, 6f5abfe55bbf3bbd2afe04365d636790ea99a306,Matvey Arye,Thu Jan 19 15:32:50 2017 -0500,Adding documentation for indexing, 8ddcfcacac080cdfc837956ea148bbc83f478d53,RobAtticus NA,Thu Jan 19 20:05:01 2017 +0000,Merged in readme (pull request #39), 29bc320cfb8ad5828209f48132417566ac751b28,Rob Kiefer,Tue Jan 17 15:25:19 2017 -0500,Add Getting Started instructions to README, 24e49136d872f819b77b25cb5ed1396c189e342c,Matvey Arye,Thu Jan 19 19:50:41 2017 +0000,Merged in fix-double-hypertable-create-2 (pull request #38),Fix double hypertable create 2 d835eadf48c9e236a74253dfbbba5940488d9854,Matvey Arye,Wed Jan 18 15:28:23 2017 -0500,fix error output to go into regression golden files; suppress some notice output, 74abecb297626af5341de334dfc6a20275fd6dec,Matvey Arye,Tue Jan 17 12:24:58 2017 -0500,Make calling hypertable_create twice throw an error, ce2926861493a911fabb2cbc1cb66845e574b6a6,Rob Kiefer,Thu Jan 19 13:06:51 2017 -0500,Change field to column in C code, d4d77db7850a7c4bc3b20d5a98935a5e4f4a4e42,RobAtticus NA,Thu Jan 19 17:58:36 2017 +0000,Merged in field-column (pull request #37), 5e117ab9c25ad352dda26560e16cf23646485e34,Rob Kiefer,Wed Jan 18 16:35:00 2017 -0500,Change field table to hypertable_column, e82cce85cf987717eb040a78a45e2887b28c7956,Rob Kiefer,Tue Jan 17 16:46:21 2017 -0500,Change instances of field to column for partitioning/time, 564c4f2bbe9d5352040dc35cdb6e8ceca08427d6,RobAtticus NA,Thu Jan 19 17:29:30 2017 +0000,Merged in remove-namespace (pull request #35), e113da7c6677546ee20131814b777bdcd8c3b4bd,Rob Kiefer,Tue Jan 17 12:21:28 2017 -0500,Rename all instances of namespace to hypertable, aab87f4f78f3f8616c867ec66d4d1731157376ee,Matvey Arye,Thu Jan 19 17:16:03 2017 +0000,Merged in combined-single-node (pull request #33),Allow single-node deployments that use one database. 3192b39db8e3e416ea7fed2c11daaec790960ee1,Matvey Arye,Wed Jan 18 18:28:33 2017 -0500,Create isolated APIs for meta and data. All communication now uses the API., 64526426c5d33bfaf44917f6548984d8c7edf8f4,Matvey Arye,Mon Jan 16 13:43:25 2017 -0500,Allow single-node deployments that use one database.,Allowing database deployments that use the same database for both the meta and node roles simplifies operations when iobeamdb is deployed on a single-node. It allows the database to operate without any cross-network communication (either through dblink or postgres_fdw) and thus offers stronger consistency guarantees. a2c181d7d044bbc608348b330aa60ba243be3e31,Olof Rensfelt,Wed Jan 18 23:16:30 2017 +0100,Merged in olof_rensfelt/backend-database/ore/drop_table (pull request #36),Fixed drop hypertable. 874276ee05d0ca5bd7c900870bd63863b21c968a,Olof Rensfelt,Wed Jan 18 20:56:48 2017 +0100,Fixed drop hypertable., 4765422b3a822db906175968b198c27855cf58d8,RobAtticus NA,Tue Jan 17 14:25:01 2017 -0500,Merged in timestamp-fix (pull request #34), 677ea3fd7d7b8eb2b9dedc7f58480e8d5c4d6577,Matvey Arye,Mon Jan 16 20:00:40 2017 -0500,Fix to timestamp conversion logic, 6cc0b103deb7717d0b3799d8ff9cdd360f57ec87,Olof Rensfelt,Mon Jan 16 21:08:44 2017 +0100,Merged in olof_rensfelt/backend-database/ore/drop_table (pull request #30),Drop hypertable added 3eec10a43cd4ff16ead1eb7d79ae9475d16e37d2,Olof Rensfelt,Mon Jan 16 10:21:20 2017 +0100,Add drop hypertable support., 21db9f62dc01529d540c49f1f1c21931053bbb8c,Matvey Arye,Mon Jan 16 10:53:54 2017 -0500,Merged in partition-exclusion-opt (pull request #28),Do parse tree transform to allow postgres to exculde partitions when quals of the form partion_key = const are present 3463a74b783057f3637474e5dbeae5fdf3e2eac1,Matvey Arye,Wed Jan 11 12:44:10 2017 -0500,Do parse tree transform to allow postgres to exculde partitions when quals of the form partion_key = const are present, 4e7a8f9fec9aff10f105b764f12e0ba4a4b2dbaf,RobAtticus NA,Mon Jan 16 10:49:32 2017 -0500,Merged in update-startup (pull request #31), 0ec5095cf5c9d12a68a47cb486a0483582c1d629,Rob Kiefer,Thu Jan 12 14:51:40 2017 -0500,"Cleanup Makefile targets, script names, and README", e419dbe62122a3a98f706ef02959eff5a4ebc5eb,enordstr NA,Mon Jan 16 14:47:09 2017 +0100,Merged in enordstr/backend-database/enordstr/add-hypertable-rename (pull request #32),Rename add_hypertable() -> create_hypertable() a27f71b614c7e18f67d5c4123dfd5949bafaa963,Erik Nordström,Fri Jan 13 16:03:31 2017 +0100,Rename add_hypertable() -> create_hypertable(), 38f0468fafcc22496eb7c2bddfbbf1cbe43cc210,Matvey Arye,Fri Jan 13 14:59:22 2017 -0500,Merged in transactional-dblink (pull request #21),This creates an infrastructure for tranactional node->meta dblink commands. 228b8eeab767d2a29cf9894d41a9c40265f5f918,Matvey Arye,Thu Jan 12 18:14:36 2017 -0500,handle dblink correctlty for meta->data too, dfd97fb5f1397cd15aca06b461a9e131f1156d90,Matvey Arye,Tue Jan 3 18:46:31 2017 -0500,This creates an infrastructure for tranactional node->meta dblink commands., 26a2e84ed4639e28a335285220672ca8b61b4f90,enordstr NA,Fri Jan 13 17:59:16 2017 +0100,Merged in enordstr/backend-database/enordstr/chunk-closing (pull request #17),Add support for closing chunks based on size 5506d80067a6b1ef70430df21985771f4ef4bb12,Erik Nordström,Fri Dec 23 15:41:39 2016 +0100,Add support for closing chunks based on size, 5a9c19d99b4afdeecb600082b5d44c2df1a8cceb,RobAtticus NA,Fri Jan 13 11:05:52 2017 -0500,Merged in remove-temp-table (pull request #29), fd0aea7c4e1cda308c71c79f98b725beb02fd7e8,Rob Kiefer,Wed Jan 11 16:01:56 2017 -0500,Remove create_temp_table func only used in tests, 90d836a5b288a9a6e30707a0e1fe56a4d37055bf,RobAtticus NA,Wed Jan 11 15:20:33 2017 -0500,Merged in tables-docs (pull request #27), 93bbba8e0b0c9acaab751303a3638468710ca39e,Rob Kiefer,Tue Jan 10 17:48:28 2017 -0500,Update docs to expand and clarify data structures and concepts, 5bee8d7ee4830f02d107ee1b53cc96822d6d972b,Rob Kiefer,Mon Jan 9 16:50:12 2017 -0500,Format SQL setup files, da6af1a6cc5ee38a83c3952568eca84d588a2b2b,RobAtticus NA,Mon Jan 9 10:30:49 2017 -0500,Merged in more-cleanup (pull request #26), 5e43448613f049fc9b59720ca59fcc08dc54ee07,Rob Kiefer,Fri Jan 6 16:03:25 2017 -0500,Add more doc comments for meta triggers and functions, 050575feb676cec7bc6d5b765e0dc280ea3f37c5,Rob Kiefer,Thu Jan 5 16:22:47 2017 -0500,Remove unused names.sql, ffc2be0a2c01d846fa92d7960badbacf726b9fee,Matvey Arye,Mon Jan 9 10:29:38 2017 -0500,Merged in multi-partition-insert-fix (pull request #25),Fix insert of batches that span multiple space partitions f7253fe12a389de08a8f54bb661a99f4ab94f7ed,Matvey Arye,Thu Jan 5 14:05:05 2017 -0500,Fix insert of batches that span multiple space partitions, 6ef973141c38ae9cf232ee84e995c66e18040748,Matvey Arye,Fri Jan 6 13:51:17 2017 -0500,adding forgotten files, e7faa8911a3de735909a26c321302abcc0fc55da,Matvey Arye,Fri Jan 6 12:41:00 2017 -0500,Regression tests cleanup to make tests repeat output less between tests (i.e. ioql_query does not repeat output from insert.sql), 74fe879150dbdb6275417c8121253910ca60b8b3,RobAtticus NA,Fri Jan 6 12:12:22 2017 -0500,Merged in error-update-main (pull request #24), 24d75909c366c3b6cf4c3959b0854579bedb229c,Rob Kiefer,Wed Jan 4 17:04:21 2017 -0500,Throw error on UPDATE/DELETE attempts to main table, dd1f32bd43de984f47c036073388eaefad56ac75,Matvey Arye,Thu Jan 5 12:37:50 2017 -0500,Merged in time-column-name (pull request #23),Allow time columns to take arbitrary names as well as be integer or timestamp(with or without time zone) types. c5d826d93199fe0bf6db0895114aaf0059cb51f3,Matvey Arye,Wed Jan 4 16:49:22 2017 -0500,support for time columns of type timestamp, 2cb4be6abd9d00c9e5c973148271c30cc4ee80ee,Matvey Arye,Mon Jan 2 17:30:06 2017 -0500,Allow time columns to take arbitrary names, 1d50810ccbb2503703412b12425e13a752557be5,RobAtticus NA,Thu Jan 5 11:48:31 2017 -0500,Merged in cleanup-insert (pull request #20), 831aa6ab9c6e1bb8c4f4f75b72b067c58d45e602,Rob Kiefer,Tue Jan 3 17:18:10 2017 -0500,"Cleanup insert functions, move helpers to _sysinternal.", 0acc544e4362710d4a7ee71677cf0b33b98df4f7,RobAtticus NA,Thu Jan 5 11:46:18 2017 -0500,Merged in cleanup (pull request #22), bee65feeb458dcfef79d8b958436873a17d541a1,Rob Kiefer,Wed Jan 4 15:29:08 2017 -0500,"Add code docs and cleanup for ddl, table creation, and field triggers.", 1645a5a97b4c40b06921a12e4afd04fbfa0c8388,RobAtticus NA,Tue Jan 3 10:37:41 2017 -0500,Merged in insert-non-bulk (pull request #19), cca1d654d7de312ddf411cef18d53d76c5bb547a,Rob Kiefer,Mon Jan 2 16:29:17 2017 -0500,"Remove insert temp table, just use main table as temp insert location.", 94f31613706f79a39ba1a85847f9860573103098,Rob Kiefer,Tue Dec 20 16:29:18 2016 -0500,Add logic for creating RULE to redirect INSERTs on root table., ff7762ea9ec5dbf45f44f12abd6bcfa1c63411ed,Rob Kiefer,Tue Dec 20 15:33:12 2016 -0500,Update hypertable metadata table to include 'insert_temp_table' and keep the insert temp table in-sync with root in terms of fields., ee36949beb5deb6d7a025138448a528125fcd2b3,Matvey Arye,Mon Jan 2 16:49:55 2017 -0500,restrict main table redirection to SELECTS, 5ee64ddc9145d35ffadec1314b41816493ff4c0d,Matvey Arye,Mon Jan 2 11:20:45 2017 -0500,"Change main table -> replica table replacement code to respect the ONLY keyword. If inheritance not used (ONLY supplied), then the replacement does not happen.", 170e030d5f62dd81029b7479f20266a508cb01d2,Matvey Arye,Fri Dec 23 23:08:22 2016 -0500,Upadting docs, 94ab3b95d668ab5c58a5d99fe41d778974762d68,Matvey Arye,Fri Dec 23 16:21:01 2016 -0500,Upadting docs, ba990d4a1048430a01f1940d133a01d6b0a2d0eb,Matvey Arye,Fri Dec 23 15:01:30 2016 -0500,Merged in mat/c-based-main-table-queries (pull request #18),Mat/c based main table queries 490019d20b41bdacefdeb1ea01f435e1f8a45c84,Matvey Arye,Fri Dec 23 02:09:51 2016 -0500,Use C parse-tree modifications to transform queries on main tables to that on replica tables, 9089fe992a2cd038603f7e1c67dc776b6d576b44,Matvey Arye,Thu Dec 22 16:56:57 2016 -0500,make hypertable create transactional, 8a72b2dd5881a7dfbc686b664b73fc6809d2b455,Matvey Arye,Fri Dec 23 14:25:57 2016 -0500,Merged in row-insert (pull request #15),Row-based temporary table insert cbcae063e49cb1a65fc3f6ff303907bb768ddd65,Rob Kiefer,Thu Dec 22 15:21:59 2016 -0500,Move to row-based temporary tables for insert., e3d9ce6d89420dad184091685dfa317c33dcd107,Rob Kiefer,Wed Dec 21 17:45:26 2016 -0500,Convert insert_data to only deal with one hypertable batch at a time. Remove unused insert_data_one_partition., 0405a3ebb4cf5d91d15ef498b787c4127b6c4b18,Matvey Arye,Fri Dec 23 10:57:32 2016 -0500,Merged in mat/ddl_extension (pull request #13),Mat/ddl extension e386a452135eca9172613a2fb181a8b62d19bace,Matvey Arye,Mon Dec 12 14:52:53 2016 -0500,Make DDL operations work on top of the main table,"Instead of having specialized functions to alter the schema of hypertables, we use event triggers on a hypertable's main table. This allows for a more natural user experience. " 19a06bc602b340db034946094dffeba164fe1d2b,Olof Rensfelt,Wed Dec 21 16:22:58 2016 +0100,Merged in olof_rensfelt/backend-database/ore/extension (pull request #14),Add test for hash function. 91765e241fafed57a38eab26f398ec489703b7e1,Olof Rensfelt,Wed Dec 21 13:45:15 2016 +0100,Add test for hash function., e76ab641012cf4665854d34cf5c640b24ffbe002,Olof Rensfelt,Tue Dec 20 18:40:54 2016 +0100,Merged in olof_rensfelt/backend-database/ore/extension (pull request #12),Ore/extension 0f3aa8d557467b0646d3854dbeddae2e456ec3aa,Olof Rensfelt,Tue Dec 13 10:38:28 2016 +0100,* Add _meta schema to allow all code to be loaded on both meta and nodes. * Split SQL into functions and setup. * Remove hash-lib dependency. * Makes code into Postgresql extension., 650eee26be81dba1b5c9282f3c2caa0b34be6214,Olof Rensfelt,Fri Dec 2 15:33:14 2016 +0000,Merged in olof_rensfelt/backend-database/ore/go_tests (pull request #10),Added query tests. 3ac5ea7fd6c5b45442d396ceb181442c488ae2db,Olof Rensfelt,Wed Nov 30 10:50:04 2016 +0100,Added query tests., 7fa37f10bcc5032ee8a6287ee530e72348d6fe52,Matvey Arye,Tue Nov 29 10:24:36 2016 -0500,Bug Fix: create index only on local tables, 77a12411d2bb5bc30bc2c350bf70e8ec5846ae7f,Matvey Arye,Mon Nov 28 22:06:36 2016 -0500,fix time period limits, f56701f7387f74c6cdde8eb3258060a3616f2255,Matvey Arye,Mon Nov 28 21:49:44 2016 -0500,Fix non existant hypertable case, 3c9c752ce4280feede5b18ac5b0f90fd951e0f4a,Olof Rensfelt,Wed Nov 23 19:25:29 2016 +0000,Merged in olof_rensfelt/backend-database/ore/unit_tests (pull request #9),Update unit tests 12f260f893893021d93b6c4dfa48b368be8593a1,Olof Rensfelt,Wed Nov 23 17:43:56 2016 +0100,Update unit tests, a61ea9fe7cf033ac57a0977a6197d9f3b4641fa0,Matvey Arye,Tue Nov 22 21:51:40 2016 +0000,Merged in DesignRefactor (pull request #6),Designrefactor 3c72689664c036842dd0cc3ae5f60b0f8e22c27c,Matvey Arye,Mon Nov 21 15:07:12 2016 -0500,Finishing refactor TODOs & Formatting. Placement and cross-epoch queries., 939a4008ad1f07195283ffbcfcf06981131f60b4,Matvey Arye,Sun Nov 20 21:59:04 2016 -0500,Cleanup after refactor, 582671f926eccaf7b4a67379e5baffbbd3036537,Matvey Arye,Sun Nov 20 21:56:30 2016 -0500,Queries again work after previous refactoring, 0ff42fad0234b3d2f6949162dee5a658265b89df,Matvey Arye,Thu Nov 17 18:40:58 2016 -0500,added ability to insert across partitions, 1fdb3d43f93e30a9bf448176e6bbe306ce3a54b4,Matvey Arye,Thu Nov 17 17:23:52 2016 -0500,remove last_time_approx, bfcb25642e13a249ad4557b63fb46e4ed228fb74,Matvey Arye,Thu Nov 17 12:22:06 2016 -0500,added meta table and communication from data nodes to meta, 42ee7c858614ed8697c5273a0de9e0f7ce93020c,Matvey Arye,Tue Nov 15 15:11:31 2016 -0500,starting refactor of clustering and naming logic,This fix allows more flexible placement of tables on a node; better and more flexible logic for remote placement of chunks on nodes. e0c6ee89437fb4d442375a5ea28c882c8e9d617b,Olof Rensfelt,Mon Nov 21 19:54:33 2016 +0000,Merged in olof_rensfelt/backend-database/ore/go_tests (pull request #5),added tests f57f34f1b285e20dd90b8882fbb8907e84fc5265,Olof Rensfelt,Thu Nov 17 21:48:56 2016 +0100,add script to generate csv, e15da09f4576abd9c882aeec2f9978841de09c68,Olof Rensfelt,Fri Nov 11 15:10:43 2016 +0100,add test from postgres-kafka-consumer,Added missing files. c9f76446daad49965098603d6c7e2e98ae9d0747,Rob Kiefer,Fri Nov 18 12:15:34 2016 -0500,Fix to start-pg-docker to first remove existing docker image, 3950aa6237be0c802c99252b466430e809208ee3,Matvey Arye,Thu Nov 10 12:30:36 2016 -0500,"rewriting the rest of non-agg queries to return sql, not records", 5f44f32f86c9c5e70d3cd081dd87bbbd8a123ac8,Matvey Arye,Wed Nov 9 16:52:31 2016 -0500,rewriting by every query to return cte-based sql., 117e5ccd47ab3df42ed815668ffd75b7de9ec525,Matvey Arye,Mon Nov 7 12:02:34 2016 -0500,remove dead code, e96509d4c0977dcddba111eb1247a3bccd42c1da,Matvey Arye,Mon Nov 7 11:25:58 2016 -0500,fix error codes, b15719f9b451baf7d9b225e8cd671db876cd1c14,Matvey Arye,Thu Nov 3 18:38:36 2016 -0400,completed aggregate query work, 60aa45cde3361c413db9749eb92f38e956c038ff,Matvey Arye,Thu Nov 3 17:07:36 2016 -0400,more work on aggregates, 7bbe78c3aca647163f9fa904f70a372d7fc02e37,Matvey Arye,Thu Nov 3 15:12:49 2016 -0400,better solution to agg queries, 99981228da3851c913c4d278a09772357517b915,Matvey Arye,Thu Nov 3 12:48:12 2016 -0400,fix in aggregate logic, ee64595155bbf8cbb1cc4417eca0eedc4144ca3c,Rob Kiefer,Wed Nov 2 15:48:55 2016 -0400,Fix README typo, 000cc33bd0fdea95477b403ac2d355a2e2e0dfc4,RobAtticus NA,Wed Nov 2 19:47:37 2016 +0000,Merged in test-pr (pull request #3), 416cc1c0414d98dff9fc19a700be570971edff4e,Rob Kiefer,Wed Nov 2 14:38:01 2016 -0400,Fix regression test file. Add Makefile and README, 29c4556402ac453a1878509501a047f402a9e90b,Rob Kiefer,Wed Nov 2 11:30:14 2016 -0400,Testing Jenkins PR, 902d632d4307e0d1a1916423ad70178ab866c256,Matvey Arye,Wed Nov 2 18:14:57 2016 +0000,Merged in olof_rensfelt/backend-database/ore/add_setup (pull request #4),Add setup scripts and cleanup test directories e199a1bcf78aea88d584a37a8aac78170cefd5f8,Olof Rensfelt,Wed Nov 2 17:23:23 2016 +0100,Add setup scripts and cleanup test directories, e6bf1047184968dade8b45ee5beface87f23cde0,Matvey Arye,Wed Nov 2 11:57:19 2016 -0400,add dockerfile with loaded scripts dir, 696e9d0304e83d688cf1402e6031a8d8fd10a002,Olof Rensfelt,Wed Nov 2 15:27:18 2016 +0100,unit_test run script bugfix, e6e88ddb9931ee68ec589f3056e667b61302ed84,Matvey Arye,Tue Nov 1 22:05:01 2016 -0400,bug fixes, c3d55d339e37de75fa2eab27d6537dca05cc6c48,Matvey Arye,Tue Nov 1 14:01:57 2016 -0400,queries now work with old consumer tests, b26e9e14e55ada2d0776817587a7a5335f75d640,Olof Rensfelt,Tue Nov 1 14:54:33 2016 +0000,Merged in olof_rensfelt/backend-database/ore/test (pull request #2),add unit tests c33e64f2da19f1961610cbf68ab9b9bac54b7cef,Olof Rensfelt,Mon Oct 24 15:13:07 2016 +0200,add unit tests, c14e1a9bc00b5c527c4d2648596d434213363894,RobAtticus NA,Mon Oct 31 19:17:07 2016 +0000,Merged in internal-schema (pull request #1), 7c6dee79dcd372a5f757969756b5b448ca2460df,Rob Kiefer,Mon Oct 31 13:11:02 2016 -0400,Add restrictions on new namespace names, 48da98b5bb6b7f3b09095b7f358f81263a8ea07e,Rob Kiefer,Mon Oct 31 12:49:07 2016 -0400,"Add trigger funcs (on_*, sync_*) to _sysinternal.", e3c1dc2eccde40c4d43b750dd8d308175c2ba0ac,Rob Kiefer,Fri Oct 28 15:28:06 2016 -0400,"Create schema _sysinternal, put underscored funcs in it.", a075899bebac19274a3d15347a6deba6107a01b2,Matvey Arye,Wed Oct 26 17:02:08 2016 -0400,test and bug fixes; getting rid of jinja2, d300471c8651a2462bd80ce65ae23ffd94f6e07f,Matvey Arye,Wed Oct 26 14:43:04 2016 -0400,move cluster fdw setup inside core, 749ba328ca3693e51a68670b6737e06813d7d292,Matvey Arye,Tue Oct 25 22:10:27 2016 -0400,bug fixes; insert test, 0f9f906facb2dcf18175d298e92aaa4945f1d10e,Matvey Arye,Mon Oct 24 21:51:33 2016 -0400,add kafka offset managment stuff, dbcaa43f48dcfbd136d16d7b5ffd0f5f41e27592,Matvey Arye,Sun Oct 23 20:20:06 2016 -0400,progress, 59cc706bdac00ecee5b6da594f56a35ae43dbd76,Matvey Arye,Sat Oct 22 22:04:09 2016 -0400,add logic for creating data tables and indexes, 82a845cf4003f5a5ed4bd828a843d934ff63e230,Matvey Arye,Thu Oct 20 14:19:58 2016 -0400,checkpoint in redoing cluster managment logic, 2241a25b8c7aeeacaa1aab8a181ad15436b03d9b,Matvey Arye,Tue Oct 18 21:16:51 2016 -0400,First Commit,