TimescaleDB'compression policy is not effective after inserting data in iot test
tomchon opened this issue · comments
haoranchen commented
[question]
compression policy cann't work in iot test when data has been inserted
[step]
insert data:
cat /data2//load_data_iot_host//data_timescaledb_iot_scale4000_2016-01-01T00:00:00Z_2016-01-05T00:00:00Z_interval10s_123.dat.gz| gunzip | tsbs_load_timescaledb --workers=12 --batch-size=10000 --db-name=benchmarkiot --host=ip-172-31-23-148 --pass=password --chunk-time=8h --hash-workers=false
add compression policy:
ALTER TABLE diagnostics SET (timescaledb.compress, timescaledb.compress_orderby = 'time DESC,fuel_state', timescaledb.compress_segmentby = 'tags_id');
ALTER TABLE readings SET (timescaledb.compress, timescaledb.compress_orderby = 'time DESC,latitude', timescaledb.compress_segmentby = 'tags_id');
SELECT add_compression_policy('diagnostics', INTERVAL '12 hours');
SELECT add_compression_policy('readings', INTERVAL '12 hours');
result
benchmarkiot=# SELECT chunk_name, range_start, range_end, is_compressed FROM timescaledb_information.chunks;
chunk_name | range_start | range_end | is_compressed
-------------------+------------------------+------------------------+---------------
_hyper_1_1_chunk | 2016-01-01 00:00:00+00 | 2016-01-01 08:00:00+00 | f
_hyper_2_2_chunk | 2016-01-01 00:00:00+00 | 2016-01-01 08:00:00+00 | f
_hyper_2_3_chunk | 2016-01-01 08:00:00+00 | 2016-01-01 16:00:00+00 | f
_hyper_1_4_chunk | 2016-01-01 08:00:00+00 | 2016-01-01 16:00:00+00 | f
_hyper_2_5_chunk | 2016-01-01 16:00:00+00 | 2016-01-02 00:00:00+00 | f
_hyper_1_6_chunk | 2016-01-01 16:00:00+00 | 2016-01-02 00:00:00+00 | f
_hyper_1_7_chunk | 2016-01-02 00:00:00+00 | 2016-01-02 08:00:00+00 | f
_hyper_2_8_chunk | 2016-01-02 00:00:00+00 | 2016-01-02 08:00:00+00 | f
_hyper_1_9_chunk | 2016-01-02 08:00:00+00 | 2016-01-02 16:00:00+00 | f
_hyper_2_10_chunk | 2016-01-02 08:00:00+00 | 2016-01-02 16:00:00+00 | f
_hyper_1_11_chunk | 2016-01-02 16:00:00+00 | 2016-01-03 00:00:00+00 | f
_hyper_2_12_chunk | 2016-01-02 16:00:00+00 | 2016-01-03 00:00:00+00 | f
_hyper_2_13_chunk | 2016-01-03 00:00:00+00 | 2016-01-03 08:00:00+00 | f
_hyper_1_14_chunk | 2016-01-03 00:00:00+00 | 2016-01-03 08:00:00+00 | f
_hyper_2_15_chunk | 2016-01-03 08:00:00+00 | 2016-01-03 16:00:00+00 | f
_hyper_1_16_chunk | 2016-01-03 08:00:00+00 | 2016-01-03 16:00:00+00 | f
_hyper_2_17_chunk | 2016-01-03 16:00:00+00 | 2016-01-04 00:00:00+00 | f
_hyper_1_18_chunk | 2016-01-03 16:00:00+00 | 2016-01-04 00:00:00+00 | f
_hyper_2_19_chunk | 2016-01-04 00:00:00+00 | 2016-01-04 08:00:00+00 | f
_hyper_1_20_chunk | 2016-01-04 00:00:00+00 | 2016-01-04 08:00:00+00 | f
_hyper_2_21_chunk | 2016-01-04 08:00:00+00 | 2016-01-04 16:00:00+00 | f
_hyper_1_22_chunk | 2016-01-04 08:00:00+00 | 2016-01-04 16:00:00+00 | f
_hyper_1_23_chunk | 2016-01-04 16:00:00+00 | 2016-01-05 00:00:00+00 | f
_hyper_2_24_chunk | 2016-01-04 16:00:00+00 | 2016-01-05 00:00:00+00 | f
(24 rows)
benchmarkiot-# WHERE proc_name='policy_compression';
job_id | application_name | schedule_interval | max_runtime | max_retries | retry_period | proc_schema | proc_name | owner | scheduled | fixed_schedule | config | next_start | initial_start | hypertable_schema | hypertable_name | check_schema | check_name
--------+---------------------------+-------------------+-------------+-------------+--------------+-----------------------+--------------------+----------+-----------+----------------+----------------------------------------------------+------------+---------------+-------------------+-----------------+-----------------------+--------------------------
1000 | Compression Policy [1000] | 04:00:00 | 00:00:00 | -1 | 01:00:00 | _timescaledb_internal | policy_compression | postgres | t | f | {"hypertable_id": 1, "compress_after": "12:00:00"} | | | public | diagnostics | _timescaledb_internal | policy_compression_check
1001 | Compression Policy [1001] | 04:00:00 | 00:00:00 | -1 | 01:00:00 | _timescaledb_internal | policy_compression | postgres | t | f | {"hypertable_id": 2, "compress_after": "12:00:00"} | | | public | readings | _timescaledb_internal | policy_compression_check
(2 rows)
expect
'is_compressed' status is t ,but it can't work normally
so how to resolve this problem, timescaledb version :
imescaledb | 2.10.1
postgresql | 14.7
haoranchen commented
After a while, it inexplicably worked again