Giters

apache / druid

Apache Druid: a high performance real-time analytics database.

Home Page:https://druid.apache.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Is druid.segmentCache.locations is used to configure the maximum storage size for hot data?

lukoou3 opened this issue · comments

commented

Is druid.segmentCache.locations is used to configure the maximum storage size for hot data?

config druid.segmentCache.locations 300g, when data increases， kafka ingestion task fail:

WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [15] Segments

Affected Version

26.0.0.

Description

  • Cluster size : 5
  • Configurations in use: config druid.segmentCache.locations 300g, deep-storage: hdfs

when data increases， kafka ingestion task fail:

WARN [coordinator_handoff_scheduled_0] org.apache.druid.segment.handoff.CoordinatorBasedSegmentHandoffNotifier - Still waiting for Handoff for [15] Segments

historical segmentCache.locations Usage : 100%

Is druid.segmentCache.locations is used to configure the maximum storage size for hot data? Delete cold data when there is not enough space, and only pull it from HDFS when querying cold data?

Do druid.segmentCache.locations need to store all undeleted data, not just hot data?