Restrict segment metadata kill query till maxInterval from last kill task time

[chetanpatidar26]

Description

In The current setup in killUnusedSegments, the segmentsMetadataManager.getUnusedSegmentIntervals queries for unused segments from the last segment kill time to the durationToRetain. This results in a expensive SQL query in some cases, where huge amount of segments are created in less time. Although we have a config druid.coordinator.kill.maxInterval which is set to 30 days by default, we are not using it. This config can be used to restrict the query to scan for only 30 days of data.

Example -
At confluent we create segments with granularity of 5 minutes, our current coordinator kill task is working on deleting segments from April 2022 and we have set durationToRetain=720 days. So everytime findIntervalToKill method runs it queries for segment data of nearly 350 days, this resulted in huge spike in handoff time.

We updated the durationToRetain=1000 days thus it resulted in scanning data of 50 days and handoff time was in control again.

Proposed fix

In findIntervalToKill method, restricting the maxEndTime to the maximum Interval from the time last segment was killed.

case 1

When a datasource is created and the first segment from it is killed we will have minStartTime as null because it is initialised as the end time of last segment killed, in this scenario the maxEndTime will be durationToRetain.

case 2

After the first segment is killed from this datasource the minStartTime will be initialised, thus now the maxEndTime will be
minimum(durationToRetain, minStartTime + maxInterval)

Release note

---

---

This PR has:

• been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[kfaraz]

Thanks for the changes, @chetanpatidar26 !
The logic makes sense to me. I have left some suggestions.

[kfaraz]

Thanks for the contribution, @chetanpatidar26 !

[chetanpatidar26]

Thanks for the review @kfaraz !
Thank you @rbankar7 for helping in fixing the issue.