Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storages: fix a series of data race or local index related issues #9962

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

Lloyd-Pottiger
Copy link
Contributor

@Lloyd-Pottiger Lloyd-Pottiger commented Mar 7, 2025

What problem does this PR solve?

Issue Number: close #9961, close #9971, close #9972

Problem Summary:

What is changed and how it works?

1. We should not reuse anything which is not thread-safe between different queries, so do not reuse `ann_query_info` in unit tests.
2. Fix `FileCache::bgDownload` may still be running after `FileCache::shutdown`.
3. Suppress false positive alarm in `~shared_ptr`  and `re2::RE2::NumberOfCapturingGroups`.
4. Suppress alarm in `TiFlashStorageTestBasic::reload` which is only used in unit tests.
5. Fix data race between `DeltaMergeStore::segmentWaitDeltaLocalIndexReady` and `ColumnFilePersistedSet::updatePersistedColumnFilesAfterAddingIndex`
6. Fix data race between `DeltaValueSpace::flush` and `DeltaMergeStore::segmentEnsureDeltaLocalIndex`
7. Fix the existing index infoes in ColumnFileTiny are missed after add new indexes.
8. Do not add task into `LocalIndexerScheduler` if the DMFile is empty.
9. Fix delta data do not build the new added local indexes.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. labels Mar 7, 2025
Copy link
Contributor

ti-chi-bot bot commented Mar 7, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from lloyd-pottiger, ensuring that each of them provides their approval before proceeding. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Mar 7, 2025
@ti-chi-bot ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Mar 10, 2025
Signed-off-by: Lloyd-Pottiger <[email protected]>
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger Lloyd-Pottiger changed the title Storages: fix TSAN error Storages: fix a series of data race or local index related issues Mar 10, 2025
Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger Lloyd-Pottiger added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Mar 10, 2025
@JaySon-Huang
Copy link
Contributor

Fix the existing index infoes in ColumnFileTiny are missed after add new indexes.

I think you need a separate issue and fix PR for this problem

@Lloyd-Pottiger
Copy link
Contributor Author

Lloyd-Pottiger commented Mar 10, 2025

Fix the existing index infoes in ColumnFileTiny are missed after add new indexes.

I think you need a separate issue and fix PR for this problem

Open a separate issue, but IMO fix in this PR is ok, since other changes also need pick to release-8.5.

Signed-off-by: Lloyd-Pottiger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
3 participants