Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WAL/blockmaps: continue blockmaps initialization after failure #241

Merged
merged 14 commits into from
Dec 11, 2023

Conversation

var77
Copy link
Collaborator

@var77 var77 commented Dec 7, 2023

Forked from PR#211 to rebase with main.

@var77 var77 force-pushed the varik/wal-blockmaps branch from 5091161 to dfac872 Compare December 7, 2023 14:29
Copy link

github-actions bot commented Dec 7, 2023

Benchmarks

metric old new pct change
recall (after create) 0.740 0.740 -
recall (after insert) 0.754 0.768 +1.86%
select bulk tps 486.766 493.939 +1.47%
select bulk latency (ms) 15.702 15.122 -3.69%
select bulk latency (stddev ms) 3.712 2.253 -39.30%
create latency (ms) 1197.828 1178.855 -1.58%
insert bulk tps 11.514 11.353 -1.40%
insert bulk latency (ms) 86.841 88.071 +1.42%
insert bulk latency (stddev ms) 1.995 1.894 -5.06%
disk usage (bytes) 6348800.000 6348800.000 -

Copy link

codecov bot commented Dec 7, 2023

Codecov Report

Merging #241 (4c280e8) into main (50c294b) will increase coverage by 0.49%.
The diff coverage is 87.83%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #241      +/-   ##
==========================================
+ Coverage   77.68%   78.17%   +0.49%     
==========================================
  Files          23       23              
  Lines        1685     1778      +93     
  Branches      415      449      +34     
==========================================
+ Hits         1309     1390      +81     
- Misses        191      194       +3     
- Partials      185      194       +9     
Files Coverage Δ
src/hnsw.c 82.37% <100.00%> (+0.39%) ⬆️
src/hnsw/scan.c 83.49% <100.00%> (ø)
src/hnsw/failure_point.c 64.00% <0.00%> (+16.00%) ⬆️
src/hnsw/validate_index.c 48.03% <60.00%> (-0.40%) ⬇️
src/hnsw/external_index.c 89.92% <90.00%> (-1.27%) ⬇️

@var77 var77 changed the title Varik/wal blockmaps WAL/blockmaps: continue blockmaps initialization after failure Dec 7, 2023
@var77 var77 marked this pull request as ready for review December 7, 2023 16:54
@var77 var77 requested a review from Ngalstyan4 December 7, 2023 16:54
medvied and others added 14 commits December 11, 2023 10:36
ContinueBlockMapGroupInitialization() description has all the details
about how it's done and how different kinds of failures are handled.
This patch adds API to trigger execution of C code from SQL to test
corner cases.

`test/sql/hnsw_failure_point.sql` has an example of how to trigger a
process crash using failure points and how to see that a space leak
happens if a crash happens after a block is allocated, but before a
record for the block is added to the index during blockmaps creation.
@var77 var77 force-pushed the varik/wal-blockmaps branch from fa2c14b to 4c280e8 Compare December 11, 2023 06:40
@var77 var77 merged commit ee2a451 into main Dec 11, 2023
@var77 var77 deleted the varik/wal-blockmaps branch December 11, 2023 06:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants