Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

slog: update to use structured logs for hashmail server #148

Merged
merged 7 commits into from
Mar 5, 2025

Conversation

ellemouton
Copy link
Member

Update aperture's btclog and lnd deps so that it can take advantage of structure logging.

Then start using the structured logs for the hashmail server.

This will make querying logs much easier & faster.

Example

Main thing to note here is the uniformity of the "stream_id" attribute in the new logs.

Old:

Screenshot 2024-09-11 at 17 16 56

New:

Screenshot 2024-09-11 at 17 16 15

@guggero guggero self-requested a review September 11, 2024 16:07
@ellemouton ellemouton force-pushed the slog branch 3 times, most recently from 550bede to 50cbe19 Compare September 16, 2024 05:02
Copy link
Member

@guggero guggero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! And cool to see a demo of the context based trace functionality 💯

LGTM pending the dependent PRs and the err value fix.

log.DebugS(ctxl, "New HashMail stream init", "auth", init.Auth)

if err := h.ValidateStreamAuth(ctxl, init); err != nil {
log.DebugS(ctxl, "Stream creation validation failed", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be "err", err, otherwise this would show as "bad key", because DebugS doesn't have an err parameter. Same further below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a yes good catch!

makes me wonder if we should switch this to a "WarnS"? but anyways, keeping it as debug for now 👍

@ellemouton ellemouton force-pushed the slog branch 3 times, most recently from 2a10761 to 8cd2863 Compare October 27, 2024 05:18
Copy link
Member

@guggero guggero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending the dependencies 🎉

@@ -621,6 +629,9 @@ func (h *hashMailServer) SendStream(readStream hashmailrpc.HashMail_SendStreamSe
return err
}

ctx := btclog.WithCtx(readStream.Context(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is a normal (non-formatting) function call, so I think the other multi-line formatting rule would apply?

Same further below.

sample-conf.yaml Outdated
Comment on lines 219 to 226
# Console and file logger settings.
logging:
console:
disable: false
call-site: short
file:
disable: true
call-site: long
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yaml parser is struggling with these 🙃

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be reserved words. Try `'short' and 'long' instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it isnt just the call-site. it is anything from that unexported empedded struct. only works for the 'style' option when compiled in 'dev' mode.

seems to be a yaml issue (cause things work for the lnd.conf file just fine)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guggero
Copy link
Member

guggero commented Feb 27, 2025

I think we need to bump the Golang version in the linter docker file.

@ellemouton ellemouton force-pushed the slog branch 5 times, most recently from 77d1c27 to b3ff32b Compare February 27, 2025 12:37
Copy link
Member

@guggero guggero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, LGTM 🎉

sample-conf.yaml Outdated
Comment on lines 219 to 226
# Console and file logger settings.
logging:
console:
disable: false
call-site: short
file:
disable: true
call-site: long
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be reserved words. Try `'short' and 'long' instead?

@ellemouton
Copy link
Member Author

struggling with this linter 🙃 will try some iterations locally

@ellemouton
Copy link
Member Author

ooo ok im winning now! gimme a min

@ellemouton
Copy link
Member Author

YAY linter happy

@guggero guggero requested a review from starius February 27, 2025 15:17
Copy link
Contributor

@starius starius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Added few comments.

_ = build.ParseAndSetDebugLevels("trace,PRXY=warn", logWriter)
logMgr := build.NewSubLoggerManager(btclog.NewDefaultHandler(os.Stdout))
SetupLoggers(logMgr, signal.Interceptor{})
_ = build.ParseAndSetDebugLevels("trace,PRXY=warn", logMgr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to check error from ParseAndSetDebugLevels. We can panic if it return an error.

@ellemouton ellemouton force-pushed the slog branch 3 times, most recently from a0289b6 to b0e0d22 Compare March 4, 2025 11:34
Copy link
Member Author

@ellemouton ellemouton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks yall 🙏
Updated.

Got the yaml parsing working but need to get lightningnetwork/lnd#9576 merged first and then point to that here before merge

cool - all good now, just waiting on LND release

@ellemouton
Copy link
Member Author

oh - I've also bumped our Yaml lib btw since the old one is not actively maintained

Update the deps so that structured logging is available in aperture.
This will make querying by stream IDs more uniform (and hence, way
easier).
@ellemouton ellemouton merged commit 18293d3 into lightninglabs:master Mar 5, 2025
5 checks passed
@ellemouton ellemouton deleted the slog branch March 5, 2025 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants