Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Log Messages with SLF4J #288

Closed
osopardo1 opened this issue Mar 22, 2024 · 0 comments · Fixed by #289
Closed

Add Log Messages with SLF4J #288

osopardo1 opened this issue Mar 22, 2024 · 0 comments · Fixed by #289
Assignees
Labels
type: enhancement Improvement of existing feature or code

Comments

@osopardo1
Copy link
Member

osopardo1 commented Mar 22, 2024

The current logging processes of qbeast-spark is only covering small cases (such as file filtering in OTreeIndex listFiles operation).

Logs are a very important part of the code, since they help users and developers to understand what is happening, facilitate bug-solving, and help with a better understanding of the code flow. Especially in a framework such as Spark, in which is easy to get lost between the APIs and Catalyst Optimizations.

My proposal is the following:

  • Add Logs for Writing Processes. Classes such as SparkDataWriter ,QbeastMetadataOperation and SparkOTreeManager should log information about, at least, the stage they are in.
  • Add Logs for Reading. Affecting QuerySpaceBuilder, QueryExecutor and OTreeIndex.
  • Add Logs on each PR / code modification. Especially if they contain sensitive processes or they touch some of the critical pieces.

Spark already has a trait for writing such messages, it's called Logging, and this is the description:

/**
 * Utility trait for classes that want to log data. Creates a SLF4J logger for the class and allows
 * logging messages at different levels using methods that only evaluate parameters lazily if the
 * log level is enabled.
 */
trait Logging
@osopardo1 osopardo1 added the type: enhancement Improvement of existing feature or code label Mar 22, 2024
@osopardo1 osopardo1 changed the title Add Log Messages Add Log Messages with SLF4J Mar 22, 2024
@fpj fpj mentioned this issue Mar 22, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement Improvement of existing feature or code
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants