Adding a periodic encoder to the DatetimeEncoder #1235

rcap107 · 2025-02-07T14:35:47Z

Draft for #907

Main points:

Adding transformers for hours in a day and days in a year
Adding the ordinal day (day in a year) as feature
Implement transformers using SplineTransformer from scikit-learn

Questions:

Should I also implement a CircularTransformer?
The main reason I am considering the CircularTransformer is because I am not sure what would be better for encoding days in a year. Should I just not bother, and keep SplineTransformer with a limited number of knots?
What should the default parameters be?
If we decide to encode the year, there is the problem of having both leap and non-leap years in a dataset. Is that a problem? I don't think it would affect the features that much.
Any more features that should be added in this PR?

Of course, tests and examples are all missing.

skrub/_datetime_encoder.py

jeromedockes · 2025-02-07T15:05:20Z

skrub/_datetime_encoder.py

+                self._required_transformers[_case] = _t
+            if _case == "year":
+                # TODO: In theory we should check that the year is leap
+                _t = PeriodicEncoder(kind="circular", period=366)


isn't it a bit surprising that the encoding is different for day and year? maybe it could be splines with 4 splines or something like that?

skrub/_datetime_encoder.py

jeromedockes · 2025-02-07T15:15:12Z

skrub/_datetime_encoder.py

@@ -256,10 +261,19 @@ class DatetimeEncoder(SingleColumnTransformer):
    timezone used during ``fit`` and that we get the same result for "hour".
    """  # noqa: E501

-    def __init__(self, resolution="hour", add_weekday=False, add_total_seconds=True):
+    def __init__(


should we not generate the values when we are doing the periodic encoding? eg if we have spline encoding of the time of day, maybe we shouldn't by default output the "hour" feature

Maybe users still need the additional columns as features for some reason? Though I guess that by the point they get to vectorizing the table they're already done with exploring the data which is what keeping those features would help with 🤔

I'll remove the old features

i'm not sure what would the best interface to make the skrub code simple and to make it easy for users to specify which features they want / figure out which features they will get 🤔

A parameter like "keep_original" would make sense to me. This reminds me somehow of coalesce in polars join https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.join.html, which pandas doesn't have.

rcap107 · 2025-02-10T15:48:14Z

I finished the base implementation and split the PeriodicEncoder into SplineEncoder and CircularEncoder

I am working locally on the examples, and now I need to write all the tests

skrub/_datetime_encoder.py

jeromedockes

thanks a lot @rcap107 ! here is a first few comments

skrub/_datetime_encoder.py

jeromedockes · 2025-02-13T14:38:58Z

skrub/_datetime_encoder.py

+        Period to be used as basis of the trigonometric function.
+    """
+
+    def __init__(self, period=24):


maybe we shouldn't have a default for the period. eg 24 only makes sense for hours

makes sense, I'll remove it

jeromedockes · 2025-02-13T14:40:26Z

skrub/_datetime_encoder.py

@@ -26,6 +29,8 @@
    "nanosecond",
 ]

+_DEFAULT_ENCODING_PERIODS = {"year": 4, "month": 30, "weekday": 7, "hour": 24}


why 4 for year? shouldn't we set the period to 12 and use the month number or something like that? or set the period to 366 if we use the day of year?

At the beginning I had 365 and the spline as default, but having 365 features for each year was too much, and I didn't have any better idea than setting it to a random number (4 seasons I guess)

Now that circular is the default, we can set it to 366

jeromedockes · 2025-02-13T14:44:22Z

skrub/_datetime_encoder.py

+        Select the strategy used to encode days in a week. By default, use a
+        CircularEncoder with period=7. If None, no encoding is performed.
+
+    hour_encoding : str, :class:``~CircularEncoder``, :class:``~SplineEncoder`` or None, default="circular"


the naming is inconsistent: does it indicate the time unit that gets encoded, or the one that defines the range?
for example here year_encoding encodes the position within the year, so the name corresponds to the range, but hour_encoding encodes the position within the day

I think we should keep the latter option, so hour_encoding will be sin(hour / 24 * 2* pi) whereas day_of_year_encoding will be sin(day_of_year / 366 * 2 * pi)

jeromedockes · 2025-02-13T15:01:24Z

skrub/_datetime_encoder.py

+            _enc_attr = [attr for attr in self.__dict__ if attr.endswith("_encoding")]
+            for _enc_name in _enc_attr:
+                _enc = self.__getattribute__(_enc_name)
+                _enc_case = _enc_name.split("_")[0]


why "case"?

jeromedockes · 2025-02-13T15:02:34Z

skrub/_datetime_encoder.py

+        # parameters
+        if self.add_periodic:
+            _enc_attr = [attr for attr in self.__dict__ if attr.endswith("_encoding")]
+            for _enc_name in _enc_attr:


detail: local variable names don't often start with an underscore. no special reason but it's one more character and I don't see what it adds

jeromedockes · 2025-02-13T15:04:48Z

skrub/_datetime_encoder.py

+        X_out = sbd.concat_horizontal(X_out, *_new_features)
+
+        # Censoring all the null features
+        X_out = sbd.where_row(X_out, self.not_nulls, _null_mask)


I think the circular and spline encoders could do that themselves on the numpy output and we wouldn't need the where_row

this

X_out[~self.not_nulls] = np.nan

works with pandas, but doesn't work with polars, is there another way?

jeromedockes · 2025-02-13T15:06:18Z

skrub/_datetime_encoder.py

+
+    def __init__(self, period=24, n_splines=None, degree=3):
+        self.period = period
+        self.n_splines = n_splines


where you had year period = 4 before I think maybe what you wanted is period of 366 (or 12 if we use the month) and number of splines = 4

yes, correct

skrub/_datetime_encoder.py

GaelVaroquaux · 2025-02-15T16:47:00Z

The SplineEncoder and the CircularEncoder are not documented in the API page, and thus they don't render correctly in the docs:
https://output.circle-artifacts.com/output/job/cc1b279e-769f-4515-bd08-30d0ebebd0f6/artifacts/0/doc/reference/generated/skrub.DatetimeEncoder.html

GaelVaroquaux · 2025-02-15T17:00:08Z

skrub/_datetime_encoder.py

+        integer.
+    """
+
+    def __init__(self, period, n_splines=None, degree=3):


In scikit-learn design, it is very frowned upon to have a parameter without a default value

I removed it after @jeromedockes suggested that having a default parameter that has nothing to do with the value to encode (e.g., 24 for month or year) would not be useful

I can add it back

yep sorry about that @rcap107 . Indeed I had said there was no sensible default here, but yeah you can add it back

GaelVaroquaux · 2025-02-15T17:03:01Z

skrub/_datetime_encoder.py

    def _more_tags(self):
        return {"preserves_dtype": []}

    def __sklearn_tags__(self):
        tags = super().__sklearn_tags__()
        tags.transformer_tags = TransformerTags(preserves_dtype=[])
        return tags
+
+
+class SplineEncoder(SingleColumnTransformer):


From a design perspective, this class seems to me really like a super thin wrapper on the scikit-learn SplineTransformer.

Would it not be possible to get rid of it, and use the SplineTransformer? I would like to minimize almost-redundant functionnality

Should I just fold the logic of both encoders back in the main DatetimeEncoder then? CircularEncoder is just wrapping a call to np.sin/np.cos

I was thinking that it could be useful to have periodic encoders for non-datetime features, but maybe I'm wrong

GaelVaroquaux · 2025-02-15T17:07:31Z

skrub/_datetime_encoder.py

+        Period to be used as basis of the trigonometric function.
+    """
+
+    def __init__(self, period):


Same comment about scikit-learn really liking defaults. Put 365 for instance

skrub/_datetime_encoder.py

GaelVaroquaux · 2025-02-15T17:12:53Z

skrub/_datetime_encoder.py

@@ -26,6 +29,8 @@
    "nanosecond",
 ]

+_DEFAULT_ENCODING_PERIODS = {"year": 366, "month": 30, "weekday": 7, "hour": 24}


Naive question: years and months don't actually have a periodicity of 366 and 30. How do we deal with this?
Is a solution to use the actually average length (365.25 days/year, I believe)?

The period must be an integer

I don't have a good answer, but my gut feeling is that choosing between period 365 and 366 (and even between 28 and 31 for months) is not going to make a noticeable difference downstream

GaelVaroquaux · 2025-02-15T17:17:07Z

I don't understand why, when I look at the generated example gallery, it looks like the relevant examples did not run:

I would really like to see the examples running. I don't have a feeling for merging a PR without the examples running.

GaelVaroquaux · 2025-02-15T17:24:14Z

skrub/_datetime_encoder.py

+
+    year_encoding : str, :class:``~CircularEncoder``, :class:``~SplineEncoder`` or None, default="circular"
+        Select the strategy used to encode days in a year. By default, use a
+        CircularEncoder with period=365. If None, no encoding is performed.


If I follow things right, this is used only if "add_periodic" is True above. We should mention this.

GaelVaroquaux · 2025-02-15T17:24:37Z

skrub/_datetime_encoder.py

+        CircularEncoder with period=365. If None, no encoding is performed.
+
+    month_encoding : str, :class:``~CircularEncoder``, :class:``~SplineEncoder`` or None, default="circular"
+        Select the strategy used to encode days in a month. By default, use a


Same comment here: this is used only if "add_periodic" is True above

GaelVaroquaux · 2025-02-15T17:31:05Z

skrub/_datetime_encoder.py

+    add_periodic : bool, default=False
+        Add periodic features with different granularities. By default, use
+        trigonometric (circular) encoding of features. Spline encoding and
+        custom encoders are also supported.


Could we have a simpler API that removes the "*_encoding" arguments below and changes this argument to accepting False, "circular", "spline"?

It would reduce the flexibility of the object, but my hunch is that it would cover most of the usecases. And it would remove the inter-dependency between arguments (aka "add_periodic=False" means that the arguments below are ignored), which make user mistake more likely and hyper-parameter selection easier)

I agree with Gael. Too many hyper-parameter options might insensitive users to grid search them all while it would probably not bring much performance lift.

Users might want to gridsearch at least some parameters for different variables (like the number of splines for encoding a year)

Either way, I can remove everything and just hardcode the defaults

If we go with the simplified implementation, there is really no reason to have the separate encoders, is there

Co-authored-by: Gael Varoquaux <[email protected]>

rcap107 · 2025-02-17T09:18:29Z

I don't understand why, when I look at the generated example gallery, it looks like the relevant examples did not run:

https://output.circle-artifacts.com/output/job/cc1b279e-769f-4515-bd08-30d0ebebd0f6/artifacts/0/doc/auto_examples/03_datetime_encoder.html#sphx-glr-auto-examples-03-datetime-encoder-py

https://output.circle-artifacts.com/output/job/cc1b279e-769f-4515-bd08-30d0ebebd0f6/artifacts/0/doc/auto_examples/01_encodings.html#sphx-glr-auto-examples-01-encodings-py

I would really like to see the examples running. I don't have a feeling for merging a PR without the examples running.

I haven't gotten to the examples yet

skrub/_datetime_encoder.py

Vincent-Maladiere · 2025-02-17T10:31:13Z

skrub/_datetime_encoder.py

+    add_periodic : bool, default=False
+        Add periodic features with different granularities. By default, use
+        trigonometric (circular) encoding of features. Spline encoding and
+        custom encoders are also supported.


I agree with Gael. Too many hyper-parameter options might insensitive users to grid search them all while it would probably not bring much performance lift.

Vincent-Maladiere · 2025-02-17T10:43:49Z

skrub/_datetime_encoder.py

@@ -256,10 +261,19 @@ class DatetimeEncoder(SingleColumnTransformer):
    timezone used during ``fit`` and that we get the same result for "hour".
    """  # noqa: E501

-    def __init__(self, resolution="hour", add_weekday=False, add_total_seconds=True):
+    def __init__(


A parameter like "keep_original" would make sense to me. This reminds me somehow of coalesce in polars join https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.join.html, which pandas doesn't have.

Co-authored-by: Vincent M <[email protected]>

GaelVaroquaux · 2025-02-17T20:56:29Z

Should I just fold the logic of both encoders back in the main DatetimeEncoder then? CircularEncoder is just wrapping a call to np.sin/np.cos

My gut feeling would be to do this

rcap107 · 2025-02-18T10:12:02Z

I had a IRL discussion with @jeromedockes about the encoder and there are some issues that we might want to discuss in person.

The main point of discussion is whether we want the user to have the flexibility to choose different parameters for the circular/spline encoders, or if it's better to keep it simple and have only None, circular, spline.

My opinion is that the flexibility is an advantage, and if we decide to remove everything we will have to either hardcode variables that a user might want to tweak (specifically, the number of splines), or expose parameters to change them anyway.

If we keep the code "as-is" (bar some refactoring), meaning with CircularEncoder and SplineEncoder and all the specific parameters:

The implementation would let the user choose (and maybe search over) specific values for the period and especially the number of splines. Also, the code is already there.
There are multiple parameters (year_encoding etc.) that bloat the docstring and that are relevant only when add_periodic=True. I don't have a better idea on how to write this.
The encoders are "there", so if someone needs a periodic encoder for something other than a datetime, they're still available (no idea if this is a realistic use case).

If instead we simplify everything, the result is folding the logic of CircularEncoder and SplineEncoder back in the DatetimeEncoder, and removing the *_encoding parameters with just add_periodic that can be either a bool, circular or spline.

This would simplify the documentation and the user-experience (I am not fully sold on this, but I get the point).
It also means that either 1) the flexibility goes out of the window and we hardcode every parameter, or 2) we try to retain some of that flexibility by exposing the number of splines as a parameter.
1. is not a problem for periods, but a user might want to tweak the number of splines for a year (or avoid having 365 features for each year feature).
1. Would just functionally be the same as providing a custom SplineEncoder, but with exposing a lower level parameter that would also be irrelevant whenever add_periodic=False.
Sidenote, if we go with 1) I would rather avoid starting a fractal discussion of the number of splines to go with for each case.
We run the risk of having someone come in and open an issue asking to provide more customization for the Encoders, which may require moving the logic back out to separate objects.

Ultimately, I will be implementing fixes for either version, what I really want to avoid is picking one version, writing all the code, and then have to rewrite it again right before merging.

The first value in a column may be a null, and the column itself might be all nulls.

glemaitre · 2025-02-24T09:55:14Z

Here, I have an example that would benefit greatly from this feature when I compare a linear model with a gradient boosting:

https://skore.probabl.ai/0.7/auto_examples/use_cases/plot_employee_salaries.html#sphx-glr-auto-examples-use-cases-plot-employee-salaries-py

It is really close from the bike rental example from scikit-learn:

https://scikit-learn.org/dev/auto_examples/applications/plot_cyclical_feature_engineering.html#sphx-glr-auto-examples-applications-plot-cyclical-feature-engineering-py

Initial commit for periodic encoder

bc9a5bc

jeromedockes reviewed Feb 7, 2025

View reviewed changes

rcap107 added 4 commits February 10, 2025 16:12

Refactoring code to split Spline and Circular encoders

176b18f

WIP Refactoring code to split Spline and Circular encoders

cc6455c

Renaming a constant

99b2d37

Finished implementation of the circular encoder

ee24931

Updating changelog

1a53da2

Vincent-Maladiere reviewed Feb 11, 2025

View reviewed changes

skrub/_datetime_encoder.py Outdated Show resolved Hide resolved

rcap107 added 6 commits February 11, 2025 11:58

✅ WIP Improving test coverage

156d397

Fixing tests

3004442

Fixing more bugs with testing

9e6e05f

Fixing a typo

542522e

Fixing all tests and docstring shenanigans

3804dc9

Fixing bugs, adding tests, adding docstrings

a981526

rcap107 marked this pull request as ready for review February 13, 2025 10:16

Delete example_circular_encoding.py

3380f83

GaelVaroquaux reviewed Feb 13, 2025

View reviewed changes

skrub/_datetime_encoder.py Outdated Show resolved Hide resolved

Adding doc classes

ed5eb46

jeromedockes reviewed Feb 13, 2025

View reviewed changes

Implementing fixes from comments

ea209d6

GaelVaroquaux reviewed Feb 15, 2025

View reviewed changes

skrub/_datetime_encoder.py Outdated Show resolved Hide resolved

GaelVaroquaux reviewed Feb 15, 2025

View reviewed changes

Update skrub/_datetime_encoder.py

af9f363

Co-authored-by: Gael Varoquaux <[email protected]>

Vincent-Maladiere reviewed Feb 17, 2025

View reviewed changes

Update skrub/_datetime_encoder.py

d0ab8bf

Co-authored-by: Vincent M <[email protected]>

rcap107 changed the title ~~Initial commit for periodic encoder~~ Adding a periodic encoder to the DatetimeEncoder Feb 17, 2025

rcap107 added 3 commits February 18, 2025 11:44

Fixing a bug with how the first value is selected

1ac5caf

Fixing a fix that didn't fix

5d98ee3

The first value in a column may be a null, and the column itself might be all nulls.

Adding back default values

c96d3fc

rcap107 added 9 commits March 10, 2025 14:16

Merge remote-tracking branch 'upstream/main' into circular_encoding

4b660e8

Simplifying code and hardcoding defaults

4dcae67

fixing an import

e160dcb

Updating the datetime encoder example to add periodic features

7be8721

Fixing example

2ab6471

wip debugging

aa88cd1

Fixing a bug that was caused by pandas indexin

72ae07b

Fixing a bug with pandas indexing, renaming variables

5bd2ffa

Updating example to add new datetimeencoder features

2d28a18

Adding a periodic encoder to the DatetimeEncoder #1235

Are you sure you want to change the base?

Adding a periodic encoder to the DatetimeEncoder #1235

Conversation

rcap107 commented Feb 7, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rcap107 commented Feb 10, 2025

jeromedockes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GaelVaroquaux commented Feb 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GaelVaroquaux commented Feb 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rcap107 commented Feb 17, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GaelVaroquaux commented Feb 17, 2025 via email

rcap107 commented Feb 18, 2025

glemaitre commented Feb 24, 2025