Skrub

GitHub - skrub-data/skrub: Machine learning with dataframes Machine learning with dataframes. Contribute to skrub-data/skrub development by creating an account on GitHub.

And even more changes and improvements!

We hope you enjoy the new release, and if you do, don't forget to 🌟 the repo 😉

github.com/skrub-data/s...

StringEncoder Gallery examples: Various string encoders: a sentiment analysis example Hands-On with Column Selection and Transformers Multiples tables: building machine learning pipelines with DataOps Hyperparam...

🔢 We changed the default high cardinality encoder: now the StringEncoder is used as high cardinality encoder by default.

skrub-data.org/stable/refer...

⚙️ A global Skrub config has been introduced, which allows to set a number of parameters to customize the behavior of Skrub.

skrub-data.org/stable/refer...

🗑️ DropUninformative is a transformer that uses various heuristics to remove columns that are unlikely to bring information for training a model.

skrub-data.org/dev/referenc...

📏 The SquashingScaler has been added: it robustly rescales and smoothly clips numerical columns, enabling more robust handling of numerical columns with neural networks.

skrub-data.org/dev/auto_exa...

🛠️ selectors, ApplyToCols and ApplyToFrame are now available, providing utilities for selecting columns to which a transformer should be applied in a flexible way.

skrub-data.org/dev/auto_exa...

📊 The TableReport has been improved with many new features: series are now supported directly, it is possible to skip generating plots when the number of columns in the dataframe exceeds a user-defined threshold. Columns with high cardinality and sorted columns are now highlighted.

Get started with DataOps in the user guide:

skrub-data.org/dev/userguid...

Form complex DataOps plans to train and tune machine learning models, then export the plans as learners, standalone objects that can be used on new data.

Tune hyperparameters where they're defined, and explore the resulting space with a parallel coordinate plot

🌟 Major feature! Skrub DataOps are a powerful new way of combining dataframe transformations over multiple tables with machine learning pipelines.

📖 Keep reading for the highlights, or check out the full changelog here: skrub-data.org/stable/CHANG...

📚 On top of that, we revamped most of the user guide, documentation, and API reference to make it easier to learn how to use the features of Skrub.

skrub-data.org/dev/document...

⚡ Release 0.6.0 is now out! ⚡

🚀 Major update! Skrub DataOps, various improvements for the TableReport, new tools for applying transformers to the columns, and a new robust transformer for numerical features are only some of the features included in this release.

skrub-data.org/dev/referenc...

DatetimeEncoder Gallery examples: Encoding: from a dataframe to a numerical matrix for machine learning Handling datetime features with the DatetimeEncoder

skrub-data.org/dev/referenc...

🔄 Finally, the DatetimeEncoder can also add periodic features: trigonometric (or circular) features, and b-spline features can be generated directly by setting the specific parameter. 4/4

Docs below!

🔢 For more extensive feature engineering, the DatetimeEncoder parses datetimes, then converts each datetime part into a numerical column (hours, minutes, seconds, days etc.). Additional features such as weekdays and time since epoch can also be added. 3/

⏱️ If you need to convert columns from strings to datetimes, then to_datetime() does that for you. It's also available as a scikit-learn compatible transformer as ToDatetime(). Both objects parse most common formats automatically, but can accept a specific time format if needed. 2/

📅 The skrub API includes various functions and objects that help with dealing with datetime strings. 1/

🚀⚡ Release: 0.5.4:
Maintenance release!
This release makes skrub compatible with scikit-learn 1.7.

Changelog:
skrub-data.org/stable/CHANG...

In the dev docs you will find examples on how to use expressions, like this:
skrub-data.org/dev/auto_exa...

⚠️ As a disclaimer, expressions are still under development and things may change. However, if you're interested in learning more or testing them out, you can do so by checking the dev docs and examples, or by cloning the main branch of the skrub repo.

Finally, results can be shown with a parallel coordinate plot to find out the impact of different hyperparameters on the prediction task.

Once you're happy with the parameter grid, it's possible to either cross-validate it with default values, or run a full randomized or grid search on the parameter grid.

Even better, choices can be nested: an estimator in a choose_from can be defined with its own set of choices, which are then expanded by the library.

With the skrub expressions, it will be possible to build complex hyperparameter grids by composing "choose_" functions: choose from a list of values or estimators, generate a linear or logarithmic distribution of integers or floats, select boolean flags.

In scikit-learn, parameter grids are often built by setting all required parameters in a dictionary, then passing the dictionary to GridSearchCV or RandomizedSearchCV. This process adds a lot of redundant code and may lead to missing some configurations.

👀 This week's post will be another sneak peek into skrub expressions, an upcoming feature that will ease the preparation and execution of machine learning pipelines on dataframes.

This time we will focus on how expressions can simplify the construction of complex hyperparameter grids.

TextEncoder Gallery examples: Various string encoders: a sentiment analysis example

Docs: skrub-data.org/stable/refer...

🐌 Just remember that language models are expensive to run. In the worst case, there's always the skrub StringEncoder 😉