Implement GT.pipe()

[jrycw]

Related to #353.

After reviewing the pipe() methods from Pandas and Polars, I propose adding GT.pipe() and GT.pipes() in this PR.

GT.pipe()

GT.pipe() will function similarly to Pandas and Polars, receiving a function along with optional positional and keyword arguments. An example is provided below:

import polars as pl
from great_tables import GT, loc, style
from great_tables.data import towny


towny_mini = pl.from_pandas(towny).head(10)
columns = ["land_area_km2", "density_2021"]
colors = ["lightgray", "lightblue"]


def tbl_style(gtbl: GT, columns: list[str], colors: list[str]) -> GT:
    for column, color in zip(columns, colors):
        gtbl = gtbl.tab_style(
            style=style.fill(color=color),
            locations=loc.body(columns=column, rows=pl.col(column).eq(pl.col(column).max())),
        )
    return gtbl


(
    GT(
        towny_mini[["name", "land_area_km2", "density_2021"]],
        rowname_col="name",
    ).pipe(tbl_style, columns, colors)
)

GT.pipes()

GT.pipe() serves as a convenient helper method for chaining multiple pre-built functions (or using partial). An example is provided below:

from functools import partial
import polars as pl
from great_tables import GT, loc, style
from great_tables.data import towny


towny_mini = pl.from_pandas(towny).head(10)
columns = ["land_area_km2", "density_2021"]
colors = ["lightgray", "lightblue"]


def tbl_style(gtbl: GT, column: str, color: str) -> GT:
    return gtbl.tab_style(
        style=style.fill(color=color),
        locations=loc.body(columns=column, rows=pl.col(column).eq(pl.col(column).max())),
    )


(
    GT(
        towny_mini[["name", "land_area_km2", "density_2021"]],
        rowname_col="name",
    ).pipes(
        *[partial(tbl_style, column=column, color=color) for column, color in zip(columns, colors)]
    )
)

Table

Both methods yield the table as follows:

Assistance Required

I hope the API design makes sense, and could the team please review my docs to ensure they render correctly? I'm relatively new to quarto.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.66%. Comparing base (c2acdbe) to head (2dafcc7).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #363      +/-   ##
==========================================
+ Coverage   90.64%   90.66%   +0.02%     
==========================================
  Files          45       46       +1     
  Lines        5355     5367      +12     
==========================================
+ Hits         4854     4866      +12     
  Misses        501      501              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jrycw]

I'm not sure if it's a good idea to support passing a list into pipes(). If we add two lines to pipes(), like:

def pipes(self: "GT", *funcs: Callable["GT", "GT"] | list[Callable["GT", "GT"]]) -> "GT":
    if isinstance(funcs[0], list) and len(funcs) == 1:  # edited
        funcs = funcs[0]
    for func in funcs:
        self = pipe(self, func)
    return self

Then, we can do:

(
    GT(
        towny_mini[["name", "land_area_km2", "density_2021"]],
        rowname_col="name",
    ).pipes(
        [partial(tbl_style, column=column, color=color) for column, color in zip(columns, colors)]
    )
)

@machow and @rich-iannone , could you share your thoughts on this idea with me?

[jrycw]

I've updated .pipes() to accept a list of functions in the latest commit. We can decide which commit to use later.

[machow]

Sorry for the wait! Thanks for taking the time to work through this, and look at what pandas and polars do. I'm noticing pandas and polars have only the one .pipe() method. Maybe we start with this, and then if people are asking for it, consider a .pipes() method?

Packages like toolz have a function---called compose()---that makes it easy to send multiple things to a single .pipe() method. For example...

from toolz.functoolz import compose, compose_left

from functools import partial
import polars as pl
from great_tables import GT, loc, style
from great_tables.data import towny


towny_mini = pl.from_pandas(towny).head(10)
columns = ["land_area_km2", "density_2021"]
colors = ["lightgray", "lightblue"]


def tbl_style(gtbl: GT, column: str, color: str) -> GT:
    return gtbl.tab_style(
        style=style.fill(color=color),
        locations=loc.body(columns=column, rows=pl.col(column).eq(pl.col(column).max())),
    )


(
    GT(
        towny_mini[["name", "land_area_km2", "density_2021"]],
        rowname_col="name",
    ).pipe(
        compose_left(*[partial(tbl_style, column=column, color=color) for column, color in zip(columns, colors)])
    )
)

[jrycw]

@machow thanks for introducing me to toolz. It's new to me, and it looks like it has a lot of great features. Sure, let's remove the .pipes() method. I've reviewed the preview docs and made some adjustments. Other than that, it looks good to me. What do you think?

[machow]

It's looking really good, I left a quick thought on how we might be able to annotate pipe using ParamSpec. But overall, it seems basically ready to go! @rich-iannone you'll probably know the right section for .pipe in the reference

[machow]

I wonder if we could use ParamSpec here? Something like...

from typing import ParamSpec, Callable

P = ParamSpec("P")
def punt(f: Callable[P, int], *args: P.args, **kwargs: P.kwargs) -> int:
    return f(*args, **kwargs)


def add(x: int, y: int) -> int:
    return x + y

# flagged in static check as z is not an arg to add
punt(add, 1, y = 2, z = 3)

https://mypy-play.net/?mypy=latest&python=3.12&gist=23dd2d435071c6e9f5639cfcadf7dd16

[jrycw]

Of course! This is somewhat new to me, so please feel free to correct me if I've misunderstood anything.

[jrycw]

@machow and @rich-iannone, this PR has been pending for a while. Could we schedule it for merging, or are there any remaining concerns?

[machow]

What about putting pipe in the helper functions section. @rich-iannone WDYT?

[rich-iannone]

I like it!

[machow]

Sorry for the wait 😓, Rich and I are pairing right now -- let me try to get this merged right now

[machow]

This LGTM, @rich-iannone do you want to merge? I simplified the test a bit, and think the lil pipeline section in the reference seems okay as is!

[rich-iannone]

Thank you @jrycw !

[jrycw]

This LGTM, @rich-iannone do you want to merge? I simplified the test a bit, and think the lil pipeline section in the reference seems okay as is!

@machow, thanks for simplifying the test—it’s much more concise now.

[jrycw]

@machow and @rich-iannone, I’ve removed GT.pipes() from the PR title since we didn’t end up merging that part into the codebase. This should help prevent potential confusion in the future.