Add plugin system and local wheels support #91

JeanChristopheMorinPerso · 2024-02-03T23:06:22Z

Fixes #64.

Add a new generic plugins system and builtin plugins. The glugin system is based on pluggy and has 5 hooks: prePipResolve, postPipResolve, groupPackages, cleanup, patches and metadata.

The 2 builtin plugins are the PySide6 and shiboken6 plugins for correctly installing PySide6>=6.3. The plugins take care of merging all the PySide6 related packages (PySide6, PySide6-Addons and PySide6-Essentials) into one rez package.

$ rez-pip2 PySide6 --prefix /tmp/asd --python 3.11 -- --find-links /tmp/wheels --no-index

$ rez-env PySide6 --paths ~/rez_packages:/tmp/asd -- python -c 'import PySide6,shiboken6; print(PySide6); print(shiboken6)'
<module 'PySide6' from '/tmp/asd/PySide6/6.6.1/9645a50b415bcaad6bcde791aeb0859a43c56501/python/PySide6/__init__.py'>
<module 'shiboken6' from '/tmp/asd/shiboken6/6.6.1/9645a50b415bcaad6bcde791aeb0859a43c56501/python/shiboken6/__init__.py'>

🎉

~~This only works on Linux and macOS. On Windows, it works with Python 3.7 but not 3.8 and above. PySide6 on Windows makes use of os.add_dll_directory...~~

Additionally, add support for installing local wheels. Considering that PySide is a big package, I needed that functionality to speed up the development of the plugin system.

TODOs:

Add option to disable plugins/hooks
Test if builtin plugins can be overridden
~~Test on Windows~~
~~Write tests~~
~~Cleanup the code~~
~~Write docs (and automatically generate docs for builtin plugins).~~

src/rez_pip/rez.py

src/rez_pip/plugins/__init__.py

JeanChristopheMorinPerso · 2024-02-03T23:15:01Z

src/rez_pip/rez.py

@@ -113,8 +136,8 @@ def make_root(variant: rez.packages.Variant, path: str) -> None:
        pkg.pip = {
            "name": dist.name,
            "version": dist.version,
-            "is_pure_python": metadata["is_pure_python"],
-            "wheel_url": wheelURL,
+            "is_pure_python": isPure,


This annoys me. I don't think we really need this...

What is annoying about it?

It's kind of useless and getting the value of "is pure" is annoying. Do you think I should keep it?

src/rez_pip/plugins/PySide6.py

src/rez_pip/cli.py

codecov · 2024-02-04T01:49:50Z

Codecov Report

Attention: Patch coverage is 89.97613% with 42 lines in your changes missing coverage. Please review.

Project coverage is 90.00%. Comparing base (71ba391) to head (97d4d08).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/rez_pip/plugins/PySide6.py	86.25%	5 Missing and 6 partials ⚠️
src/rez_pip/install.py	84.21%	4 Missing and 5 partials ⚠️
src/rez_pip/patch.py	83.33%	6 Missing and 3 partials ⚠️
src/rez_pip/pip.py	88.57%	3 Missing and 1 partial ⚠️
src/rez_pip/plugins/__init__.py	94.82%	2 Missing and 1 partial ⚠️
src/rez_pip/cli.py	95.12%	1 Missing and 1 partial ⚠️
src/rez_pip/compat.py	75.00%	1 Missing and 1 partial ⚠️
src/rez_pip/rez.py	93.75%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #91      +/-   ##
==========================================
+ Coverage   81.14%   90.00%   +8.85%     
==========================================
  Files           8       13       +5     
  Lines         716     1050     +334     
  Branches      133      202      +69     
==========================================
+ Hits          581      945     +364     
+ Misses        121       68      -53     
- Partials       14       37      +23

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

BryceGattis

Left a couple questions/comments.

docs/source/conf.py

docs/source/plugins.rst

src/rez_pip/pip.py

src/rez_pip/plugins/PySide6.py

BryceGattis · 2024-04-30T13:18:44Z

src/rez_pip/plugins/PySide6.py

+    packages: typing.List[str],
+) -> None:
+    pyside6Seen = False
+    variantsSeens = []


Should this just be variantsSeen?

BryceGattis · 2024-04-30T13:23:29Z

src/rez_pip/rez.py

@@ -113,8 +136,8 @@ def make_root(variant: rez.packages.Variant, path: str) -> None:
        pkg.pip = {
            "name": dist.name,
            "version": dist.version,
-            "is_pure_python": metadata["is_pure_python"],
-            "wheel_url": wheelURL,
+            "is_pure_python": isPure,


What is annoying about it?

Signed-off-by: Jean-Christophe Morin <[email protected]>

…e arguments are immutable The code is also a little bit cleaner IMO Signed-off-by: Jean-Christophe Morin <[email protected]>

Signed-off-by: Jean-Christophe Morin <[email protected]>

…force immutability at runtime. Signed-off-by: Jean-Christophe Morin <[email protected]>

Signed-off-by: Jean-Christophe Morin <[email protected]>

… positional arguments instead of kwargs Signed-off-by: Jean-Christophe Morin <[email protected]>

Signed-off-by: Jean-Christophe Morin <[email protected]>

… is responsible of performing the actions Signed-off-by: Jean-Christophe Morin <[email protected]>

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso mentioned this pull request Feb 3, 2024

Fail to install PySide6 #64

Closed

JeanChristopheMorinPerso commented Feb 3, 2024

View reviewed changes

JeanChristopheMorinPerso force-pushed the plugins branch from 5163f94 to c7edc67 Compare February 4, 2024 00:36

Repository owner deleted a comment from sonarqubecloud bot Mar 6, 2024

JeanChristopheMorinPerso force-pushed the plugins branch 3 times, most recently from 178c63c to f9b6577 Compare April 28, 2024 23:38

JeanChristopheMorinPerso changed the title ~~Initial commit adding a plugin system and a PySide6 plugin~~ Plugin system Apr 28, 2024

JeanChristopheMorinPerso force-pushed the plugins branch from f9b6577 to e57ce92 Compare April 28, 2024 23:58

BryceGattis reviewed Apr 30, 2024

View reviewed changes

JeanChristopheMorinPerso mentioned this pull request Nov 17, 2024

rez-pip with custom package.py AcademySoftwareFoundation/rez#1871

Open

JeanChristopheMorinPerso changed the title ~~Plugin system~~ Add plugin system Jan 1, 2025

JeanChristopheMorinPerso force-pushed the plugins branch from cd5f530 to 27b6a5c Compare January 1, 2025 17:34

JeanChristopheMorinPerso added 16 commits January 1, 2025 13:33

Initial commit adding a plugin system and a PySide6 plugin

209e30f

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix bug with requirements of merged packages

b503662

Signed-off-by: Jean-Christophe Morin <[email protected]>

Simplify registration of builtin plugins

7006ab9

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add pluggy to project deps

5a7ff04

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix tests

d854773

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix remaining issues (I think)

f41fb6d

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix download tests and add a test for skipped downloads

2915723

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix typing

d184853

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix typing

5bd4d54

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add basic test for plugins module

f93a87d

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix typing again

a8fa6d0

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add type hints and new dataclasses to express plugin authors that som…

91c7aca

…e arguments are immutable The code is also a little bit cleaner IMO Signed-off-by: Jean-Christophe Morin <[email protected]>

Improve plugins docs and autogenerate signatures

788fb4f

Signed-off-by: Jean-Christophe Morin <[email protected]>

Explicitly use tuples in hookspecs to improve typing even more and en…

13667aa

…force immutability at runtime. Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix some mypy errors

b798d8d

Signed-off-by: Jean-Christophe Morin <[email protected]>

More typing fixes and fix regression introduced by calling hooks with…

8e53cf3

… positional arguments instead of kwargs Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso force-pushed the plugins branch from 58e81fd to 32dff3b Compare January 2, 2025 03:20

JeanChristopheMorinPerso added 7 commits January 5, 2025 14:56

Some more docs

a3df115

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix order of hooks in docs

90e8b99

Signed-off-by: Jean-Christophe Morin <[email protected]>

Ignore /patches dir

d051a5e

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add .gitattributes

3e3bee0

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add a new patching mechanism by introducing a new patches hook

ca8d3b9

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add patches for PySide6

7924a55

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix test_getHookImplementations and test_list_plugins

3e8939d

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso force-pushed the plugins branch from 3cf2620 to 3e8939d Compare January 11, 2025 20:57

JeanChristopheMorinPerso added 11 commits January 11, 2025 16:10

Try to set TERM to dumb

996e0f2

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix rich output in CI and tests

458b340

Signed-off-by: Jean-Christophe Morin <[email protected]>

Use fuzz=True when applying patches

8817b02

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add more PySide6 patches and don't apply patch with fuzzing

cece022

Signed-off-by: Jean-Christophe Morin <[email protected]>

Rework how the cleanup hook works. It now returns actions and rez-pip…

55c2b62

… is responsible of performing the actions Signed-off-by: Jean-Christophe Morin <[email protected]>

Some tying fixes

107bcec

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add note about future breaking changes in the plugins doc

57484c3

Signed-off-by: Jean-Christophe Morin <[email protected]>

Adjust shiboken cleanup hook

5b76ca7

Fix tests

961d95e

Signed-off-by: Jean-Christophe Morin <[email protected]>

Improve docs and fix typos in docs

316f066

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add some more logs to debug stuff

9349997

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso force-pushed the plugins branch from 18cf87d to 9349997 Compare January 12, 2025 20:13

JeanChristopheMorinPerso added 3 commits January 12, 2025 15:23

Try to normalize path to posix

ec51e08

Signed-off-by: Jean-Christophe Morin <[email protected]>

Log patch_ng logs if applying a patch fails

887b1bd

Signed-off-by: Jean-Christophe Morin <[email protected]>

Remove superfluous logs

5bdd697

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso mentioned this pull request Jan 15, 2025

TSC Meeting 2025-01-16 AcademySoftwareFoundation/rez#1907

Open

JeanChristopheMorinPerso added the enhancement New feature or request label Jan 16, 2025

Fix Windows console script test

97d4d08

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso marked this pull request as ready for review January 18, 2025 20:34

JeanChristopheMorinPerso merged commit ce91594 into main Jan 18, 2025
17 of 21 checks passed

JeanChristopheMorinPerso deleted the plugins branch January 18, 2025 20:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add plugin system and local wheels support #91

Add plugin system and local wheels support #91

JeanChristopheMorinPerso commented Feb 3, 2024 •

edited

Loading

JeanChristopheMorinPerso Feb 3, 2024

BryceGattis Apr 30, 2024

JeanChristopheMorinPerso May 1, 2024

codecov bot commented Feb 4, 2024 •

edited

Loading

BryceGattis left a comment

BryceGattis Apr 30, 2024

BryceGattis Apr 30, 2024

Add plugin system and local wheels support #91

Add plugin system and local wheels support #91

Conversation

JeanChristopheMorinPerso commented Feb 3, 2024 • edited Loading

JeanChristopheMorinPerso Feb 3, 2024

Choose a reason for hiding this comment

BryceGattis Apr 30, 2024

Choose a reason for hiding this comment

JeanChristopheMorinPerso May 1, 2024

Choose a reason for hiding this comment

codecov bot commented Feb 4, 2024 • edited Loading

Codecov Report

BryceGattis left a comment

Choose a reason for hiding this comment

BryceGattis Apr 30, 2024

Choose a reason for hiding this comment

BryceGattis Apr 30, 2024

Choose a reason for hiding this comment

JeanChristopheMorinPerso commented Feb 3, 2024 •

edited

Loading

codecov bot commented Feb 4, 2024 •

edited

Loading