[four-point-oh?] Coordinated 3.3 ⇾ 3.8 upgrade of packages with plugins

[LDAP]

With the upcoming PC4 release, packages with plugins such as LSP (@sublimelsp) or SublimeLinter (@kaste) are going to switch to the Python 3.8 plugin host (see Discord discussion).

One problem is the coordinated upgrade of the "main package" and it's plugins (LSP-helpers, SublimeLinter-adapters):
Upgrading the main package to 3.8 breaks all plugins that are not yet ready to update and vice-versa. This is especially difficult because some plugins are unmaintained or maintained by different members of the community.

My vague solution (inspired by the poetry package manager):

• As already proposed in [four-point-oh?] Unload packages that depend on a plugin that is updated #1628 plugins should declare their dependency to the main package and the version that is required. (In the channel so that this can be influenced for unmaintained packages?)

• Plugins should declare the required python version similar how they declare their supported ST version. (In the channel?)

• Similarly to python package managers (e.g. poetry), when installing a new package (for example a 3.3 plugin) check if all requirements can be fulfilled and possibly downgrade dependencies (main package and its plugins) after asking the user for permission. Example: LSP and LSP-helper are 3.8, user installs LSP-helper2 which is 3.3 ⇾ PC asks for permission and then downgrades LSP and LSP-helper to the latest 3.3 version and installs LSP-helper2.

• On update, PC builds a dependency graph and updates plugins only if the whole component in the dependency graph can be updated.

• Optionally: Ask the user if packages should be "Updated anyway"

• Optionally: Ask the user if the plugins that do not provide a 3.8 version should be disabled

It is questionable if this is worth the development effort. Also, there may be simpler solutions, maybe (parts of) an existing python package manager can be reused?

[kaste]

It is probably only LSP and SublimeLinter with that architecture. SublimeLinter ships an auto-loader-after-update. And LSP not, I guess?

On discourse I made the "proposal" that users decide this. Generally, you maintain a configuration locally, which holds a list of plugins you want to promote. And then we just create a .python-version file with the correct content after install/upgrade.

It begs the question when exactly this file has to be created so that the plugins don't load twice and maybe throw when they first try on py3.3 (with their framework/core already on py3.8).

[deathaxe]

It begs the question when exactly this file has to be created

A .python-version needs to be created before a package is reenabled during installation/upgrade. That's basically something PC needs to do as there's no valuable event mechanism which works between py33 and py38 synchronously.

Plugins should declare the required python version similar how they declare their supported ST version.

They already do so via .python-version.

There's an idea of forcing packages to run on certain plugin host locally or via declaration in repositories.json, to be able to push unmaintained packages forwared, if they are known to support py3.8. But keeping a set of packages which needs to run on the same plugin_host together is somewhat challanging as it is not part of the design.

Similarly to python package managers (e.g. poetry) ...

I had a look at how pip resolves requirements and handles dependencies when looking into how to handle python wheels (which was started by Will vith pep508 module).

I must confess - creating a full fledged package manager with all the capabilities of what poetry, npm, pip etc. can do is far beyound the effort I am willing to put into this project, especially as we can't rely on any 3rd-party package as none of them support python 3.3 anymore, which will need to be maintained for various years if ST3 1.0 (3143) is the first version to be supported.

So I guess, I won't invent something like that for just 2 packages, too. The major design of ST is to run independend plugins. The construct of a major package and various depending helpers is not part of any official design.

Version pinning etc. is not within the scope of PC. It always updates packages to the latest available version. All meta data downloaded from packagecontrol.io contain information of at most 3 most recent releases, so something like universal version pinning is not possible without major refactoring of the overall communication between PC client and packagecontrol.io.

The only thing I could imagine was to force depending packages to the same plugin_host by injecting a .python-version file, to keep all members on the same host.