Pypi

[constantinpape]

Publish on pypi

[clbarnes]

Vigra and nifty will need to be handled. I really wish there were a pip-installable equivalent of vigra.VigraArray, it's such a useful data structure and probably wouldn't be that hard to do. Maybe xarray.DataArray would work, but the API is a bit more awkward and its handling of resolution is pretty inefficient.

[constantinpape]

Vigra and nifty will need to be handled.

Yes, vigra and nifty are optional dependencies and can't be intalled via pip.
Note that the package is designed in such a way that the imports are only done when necessary.
So it's possible to use elf without vigra and nifty, but the functionality from elf.segmentation will not be available.

[clbarnes]

Mentioned in #10: it's nice that conda reads setuptools' data for picking up dependencies, but there are a couple which are on conda but not setuptools, so attempting to install those extras from pip will break. The more subtle breakage is nifty, which clashes with the name of an entirely unrelated PyPI package.

[constantinpape]

Yes, that is a good point.
The idea @k-dominik and me had was to put everything that is on conda, but not on pip into extras. This is not done consistently right now and should be fixed. But we might have the opposite case that something is on pip and not on conda, so I would suggest the following:

# all packages that are available on pip and on conda
requires = [...]
# packages that are only on conda, and will be added as requirements in the conda recipe
extras_conda = {....}
# packages that are only on pip. will be used in setup and could be added as pip dependencies in the conda recipe
extras_pip = {...}

setup(...,
  extras_require=extras_pip
)

Do you see any issues with this pattern @clbarnes?

[clbarnes]

It depends where load_setup_py_data in the meta.yaml gets its information from. I think it's probably getting it from the metadata setuptools generates, in which case it'll see exactly what pip sees (where we want it to see the opposite).

One such pip-not-conda dependency is pyn5 (as a pip-friendly alternative for N5 access). The tool we use to build it publishes wheels to pypi, but can't build conda packages.

[constantinpape]

It depends where load_setup_py_data in the meta.yaml gets its information from. I think it's probably getting it from the metadata setuptools generates, in which case it'll see exactly what pip sees (where we want it to see the opposite).

You're right, my initial suggestion doesn't work.
I changed it bit and have a solution that would work for me now, see my PR on your PR :D
clbarnes#1

[constantinpape]

Ok, the dependency handling is improved a bit thanks to #10.
I will look into publishing a new release soon.
Maybe it would be a good point to push it on pypi then.
@clbarnes do you have a good pipeline in place to publish packages from github to pypi?

[clbarnes]

h5py-like builds and uploads wheels and sdists automatically from travis, on tagged commits to master.

https://github.com/clbarnes/h5py_like/blob/master/.travis.yml

To add the encrypted pypi password once, you need ruby and the Travis gem locally. You can also add it as an environment variable on the Travis website, I think.

[constantinpape]

h5py-like builds and uploads wheels and sdists automatically from travis, on tagged commits to master.

https://github.com/clbarnes/h5py_like/blob/master/.travis.yml

To add the encrypted pypi password once, you need ruby and the Travis gem locally. You can also add it as an environment variable on the Travis website, I think.

Thanks for sharing this! I will have a look once I come around to make the release.