Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
@phihos
phihos committed Feb 14, 2024
1 parent 90c6794 commit e5addbf
Show file tree
Hide file tree
Showing 2 changed files with 88 additions and 59 deletions.
142 changes: 85 additions & 57 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
# PyDuckling
[![Project License - MIT](https://img.shields.io/pypi/l/pyduckling-native.svg)](https://raw.githubusercontent.com/treble-ai/pyduckling-native/master/LICENSE)
[![pypi version](https://img.shields.io/pypi/v/pyduckling-native.svg)](https://pypi.org/project/pyduckling-native/)
[![conda version](https://img.shields.io/conda/vn/treble-ai/pyduckling.svg)](https://www.anaconda.com/download/)
[![download count](https://img.shields.io/conda/dn/treble-ai/pyduckling.svg)](https://www.anaconda.com/download/)
[![Downloads](https://pepy.tech/badge/pyduckling-native)](https://pepy.tech/project/pyduckling-native)
[![PyPI status](https://img.shields.io/pypi/status/pyduckling-native.svg)](https://github.com/treble-ai/pyduckling-native)
![Linux Tests](https://github.com/treble-ai/pyduckling/workflows/Linux%20Tests/badge.svg?branch=master)
![Mac Tests](https://github.com/treble-ai/pyduckling/workflows/Mac%20Tests/badge.svg?branch=master)
[![Project License - MIT](https://img.shields.io/pypi/l/pyduckling-native.svg)](https://raw.githubusercontent.com/phihos/pyduckling-native/master/LICENSE)
[![pypi version](https://img.shields.io/pypi/v/pyduckling-native-phihos.svg)](https://pypi.org/project/pyduckling-native-phihos/)
[![Downloads](https://pepy.tech/badge/pyduckling-native-phihos)](https://pepy.tech/project/pyduckling-native-phihos)
[![PyPI status](https://img.shields.io/pypi/status/pyduckling-native-phihos.svg)](https://github.com/phihos/pyduckling-native)
![Linux Tests](https://github.com/phihos/pyduckling/workflows/build/badge.svg?branch=master)

*Copyright © 2020– Treble.ai*

> ℹ️ This is a fork of the original pyduckling-native library. There are differences to the original:
> * Build against the latest version of [Duckling](https://github.com/facebook/duckling)
> * x86_64 Linux only, but contributions welcome if you dare to take on the challenge
## Overview
This package provides native bindings for Facebook's [Duckling](https://github.com/facebook/duckling) in Python. This package supports all dimensions and languages available on the original library, and it does not require to spawn a Haskell server and does not use HTTP to call the Duckling API.

**Note:** This package is completely Haskell-less
> ℹ️ This package is completely Haskell-less
## Installing
To install pyduckling, you can use both conda and pip package managers:
Expand All @@ -23,56 +24,10 @@ To install pyduckling, you can use both conda and pip package managers:
pip install pyduckling-native-phihos
```

**Notes:** Right now, we only provide package distributions for Linux (x86_64). We will provide Windows and Mac distributions on the next release


## Dependencies
To compile pyduckling, you will require the latest nightly release of [Rust](https://rustup.rs/), alongside [Cargo](https://crates.io/). Also, it requires a Python distribution with its corresponding development headers. Finally, this project depends on the following Cargo crates:

* [PyO3](https://github.com/PyO3/pyo3): Library used to produce Python bindings from Rust code.
* [Maturin](https://github.com/PyO3/maturin): Build system to build and publish Rust-based Python packages

Additionally, this package depends on [Duckling-FFI](https://github.com/treble-ai/duckling-ffi), used to compile the native interface to Duckling on Haskell. In order to compile Duckling-FFI, you will require the [Stack](https://haskell-lang.org/get-started) Haskell manager.


## Installing locally
Besides Rust and Stack, you will require the latest version of maturin installed to compile this project locally:

```bash
pip install maturin toml
```

First, you will need to compile Duckling-FFI in order to produce the shared library ``libducklingffi``, to do so, you can use the git submodule found at the root of this repository:

```bash
cd duckling-ffi
stack build
```

Then, you will need to move the resulting binary ``libducklingffi.so`` to the ``ext_lib`` folder:
> ℹ️ Right now, we only provide package distributions for Linux (x86_64).
```bash
cp duckling-ffi/libducklingffi.so ext_lib
```

After completing this procedure, it is possible to execute the following command to compile pyduckling:

```bash
maturin develop
```

In order to produce wheels, ``maturin build`` can be used instead. This project supports [PEP517](https://www.python.org/dev/peps/pep-0517/), thus pip can be used to install this package as well:

```bash
pip install -U .
```

## Running tests
We use pytest to run tests as it follows (after calling ``maturin develop``):

```bash
pytest -v duckling/tests
```

## Package usage
PyDuckling provides access to the parsing capabilities of Duckling used to extract structured data from text.
Expand Down Expand Up @@ -130,9 +85,82 @@ This wrapper allows access to all the dimensions and languages available on Duck
| `url` | "https://api.wit.ai/message?q=hi" | `{"value":"https://api.wit.ai/message?q=hi","domain":"api.wit.ai"}` |
| `volume` | "4 gallons" | `{"value":4,"type":"value","unit":"gallon"}` |

## Dependencies
To compile pyduckling, you will require the latest nightly release of [Rust](https://rustup.rs/), alongside [Cargo](https://crates.io/). Also, it requires a Python distribution with its corresponding development headers. Finally, this project depends on the following Cargo crates:

* [PyO3](https://github.com/PyO3/pyo3): Library used to produce Python bindings from Rust code.
* [Maturin](https://github.com/PyO3/maturin): Build system to build and publish Rust-based Python packages

Additionally, this package depends on [Duckling-FFI](https://github.com/treble-ai/duckling-ffi), used to compile the native interface to Duckling on Haskell. In order to compile Duckling-FFI, you will require the [Stack](https://haskell-lang.org/get-started) Haskell manager.


## Installing locally

### Via Docker

The only thing you need is a running [Docker](https://docs.docker.com/engine/install/) daemon
and permission to run `docker build` and `docker run`.
Then just run
```shell
./build.sh
```

All build dependencies are already installed in container images. The build script will get them and start
containers for building the Haskell lib and the Python lib. The directories `.cache`, `duckling-ffi/.stack-work`
and `target` will appear. The first two are for accelerating future builds and the last one will contain your final
build result.

After the `build.sh` completed successfully, you can find wheel files (binary distributions of the library) inside `target/wheels`.
The Python version (e.g. Python 3.11 = `cp311`), the libc variant
(`manylinux` or `musllinux`, if you are unsure you probably need manylinux) and the CPU architecture
(currently only `x86_64`) are encoded in the file name. Just pick the file matching to your system and install it with:

```shell
pip install -U target/wheels/<myfile>.whl
```

### Manually

Besides Rust and Stack, you will require the latest version of maturin installed to compile this project locally:

```bash
pip install maturin toml
```

First, you will need to compile Duckling-FFI in order to produce the shared library ``libducklingffi``, to do so, you can use the git submodule found at the root of this repository:

```bash
cd duckling-ffi
stack build
```

Then, you will need to move the resulting binary ``libducklingffi.a`` to the ``ext_lib`` folder:

```bash
cp duckling-ffi/libducklingffi.a ext_lib
```

After completing this procedure, it is possible to execute the following command to compile pyduckling:

```bash
maturin develop
```

In order to produce wheels, ``maturin build`` can be used instead. This project supports [PEP517](https://www.python.org/dev/peps/pep-0517/), thus pip can be used to install this package as well:

```bash
pip install -U .
```

## Running tests
We use pytest to run tests as it follows (after calling ``maturin develop``):

```bash
pytest -v duckling/tests
```

## Changelog
Please see our [CHANGELOG](https://github.com/treble-ai/pyduckling/blob/master/CHANGELOG.md) file to learn more about our new features and improvements.
Please see our [CHANGELOG](https://github.com/phihos/pyduckling/blob/master/CHANGELOG.md) file to learn more about our new features and improvements.


## Contribution guidelines
Expand Down
5 changes: 3 additions & 2 deletions build.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ echo -n $PYTHON3_VERSION_RANGE | parallel -j0 --halt now,fail=1 -d ' ' 'build_p
mkdir -p "${USER_HOME_CACHE_PATH}"

# --- build the statically linked library file
echo "---"
echo "Building statically linked \"libducklingffi.a\"..."
docker run \
--mount type=bind,source="$(pwd)"/duckling-ffi,target=/duckling-ffi \
Expand All @@ -100,12 +101,12 @@ docker run \
"${BUILD_IMAGE_DUCKLING_FFI}" \
sleep infinity
docker exec "${STATIC_LIB_CONTAINER_NAME}" bash -c 'cd /duckling-ffi && stack build --no-install-ghc --system-ghc --allow-different-user'
docker exec "${STATIC_LIB_CONTAINER_NAME}" bash -c 'cd /duckling-ffi && ghc --numeric-version > ghc-version'
docker cp "${STATIC_LIB_CONTAINER_NAME}:/duckling-ffi/libducklingffi.a" ext_lib/libducklingffi.a

# --- build binary distributions
echo "---"
echo "Building GLIBC wheels for all python versions..."
echo -n $PYTHON3_VERSION_RANGE | parallel -j0 --halt now,fail=1 -d ' ' 'build_python_package {} glibc'

echo "---"
echo "Building MUSL wheels for all python versions..."
echo -n $PYTHON3_VERSION_RANGE | parallel -j0 --halt now,fail=1 -d ' ' 'build_python_package {} musl'

0 comments on commit e5addbf

Please sign in to comment.