Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manage project with poetry #114

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ jobs:
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Install pip packages from dev_requirements.txt
- name: Install pip packages from requirements-dev.txt
run: |
pip install -r dev_requirements.txt
pip install -r requirements-dev.txt

- name: Test with unittest
run: |
Expand Down
59 changes: 59 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@

---
name: 🕊️ Release Workflow

on:
workflow_dispatch:

env:
ALLOWED_PUBLISHERS: "${{ github.actor == 'hmartiro' || github.actor == 'SethForsgren' }}"
STABLE_PYTHON_VERSION: '3.9'
PYTHONUNBUFFERED: "1"
FORCE_COLOR: "1"


concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

jobs:
release-workflow:
name: 🏹 Release Workflow
runs-on: ubuntu-latest

timeout-minutes: 10

steps:
- name: 📩 Checkout the source w/ full depth(for tags and SCM)
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: 🐍 Setup Python ${{ env.STABLE_PYTHON_VERSION }}
uses: actions/setup-python@v4
with:
python-version: ${{ env.STABLE_PYTHON_VERSION }}

- name: 🧱 Build wheel
run: |
pip install poetry
poetry install
poetry build

- name: 🐦 Release wheel
if: ${{ env.ALLOWED_PUBLISHERS && !failure() }}
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ SECRETS.PYPI_TOKEN }}
packages_dir: dist
verbose: true

- name: 📁 Archiving Artifacts
uses: actions/upload-artifact@v3
if: ${{ !failure() }}
continue-on-error: true
with:
name: build-artifacts
retention-days: 1
path: |
dist
5 changes: 5 additions & 0 deletions .pylintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[LOGGING]

# Format style used to check logging format string. `old` means using %
# formatting, `new` is for `{}` formatting,and `fstr` is for f-strings.
logging-format-style=fstr
36 changes: 36 additions & 0 deletions .streamlit/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Riffusion project streamlit configuration file

[global]
disableWatchdogWarning = false
showWarningOnDirectExecution = true

[client]
caching = true
displayEnabled = true


[runner]
magicEnabled = true
installTracer = false
fixMatplotlib = true


[server]
folderWatchBlacklist = []
headless = true
runOnSave = true
port = 8501
enableCORS = true


[browser]
serverAddress = "127.0.0.1"
gatherUsageStats = false
serverPort = 8501


[logger]
level = "info"

[ui]
hideTopBar = false
93 changes: 62 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# :guitar: Riffusion

<!-- markdownlint-disable MD033 MD034 -->

<a href="https://github.com/riffusion/riffusion/actions/workflows/ci.yml?query=branch%3Amain"><img alt="CI status" src="https://github.com/riffusion/riffusion/actions/workflows/ci.yml/badge.svg" /></a>
<img alt="Python 3.9 | 3.10" src="https://img.shields.io/badge/Python-3.9%20%7C%203.10-blue" />
<a href="https://github.com/riffusion/riffusion/tree/main/LICENSE"><img alt="MIT License" src="https://img.shields.io/badge/License-MIT-yellowgreen" /></a>
Expand All @@ -10,22 +12,23 @@ Read about it at https://www.riffusion.com/about and try it at https://www.riffu

This is the core repository for riffusion image and audio processing code.

* Diffusion pipeline that performs prompt interpolation combined with image conditioning
* Conversions between spectrogram images and audio clips
* Command-line interface for common tasks
* Interactive app using streamlit
* Flask server to provide model inference via API
* Various third party integrations
* Diffusion pipeline that performs prompt interpolation combined with image conditioning
* Conversions between spectrogram images and audio clips
* Command-line interface for common tasks
* Interactive app using streamlit
* Flask server to provide model inference via API
* Various third party integrations

Related repositories:

* Web app: https://github.com/riffusion/riffusion-app
* Model checkpoint: https://huggingface.co/riffusion/riffusion-model-v1

## Citation

If you build on this work, please cite it as follows:

```
```txt
@article{Forsgren_Martiros_2022,
author = {Forsgren, Seth* and Martiros, Hayk*},
title = {{Riffusion - Stable diffusion for real-time music generation}},
Expand All @@ -39,18 +42,29 @@ If you build on this work, please cite it as follows:
Tested in CI with Python 3.9 and 3.10.

It's highly recommended to set up a virtual Python environment with `conda` or `virtualenv`:
```

```shell
conda create --name riffusion python=3.9
conda activate riffusion
```

Install Python dependencies:
Install Python package:

```shell
pip install -U riffusion
```
python -m pip install -r requirements.txt

or clone the repository and install from source:

```shell
git clone https://github.com/riffusion/riffusion.git
cd riffusion
python -m pip install --editable .
```

In order to use audio formats other than WAV, [ffmpeg](https://ffmpeg.org/download.html) is required.
```

```shell
sudo apt-get install ffmpeg # linux
brew install ffmpeg # mac
conda install -c conda-forge ffmpeg # conda
Expand All @@ -61,14 +75,17 @@ If torchaudio has no backend, you may need to install `libsndfile`. See [this is
If you have an issue, try upgrading [diffusers](https://github.com/huggingface/diffusers). Tested with 0.9 - 0.11.

Guides:

* [Simple Install Guide for Windows](https://www.reddit.com/r/riffusion/comments/zrubc9/installation_guide_for_riffusion_app_inference/)

## Backends

### CPU

`cpu` is supported but is quite slow.

### CUDA

`cuda` is the recommended and most performant backend.

To use with CUDA, make sure you have torch and torchaudio installed with CUDA support. See the
Expand All @@ -80,12 +97,13 @@ steps in under five seconds, such as a 3090 or A10G.

Test availability with:

```python3
```python
import torch
torch.cuda.is_available()
```

### MPS

The `mps` backend on Apple Silicon is supported for inference but some operations fall back to CPU,
particularly for audio processing. You may need to set
`PYTORCH_ENABLE_MPS_FALLBACK=1`.
Expand All @@ -94,7 +112,7 @@ In addition, this backend is not deterministic.

Test availability with:

```python3
```python
import torch
torch.backends.mps.is_available()
```
Expand All @@ -104,27 +122,31 @@ torch.backends.mps.is_available()
Riffusion comes with a command line interface for performing common tasks.

See available commands:
```
python -m riffusion.cli -h

```shell
riffusion -h
```

Get help for a specific command:
```
python -m riffusion.cli image-to-audio -h

```shell
riffusion image-to-audio -h
```

Execute:
```
python -m riffusion.cli image-to-audio --image spectrogram_image.png --audio clip.wav

```shell
riffusion image-to-audio --image spectrogram_image.png --audio clip.wav
```

## Riffusion Playground

Riffusion contains a [streamlit](https://streamlit.io/) app for interactive use and exploration.

Run with:
```
python -m streamlit run riffusion/streamlit/playground.py --browser.serverAddress 127.0.0.1 --browser.serverPort 8501

```shell
riffusion-playground
```

And access at http://127.0.0.1:8501/
Expand All @@ -137,8 +159,8 @@ Riffusion can be run as a flask server that provides inference via API. This ser

Run with:

```
python -m riffusion.server --host 127.0.0.1 --port 3013
```shell
riffusion-server --host 127.0.0.1 --port 3013
```

You can specify `--checkpoint` with your own directory or huggingface ID in diffusers format.
Expand All @@ -148,7 +170,8 @@ Use the `--device` argument to specify the torch device to use.
The model endpoint is now available at `http://127.0.0.1:3013/run_inference` via POST request.

Example input (see [InferenceInput](https://github.com/hmartiro/riffusion-inference/blob/main/riffusion/datatypes.py#L28) for the API):
```

```json
{
"alpha": 0.75,
"num_inference_steps": 50,
Expand All @@ -171,45 +194,53 @@ Example input (see [InferenceInput](https://github.com/hmartiro/riffusion-infere
```

Example output (see [InferenceOutput](https://github.com/hmartiro/riffusion-inference/blob/main/riffusion/datatypes.py#L54) for the API):
```

```json
{
"image": "< base64 encoded JPEG image >",
"audio": "< base64 encoded MP3 clip >"
}
```

## Tests

Tests live in the `test/` directory and are implemented with `unittest`.

To run all tests:
```

```shell
python -m unittest test/*_test.py
```

To run a single test:
```

```shell
python -m unittest test.audio_to_image_test
```

To preserve temporary outputs for debugging, set `RIFFUSION_TEST_DEBUG`:
```

```shell
RIFFUSION_TEST_DEBUG=1 python -m unittest test.audio_to_image_test
```

To run a single test case within a test:
```

```shell
python -m unittest test.audio_to_image_test -k AudioToImageTest.test_stereo
```

To run tests using a specific torch device, set `RIFFUSION_TEST_DEVICE`. Tests should pass with
`cpu`, `cuda`, and `mps` backends.

## Development Guide
Install additional packages for dev with `python -m pip install -r dev_requirements.txt`.

* Linter: `ruff`
Install additional packages for dev with `python -m pip install -r requirements-dev.txt`.

* Linters: `ruff`, `flake8`, `pylint`
* Formatter: `black`
* Type checker: `mypy`
* Docstring checker: `pydocstyle`

These are configured in `pyproject.toml`.

Expand Down
8 changes: 0 additions & 8 deletions dev_requirements.txt

This file was deleted.

Loading