Skip to content

1 Setup

Jump to bottom
Andrew Plaza edited this page Jan 25, 2022 · 8 revisions

Before running substrate-archive, a few things are required from the chain you plan to index:

  • It uses RocksDB as its backend
  • it is running with --pruning=archive

RocksDB is the default backend for a substrate chain. However, if you have not ran the chain in archive mode previously, you will need to re-sync your chain. For polkadot, that's two commands:

polkadot purge-chain --chain=polkadot
polkadot --pruning=archive --chain=polkadot

The first command removes previous chain data stored locally on your machine, the second begins to re-sync polkadot in archive mode. This turns off any pruning optimizations to remove unused state from the rocksdb backend. So, make sure you have enough disk space before continuing.

Setting up services external to Substrate Archive

Using Docker

Alternatively to installing & running Postgres and RabbitMq manually, a docker-compose file starting the required external services for Archive is provided here. This Dockerfile exposes:

Postgres @ port 6432
RabbitMq @ port 5672
Postgres Management Interface (Optional) (PgAdmin) @ port 16543
    user: [email protected], password: admin
RabbitMq Management Interface @ port 15672
    user: guest, pass: guest

PgAdmin is a useful web-interface for running queries and checking postgres database statistics. The docker postgres port is non-standard in case of any existing postgres-installations.

Manually

Note

These instructions differ based on OS -- there may be subtle differences between how commands work between Operating Systems and which packages are required to be installed before archive use is functional.

**Required Dependencies ** Ubuntu: postgresql, postgresql-contrib, libpq-dev, rabbitmq

RabbitMq and Postgres must be started as a service prior to archive:

sudo systemctl enable postgresql && sudo systemctl start postgresql
sudo systemctl enable rabbitmq && sudo systemctl start rabbitmq
  • a Postgres Database with the components exposed under environment variables or in a config file (like for polkadot-archive). Variables that may be defined:
    • DATABASE_URL the url to the postgres database. By setting this, the below variables are not required. No Default
    • CHAIN_DATA_DB path to backend RocksDB database where a substrate node keeps it's files. Example: /home/alice/.local/share/polkadot/chains/ksmcc3/db/full/
    • note AMQP URL is not configurable via environment variables -- it needs to be configured via the Archive builder or with a toml file. See #431

Migrations are applied automatically upon running substrate-archive.

Example: Setting Up the Database

The postgres user should exist by default on most systems, and comes with the standard postgres installation. If it does not exist, you can try creating it:

# Create a postgres user if it does not yet exist
createuser -s postgres

Creating a database from the command line as postgres user

sudo -u postgres createdb "polkadot-archive"

The resulting database URL would look like this: postgres://postgres:123@localhost/polkadot-archive

Set the postgres pasword for the user postgres

sudo -u postgres psql -U postgres -d postgres -c "ALTER USER postgres WITH PASSWORD '123';" # we recommend better password security

Compiling without a database

If you wish to compile substrate-archive without a database setup, for instance in order to satisfy a CI setup, make sure to set SQLX_OFFLINE=true.

For instance, SQLX_OFFLINE=true cargo check

Clone this wiki locally