Installing and Running for Developers

StoryBoard has two main components: the API server, and the Javascript-based web client. The API server is essential, but the webclient can be swapped out for an alternative if an alternative is available. This means it is possible to use a different user interface with the StoryBoard API; install instructions for those are detailed in their own repos (eg: boartty, a commandline interface, is available here: https://git.openstack.org/cgit/openstack/boartty/).

This install guide will cover the API and the most widely-used StoryBoard webclient, and assumes being run on Ubuntu 16.04 or newer. The instructions are mostly portable to other distributions.

The recommended way to set up your machine for developing StoryBoard is to use the docker-compose.yml file provided. However, we also provide instructions for a manual setup if preferred.

Using Docker

This approach uses Docker to run the services required by StoryBoard, such as MySQL and RabbitMQ. The StoryBoard API and webclient are run on the host machine directly, to reduce cycle time when developing. They use tox to run using virtualenvs to minimise the amount of manual installation required.

Upon completion of these steps, you should have a usable StoryBoard API running at http://localhost:8080/ and a usable StoryBoard webclient served at http://localhost:9000/.

1. Install docker

Follow the docker installation instructions for your platform.

Note

On Linux, be sure to add your user to the docker group to avoid needing sudo:

sudo usermod -aG docker your-user

You’ll need to log out and in again for this to take effect.

2. Install docker-compose

Either install using pip:

pip3 install --user docker-compose

or follow the instructions for your platform.

3. Get the code

The code is stored using git, so you’ll need to have git installed:

sudo apt install git

The code for the API and webclient can then be cloned:

git clone https://git.openstack.org/openstack-infra/storyboard
git clone https://git.openstack.org/openstack-infra/storyboard-webclient
cd storyboard

4. Run containers

Currently the docker-compose.yml file sets up 3 containers to provide the following services

  • MySQL
  • Swift
  • RabbitMQ

The containers can be started by doing the following, starting in the root of the storyboard repository:

cd docker
docker-compose up

Note

You can make the docker-compose process run in the background by instead doing:

cd docker
docker-compose up -d

5. Install dependencies

Some dependencies are needed to run the API and build the webclient. On Ubuntu, you can install these with:

sudo apt install build-essential python3-dev
pip3 install --user tox

6. Migrate the database

At this point you could run StoryBoard, but its useless with an empty database. The migrations are run using the storyboard-db-manage script, which you can run using tox in the root of the storyboard repository:

tox -e venv -- storyboard-db-manage --config-file ./docker/storyboard.conf upgrade head

This command runs all the database migrations in order. Under the hood it uses alembic, and has a similar CLI.

7. Run the API

The API is run using the storyboard-api command. Again this can be run using tox in the root of the storyboard repository:

tox -e venv -- storyboard-api --config-file ./docker/storyboard.conf

The docker/storyboard.conf configuration file is contains config which is already set up to use the containers created earlier, so there is no need for manual configuration.

The output of this command should finish with something like:

2019-03-20 11:25:44.862 22047 INFO storyboard.api.app [-] Starting server in PID 22047
2019-03-20 11:25:44.863 22047 INFO storyboard.api.app [-] Configuration:
2019-03-20 11:25:44.863 22047 INFO storyboard.api.app [-] serving on 0.0.0.0:8080, view at http://127.0.0.1:8080

At that point, the API is running successfully. You can stop it using Ctrl+C or by closing your terminal.

8. Serve the webclient

The storyboard-webclient repository provides a tox target which builds the webclient and serves it using a development server. You can run it using tox in the root of the storyboard-webclient repository:

tox -e grunt_no_api -- serve

This will take a little while to run as it obtains the required dependencies using npm, and builds node-sass.

The output of this command should finish with something like:

Running "connect:livereload" (connect) task
Started connect web server on http://localhost:9000

Running "watch" task
Waiting...

At that point the webclient is being served successfully. You can stop it using Ctrl+C or by closing the terminal. Any changes to existing files in the codebase will cause it to automatically rebuild the webclient and refresh the page in your browser, to help streamline the development workflow.

You can view it in a browser at http://localhost:9000/. You should also be able to log in here. The provided configuration file uses Ubuntu One as the OpenID provider, so you’ll need an Ubuntu One account to do so.

9. Enable notifications

Notifications in StoryBoard are handled by workers which subscribe to events on a message queue. Currently only RabbitMQ is supported. The docker-compose.yml file runs a RabbitMQ server, and the provided config file is already set up to enable notifications.

To run the workers so that notifications are actually created, use tox in the root of the storyboard repository:

tox -e venv -- storyboard-worker-daemon --config-file ./docker/storyboard.conf

This will start 5 workers to listen for events and create any relevant notifications.

Manual Installation

1. Install dependencies

To start the API server, make sure you have the following packages installed locally:

  • build-essential

  • python3-dev

  • python3-pip

  • MySQL

    sudo apt update
sudo apt install build-essential python3-dev python3-pip
sudo apt install mysql-server-5.7    # Here you should be asked to set a password
mysql --version

Note

MySQL must be >= 5.6, to support fulltext indexes on InnoDB tables

Warning

On Ubuntu 17.10 or newer mysql won’t ask to set a root password. You’ll need to manually set one, which you can read how to do here.

2. Get the code

The code is stored using git, so you’ll need to have git installed:

sudo apt install git

The code for the API and webclient can then be cloned:

git clone https://git.openstack.org/openstack-infra/storyboard
git clone https://git.openstack.org/openstack-infra/storyboard-webclient
cd storyboard

3. Create database

StoryBoard requires a single database. These commands create one called storyboard, and delete any existing database with the same name.

mysql -u root -p -e 'DROP DATABASE IF EXISTS storyboard;'
mysql -u root -p -e 'CREATE DATABASE storyboard;'

Note

If you want to use a non-root user, change root to the desired username.

4. Create a config file

StoryBoard needs a configuration file to run. The minimum useable configuration needs to contain an uncommented connection line in addition to the sample content, to allow a database connection.

cp ./etc/storyboard.conf.sample ./etc/storyboard.conf

To make this into a useable config file, some changes are needed. Edit ./etc/storyboard.conf and make the following changes:

  • Update the connection line in the database section to replace root:pass with the username and password you’re using to connect to the database. This is likely root and the password you chose when installing MySQL.

Uncomment this line by removing the #, ensuring there is no whitespace at the start of the line.

Warning

If you are running the API in a VM, and plan to access it remotely, ie. by its IP address or hostname, you also need to add that IP address or hostname to the valid_oauth_clients line in the oauth section. Uncomment this line too. It can be a regular expression as well if started with a ^ character.

5. Install tox

StoryBoard uses tox for both running tests and managing a virtualenv for running the development servers.

pip3 install --user tox

Note

If this is your first time passing --user to pip, you will likely need to add ~/.local/bin to your PATH.

6. Migrate the database

At this point you could run StoryBoard, but its useless with an empty database. The migrations are run using the storyboard-db-manage script, which you can run using tox in the root of the storyboard repository:

tox -e venv -- storyboard-db-manage --config-file ./etc/storyboard.conf upgrade head

This command runs all the database migrations in order. Under the hood it uses alembic, and has a similar CLI.

7. Run the API

The API is run using the storyboard-api command. Again this can be run using tox in the root of the storyboard repository:

tox -e venv -- storyboard-api --config-file ./etc/storyboard.conf

The output of this command should finish with something like:

2019-03-20 11:25:44.862 22047 INFO storyboard.api.app [-] Starting server in PID 22047
2019-03-20 11:25:44.863 22047 INFO storyboard.api.app [-] Configuration:
2019-03-20 11:25:44.863 22047 INFO storyboard.api.app [-] serving on 0.0.0.0:8080, view at http://127.0.0.1:8080

At that point, the API is running successfully. You can stop it using Ctrl+C or by closing your terminal.

8. Serve the webclient

The storyboard-webclient repository provides a tox target which builds the webclient and serves it using a development server. You can run it using tox in the root of the storyboard-webclient repository:

tox -e grunt_no_api -- serve

This will take a little while to run as it obtains the required dependencies using npm, and builds node-sass.

The output of this command should finish with something like:

Running "connect:livereload" (connect) task
Started connect web server on http://localhost:9000

Running "watch" task
Waiting...

At that point the webclient is being served successfully. You can stop it using Ctrl+C or by closing the terminal. Any changes to existing files in the codebase will cause it to automatically rebuild the webclient and refresh the page in your browser, to help streamline the development workflow.

You can view it in a browser at http://localhost:9000/. You should also be able to log in here. The provided configuration file uses Ubuntu One as the OpenID provider, so you’ll need an Ubuntu One account to do so.

Optional: Enable notifications

Notifications in StoryBoard are handled by workers which subscribe to events on a message queue. Currently only RabbitMQ is supported.

1. Install rabbitmq on your development machine

The first step to enable notifications is to get RabbitMQ installed on your development instance.

sudo apt install rabbitmq-server

2. Set up rabbitmq

Create a rabbitmq user/password for StoryBoard (more information can be found in the rabbitmq manpages):

#                         (username) (password)
sudo rabbitmqctl add_user storyboard storyboard
sudo rabbitmqctl set_permissions -p / storyboard ".*" ".*" ".*"

3. Configure notifications

StoryBoard needs various keys set in its configuration file to know how to use RabbitMQ, and the notifications functionality needs to be explicitly enabled.

Set the following configuration in your storyboard.conf file, replacing the userid and password as needed if you used something different in the previous step:

[DEFAULT]
enable_notifications = True

[notifications]
rabbit_host=127.0.0.1
rabbit_login_method = AMQPLAIN
rabbit_userid = storyboard
rabbit_password = storyboard
rabbit_port = 5672
rabbit_virtual_host = /

4. Restart the API

If the API is running, restart it now to pick up the configuration changes. Use Ctrl+C to stop the existing process, and again use tox in the root of the storyboard repository:

tox -e venv -- storyboard-api --config-file ./etc/storyboard.conf

5. Run the workers

To run the workers so that notifications are actually created, use tox in the root of the storyboard repository:

tox -e venv -- storyboard-worker-daemon --config-file ./etc/storyboard.conf

This will start 5 workers to listen for events and create any relevant notifications.

Using your development StoryBoard

Once the API and the webclient development server are running, you can use your development instance of StoryBoard in a few ways.

By default, the webclient development server uses port 9000, and so can be accessed by navigating to http://localhost:9000/ in a web browser running on the same machine.

If your browser is on a different machine, the hostname or IP address of the machine running the API will need to be in the valid_oauth_clients key of ./etc/storyboard.conf for the API in order to log in. It can be a regular expression as well if started with a ^ character.

By default, the API server uses port 8080, and so the API can be accessed at http://localhost:8080/. That will produce a 404 as the API doesn’t actually serve anything on the / endpoint, but that counter-intuitively means its probably working. The API endpoints that are available are documented on the V1 Web API page.

The webclient server also forwards /api to the API server, so it is also possible to use the API by sending requests to http://localhost:9000/api/.

The first user who logs in to the StoryBoard instance will be made a superuser with full admin privileges, to allow easy configuration of the development instance.

Optional steps: Seed database with base data

  1. If you want to define superusers in the database, copy ./etc/superusers.yaml.sample to ./etc/superusers.yaml and define a few superuser IDs.

  2. Enable the superusers in the database:

    tox -e venv -- storyboard-db-manage --config-file ./etc/storyboard.conf load_superusers ./etc/superusers.yaml

  3. If you want to quickly set up a set of projects and project groups in the database, copy ./etc/projects.yaml.sample to ./etc/projects.yaml and define a few projects and project groups.

  4. Create the projects and projectgroups in the DB:

    tox -e venv -- storyboard-db-manage --config-file ./etc/storyboard.conf load_projects ./etc/projects.yaml