Skip to content
Martin Kutlak edited this page Mar 21, 2020 · 3 revisions

Installation guide for ABRT Analytics

Quick links


How to install ABRT Analytics

Using ansible

Preferred way to install and manage FAF instances is using ansible recipes available at https://github.com/abrt/abrt-ansible repository.

Manual installation

This guide will walk you through the installation of FAF assuming you need no special tweaks and you do not need to change default paths.
The guide will focus on Fedora operating system as it is the only one supported at the moment. Every command will also be prepended with sudo -u faf so that it is clear that they should be executed with both user and group faf. It may work with other credentials as well, but executing as faf will prevent future permisson problems.

  1. Set up a database

FAF requires a working relational database such as PostgreSQL. If not already done, set up a database for FAF and install the appropriate python connector package (e.g. python3-psycopg2).

  1. Install FAF

The preferred way of installing FAF is from packages. At the moment only RPM is supported. There are pre-built packages available for Fedora 21 and EPEL 7 (Contains a few custom packages that override the ones in EPEL 7). These were tested on the appropriate systems.
Note: At the moment you probably want to install all FAF parts, so use

# dnf install "faf-*"

Other systems may also work, but have not been tested. These can build the packages directly from the git repository:

$ git clone git://github.com/abrt/faf
cd faf
git checkout new
tito build --rpm --test

or to install:

$ tito install
  1. Configure FAF

Open /etc/faf/faf.conf and edit options starting with DB within Storage section to reflect your actual database connection details.

  1. Initialize FAF

First create the database schema:

$ sudo -u faf faf-migrate-db --create-all
$ sudo -u faf faf-migrate-db --stamp-only

The majority of FAF is controlled by the script faf followed by the name of action. The faf script should always be executed with both user and group faf.
The action init is designed to fill the database tables by pre-defined constants and enable all installed plugins. Become faf user and execute faf init command. One of the ways to do this is to use sudo command:

$ sudo -u faf faf init
  1. Add repository definitions

In order to process reports, FAF needs to have the packages downloaded in its storage. FAF gets these packages from standard package repositories. You need to define the repositories that FAF should use. The repoadd and repoimport actions are handling this.

$ sudo -u faf faf repoadd f31 dnf http://dl.fedoraproject.org/pub/fedora/linux/releases/31/Everything/x86_64/os/

You can use $releasever and $basearch variables in the repository definition. To make these expand correctly, you also need to assign your repository with an operating system and architecture with the repoassign action.

$ sudo -u faf faf repoadd fedora_repo dnf http://dl.fedoraproject.org/pub/fedora/linux/releases/\$releasever/Everything/\$basearch/os/
$ sudo -u faf faf repoassign fedora_repo "Fedora 31" x86_64
  1. Pull data from remote services

To be able to receive uReports, you need to fill the database with operating system metadata - releases and components. These are mandatory parts of uReport that must match against the data in storage.

First you need to download operating system releases. There is pull-releases action which serves this purpose:

$ sudo -u faf faf pull-releases -o fedora

Then you need to download components for the releases. This is handled by pull-components action. You can specify the operating system with -o and the release with --opsys-release. If none of these are included, the script will iterate through all available operating systems and releases and download the components for each of them.

$ sudo -u faf faf pull-components -o fedora --opsys-release 31 --opsys-release 32 --opsys-release rawhide

You can also define custom components:

  • Add them manually with faf compadd. This allows you to have a whitelist of "supported" components. uReports reported against an unsupported component will not be processed.
  • Set CreateComponents = True in the uReport section of /etc/faf/faf.conf. This will automatically create a new component in the storage whenever it is missing. This should however only be used in a safe environment, as spoofed uReports may easily fill the storage with fake components.

Next you need to download packages from previously defined repositories using the reposync action. This may take quite a while depending on the size of repositories.

$ sudo -u faf faf reposync
  1. Ready to save reports

Now the system is ready for saving reports. Incoming uReport files placed at /var/spool/faf/reports/incoming can be saved by calling save-reports action.

$ sudo -u faf faf save-reports

This iterates through all the incoming reports. If the report is valid it is saved into storage and the file is moved to /var/spool/faf/reports/saved. Otherwise the file is moved to /var/spool/faf/reports/deferred.

  1. WebUI

If you have installed the webUI, Apache configs should be placed to the correct directory and you only need to (re)start Apache, which will start the webUI at http://localhost/faf/.

  1. Set up cron jobs

As the system is quite dynamic, the synchronization with external services should be executed regularly. Server administrators are expected to put these command into faf user's crontab, which can result into something like:

*/5 * * * * faf save-reports >> /var/log/faf/save-reports.log 2>&1

55 0 * * * faf pull-releases -o fedora >> /var/log/faf/pull-releases.log 2>&1
55 2 * * * faf pull-components -o fedora >> /var/log/faf/pull-components.log 2>&1

5 3 * * * faf reposync >> /var/log/faf/reposync.log 2>&1

Useful actions

The following actions should be set to cron to be called on a regular basis:

  • faf save-reports (every few minutes) - reports accepted via WebUI are not automatically processed. They are just stored to be batch-processed by this action.
  • faf create-problems (a few times a day) - combines similar reports into problems, generates "Hot Problems" and "Long-Term Problems".
  • faf reposync (once a day) - if you need to synchronize repositories for retracing.
  • faf retrace (once a day) - to apply the actual retracing if required. Caution, this can be quite - memory and CPU consuming if there are enough (100,000+) reports. faf find-crashfn (once a day) - after retracing the crash function is sometimes not obvious and not updated automatically, this actions tries harder to find it. It is also able to fix the "unknown function" fields in WebUI.
  • faf archive-reports (once a day) - packs reports into a tarball so that they 1) get compressed and do not occupy too much space, 2) do not waste inodes (every single report is saved as new file when accepted).
  • faf pull-releases, faf pull-components, faf pull-associates (every few days) - to automatically pull the given operating system metadata. This is only implemented for Fedora at the moment.

How to setup database for ABRT Analytics

This guide will show you how to set up a database for FAF from scratch if you are not familiar with any of the database engines. If you need an advanced database setup, you most probably know how to do it. The only requirements on FAF side are a read-write database and an appropriate RFC1738-formatted URL in /etc/faf/faf.conf (Storage section, ConnectString option).

If you are using a database engine not mentioned here and you would like to add a simplistic setup guide, feel free to ping us so that we can add it.

Only Fedora 21 and RHEL/CentOS/Scientific 7 were tested. You may need to use equivalent alternatives when using a different system.

PostgreSQL

PostgreSQL is the preferred (and best tested) database engine. You need to install postgresql-server, postgresql, python3-psycopg2 and pg-semver packages:

# dnf install postgresql-server postgresql python3-psycopg2 pg-semver

Next you need to initialize the postgresql database:

# postgresql-setup --initdb --unit postgresql

Then start (and enable) the service:

# systemctl enable --now postgresql

When the database enging is running, you need to create a user and an actual database for FAF. Become postgres user and use the command line utility psql to do that. Type in the following commands:

CREATE USER faf;
CREATE DATABASE faf;
GRANT ALL PRIVILEGES ON DATABASE faf TO faf;

As the faf user in the faf database

CREATE EXTENSION semver;

You might need to make the faf user superuser:

ALTER USER faf WITH superuser;

If everything works you should see something like this:

# sudo -u postgres psql
psql (9.2.4)
Type "help" for help.

postgres=# CREATE USER faf;
CREATE ROLE
postgres=# CREATE DATABASE faf;
CREATE DATABASE
postgres=# GRANT ALL PRIVILEGES ON DATABASE faf TO faf;
GRANT
postgres=# \q

# sudo -u faf psql
psql (9.2.4)
Type "help" for help.

postgres=# CREATE EXTENSION semver;
CREATE EXTENSION
postgres=# \q

As PostgreSQL defaults to ident authentication, which will map unix user names into PostgreSQL user names, unix user faf has now all required privileges on faf database. The last thing you need to do is update the /etc/faf/faf.conf file. Under Storage section, there is ConnectString option, which in this case should equal to

postgresql:///faf

This is the default, so if you did not touch the default config at all, there is a high chance it will work without touching /etc/faf/faf.conf.

Other database engines

MySQL / MariaDB

Despite their high popularity, FAF is not able to run on neither MySQL nor MariaDB at the moment.
The first reason is the lack of a semantic versioning extension. Also, there are several engine-specific limitations that FAF does not respect (64K maximum row size, 768B maximum primary key size, and probably more of them). Although we would like to add the support, the issue has a quite low priority. Anybody interested in porting FAF to MySQL / MariaDB is welcome.

SQLite

SQLite is not supported due to the lack of a semantic versioning extension.