Migration

FAF 0 to FAF 1

An instance of FAF 0 (old FAF) is running and accepting uReports. The goal is to replace FAF 0 by FAF 1 (new FAF) so that the transition is as transparent as possible for all the end users.
Even though FAF 0 and FAF 1 share the majority of storage backend, there are a few differences that need to be considered:

• FAF 1 saves more information than FAF 0.
• FAF 1 internally operates on uReport2. As there are many clients still sending uReport1, a mechanism converting between versions is implemented, however uReport1 usually contains less information and thus some properties/values may be lost.
• FAF 1 does not implement all the functionality that FAF 0 has. There are many dead tools in FAF 0 (either broken or not used for a long time). These were not ported and probably will never be - unless somebody shows interest.
• FAF 1 is not able to interact with Koji in any way. FAF 0 uses Koji as a source of builds and packages. This functionality has been intentionally retired and FAF 1 uses YUM repositories instead.

Plan

Stage 1 - Deploy a standalone FAF 1 instance

• Deploy FAF 1 from scratch on a standalone host.
• Define operating systems, repositories, problem types etc.
• Run the initial synchronization - releases, components, packages.
• Set up cron.

At the end of Stage 1, the FAF 1 instance is ready to accept reports from the latest packages. Both FAF 0 and FAF 1 run in parallel, FAF 0 still being the master and having more data (history) than FAF 0. At this point if FAF 0 dies, the history is lost, but all of the incoming reports can be handled properly.

Stage 2 - Backport the data from FAF 0 to FAF 1

• Package data and metadata - Two separate problems. All of the packages ever seen have metadata in the DB, but some of them do not have the raw package files associated. This is mostly because of lack of storage.
• Case 1 - The package file is available
  Create a repository containing all these files and run the standard mechanism to import all of them to new storage. This may take a while, but it is quite simple.
• Case 2 - The package file is not available
  Two possibilities - create a migration script that will generate an appropriate SQL query or try to find the package files somewhere in koji/mirrors/wherever and to the same as above. Each of them has its pros and cons, undecided yet.
• Reports - All reports are archived, just push all of them through the standard mechanism.
• Attachments - Tricky. BTHASH is computed differently, the old algorithm needs to be used when porting old reports.
• RHBZ Bugs - No need to port at all. All required bugs will be downloaded first time they are needed.
• Run initial retracing and clustering.

At the end of Stage 2 both FAF 0 and FAF 1 run in parallel, FAF 0 still being the master. Both instances have the same data and are interchangeable.

Stage 3 - Flip the switch

• Update the host to FAF 1, update configs to use the new storage.
• A fallback - switch back to FAF 0 - must be available at any moment.
• Apply hotfixes in real time - ideally this will not be required, but we all know how it works...
• Check for performance issues and bottlenecks. A few optimizations have been lost, need to investigate whether this is critical or not.

At the end of Stage 3 both FAF 0 and FAF 1 run in parallel, FAF 1 being the master. FAF 1 may have more data as some stuff in FAF 0 will probably be disabled due to the lack of storage space.

Stage 4 - Remove FAF 0

• Port configs to new format - Although FAF 1 has some backwards compatibility when reading configs, it is not a good idea to rely on it.
• Clean up FAF 0 garbage - Old storage (there are even the remains of cache), old config files, database, packages, httpd aliases...
• Clean up migration garbage - Temporary repositories, hacks...

At the end of Stage 4 FAF 1 is the only instance running and there are no remains of FAF 0.