Skip to content

Bitstream update for CAPI cards

Jump to bottom Edit New page
Frank Haverkamp edited this page Nov 10, 2016 · 14 revisions

There is now a new tool for updating CAPI FPGA cards available. Please have a look at the projects webpage to see how to install and operate it:

To update the CAPI version of our compression FPGA card the following tool can be used:

The Nallatech card can store two bitstreams. One (default:0) is loaded after power on. The other one is used to recover problems with the default bitstream (backup: 1). As consequence we recommend updating the default bitstream, reload and check if you get the expected results. If not, something has gone wrong and you can use the backup bitstream to recover the situation. Only update the backup bitstream, if you are absolutely sure that it works.

The currently supported cards are manufactured by Nallatech and use Altera FPGAs. Those require *.rbf files to be used. Depending on which personalization of the card one has, either the plain rbf, the compressed, or the encrypted + compressed rbf must be used.

Here is how it normally looks like:

# ./capi_flash.pl -p1 -f bitstreams/cgzip.603_0.20160316.r844-000_debug_basic_compressed.rbf  -t /sys/bu/pci/devices/0000:01:00.0
Target specified: /sys/bus/pci/devices/0000:01:00.0

CAPI Adapter is : /sys/bus/pci/devices/0000:01:00.0
  Device/Vendor ID: 0x04771014

 Image has no header

  VSEC Length/VSEC Rev/VSEC ID: 0x08001280
    Version 0.12

Programming User Partition with bitstreams/cgzip.603_0.20160316.r844-000_debug_basic_compressed.rbf
  Program -> Address: 0x850000 for Size: 58 in blocks (32K Words or 128K Bytes)

Erasing Flash
.......

Programming Flash
Writing Buffer: 7423        

Verifying Flash
Reading Block: 57        

Erase Time:   46 seconds
Program Time: 19 seconds
Verify Time:  16 seconds
Total Time:   81 seconds

After the bitstream is updated, a reload of the bitstream needs to be triggered:

echo 1 > /sys/class/cxl/card0/reset

This needs to be done as superuser and please ensure that you do not run something important on the system you are using that operation. We saw older CAPI versions having problems with that. Once you tried that out a couple of times and it works, it should be stable.

If for whatever reason this soft-reload is not working, you can powercycle/reboot the system to enforce the reload of the card.

The Linux CAPI subsystem offers several sysfs interfaces. Please review the following documentation for details: cxl driver sysfs interfaces.

Multiple CAPI cards in your system

If you have multiple CAPI cards in your system, it is not obvious at the first glance which one is which:

$ lspci | grep accel
0002:01:00.0 Processing accelerators: IBM Device 0477 (rev 01)
0004:01:00.0 Processing accelerators: IBM Device 0477 (rev 01)

You can try using the psl_revsion sysfs entry to figure out, which card is supporting our compression feature:

$ cat /sys/class/cxl/card0/psl_revision 
12289
$ cat /sys/class/cxl/card1/psl_revision 
1

In this case card1 would be a Nallatech card. Next thing you should check is if you have a CGZIP bitstream on the card. The next section shows how this is to be done.

Query the CAPI bitstream version for CGZIP

To figure out if you have a CGZIP CAPI card you can query the AFU device/vendor ids:

haver@tul2eth3:~$ cat /sys/class/cxl/afu1.0s/device/cr0/device
0x0602

The 0x0602 is unique for the CGZIP CAPI card.

You can use the genwqe_echo command to get the detailed bitstream version information. Use it as follows:

haver@tul2eth3:~$ genwqe_echo -ACAPI -C0 -H
 cr_device:          0x0000000000000602
 cr_vendor:          0x0000000000001014
 cr_class:           0x0000000000120000
 Version Reg:        0x0000465016031600
 Appl. Reg:          0x06030003475a4950
 Afu Config Reg:     0x0000000000020000
 Afu Status Reg:     0x0000000000002400
 Afu Cmd Reg:        0x0000000000000000
 Free Run Timer:     0x00012005a8ca2cd8
 DDCBQ Start Reg:    0x0000000010070000
 DDCBQ Conf Reg:     0xf00d000000030000
 DDCBQ Cmd Reg:      0x0000000000000000
 DDCBQ Stat Reg:     0xf00d000000000000
 DDCBQ Context ID:   0x0000000000000000
 DDCBQ WT Reg:       0x0000000000000000
 FIR Reg [00001000]: 0x0001100000000000
 FIR Reg [00001008]: 0x0001100800000000
 FIR Reg [00001010]: 0x0001101000000000
 FIR Reg [00001018]: 0x0001101800000000
 FIR Reg [00001020]: 0x0001102000000000
 FIR Reg [00001028]: 0x0001102800000000

--- UNIT #1 echo statistics ---
0 packets transmitted, 0 received, 0 lost, 100% packet loss, queue 0 usec

IBM Deutschland Forschung und Entwicklung

Clone this wiki locally