Should we add $\alpha$ and $\beta$ to Op interfaces?

[ravil-mobile]

Hi all,

As far as I understood, we have the GEMM interface defined as $C = A \cdot B$ plus some attributes. There is an ambiguous situation arising from the Split-K. When Split-K is used, one need to use atomicAdd as the store method. In general, it demands zero-initialization of matrix $C$. However, we also convert Conv operations to GEMMs and, sometimes (e.g., weights updates during the back propagation), the content of matrix $C$ must be preserved.

We can solve this problem by adding some additional (probably optional) Boolean attribute which the user should provide to tell us whether the original content of $C$ must be re-written or taken into account.

However, this problem can be solved generically if we stick to the BLAS interface - i.e., $C = \alpha A \cdot B + \beta C$. We can probably do the same with conv. ops. - i.e., $Out = \alpha ( Img \otimes Kernel) + \beta ( Out )$.

In our case, we can demand that $\alpha$ and $\beta$ must be compile-time constants.

What do you think?

[krzysz00]

In the interface we've historically exposed, there isn't a kernel that preserves the existing weights? So that'd be a new generalization?

That being said, I'd be open to making $\alpha$ and $\beta$ a thing - I think I had an ancient ticket about it.

I agree with you that restriction to a compile-time scalar constant (so an attribute) makes sense.

As for implementation, I see the following cases

• $\alpha \neq 1$ is an output fusion with $\beta = 0$ and probably works best as an input fusion otherwise
• $\beta = 0$ means we have to zero-initialize for split-K
• $\beta = 1, \alpha = 1$ is generally an atomic_add kernel
• But, when we don't want atomic_add (or when $\beta \not\in {0, 1}$), we can implement it by replacing the logic that starts our matrix C accumulators with 0s with loads from matrix C. That should be reasonably simple to add to GridwiseGemmToBlockwise

Or, in other words, this is a reasonable generalization we could do.

However, on the interface note:
Back in the MIOpen days, the API was that we'd perform $C \leftarrow AB$. However, we had the right to send back multiple kernels, so we could implement $C = AB$ with $C \leftarrow 0$ followed by $C \leftarrow AB + C$

For the current case, what we want - since we can't send back a zero-init kernel - is to tell MIGraphX we want that initilalization step.