Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nightly Build Failure 2024-09-21 #178

Open
github-actions bot opened this issue Sep 21, 2024 · 141 comments · May be fixed by #223
Open

Nightly Build Failure 2024-09-21 #178

github-actions bot opened this issue Sep 21, 2024 · 141 comments · May be fixed by #223
Labels
nightly-build-failure Failure during nightly testing of main-main

Comments

@github-actions
Copy link

cc @microsoft/aifx-compilers
The nightly build has failed. See: https://github.com/microsoft/triton-shared/actions/runs/10971860915

@github-actions github-actions bot added the nightly-build-failure Failure during nightly testing of main-main label Sep 21, 2024
Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/10980406267

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/10992497052

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11011978501

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11031248472

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11050462552

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11069050269

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11083218385

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11091917035

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11104325608

Copy link
Author

github-actions bot commented Oct 1, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11123545175

Copy link
Author

github-actions bot commented Oct 2, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11142210626

Copy link
Author

github-actions bot commented Oct 3, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11160347600

Copy link
Author

github-actions bot commented Oct 4, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11178383563

Copy link
Author

github-actions bot commented Oct 5, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11192523575

Copy link
Author

github-actions bot commented Oct 6, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11201108289

Copy link
Author

github-actions bot commented Oct 7, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11213687750

Copy link
Author

github-actions bot commented Oct 8, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11233614009

Copy link
Author

github-actions bot commented Oct 9, 2024

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11253175524

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11272276180

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11290789094

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11304945070

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11313619058

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11325757696

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11344267877

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11363864630

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11383183824

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11402089868

@nhat-nguyen
Copy link
Collaborator

We're prioritizing some feature work, so updating triton-shared to work with the latest triton will be less of a priority.

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/11416836116

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12745331664

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12765784840

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12786415610

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12806980456

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12827315570

zhaoshiz added a commit to zhaoshiz/triton-shared that referenced this issue Jan 17, 2025
Fixing below compilation errors in include/triton-shared/Conversion/TritonArithToLinalg/ConversionPatterns.hpp:

  /home/runner/work/triton-shared/triton-shared/triton_shared/include/triton-shared/Conversion/TritonArithToLinalg/ConversionPatterns.hpp:839:68: error: no member named 'getFile' in 'mlir::triton::AssertOp'
          llvm::formatv("{0}.py:{1}: {2} Assertion `{3}` failed", op.getFile(),
                                                                  ~~ ^
  /home/runner/work/triton-shared/triton-shared/triton_shared/include/triton-shared/Conversion/TritonArithToLinalg/ConversionPatterns.hpp:840:26: error: no member named 'getLine' in 'mlir::triton::AssertOp'
                        op.getLine(), op.getFunc(), op.getMessage());
                        ~~ ^
  /home/runner/work/triton-shared/triton-shared/triton_shared/include/triton-shared/Conversion/TritonArithToLinalg/ConversionPatterns.hpp:840:40: error: no member named 'getFunc' in 'mlir::triton::AssertOp'
                        op.getLine(), op.getFunc(), op.getMessage());
                                      ~~ ^
  3 errors generated.

This fix builds with triton @ab07e5472bcb414a0c8dd7ecab80f84370c4894e,
and llvm @cfd3289a1f9a87e220737a634904a886a82d424a.
zhaoshiz added a commit to zhaoshiz/triton-shared that referenced this issue Jan 17, 2025
Fix compilation errors caused by LLVM commits:
f18c3e4e7335df282c468b6dff3d29be1822a96d [mlir][Transforms] Dialect Conversion: Simplify materialization fn result type (#113031)
8c4bc1e75de27adfbaead34b895b0efbaf17bd02 [mlir][Transforms] Merge 1:1 and 1:N type converters (#113032)

Update Triton to 94684d326723b67b146f23f342623ea058a32098
Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12843230995

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12852718609

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12866061451

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12885039127

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12906268385

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12927284670

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12947877749

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12964163456

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12973809579

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/12986995597

nhat-nguyen pushed a commit that referenced this issue Jan 27, 2025
Fixing below compilation errors in
include/triton-shared/Conversion/TritonArithToLinalg/ConversionPatterns.hpp:


/home/runner/work/triton-shared/triton-shared/triton_shared/include/triton-shared/Conversion/TritonArithToLinalg/ConversionPatterns.hpp:839:68:
error: no member named 'getFile' in 'mlir::triton::AssertOp'
llvm::formatv("{0}.py:{1}: {2} Assertion `{3}` failed", op.getFile(),
                                                                  ~~ ^

/home/runner/work/triton-shared/triton-shared/triton_shared/include/triton-shared/Conversion/TritonArithToLinalg/ConversionPatterns.hpp:840:26:
error: no member named 'getLine' in 'mlir::triton::AssertOp'
                        op.getLine(), op.getFunc(), op.getMessage());
                        ~~ ^

/home/runner/work/triton-shared/triton-shared/triton_shared/include/triton-shared/Conversion/TritonArithToLinalg/ConversionPatterns.hpp:840:40:
error: no member named 'getFunc' in 'mlir::triton::AssertOp'
                        op.getLine(), op.getFunc(), op.getMessage());
                                      ~~ ^
  3 errors generated.

This fix builds with triton @ab07e5472bcb414a0c8dd7ecab80f84370c4894e,
and llvm @cfd3289a1f9a87e220737a634904a886a82d424a.

---------

Co-authored-by: Zhaoshi Zheng <[email protected]>
Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/13008487182

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/13029373762

@zhaoshiz
Copy link
Contributor

cc @microsoft/aifx-compilers The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/13029373762

seems the nightly build is checkout Triton main after git submodule, pushed a fix: #223

@parsifal-47
Copy link
Contributor

seems the nightly build is checkout Triton main after git submodule, pushed a fix: #223

I was thinking this is on purpose

@nhat-nguyen
Copy link
Collaborator

@zhaoshiz Thanks for your PR. This is on purpose though -- it lets us know that we need to fix triton-shared to work with triton upstream. Basically, every night, we would run triton-shared with the latest triton. If it fails, an issue is created. From the newest logs, it looks like upstream triton just updated llvm which renamed some methods.

@zhaoshiz
Copy link
Contributor

@zhaoshiz Thanks for your PR. This is on purpose though -- it lets us know that we need to fix triton-shared to work with triton upstream. Basically, every night, we would run triton-shared with the latest triton. If it fails, an issue is created. From the newest logs, it looks like upstream triton just updated llvm which renamed some methods.

I see, thanks for clarifying that. I tried building with latest Triton locally, there's more than just LLVM API change needed. I got a link error about TritonGPUTC05MMAPipeline (https://github.com/triton-lang/triton/blob/1186806d96be85e1d95d2535319f5e257024490a/include/triton/Dialect/TritonGPU/Transforms/Passes.td#L26) and a bunch of test failures from test_core.py.

I'll try to resolve them.

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/13050420552

@zhaoshiz
Copy link
Contributor

I'll try to resolve them.

I managed to figure out and updated #223

Copy link
Author

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/13070964898

Copy link
Author

github-actions bot commented Feb 1, 2025

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/13087759547

Copy link
Author

github-actions bot commented Feb 2, 2025

cc @microsoft/aifx-compilers
The nightly build has failed again. See: https://github.com/microsoft/triton-shared/actions/runs/13097955492

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
nightly-build-failure Failure during nightly testing of main-main
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants