[pull] master from tensorflow:master #169
Open
+223,421
−127,255
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
PiperOrigin-RevId: 723561813
… instructions PiperOrigin-RevId: 723565430
…` from TSL TSL is no longer independently buildable, so these files have no meaning PiperOrigin-RevId: 723568791
…ing operand or result shapes / returning capsule descriptor. PiperOrigin-RevId: 723571011
PiperOrigin-RevId: 723578794
This file has no meaning now that TSL is no longer independently buildable PiperOrigin-RevId: 723595198
PiperOrigin-RevId: 723596060
python3.13-nogil is a free-threaded build of python3.13. PiperOrigin-RevId: 723604422
…ilename and --xla-hlo-enable-passes-only). PiperOrigin-RevId: 723606669
…collective-permute into send/recv This is to ensure that conflicting collectives are not scheduled in between the decomposed send/recv ops, which would cause deadlocks. PiperOrigin-RevId: 723614014
…actored in) to MSA. PiperOrigin-RevId: 723618095
…y.dtypes.StringDType`) in JAX. Currently this only supports making of a string array by means of either `jax.numpy.asarray` or `jax.device_put` and reading it back with `jax.device_get`. PiperOrigin-RevId: 723620391
Updates LLVM usage to match [f8287f6c373f](llvm/llvm-project@f8287f6c373f) PiperOrigin-RevId: 723639584
…y default to `"PY3"` PiperOrigin-RevId: 723643353
PiperOrigin-RevId: 723647280
The include paths for headers within the ml_dtypes package have changed. We therefore need to adjust the TF/XLA build rules and paths to account for this. Also updated the pip ml_dtypes version to match. The main ml_dtypes repo name needed to be changed to avoid conflicts with the ml_dtypes subfolder. The subfolder contains the main python package that needs to be added to the PYTHON_PATH. PiperOrigin-RevId: 723654395
PiperOrigin-RevId: 723657238
non-unit slice sizes. PiperOrigin-RevId: 723669805
The c-api support will be added in: openxla/xla#22272 . PiperOrigin-RevId: 723682207
PiperOrigin-RevId: 723683662
…fer (tied to an HloValue). Our best log that prints the buffer is "Creating AllocationValues for"; however, we don't always print that line. So, 2 changes were made. 1) We indicate with VLOG(3) when we start processing a new buffer, and we print that buffer. 2) Since, we're print the buffer in 1, we no longer need to print it when printing "Creating AllocationValues for" PiperOrigin-RevId: 723685932
(ifrt|pjrt)::Future already have correct move semantics and act as a move-only value if payload type is move-only. PiperOrigin-RevId: 723686554
Reverts 6ae9c29 PiperOrigin-RevId: 723688407
…ray shapes. Prior to this change, `HloRunnerPjRt` would fail during execution of `FfiCustomCallTest.Tokens` (on CPU) because the `GenerateExecutionOptions` function called `ComputationLayout::FlattenedResultLayouts`. This function returns an error if any of the subshapes of the tuple are not arrays. While it is possible to adapt the implementation of this particular function to return only array shapes and skip over the non-array shapes, it would also require changes in how the output buffers are allocated in the CPU compiler. PiperOrigin-RevId: 723691850
PiperOrigin-RevId: 723696496
PiperOrigin-RevId: 723698246
PiperOrigin-RevId: 723704575
IFRT is moving towards runtime-controlled device list creation. This CL moves most of explicit device list creation from `BasicDeviceLIst::Create()` to `Client::MakeDeviceList()`. Once the migration is done, `BasicDeviceList::Create()` will be reserved only for IFRT implementations and all IFRT users will be expected to use `Client::MakeDeviceList::Create()` to create device lists. PiperOrigin-RevId: 725356476
…-ext-flatten-tuple` pass. ODS is generating it anyway. PiperOrigin-RevId: 725357437
…y default to `"PY3"` PiperOrigin-RevId: 725360453
…ory_fitting_level options when available. We temporarily continue to respect the legacy exec_time_optimization_effort option if the new one is not specified. This will be phased out once all users have been migrated away from the old option. PiperOrigin-RevId: 725367749
PiperOrigin-RevId: 725373453
… a complete list of resources used by instructions in a while loop. This will make async `done` and while ops have similar priority (in terms of occupying resource types) and avoid delaying the while loops only because they cross the overlap limit (even though they have a higher async depth). This CL also fixes the double counting of a resource in `GetNumResourcesPerInstruction` because of multiple async `done` ops in the while body. PiperOrigin-RevId: 725375955
PiperOrigin-RevId: 725390834
PiperOrigin-RevId: 725399735
PiperOrigin-RevId: 725401102
It started failing with openxla/xla@b39e89e. PiperOrigin-RevId: 725405400
Updates LLVM usage to match [1c583c19bb79](llvm/llvm-project@1c583c19bb79) PiperOrigin-RevId: 725406973
The method was effectively unused, since it wasn't overridden by SequentialThunk, and so SequentialThunk wouldn't call Cleanup on its subthunks. NcclRaggedAllToAllStartThunk overrode Cleanup to free some device buffers, but these were never freed since Cleanup was not called. The memory is now stored in DeviceMemoryHandles, which automatically free the buffers in the destructor. PiperOrigin-RevId: 725409574
… weights buffer stuff. PiperOrigin-RevId: 725453264
PiperOrigin-RevId: 725457151
PiperOrigin-RevId: 725458294
PiperOrigin-RevId: 725473466
PiperOrigin-RevId: 725477920
IFRT is moving towards runtime-controlled device list creation. This CL moves most of explicit device list creation from `BasicDeviceLIst::Create()` to `Client::MakeDeviceList()`. Once the migration is done, `BasicDeviceList::Create()` will be reserved only for IFRT implementations and all IFRT users will be expected to use `Client::MakeDeviceList::Create()` to create device lists. PiperOrigin-RevId: 725479063
PiperOrigin-RevId: 725509207
…e manager. PiperOrigin-RevId: 725511326
…anager. This will substantially decrease the number of copies during partition step. PiperOrigin-RevId: 725513804
PiperOrigin-RevId: 725520770
PiperOrigin-RevId: 725520771
PiperOrigin-RevId: 725521622
It seems useful if we have helper methods to evaluate offset, stride and size of a symbolic tile even without a SymbolicTiledHloInstruction. Therefore also create the methods EvaluateTileOffsets(), EvaluateTileStrides() and EvaluateTileSizes() and migrate callers of the old helper functions to the new ones. PiperOrigin-RevId: 725564198
…nd config. Imported from GitHub PR openxla/xla#22376 Collective backend config is expected to be under a key `"collective_backend_config"` under `backend_config`. In many testcases, this was not the case. This was not flagged by the testcases because the function `IsGPUSyncCollective` supressed any errors in the collective config and assumed `is_sync` to be `false` in those cases (which is generally the case we would like to test with collectives). Hence this went unnoticed so far. We have also stopped supressing the error in that function now. Copybara import of the project: -- 1ebd7c8a2859ff9ff91cf65b334355e1f213681b by Shraiysh Vaishay <[email protected]>: Fix erroneous HLO in testcases related to collective backend config. Collective backend config is expected to be under a key `"collective_backend_config"` under `backend_config`. In many testcases, this was not the case. This was not flagged by the testcases because the function `IsGPUSyncCollective` supressed any errors in the collective config and assumed `is_sync` to be `false` in those cases (which is generally the case we would like to test with collectives). Hence this went unnoticed so far. We have also stopped supressing the error in that function now. -- 1151e05ed74f57722d2599a837520773f46a7b77 by Shraiysh Vaishay <[email protected]>: Addressed comments Merging this change closes #22376 PiperOrigin-RevId: 725564481
PiperOrigin-RevId: 725565602
PiperOrigin-RevId: 725572380
PiperOrigin-RevId: 725584376
…pu:determinism_test` test. PiperOrigin-RevId: 725591796
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.1)
Can you help keep this open source service alive? 💖 Please sponsor : )