Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] add dlpack support to to_table #2275

Open
wants to merge 84 commits into
base: main
Choose a base branch
from

Conversation

icfaust
Copy link
Contributor

@icfaust icfaust commented Jan 27, 2025

Description

This PR introduces __dlpack__ tensor (https://github.com/dmlc/dlpack) consumption by to_table allowing for zero-copy use of data in oneDAL. This is important for enabling array_api support and is a pre-requisite for #2096 (array api dispatching). That PR is then a pre-requisite for #2100 #2106 #2189 #2206 #2207 and #2209. Sklearn provides array_api support for some algorithms. If we wish to fully support zero copy of sycl_usm inputs, we need to be able to consume array_api inputs due to underlying sklearn dependencies (validate_data, check_array, etc.). While we support Sycl usm ndarrays (dpctl, dpnp) via the __sycl_usm_array_interface__ method in the onedal folder estimators, to properly interface estimators in the sklearnex folder, we need to support the __dlpack__ method of arrays/tensors. This PR does that and greatly simplifies the necessary logic in #2096 and the follow-up PRs. This PR also provides the added benefit of working with other frameworks which support SYCL gpu data which have __dlpack__ interfaces (i.e. PyTorch).

NOTES:

TODO: add a onedal function which checks a dlpack tensor for C-contiguity or F-contiguity similar to the flags attribute of numpy/dpctl/dpnp. This is out of the scope of this PR, but is necessary for assert_all_finite support for the next step in array_api work.


PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).

Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

Performance

  • I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
  • I have provided justification why performance has changed or why changes are not expected.
  • I have provided justification why quality metrics have changed or why changes are not expected.
  • I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

@icfaust
Copy link
Contributor Author

icfaust commented Jan 31, 2025

Something weird is happening to codecov with respect to seeing the new files. Will need to be investigated at some point

@icfaust
Copy link
Contributor Author

icfaust commented Jan 31, 2025

/intelci: run

@icfaust icfaust changed the title [WIP, enhancement] add dlpack support to to_table [enhancement] add dlpack support to to_table Jan 31, 2025
@icfaust icfaust marked this pull request as ready for review January 31, 2025 09:01
@@ -0,0 +1,332 @@
/*!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this file be added through a build dependency instead of being committed into the repository?

Otherwise, could it be linked from a git module of the repository where it came from?

Copy link
Contributor Author

@icfaust icfaust Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@icfaust
Copy link
Contributor Author

icfaust commented Jan 31, 2025

/intelci: run

@icfaust
Copy link
Contributor Author

icfaust commented Feb 2, 2025

/intelci: run

icfaust added a commit to icfaust/scikit-learn-intelex that referenced this pull request Feb 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants