Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ORM: Use skip_orm as the default implementation for SqlaGroup.add_nodes and SqlaGroup.remove_nodes #6720

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

rabbull
Copy link
Contributor

@rabbull rabbull commented Jan 21, 2025

This PR removes the skip_orm flag and adopts the optimized approach behind it as the default behavior for the add_nodes and remove_nodes methods.

As discussed in #5453, the ORM-based implementations of add_nodes and remove_nodes were found to be inefficient, and to address this, the skip_orm flag was introduced in b40b2ca and bced84e. This flag enabled a faster, core API-based approach for these operations.

Over the years, no users have reported bugs related to this method, and the existing tests are sufficiently comprehensive to ensure its safety. Consequently, it is a reasonable decision to make the skip_orm implementation the default for bulk insertion and removal of elements within a group.

Closes #5453.

@rabbull
Copy link
Contributor Author

rabbull commented Jan 21, 2025

The existing tests related are:

    def test_add_nodes(self):
        """Test different ways of adding nodes."""
        node_01 = orm.Data().store()
        node_02 = orm.Data().store()
        node_03 = orm.Data().store()
        nodes = [node_01, node_02, node_03]

        group = orm.Group(label='test_adding_nodes').store()
        # Single node
        group.add_nodes(node_01)
        # List of nodes
        group.add_nodes([node_02, node_03])

        # Check
        assert set(_.pk for _ in nodes) == set(_.pk for _ in group.nodes)

        # Try to add a node that is already present: there should be no problem
        group.add_nodes(node_01)
        assert set(_.pk for _ in nodes) == set(_.pk for _ in group.nodes)

    def test_remove_nodes(self):
        """Test node removal."""
        node_01 = orm.Data().store()
        node_02 = orm.Data().store()
        node_03 = orm.Data().store()
        node_04 = orm.Data().store()
        nodes = [node_01, node_02, node_03]
        group = orm.Group(label=uuid.uuid4().hex).store()

        # Add initial nodes
        group.add_nodes(nodes)
        assert set(_.pk for _ in nodes) == set(_.pk for _ in group.nodes)

        # Remove a node that is not in the group: nothing should happen
        group.remove_nodes(node_04)
        assert set(_.pk for _ in nodes) == set(_.pk for _ in group.nodes)

        # Remove one orm.Node
        nodes.remove(node_03)
        group.remove_nodes(node_03)
        assert set(_.pk for _ in nodes) == set(_.pk for _ in group.nodes)

        # Remove a list of Nodes and check
        nodes.remove(node_01)
        nodes.remove(node_02)
        group.remove_nodes([node_01, node_02])
        assert set(_.pk for _ in nodes) == set(_.pk for _ in group.nodes)

This looks sufficient to me.

Copy link

codecov bot commented Jan 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.05%. Comparing base (aa0aa26) to head (46085a5).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6720      +/-   ##
==========================================
- Coverage   78.05%   78.05%   -0.00%     
==========================================
  Files         563      563              
  Lines       41790    41770      -20     
==========================================
- Hits        32616    32598      -18     
+ Misses       9174     9172       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rabbull rabbull marked this pull request as ready for review January 21, 2025 23:28
@rabbull
Copy link
Contributor Author

rabbull commented Jan 21, 2025

A test case related to cmd tools found that skip_orm doesn't support adding nothing very well, so a few more test cases are added and the bug is fixed.

@rabbull rabbull requested a review from GeigerJ2 January 21, 2025 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use/remove SqlaGroup.add_nodes/remove_nodess skip_orm flag?
1 participant