Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PLT-2568 optimise vertex fetch from janusgraph #3740

Closed
wants to merge 1 commit into from
Closed

PLT-2568 optimise vertex fetch from janusgraph #3740

wants to merge 1 commit into from

Conversation

sriram-atlan
Copy link

@sriram-atlan sriram-atlan commented Nov 13, 2024 •
edited
Loading

Change description

Post elastic fetch optimisation to reduce mapVertexToAtlasEntityHeader number
Janus graph lookup optimisation with batching and memoization
removing recursive code

Type of change

  • Bug fix (fixes an issue)
  • New feature (adds functionality)

Related issues

https://atlanhq.atlassian.net/browse/PLT-2568

Fix #1

Checklists

Development

  • Lint rules pass locally
  • Application changes have been tested thoroughly
  • Automated tests covering modified code pass

Security

  • Security impact of change has been considered
  • Code follows company security practices and guidelines

Code review

  • Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
  • "Ready for review" label attached and reviewers assigned
  • Changes have been reviewed by at least one other contributor
  • Pull request linked to task tracker where applicable

sriram-atlan
sriram-atlan commented Nov 13, 2024
View reviewed changes
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
import static org.apache.atlas.util.AtlasGremlinQueryProvider.AtlasGremlinQuery.BASIC_SEARCH_STATE_FILTER;
import static org.apache.atlas.util.AtlasGremlinQueryProvider.AtlasGremlinQuery.TO_RANGE_LIST;

@Component
public class EntityDiscoveryService implements AtlasDiscoveryService {
private static final Logger LOG = LoggerFactory.getLogger(EntityDiscoveryService.class);
private static final String DEFAULT_SORT_ATTRIBUTE_NAME = "name";
private static final int AVAILABLEPROCESSORS = Runtime.getRuntime().availableProcessors();
private static final ForkJoinPool CUSTOMTHREADPOOL = new ForkJoinPool(AVAILABLEPROCESSORS/2); // Use half of available cores
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

once we test and check that things are intact and this change brings value, we can parameterize and better this change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mukcool mukcool Awaiting requested review from mukcool

@mehtaanshul mehtaanshul Awaiting requested review from mehtaanshul

@sumandas0 sumandas0 Awaiting requested review from sumandas0

@aarshi0301 aarshi0301 Awaiting requested review from aarshi0301

@arniesaha arniesaha Awaiting requested review from arniesaha

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@sriram-atlan