[META] OpenSearch 3.0 Core Release Tracker

[expani]

Please describe the end goal of this project

OpenSearch 3.0 will use Lucene 10 in accordance with its release policy of maintaining semver with every major Lucene version release covered more in detail at #250

The published release schedule #252 commits to 17th Feb 2025 as the first alpha release of 3.0

We have upgraded to Lucene 10.1.0 and started using baseline JDK 21 for OpenSearch 3.0 in this PR

The purpose of this meta is to track all the efforts towards releasing OpenSearch 3.0 ( JDK 21/23 + Lucene 10 ) and the order of milestones looks as follows :

Milestone	Description	Status	Issue
1	Efforts to resolve compilation issues caused by removed APIs in Lucene, new methods to override; etc.	Completed
2	Efforts to resolve unit and integration test failures ( REST Yaml / InternalCluster / QA )	Completed	#16933
3	Depcreated Code Removal and Package Split Completion	Not started	#8110 and #5214
4	Efforts to resolve any performance issues that arise with Lucene 10 Upgrade in core OpenSearch.	Ongoing	#16934
5	Releasing 3.0.0-alpha for plugins to start the same process for themselves to integrate with OS 3.0	Not started
6	Efforts to fix issue in core that arise with plugin integrations after Lucene 10 Upgrade.	Not started

Issues

• [BUG] Fix Unit and Integration tests for Lucene 10 Upgrade #16933
• [BUG] Performance Comparison of OS 3.0 Lucene 10 with Mainline #16934
• [META] Split and modularize the server monolith #8110
• Remove deprecated API that uses non-inclusive terminology #5214

Contribution

If you wish to contribute by fixing a failing test in #16366 please drop a message on the PR with the test you are planning to pick so that others can focus on different one's.

Create a PR against the issue-11415 branch in reta's fork. Link this meta on the PR as well.

If it's a very involved test failure or a change that needs to be on the radar for Performance runs, make sure to create an issue and link with this meta.

We will keep posting messages for help required in contributions with title Help Needed - Testing/Performance

These will be tests or issues that primary owners don't have bandwidth to actively look into for sometime.

Stakeholder Notifications

We will send weekly updates on this meta on the progress towards alpha release candidate readiness of OpenSearch core.

Look out for messages with title ALPHA OS 3.0 RC - CORE READINESS STATUS

This will contain the number of tests fixed, highlight significant changes and capture any callouts for performance runs.

Supporting References

Lucene 10 Update Feature Request

Baseline JDK 21

Breaking Changes in OpenSearch 3.0

Related component

Other

[expani]

ALPHA OS 3.0 RC - CORE READINESS STATUS

Milestone-2 Completed - Unit and Integration tests are passing in OpenSearch with Lucene 10 changes.

Milestone-3 Ongoing

[andrross]

@expani @getsaurabh02 This issue appears to be focused on the Lucene 10 upgrade in core. There are a few other significant things that have been discussed for 3.0 in core, namely resolving top level package split across modules (#8110) and removal of all deprecated code (much of this around inclusive naming, such as #5214). Should we formalize that list of work and track progress here as well?

[expani]

@andrross Was planning to close this once we have OS 3.0 released.
Will the linked feature requests/Meta need to be completed before OS 3.0 release ?

[andrross]

Was planning to close this once we have OS 3.0 released. Will the linked feature requests/Meta need to be completed before OS 3.0 release ?

@expani Yes, breaking changes by definition must go in before the 3.0 release.

[getsaurabh02]

@expani can we update the list above to incorporate the code cleanup (#5214) and renaming that might be needed

[expani]

Thanks for the inputs @andrross
@getsaurabh02 Updated the milestones.

[expani]

From #8110 we will just be covering the items in
Phase 0 - JPMS Support (Eliminate top level split packages, see #1838)