Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch away from getMergeInstance for derived source #2472

Merged
merged 1 commit into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -107,12 +107,12 @@ public void close() throws IOException {

/**
* For merging, we need to tell the derived source stored fields reader to skip injecting the source. Otherwise,
* on merge we will end up just writing the source to disk
* on merge we will end up just writing the source to disk. We cant override
* {@link StoredFieldsReader#getMergeInstance()} because it is used elsewhere than just merging.
*
* @return Merged instance that wont inject by default
*/
@Override
public StoredFieldsReader getMergeInstance() {
private StoredFieldsReader cloneForMerge() {
try {
return new DerivedSourceStoredFieldsReader(
delegate.getMergeInstance(),
Expand All @@ -125,4 +125,18 @@ public StoredFieldsReader getMergeInstance() {
throw new RuntimeException(e);
}
}

/**
* For merging, we need to tell the derived source stored fields reader to skip injecting the source. Otherwise,
* on merge we will end up just writing the source to disk
*
* @param storedFieldsReader stored fields reader to wrap
* @return wrapped stored fields reader
*/
public static StoredFieldsReader wrapForMerge(StoredFieldsReader storedFieldsReader) {
if (storedFieldsReader instanceof DerivedSourceStoredFieldsReader) {
return ((DerivedSourceStoredFieldsReader) storedFieldsReader).cloneForMerge();
}
return storedFieldsReader;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ public void writeField(FieldInfo info, DataInput value, int length) throws IOExc

@Override
public int merge(MergeState mergeState) throws IOException {
// We have to wrap these here to avoid storing the vectors during merge
for (int i = 0; i < mergeState.storedFieldsReaders.length; i++) {
navneet1v marked this conversation as resolved.
Show resolved Hide resolved
mergeState.storedFieldsReaders[i] = DerivedSourceStoredFieldsReader.wrapForMerge(mergeState.storedFieldsReaders[i]);
}
return delegate.merge(mergeState);
}

Expand Down
Loading