Librilight Preprocess Scripts Revision

[HarryHe11]

✨ Description

This PR aims to enhance the compatibility and efficiency of the librilight preprocessing scripts with the new NS2 framework to be updated by @HeCheng0625. By updating the librilight preprocessing scripts, we ensure they align with the new NS2 dataloader's expectations, facilitating smoother NS2 training processes. Besides, this PR also improves the usability of the original librilight preprocess scripts.

🚧 Related Issues

#115

👨‍💻 Changes Proposed

• Updated librilight preprocess scripts for better alignment with the NS2 dataloader, ensuring compatibility and efficiency.
• Enhanced the original librilight preprocess scripts, improving usability.

🧑‍🤝‍🧑 Who Can Review?

@HeCheng0625 @lmxue

🛠 TODO

• Conduct a comprehensive test of the entire NS2 training pipeline with the updated preprocessing scripts, as coordinated with @HeCheng0625, to validate the enhancements and ensure seamless integration and functionality within the new NS2 framework to be updated.

✅ Checklist

• Code has been reviewed
• Code complies with the project's code standards and best practices
• Code has passed all tests
• Code does not affect the normal use of existing features
• Code has been commented properly
• Documentation has been updated (if applicable)
• Demo/checkpoint has been attached (if applicable)

[lmxue]

Thanks for your efforts. Please check the comments.

[lmxue]

I recommend modifying this to separately check if different files exist. Otherwise, it's unclear which specific file is missing.

[lmxue]

These two lines of code imply that the LibriLight dataset does not undergo subsequent feature extraction; does it use online feature extraction? Can we add a condition here to determine whether the feature extraction method is online extraction or pre-extraction? This means our system will support two types of feature extraction methods.
P.S., we need to integrate the online feature extraction process later.