Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a feeder module #232

Open
Glutexo opened this issue May 19, 2024 · 0 comments
Open

Create a feeder module #232

Glutexo opened this issue May 19, 2024 · 0 comments

Comments

@Glutexo
Copy link
Owner

Glutexo commented May 19, 2024

Maybe the whole architecture can be different. The current idea of having three main components directly in Onigumo would result in Onigumo.Parser and Onigumo.Operator only discovering the files for parsing/handling, pasing them to the respective Spider modules. This discovery can be rather handled by a standalone module, directly feeding the Spider modules. The directory listing logic doesn’t need to be duplicated in thin modules doing essentialy nothing.

Downloader is the only component from the main trinity that doesn’t pass its items to a Spider. It still needs to be feeded by the new module though.

This would require to update the diagram in the README. Related to #228 and #233.

What about the name? Ideas: feeder, supplier, discovery. I personally like the -er/-or suffix. Look at queue jargon, what’s being used in the wild.

@Glutexo Glutexo changed the title Is Parser a good name? Create a feeder/discovery module May 25, 2024
@Glutexo Glutexo changed the title Create a feeder/discovery module Create a feeder module May 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants