Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenPechaS3 class #234

Open
eroux opened this issue Jan 9, 2023 · 1 comment
Open

OpenPechaS3 class #234

eroux opened this issue Jan 9, 2023 · 1 comment

Comments

@eroux
Copy link
Contributor

eroux commented Jan 9, 2023

It seems our current workflow won't scale very well because of Github limitations (see this example). An alternative is to store the opf files on an s3 bucket. The bucket name could be opf.openpecha.org and there could be an intermediate step of folders like what we have on archive.tbrc.org or ocr.bdrc.io, using the last two letters of the pecha id for instance. Once it's done we should copy the opfs currently on Github on s3.

@eroux
Copy link
Contributor Author

eroux commented Jan 9, 2023

Now, obviously, the issue is that these won't be git repositories anymore. Some ideas:

  • perhaps we don't always need git repositories, but I think we should keep git repos
  • we could store the git repos as bare git repositories (there's already a class for that but it could be improved), this will lower the storage cost but will be a little less convenient. Since we'll replace the files after each change, conflicts, etc. won't be handled but I think we can live with that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant