Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQLite API: Make Row Reader more efficient and backward compatible #566

Open
fklebert opened this issue Jan 31, 2024 · 1 comment
Open
Labels
c++ C++ language generator java Java language generator python Python language generator
Milestone

Comments

@fklebert
Copy link
Contributor

fklebert commented Jan 31, 2024

At the moment the zserio generated C++ API for Rows of a SQLite table does kind of a "SELECT *" from the relevant table. It does not really use "*", but all the column names that are available in the schema.
This is in two cases not optimal:

  1. It makes SQLite fetch all columns, which in many cases is not necessary. In case of large blobs in columns, it will generate unnessecary memory pressure and performance degradation.
  2. It breaks backward compatibility with older databases in case columns have been added in subsequent versions.

It would be beneficial if we can extend ::createReader(condition) to also allow to specify the colunms to be fetched. Like
::createReader(columns, condition)

This has been observed in the C++ API. I haven't checked the others...

@mikir
Copy link
Contributor

mikir commented Jan 31, 2024

Yes, I agree. This is a good enhancement of SQLite API. The same problem is in Java and Python API(s).

@mikir mikir added python Python language generator java Java language generator c++ C++ language generator labels Jan 31, 2024
@mikir mikir added this to the Backlog milestone Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ C++ language generator java Java language generator python Python language generator
Projects
None yet
Development

No branches or pull requests

2 participants