Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-adding SqlUpdatesWithProcedures (with build fixes) #40

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

hulien22
Copy link
Contributor

@hulien22 hulien22 commented Sep 30, 2020

Same as #37 but with fixes for the build (previous pull request was missing the latest commit from master, and so there was a build error around apps/SqlUpdatesWithProcedures.java:[69,3] method does not override or implement a method from a supertype).

Old PR message from #37:

Adding a new workload that is similar to SqlUpdates but that uses sql procedures in order to batch the updates together (see batching of updates in yugabyte/yugabyte-db#5257).


As mentioned in the comments of #37, SqlUpdatesWithProcedures differs from SqlUpdates in that this workload will only send num_writes unique updates (the uniqueness is to avoid transaction conflicts). Thus this workload can also be used for performance timing, as it will exit after it has finished all its updates.

Typical use case of this workload is via first running SqlInserts:

java -jar yb-sample-apps.jar \
--workload SqlInserts \
--nodes a:5433,b:5433,c:5433 \
--num_unique_keys 5000000 \
--num_reads 0 \
--num_threads_write 128 \
--uuid 'ffffffff-ffff-ffff-ffff-ffffffffffff'

Then by running SqlUpdatesWithProcedures:

java -jar yb-sample-apps.jar \
--workload SqlUpdatesWithProcedures \
--nodes a:5433,b:5433,c:5433 \
--uuid 'ffffffff-ffff-ffff-ffff-ffffffffffff' \
--max_written_key 5000000 \
--num_writes 1000000 \
--num_reads 0 \
--num_threads_read 0 \
--num_threads_write XXX \
--update_batch_size YYY

A few things to note:

  • Notice that num_writes is lower than max_written_key, this is mostly to ensure that we don't have issues randomly selecting the last remaining keys to update at the end.
  • With the way the workload is currently written, the max update_batch_size is 50 since postgres only allows procedures to have at most 100 parameters, and we are currently just passing in key/value pairs to be updated (100/2 = 50).
  • Performance can also be improved by forcing all writes to a single tablet, for example by running \c postgres and create table postgresqlkeyvalue(k text primary key, v text) split into 1 tablets; in ysqlsh prior to running SqlUpdates

@psudheer21
Copy link
Contributor

LGTM!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants