-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathREADME
116 lines (79 loc) · 4.76 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
NoWog (Not only Workload Generator)
NoWog
-----
NoWog stands for "Not only Workload Generator", which is a workload generator able to generate and execute customizable MongoDB workload based on a specific input format defined in doc/EBNF_gammar.txt
NoWog consist of three parts:
- Parser: use pyparsing to parse the BNF input.
Parser result consist of ID of each rule, distribution and operation information.
- Mapping: map the parser result into MongoDB command.
This part use distribution.py to draw sampled from specified distribution.
And use values.py to generate all random values in each command.
Each rule will be mapped as a session, which consist of its ID and a
execution time table.
- Execution: execute all sessions.
This part are also able to display histogram of specific type or IDs of workload.
Usage without scenarios:
------------------------
1. Adapt config.ini:
a) For [inputs]:
- input_files: a list of BNF input file names, separated by comma ",".
b) For [outputs]:
- save_parser: the output json file name of parser result.
- save_sessions: the output json file name of mapping result
Note:
All items in this section is not strictly required.
c) For [connection]:
- db_name: database name in which workload will be executed.
- coll_name: collection name in which workload will be executed
- URL: MongoDB URL, which provide all information required by connection.
d) For [seed]:
- seed: seed used in drawing samples from specific distribution and generating
random value for operations
d) For [additional_value_setting]:
Add additional setting of generating random values for operation. Read more in values.py
e) For [additional_execution_setting]:
Add additional setting of executor. Read more in execution.py
2. Run main.py with or without arguments:
--show: display the histogram of designated type of operation: {all, find, update, insert, delete}
--showid: display designated histogram of IDs
--drop: drop whole collection before "try" and "run"
--try (-t): run command in each session once. In order to test the correctness of parameter.
--run (-r): run all sessions
Example:
If run without any arguments, program will exit after generate (or write) sessions file.
$ python main.py
Display workload histogram for all operations of all sessions.
$ python main.py --show
Only run one operation in each session once, in order to test the correctness
and runnability of the generated command.
$ python main.py --try
Run all operations in all sessions under prearranged distribution.
$ python main.py --run
Usage with scenarios:
---------------------
1. Adopt scenario.ini:
- load_BNF : output BNF file name of LOADING stage.
- workload_BNF : output BNF file name of WORKLOAD stage
- coll_size: collection size
- write: write phrase for loading BNF, also the template of generated dataset
- complexity: indicate the attributes in output BNF are "simple" or "complicate"
- start_time: starting time of all workloads
- end_time: ending time of all workloads
- total_worklaod: total amount of workload
- type: type of workloads:
* read_only: 100% read
* update_only: 100% update
* insert_only: 100% insert
* read_mostly: 95% read and 5% update
* update_mostly: 50% read and 50% update
* custom: customize your ratio in [ratio] section
- ratio: different ratio of read, insert and update. Only valid when workload type is "custom"
Note: If sum of all ratio greater than 100, raise value exception.
2. Run senario.py without any arguments
Dependencies
------------
Python version: higher than 2.7.10.
MongoDB version: 3.2.1
Require modules:
pymongo
pyparsing