Skip to content

Commit

Permalink
Move SDB into RzUtil
Browse files Browse the repository at this point in the history
* Make SDB code SPDX compliant
* Use some RzUtil functions inside SDB

util: compile natively without any extra dependency
  • Loading branch information
ret2libc committed Jul 19, 2022
1 parent 35cf52e commit faac243
Show file tree
Hide file tree
Showing 63 changed files with 11,520 additions and 159 deletions.
8 changes: 8 additions & 0 deletions .reuse/dep5
Original file line number Diff line number Diff line change
Expand Up @@ -255,3 +255,11 @@ License: LGPL-3.0-only
Files: librz/reg/p/x86-linux.regs
Copyright: 2009 pancake <[email protected]>
License: LGPL-3.0-only

Files: librz/util/sdb/src/json/README
Copyright: pancake <[email protected]>
License: MIT

Files: librz/util/sdb/src/json/test.json
Copyright: pancake <[email protected]>
License: MIT
8 changes: 8 additions & 0 deletions LICENSES/CC-PDDC.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@

The person or persons who have associated work with this document (the "Dedicator" or "Certifier") hereby either (a) certifies that, to the best of his knowledge, the work of authorship identified is in the public domain of the country from which the work is published, or (b) hereby dedicates whatever copyright the dedicators holds in the work of authorship identified below (the "Work") to the public domain. A certifier, moreover, dedicates any copyright interest he may have in the associated work, and for these purposes, is described as a "dedicator" below.

A certifier has taken reasonable steps to verify the copyright status of this work. Certifier recognizes that his good faith efforts may not shield him from liability if in fact the work certified is not in the public domain.

Dedicator makes this dedication for the benefit of the public at large and to the detriment of the Dedicator's heirs and successors. Dedicator intends this dedication to be an overt act of relinquishment in perpetuity of all present and future rights under copyright law, whether vested or contingent, in the Work. Dedicator understands that such relinquishment of all rights includes the relinquishment of all rights to enforce (by lawsuit or otherwise) those copyrights in the Work.

Dedicator recognizes that, once placed in the public domain, the Work may be freely reproduced, distributed, transmitted, used, modified, built upon, or otherwise exploited by anyone for any purpose, commercial or non-commercial, and in any way, including by methods that have not yet been invented or conceived.
87 changes: 80 additions & 7 deletions librz/util/meson.build
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
subdir('sdb/src')

rz_util_sources = [
'alloc.c',
Expand Down Expand Up @@ -81,7 +82,7 @@ rz_util_sources = [
'x509.c',
]

rz_util_deps = [ldl, lrt, mth, th, utl, sdb_dep] + platform_deps
rz_util_deps = [ldl, lrt, mth, th, utl] + platform_deps
if zlib_dep.found()
rz_util_deps += [zlib_dep]
endif
Expand All @@ -92,6 +93,13 @@ if ['freebsd', 'netbsd', 'haiku', 'dragonfly'].contains(host_machine.system())
# backtrace_symbols_fd requires -lexecinfo
rz_util_deps += [cc.find_library('execinfo', static: is_static_build)]
endif
if meson.is_cross_build()
execinfo_native = disabler()
if ['freebsd', 'netbsd', 'haiku', 'dragonfly'].contains(build_machine.system())
# backtrace_symbols_fd requires -lexecinfo
execinfo_native = cc_native.find_library('execinfo', static: is_static_build)
endif
endif

if sys_openssl.found()
rz_util_deps += [sys_openssl]
Expand All @@ -100,8 +108,10 @@ else
rz_util_sources += ['big.c']
endif

rz_util = library('rz_util', rz_util_sources,
include_directories: platform_inc,
rz_util_includes = [platform_inc, include_directories('sdb/src', 'sdb')]

rz_util = library('rz_util', rz_util_sources, libsdb_sources,
include_directories: rz_util_includes,
dependencies: rz_util_deps,
install: true,
implicit_include_directories: false,
Expand All @@ -114,13 +124,61 @@ rz_util = library('rz_util', rz_util_sources,

rz_util_dep = declare_dependency(
link_with: rz_util,
include_directories: platform_inc,
dependencies: [
sdb_dep.partial_dependency(includes: true),
]
include_directories: rz_util_includes,
)
meson.override_dependency('rz_util', rz_util_dep)

if meson.is_cross_build()
# this is a bit messy, it duplicates most of the code required to get the
# regular dependencies but for the build_system. This is required to build
# sdb_native, used at build time to compile .sdb.txt files into .sdb ones.
platform_native_deps = []
if build_machine.system() == 'windows'
platform_native_deps = [
cc_native.find_library('ws2_32'),
cc_native.find_library('wininet'),
cc_native.find_library('psapi'),
]
endif
rz_util_native_deps = [ldl_native, lrt_native, mth_native, th_native, utl_native] + platform_native_deps
if execinfo_native.found()
rz_util_native_deps += [execinfo_native]
endif
# do not use external libs, we do not need them
userconf_native.set10('HAVE_LIB_MAGIC', false)
userconf_native.set10('USE_LIB_MAGIC', false)
userconf_native.set10('HAVE_LIB_XXHASH', false)
userconf_native.set10('USE_LIB_XXHASH', false)
userconf_native.set10('HAVE_OPENSSL', false)
userconf_native.set10('HAVE_LIBUV', false)
userconf_native.set10('HAVE_LZMA', false)
userconf_native.set10('HAVE_ZLIB', false)

rz_userconf_native_h = configure_file(
input: rz_userconf_h_in,
output: 'rz_userconf.h',
configuration: userconf_native,
install_dir: rizin_incdir
)
rz_util_native_includes = [include_directories('.'), rz_util_includes]

rz_util_native = static_library('rz_util_native', rz_util_sources, libsdb_sources,
include_directories: rz_util_native_includes,
dependencies: rz_util_native_deps,
implicit_include_directories: false,
install_rpath: rpath_lib,
install: false,
native: true,
)

rz_util_native_dep = declare_dependency(
link_with: rz_util_native,
include_directories: rz_util_native_includes,
)
else
rz_util_native_dep = rz_util_dep
endif

pkgconfig_mod.generate(rz_util,
subdirs: ['librz', 'librz/sdb'],
version: rizin_version,
Expand Down Expand Up @@ -149,3 +207,18 @@ if not is_static_libs_only
configuration: conf,
)
endif

sdb_exe = executable('sdb_native', 'sdb/src/main.c',
dependencies: rz_util_native_dep,
install: false,
native: true,
implicit_include_directories: false,
install_rpath: rpath_exe,
)

sdb_gen_cmd = [
sdb_exe,
'@OUTPUT@',
'==',
'@INPUT@'
]
88 changes: 88 additions & 0 deletions librz/util/sdb/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
SDB (string database)
=====================

sdb is a simple string key/value database based on djb's cdb
disk storage and supports JSON and arrays introspection.

Author
------
pancake <[email protected]>

Contains
--------
* namespaces (multiple sdb paths)
* atomic database sync (never corrupted)
* commandline frontend for sdb databases
* arrays support (syntax sugar)
* json parser/getter (js0n.c)

Rips
----
* disk storage based on cdb code
* linked lists from rizin api

Compilation
-----------
SDB requires [Meson](https://mesonbuild.com/) and [Ninja](https://ninja-build.org/) buildsystems to be built:
```
meson build
ninja -C build
```

Changes
-------
I have modified cdb code a little to create smaller databases and
be memory leak free in order to use it from a library.

The sdb's cdb database format is 10% smaller than the original
one. This is because keylen and valuelen are encoded in 4 bytes:
1 for the key length and 3 for the value length.

In a test case, a 4.3MB cdb database takes only 3.9MB after this
file format change.

Usage example
-------------
Let's create a database!
```
$ sdb d hello=world
$ sdb d hello
world
```
Using arrays (>=0.6):
```
$ sdb - '[]list=1,2' '[0]list' '[0]list=foo' '[]list' '[+1]list=bar'
1
foo
2
```
Let's play with json:
```
$ sdb d g='{"foo":1,"bar":{"cow":3}}'
$ sdb d g:bar.cow
3
$ sdb - user='{"id":123}' user:id=99 user:id
99
```
Using the commandline without any disk database:
```
$ sdb - foo=bar foo a=3 +a -a
bar
4
3
```
```
$ sdb -
foo=bar
foo
bar
a=3
+a
4
-a
3
```
Remove the database
```
$ rm -f d
```
Loading

0 comments on commit faac243

Please sign in to comment.