Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata memory layout refactor #41

Open
wants to merge 68 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
40417be
work
jpihl Dec 17, 2024
7317897
work
jpihl Dec 20, 2024
552cea5
working on it
jpihl Dec 23, 2024
f8d1bb2
working on it
jpihl Jan 6, 2025
27d897a
working on it
jpihl Jan 7, 2025
7d9fd27
cleanup
jpihl Jan 8, 2025
9b16c83
try fix docs
jpihl Jan 8, 2025
aad11ac
try fix docs
jpihl Jan 8, 2025
9e170f4
try fix docs
jpihl Jan 8, 2025
c6ae2df
initial work on benchmark
jpihl Jan 8, 2025
63de345
better bench
jpihl Jan 9, 2025
335ee37
work
jpihl Jan 9, 2025
3738eb4
work
jpihl Jan 9, 2025
e27e6bb
include guarding
jpihl Jan 9, 2025
cdcab10
Mvp experiment (#42)
mortenvp Jan 20, 2025
c6fabc6
work
jpihl Jan 21, 2025
b30f86e
work
jpihl Jan 21, 2025
7ece9ec
more asserts
jpihl Jan 21, 2025
b19294f
type -> info
jpihl Jan 21, 2025
92228d9
working
jpihl Jan 21, 2025
53236b1
fin
jpihl Jan 22, 2025
6106391
work
jpihl Jan 22, 2025
39873ec
work
jpihl Jan 22, 2025
42e9720
work
jpihl Jan 22, 2025
f7f846a
fix memory leak
jpihl Jan 23, 2025
8d9034c
Value data in protobuf (#43)
jpihl Jan 24, 2025
e377dbc
fix docs
jpihl Jan 24, 2025
39f8fa0
fix docs
jpihl Jan 24, 2025
b5e102d
fix docs
jpihl Jan 24, 2025
2bc7728
availablity -> availability
jpihl Jan 24, 2025
4a8e7ad
availablity -> availability
jpihl Jan 24, 2025
490c9bf
fix docs
jpihl Jan 24, 2025
1403970
work
jpihl Jan 27, 2025
77cedfc
add parser for Metrics
jpihl Jan 27, 2025
c1ab0fc
fix build
jpihl Jan 27, 2025
533c0df
work
jpihl Jan 27, 2025
021cd2b
work
jpihl Jan 27, 2025
03095cc
work
jpihl Jan 27, 2025
1525128
work
jpihl Jan 27, 2025
f6435d7
work
jpihl Jan 27, 2025
f502410
is this ub?
jpihl Jan 28, 2025
cd8c6e3
work
jpihl Jan 28, 2025
b91b188
Merge branch 'master' into metadata-memory-layout-refactor
jpihl Jan 28, 2025
63fa6a0
work
jpihl Jan 28, 2025
ed5b082
pynte
jpihl Jan 28, 2025
0c529a1
wokr
jpihl Jan 28, 2025
c36971b
work
jpihl Jan 29, 2025
69877b4
work
jpihl Jan 29, 2025
8c6937a
work
jpihl Jan 31, 2025
99a923e
move offset
jpihl Jan 31, 2025
f8f6e2d
working on it
jpihl Jan 31, 2025
3152d6e
working on it
jpihl Feb 1, 2025
59c38c9
working on it
jpihl Feb 2, 2025
56554ee
suggestions (#44)
jpihl Feb 4, 2025
18db3bc
fixed tests
jpihl Feb 4, 2025
994d96d
rm required
jpihl Feb 4, 2025
110563e
no more optional
jpihl Feb 4, 2025
2b08c5d
fix oldy
jpihl Feb 4, 2025
c77bfee
fix oldy
jpihl Feb 4, 2025
fe36c6e
fix oldy
jpihl Feb 4, 2025
4d9f50c
fix docs
jpihl Feb 4, 2025
2b89e5c
fix docs
jpihl Feb 4, 2025
2fff7ed
fix docs
jpihl Feb 4, 2025
23f44ef
work
jpihl Feb 4, 2025
cfdd2b6
added docs
jpihl Feb 4, 2025
0e758b2
minor changes
jpihl Feb 4, 2025
f24ca61
simplify unit tests
jpihl Feb 5, 2025
338e6e1
fix unit test names
jpihl Feb 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ if(${CMAKE_PROJECT_NAME} STREQUAL ${PROJECT_NAME})
# Build executables
add_executable(metrics_simple examples/metrics_simple.cpp)
target_link_libraries(metrics_simple abacus)
target_link_libraries(metrics_simple steinwurf::endian)
target_link_libraries(metrics_simple steinwurf::protobuf)

enable_testing()
Expand All @@ -107,4 +108,23 @@ if(${CMAKE_PROJECT_NAME} STREQUAL ${PROJECT_NAME})
steinwurf::protobuf)

add_test(abacus_tests abacus_tests)

# Google Benchmark dependency
set(BENCHMARK_ENABLE_TESTING
OFF
CACHE INTERNAL "Disable gbenchmark test" FORCE)

if(NOT TARGET benchmark)
add_subdirectory("${STEINWURF_RESOLVE}/gbenchmark-source")
endif()

# Build benchmark executable
file(GLOB_RECURSE abacus_benchmark_sources ./benchmark/main.cpp)
add_executable(abacus_benchmark ${abacus_benchmark_sources})
target_link_libraries(abacus_benchmark benchmark)
target_link_libraries(abacus_benchmark
steinwurf::abacus
steinwurf::endian
steinwurf::bourne
steinwurf::protobuf)
endif()
9 changes: 8 additions & 1 deletion NEWS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,14 @@ every change, see the Git log.

Latest
------
* tbd
* Major: Changed protocol format to version 2.
* Minor: Added benchmark.
* Major: Changed the metadata to be specified using protobuf.
* Major: Changed API of how metrics are specified. All attributes are now
strongly typed, and specified using a std::map of abacus::name to abacus::info.
* Minor: Added support for enums.
* Minor: Added support for constants.
* Minor: Added support for more metric types: int32, uint32, float32, and enum8.

6.0.1
-----
Expand Down
142 changes: 142 additions & 0 deletions benchmark/main.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
#include <abacus/metrics.hpp>
#include <benchmark/benchmark.h>
#include <map>
#include <string>

enum class test_enum
{
value0 = 0,
value1 = 1,
value2 = 2,
value3 = 3
};

// Helper function to create metric definitions
std::map<abacus::name, abacus::info> create_metric_infos()
{
return {
{abacus::name{"0"}, abacus::boolean{abacus::description{""}}},
{abacus::name{"1"},
abacus::uint64{abacus::kind::gauge, abacus::description{""}}},
{abacus::name{"2"},
abacus::int64{abacus::kind::gauge, abacus::description{""}}},
{abacus::name{"3"},
abacus::float64{abacus::kind::gauge, abacus::description{""}}},
{abacus::name{"4"}, abacus::boolean{abacus::description{""}}},
{abacus::name{"5"},
abacus::float64{abacus::kind::gauge, abacus::description{""}}},
{abacus::name{"6"}, abacus::enum8{abacus::description{""},
{{test_enum::value0, {"", ""}},
{test_enum::value1, {"", ""}},
{test_enum::value2, {"", ""}},
{test_enum::value3, {"", ""}}}}}};
}

// Benchmark for metric initialization
static void BM_MetricInitialization(benchmark::State& state)
{
state.SetLabel("Metric Initialization");
for (auto _ : state)
{
abacus::metrics metrics(create_metric_infos());
auto m0 = metrics.initialize<abacus::boolean>("0").set_value(false);
auto m1 = metrics.initialize<abacus::uint64>("1").set_value(0);
auto m2 = metrics.initialize<abacus::int64>("2").set_value(0);
auto m3 = metrics.initialize<abacus::float64>("3").set_value(0.0);
auto m4 = metrics.initialize<abacus::boolean>("4").set_value(true);
auto m5 = metrics.initialize<abacus::float64>("5").set_value(3.14);
auto m6 =
metrics.initialize<abacus::enum8>("6").set_value(test_enum::value1);

(void)m0;
(void)m1;
(void)m2;
(void)m3;
(void)m4;
(void)m5;
(void)m6;
}
state.SetItemsProcessed(state.iterations());
}

// Benchmark for assignment operations
static void BM_AssignMetrics(benchmark::State& state)
{
state.SetLabel("Assign Metrics");
abacus::metrics metrics(create_metric_infos());
auto m0 = metrics.initialize<abacus::boolean>("0").set_value(false);
auto m1 = metrics.initialize<abacus::uint64>("1").set_value(0);
auto m2 = metrics.initialize<abacus::int64>("2").set_value(0);
auto m3 = metrics.initialize<abacus::float64>("3").set_value(0.0);
auto m4 = metrics.initialize<abacus::boolean>("4").set_value(true);
auto m5 = metrics.initialize<abacus::float64>("5").set_value(3.14);
auto m6 =
metrics.initialize<abacus::enum8>("6").set_value(test_enum::value1);

for (auto _ : state)
{
m0 = true;
m1 += 1;
m2 -= 1;
++m3;
m4 = false;
--m5;
m6 = 4;
}

state.SetItemsProcessed(state.iterations());
}

// Benchmark for accessing metrics
static void BM_AccessMetrics(benchmark::State& state)
{
state.SetLabel("Access Metrics");
abacus::metrics metrics(create_metric_infos());
auto m0 = metrics.initialize<abacus::boolean>("0").set_value(false);
auto m1 = metrics.initialize<abacus::uint64>("1").set_value(0);
auto m2 = metrics.initialize<abacus::int64>("2").set_value(0);
auto m3 = metrics.initialize<abacus::float64>("3").set_value(0.0);

for (auto _ : state)
{
bool val0 = m0.value();
uint64_t val1 = m1.value();
int64_t val2 = m2.value();
double val3 = m3.value();

benchmark::DoNotOptimize(val0);
benchmark::DoNotOptimize(val1);
benchmark::DoNotOptimize(val2);
benchmark::DoNotOptimize(val3);
}

state.SetItemsProcessed(state.iterations());
}

// Benchmark for incrementing uint64 metric
static void BM_IncrementUint64(benchmark::State& state)
{
state.SetLabel("Increment Uint64 Metric");
abacus::metrics metrics(create_metric_infos());
auto m1 = metrics.initialize<abacus::uint64>("1").set_value(0);

for (auto _ : state)
{
m1 += 1;
}

state.SetItemsProcessed(state.iterations());
}

// Apply custom arguments to all benchmarks
static void CustomArguments(benchmark::internal::Benchmark* b)
{
b->Unit(benchmark::kMicrosecond);
}

BENCHMARK(BM_MetricInitialization)->Apply(CustomArguments);
BENCHMARK(BM_AssignMetrics)->Apply(CustomArguments);
BENCHMARK(BM_AccessMetrics)->Apply(CustomArguments);
BENCHMARK(BM_IncrementUint64)->Apply(CustomArguments);

BENCHMARK_MAIN();
25 changes: 20 additions & 5 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,29 @@
wurfapi = {
"source_paths": [
# API
"../src/abacus/type.hpp",
"../src/abacus/metric.hpp",
"../src/abacus/metric_info.hpp",
"../src/abacus/boolean.hpp",
"../src/abacus/constant.hpp",
"../src/abacus/description.hpp",
"../src/abacus/enum8.hpp",
"../src/abacus/float32.hpp",
"../src/abacus/float64.hpp",
"../src/abacus/info.hpp",
"../src/abacus/int32.hpp",
"../src/abacus/int64.hpp",
"../src/abacus/kind.hpp",
"../src/abacus/unit.hpp",
"../src/abacus/max.hpp",
"../src/abacus/metric.hpp",
"../src/abacus/metrics.hpp",
"../src/abacus/view.hpp",
"../src/abacus/min.hpp",
"../src/abacus/name.hpp",
"../src/abacus/parse_metadata.hpp",
"../src/abacus/protocol_version.hpp",
"../src/abacus/to_json.hpp",
"../src/abacus/uint32.hpp",
"../src/abacus/uint64.hpp",
"../src/abacus/unit.hpp",
"../src/abacus/version.hpp",
"../src/abacus/view.hpp",
],
"recursive": False,
"include_paths": ["../src"],
Expand Down
2 changes: 1 addition & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -115,5 +115,5 @@ versjon==2.3.0
# via -r docs/requirements.in
virtualenv==20.17.1
# via giit
wurfapi==9.0.0
wurfapi==9.1.1
# via -r docs/requirements.in
2 changes: 2 additions & 0 deletions docs/user_api/description.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::description
2 changes: 0 additions & 2 deletions docs/user_api/kind.rst

This file was deleted.

3 changes: 1 addition & 2 deletions docs/user_api/metrics.rst → docs/user_api/max.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::metrics

:selector: abacus::max
2 changes: 0 additions & 2 deletions docs/user_api/metric_info.rst

This file was deleted.

2 changes: 2 additions & 0 deletions docs/user_api/metric_info/boolean.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::boolean
2 changes: 2 additions & 0 deletions docs/user_api/metric_info/constant.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::constant
2 changes: 2 additions & 0 deletions docs/user_api/metric_info/enum8.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::enum8
2 changes: 2 additions & 0 deletions docs/user_api/metric_info/float32.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::float32
2 changes: 2 additions & 0 deletions docs/user_api/metric_info/float64.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::float64
2 changes: 2 additions & 0 deletions docs/user_api/metric_info/int32.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::int32
2 changes: 2 additions & 0 deletions docs/user_api/metric_info/int64.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::int64
20 changes: 20 additions & 0 deletions docs/user_api/metric_info/metric_info.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@

.. _metric_specializations:

===========
Metric Info
===========

Overview of the metric info types.

.. toctree::
:maxdepth: 2

constant
uint64
uint32
int64
int32
float64
boolean
enum8
2 changes: 2 additions & 0 deletions docs/user_api/metric_info/uint32.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::uint32
2 changes: 2 additions & 0 deletions docs/user_api/metric_info/uint64.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::uint64
2 changes: 0 additions & 2 deletions docs/user_api/metric_specializations/boolean.rst

This file was deleted.

2 changes: 0 additions & 2 deletions docs/user_api/metric_specializations/float64.rst

This file was deleted.

2 changes: 0 additions & 2 deletions docs/user_api/metric_specializations/int64.rst

This file was deleted.

16 changes: 0 additions & 16 deletions docs/user_api/metric_specializations/metric_specializations.rst

This file was deleted.

2 changes: 0 additions & 2 deletions docs/user_api/metric_specializations/uint64.rst

This file was deleted.

2 changes: 2 additions & 0 deletions docs/user_api/min.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::min
2 changes: 2 additions & 0 deletions docs/user_api/name.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.. wurfapi:: class_synopsis.rst
:selector: abacus::name
2 changes: 0 additions & 2 deletions docs/user_api/type.rst

This file was deleted.

11 changes: 6 additions & 5 deletions docs/user_api/user_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,13 @@ Overview of the public API.
.. toctree::
:maxdepth: 3

metric
metric_specializations/metric_specializations
metric_info
type
metrics
name
metric_info/metric_info
kind
description
unit
metrics
min
max
view
functions
Loading
Loading