Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: mirage/capnp-rpc
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v1.2.1
Choose a base ref
...
head repository: mirage/capnp-rpc
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Loading
Showing with 6,380 additions and 4,915 deletions.
  1. +43 −0 .github/workflows/main.yml
  2. +2 −1 .gitignore
  3. +136 −0 CHANGES.md
  4. +1 −8 Makefile
  5. +519 −385 README.md
  6. +5 −13 capnp-rpc-lwt.opam
  7. +8 −106 capnp-rpc-lwt/capability.ml
  8. +0 −24 capnp-rpc-lwt/capnp_core.ml
  9. +11 −69 capnp-rpc-lwt/capnp_rpc_lwt.ml
  10. +18 −48 capnp-rpc-lwt/capnp_rpc_lwt.mli
  11. +1 −13 capnp-rpc-lwt/dune
  12. +7 −41 capnp-rpc-lwt/persistence.ml
  13. +5 −76 capnp-rpc-lwt/service.ml
  14. +5 −11 capnp-rpc-lwt/sturdy_ref.ml
  15. +0 −37 capnp-rpc-mirage.opam
  16. +7 −11 capnp-rpc-net.opam
  17. +13 −13 capnp-rpc-net/auth.ml
  18. +2 −5 capnp-rpc-net/auth.mli
  19. +75 −154 capnp-rpc-net/capTP_capnp.ml
  20. +11 −6 capnp-rpc-net/capTP_capnp.mli
  21. +3 −6 capnp-rpc-net/capnp_rpc_net.ml
  22. +18 −16 capnp-rpc-net/capnp_rpc_net.mli
  23. +2 −2 capnp-rpc-net/dune
  24. +97 −33 capnp-rpc-net/endpoint.ml
  25. +21 −21 capnp-rpc-net/endpoint.mli
  26. +15 −17 capnp-rpc-net/parse.ml
  27. +2 −2 capnp-rpc-net/parse.mli
  28. +74 −74 capnp-rpc-net/restorer.ml
  29. +24 −23 capnp-rpc-net/s.ml
  30. +3 −3 capnp-rpc-net/serialise.ml
  31. +1 −1 capnp-rpc-net/serialise.mli
  32. +32 −49 capnp-rpc-net/tls_wrapper.ml
  33. +12 −16 capnp-rpc-net/tls_wrapper.mli
  34. +1 −1 capnp-rpc-net/two_party_network.ml
  35. +87 −99 capnp-rpc-net/vat.ml
  36. +14 −10 capnp-rpc-unix.opam
  37. +10 −8 capnp-rpc.opam
  38. +130 −0 capnp-rpc/capability.ml
  39. +18 −0 capnp-rpc/capnp_core.ml
  40. +84 −14 capnp-rpc/capnp_rpc.ml
  41. +345 −17 capnp-rpc/capnp_rpc.mli
  42. +11 −1 capnp-rpc/dune
  43. +54 −0 capnp-rpc/leak_handler.ml
  44. +22 −0 capnp-rpc/leak_handler.mli
  45. +5 −5 {capnp-rpc-lwt → capnp-rpc}/msg.ml
  46. +4 −4 {capnp-rpc-lwt → capnp-rpc}/msg.mli
  47. +53 −0 capnp-rpc/persistence.ml
  48. 0 {capnp-rpc-lwt → capnp-rpc}/persistent.capnp
  49. 0 {capnp-rpc-lwt → capnp-rpc}/private.ml
  50. +38 −32 capnp-rpc/{ → proto}/capTP.ml
  51. +5 −3 capnp-rpc/{ → proto}/capTP.mli
  52. 0 capnp-rpc/{ → proto}/cap_proxy.ml
  53. 0 capnp-rpc/{ → proto}/cap_proxy.mli
  54. +14 −0 capnp-rpc/proto/capnp_rpc_proto.ml
  55. +17 −0 capnp-rpc/proto/capnp_rpc_proto.mli
  56. +2 −1 capnp-rpc/{ → proto}/core_types.ml
  57. 0 capnp-rpc/{ → proto}/debug.ml
  58. +1 −0 capnp-rpc/{ → proto}/debug.mli
  59. +4 −0 capnp-rpc/proto/dune
  60. 0 capnp-rpc/{ → proto}/dyn_array.ml
  61. 0 capnp-rpc/{ → proto}/error.ml
  62. 0 capnp-rpc/{ → proto}/exception.ml
  63. 0 capnp-rpc/{ → proto}/id.ml
  64. 0 capnp-rpc/{ → proto}/local_struct_promise.ml
  65. 0 capnp-rpc/{ → proto}/local_struct_promise.mli
  66. 0 capnp-rpc/{ → proto}/message_types.ml
  67. +4 −4 capnp-rpc/{ → proto}/rC.ml
  68. 0 capnp-rpc/{ → proto}/rC.mli
  69. 0 capnp-rpc/{ → proto}/rO_array.ml
  70. 0 capnp-rpc/{ → proto}/rO_array.mli
  71. +5 −5 capnp-rpc/{ → proto}/s.ml
  72. 0 capnp-rpc/{ → proto}/stats.ml
  73. +10 −11 capnp-rpc/{ → proto}/struct_proxy.ml
  74. 0 capnp-rpc/{ → proto}/table.ml
  75. 0 capnp-rpc/{ → proto}/weak_ptr.ml
  76. 0 capnp-rpc/{ → proto}/weak_ptr.mli
  77. +1 −1 {capnp-rpc-lwt → capnp-rpc}/request.ml
  78. +1 −1 {capnp-rpc-lwt → capnp-rpc}/request.mli
  79. +1 −1 {capnp-rpc-lwt → capnp-rpc}/response.ml
  80. 0 {capnp-rpc-lwt → capnp-rpc}/rpc_schema.capnp
  81. 0 {capnp-rpc-lwt → capnp-rpc}/schema.ml
  82. +72 −0 capnp-rpc/service.ml
  83. +25 −0 capnp-rpc/sturdy_ref.ml
  84. 0 {capnp-rpc-lwt → capnp-rpc}/xform.ml
  85. 0 {capnp-rpc-lwt → capnp-rpc}/xform.mli
  86. +8 −0 dune
  87. +3 −1 dune-project
  88. +0 −1 examples/api.ml
  89. +10 −14 examples/dune
  90. +8 −0 examples/pipelining/dune
  91. +3 −0 examples/pipelining/dune-project
  92. +98 −0 examples/pipelining/echo.ml
  93. +11 −0 examples/pipelining/echo_api.capnp
  94. +43 −0 examples/pipelining/main.ml
  95. +21 −0 examples/prelude.ml
  96. +6 −0 examples/sturdy-refs-2/api.capnp
  97. +8 −0 examples/sturdy-refs-2/dune
  98. +3 −0 examples/sturdy-refs-2/dune-project
  99. +40 −0 examples/sturdy-refs-2/logger.ml
  100. +43 −0 examples/sturdy-refs-2/main.ml
  101. +6 −0 examples/sturdy-refs-3/api.capnp
  102. +8 −0 examples/sturdy-refs-3/dune
  103. +3 −0 examples/sturdy-refs-3/dune-project
  104. +47 −0 examples/sturdy-refs-3/logger.ml
  105. +56 −0 examples/sturdy-refs-3/main.ml
  106. +3 −0 examples/sturdy-refs-4/.gitignore
  107. +6 −0 examples/sturdy-refs-4/api.capnp
  108. +48 −0 examples/sturdy-refs-4/db.ml
  109. +16 −0 examples/sturdy-refs-4/db.mli
  110. +13 −0 examples/sturdy-refs-4/dune
  111. +3 −0 examples/sturdy-refs-4/dune-project
  112. +48 −0 examples/sturdy-refs-4/logger.ml
  113. +104 −0 examples/sturdy-refs-4/main.ml
  114. 0 test/test.mli → examples/sturdy-refs-4/main.mli
  115. +9 −0 examples/sturdy-refs-4/store.capnp
  116. +5 −0 examples/sturdy-refs/api.capnp
  117. +8 −0 examples/sturdy-refs/dune
  118. +3 −0 examples/sturdy-refs/dune-project
  119. +24 −0 examples/sturdy-refs/logger.ml
  120. +53 −0 examples/sturdy-refs/main.ml
  121. +1 −0 examples/testlib/api.ml
  122. +37 −35 examples/{ → testlib}/calc.ml
  123. +7 −6 examples/{ → testlib}/calc.mli
  124. 0 examples/{ → testlib}/calculator.capnp
  125. +13 −0 examples/testlib/dune
  126. +9 −10 examples/{ → testlib}/echo.ml
  127. +4 −4 examples/{ → testlib}/echo.mli
  128. +11 −12 examples/{ → testlib}/registry.ml
  129. +7 −6 examples/{ → testlib}/registry.mli
  130. +10 −12 examples/{ → testlib}/store.ml
  131. +4 −4 examples/{ → testlib}/store.mli
  132. 0 examples/{ → testlib}/test_api.capnp
  133. +8 −0 examples/v1/dune
  134. +3 −0 examples/v1/dune-project
  135. +29 −0 examples/v1/echo.ml
  136. +5 −0 examples/v1/echo_api.capnp
  137. +11 −0 examples/v1/main.ml
  138. +8 −0 examples/v2/dune
  139. +3 −0 examples/v2/dune-project
  140. +84 −0 examples/v2/echo.ml
  141. +10 −0 examples/v2/echo_api.capnp
  142. +21 −0 examples/v2/main.ml
  143. +8 −0 examples/v3/dune
  144. +3 −0 examples/v3/dune-project
  145. +84 −0 examples/v3/echo.ml
  146. +10 −0 examples/v3/echo_api.capnp
  147. +36 −0 examples/v3/main.ml
  148. +37 −0 examples/v4/client.ml
  149. +8 −0 examples/v4/dune
  150. +3 −0 examples/v4/dune-project
  151. +78 −0 examples/v4/echo.ml
  152. +10 −0 examples/v4/echo_api.capnp
  153. +36 −0 examples/v4/server.ml
  154. +1 −1 fuzz/direct.ml
  155. +1 −1 fuzz/direct.mli
  156. +1 −1 fuzz/dune
  157. +28 −27 fuzz/fuzz.ml
  158. +0 −54 mirage/capnp_rpc_mirage.ml
  159. +0 −72 mirage/capnp_rpc_mirage.mli
  160. +0 −4 mirage/dune
  161. +0 −93 mirage/network.ml
  162. +0 −37 mirage/network.mli
  163. +0 −57 mirage/vat_config.ml
  164. +36 −37 test-bin/calc.ml
  165. +42 −57 test-bin/calc_direct.ml
  166. +2 −1 test-bin/dune
  167. +1 −2 test-bin/echo/dune
  168. +3 −4 test-bin/echo/echo.ml
  169. +18 −18 test-bin/echo/echo_bench.ml
  170. +0 −5 test-lwt/dune
  171. +0 −729 test-lwt/test_lwt.ml
  172. +0 −1 test-lwt/test_lwt.mli
  173. +0 −6 test-mirage/dune
  174. +0 −142 test-mirage/test_mirage.ml
  175. +0 −1 test-mirage/test_mirage.mli
  176. +3 −2 test/dune
  177. +4 −0 test/proto/dune
  178. +1,612 −0 test/proto/test.ml
  179. 0 test/proto/test.mli
  180. +7 −7 test/{ → proto}/testbed/capnp_direct.ml
  181. +15 −13 test/{ → proto}/testbed/connection.ml
  182. +4 −4 test/{ → proto}/testbed/connection.mli
  183. +3 −0 test/proto/testbed/dune
  184. +4 −4 test/{ → proto}/testbed/services.ml
  185. +1 −1 test/{ → proto}/testbed/test_utils.ml
  186. +789 −1,590 test/test.ml
  187. +0 −3 test/testbed/dune
  188. +76 −95 unix/capnp_rpc_unix.ml
  189. +28 −21 unix/capnp_rpc_unix.mli
  190. +1 −2 unix/dune
  191. +24 −31 unix/file_store.ml
  192. +64 −54 unix/network.ml
  193. +8 −5 unix/network.mli
  194. +0 −91 unix/unix_flow.ml
  195. +0 −7 unix/unix_flow.mli
  196. +26 −27 unix/vat_config.ml
  197. +1 −1 unix/vat_network.ml
43 changes: 43 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Main workflow

on:
pull_request:
push:
schedule:
# Prime the caches every Monday
- cron: 0 1 * * MON

jobs:
build:
strategy:
fail-fast: false
matrix:
os:
- macos-latest
- windows-latest

runs-on: ${{ matrix.os }}

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set-up OCaml
uses: ocaml/setup-ocaml@v3
with:
ocaml-compiler: 5

- name: Install Cap'n Proto (on Windows)
if: runner.os == 'Windows'
shell: bash
run: |
curl -LO https://capnproto.org/capnproto-c++-win32-$CAPNP_VERSION.zip && \
unzip -j capnproto-c++-win32-$CAPNP_VERSION.zip capnproto-tools-win32-$CAPNP_VERSION/capnp.exe -d /usr/bin
env:
CAPNP_VERSION: 1.1.0

- run: opam install . --deps-only --with-test

- run: opam exec -- dune build

- run: opam exec -- dune runtest
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.merlin
_build
_opam
.merlin
*.install
136 changes: 136 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,139 @@
### v2.0

capnp-rpc 2.0 switches from using Lwt to Eio.
The `capnp-rpc-lwt` package is now just Lwt wrappers around the Eio functions in `capnp-rpc`.
This allows libraries using the old Lwt API to be used with applications and libraries using the new Eio one,
in the same binary.
Because Lwt libraries (using `capnp-rpc-lwt`) can be used with Eio applications (using `capnp-rpc-unix`),
you should first upgrade your application and then upgrade the libraries afterwards.

It is recommended to upgrade in stages, checking your application still works after each step:

1. Upgrade to the latest Lwt version (`opam install capnp-rpc-unix.1.2.4`).
This uses the new version of mirage-crypto, tls, etc,
which may be incompatible with other libraries you are using.
Get that sorted out before switching to capnp-rpc 2.0.

2. Use [lwt_eio][] to allow using Eio and Lwt together in your application.
This means replacing your call to `Lwt_main.run` with code to run Eio and Lwt together,
as explained at the start of the `lwt_eio` README.

3. Upgrade your main application to capnp-rpc-unix 2.0.
Calls to functions in the `Capnp_rpc_unix` and `Capnp_rpc_net` modules will need minor updates.
They were previously called from Lwt context, so you'll need to wrap them with `Lwt_eio.run_eio`
(or remove a `Lwt_eio.run_lwt` if you already updated the surrounding code).
You should also use `mirage-crypto-rng-eio` to ensure randomness is available
(`capnp-rpc-unix` no longer does this, although some other library might).

4. Upgrade code and libraries using `Capnp_rpc_lwt`:

1. Replace `open Capnp_rpc_lwt` with `open Capnp_rpc.Std`.
2. Replace all other uses of `Capnp_rpc_lwt` with just `Capnp_rpc`.
3. In `dune` and `opam` files, replace `capnp-rpc-lwt` with `capnp-rpc`.
4. Some modules are in `Capnp_rpc` but not the `Capnp_rpc.Std` subset.
Those should now be fully qualified (e.g. replace `Persistence` with
`Capnp_rpc.Persistence`).
5. Replace `Service.return_lwt` with `Lwt_eio.run_lwt`.
Replace `Lwt.return (Ok x)` with `Service.return x`.

Once all Lwt code is gone, you can remove the dependencies on `lwt` and `lwt_eio`.

New features:

- Allow numeric IPv6 listen addresses (@talex5 #296, requested by @BChabanne).

API changes:

- Eio port (@talex5 #280 #284 #298 #292 #297 #300 #304).

This switches capnp-rpc from Lwt to Eio.
One particularly nice side effect of this is that `Service.return_lwt` has gone,
as there is no distinction now between concurrent and non-concurrent service methods.

Uses of `Capnp_rpc_lwt` should be replaced by uses of `Capnp_rpc`
(and the old `Capnp_rpc` is now an internal module, `Capnp_rpc_proto`).
The new `Capnp_rpc` now provides `Error`, `Exception` and `Debug` aliases to the same modules in `Capnp_rpc_proto`,
so that `Capnp_rpc_proto` doesn't need to be used directly.

`Capnp_rpc_lwt` now provides (deprecated) compatibility wrappers, using `lwt_eio`.

This also adds `Capnp_rpc.Std` with some common module aliases,
to reduce the need to `open Capnp_rpc` (which is rather large).

- Deprecate `Debug.failf` (@talex5 #291).

Performance and bug fixes:

- Add buffering of outgoing messages (@talex5 #287 #303).
Sending each message in its own TCP packet isn't very efficient, and also interacts very badly with Nagle's algorithm.
See <https://roscidus.com/blog/blog/2024/07/22/performance/> for details.

- Only disconnect socket when sending is done (@talex5 #295).
Allows sending the reason for the disconnection in some cases.

- Fix type of `Capability.call` (@talex5 #293).
It was previously unusable.

- Work around FreeBSD `close` problem (@talex5 #302).

Build:

- Update to capnp 3.6.0 (@talex5 #285).

- Update to dune 3.16 and fix warnings (@talex5 #282).

- Use `String.starts_with` instead of `Astring` (@talex5 #290).

- Remove unused `tls-mirage` dependency (@talex5 #289).

- Remove `asetmap` dependency (@talex5 #288).

[lwt_eio]: https://github.com/ocaml-multicore/lwt_eio

### v1.2.4

- Update dependencies and remove capnp-rpc-mirage (@talex5 #276).
- Update for breaking changes in tls, mirage-crypto and mirage-flow.
- capnp-rpc-mirage isn't used much, and won't be needed after switching to Eio.

- Update tests to Cap'n Proto 1.0.1 (@MisterDA #274).

- Update links to ocaml-ci and capnp-ocaml (@tmcgilchrist #271).

- Opam doesn't allow with-test in conflicts (@talex5 #269).

- Disable opam tests on macos (@talex5 #268).

- Update CI (@MisterDA #272, @talex5 #277).

### v1.2.3

- Update to cmdliner 1.1.0 (@MisterDA #249).

- Fix tests on Windows, avoid using Unix domain sockets (@talex5 #251).

- Fix warning on `Unix.ENOTCONN` under macos (@talex5 #252).

- Add `Capnp_rpc_unix.Cap_file.save_uri` (@talex5 #253).

- Update README to use MDX (@talex5 #254).

- Update tests to Cap'n Proto 0.10.3 (@MisterDA #257 #260).

- Update to mirage-crypto-rng 0.11 and dns 7.0.0 API changes (@hannesm #261).

- Update for opam 2.1+ installations (@frumioj #259).

- Install Cap'n Proto binaries in Windows GHA (@MisterDA #248).

### v1.2.2

- Switch from mirage-{stack,protocols} to tcpip (@MisterDA #246).

- Remove deprecated tcpip.unix dependency (@hannesm #245).

- Add OCaml 4.14 support (@kit-ty-kate #244).

### v1.2.1

- Fix Fmt deprecation warnings (@tmcgilchrist #239).
9 changes: 1 addition & 8 deletions Makefile
Original file line number Diff line number Diff line change
@@ -2,11 +2,6 @@

default: test build-fuzz

all:
dune build @install test/test.exe test-lwt/test_lwt.exe test-bin/calc.exe test-mirage/test_mirage.exe
rm -rf _build/_tests
dune runtest --no-buffer -j 1

build-fuzz:
dune build fuzz/fuzz.exe

@@ -19,7 +14,5 @@ clean:

test:
rm -rf _build/_tests
dune build test/test.exe test-lwt/test_lwt.exe test-bin/calc.exe test-mirage/test_mirage.exe test-bin/echo/echo_bench.exe @install
#./_build/default/test/test.bc test core -ev 36
#./_build/default/test-lwt/test.bc test lwt -ev 3
dune build test test-bin examples @install
dune build @runtest --no-buffer -j 1
Loading