RFC001 updates

text/001-wasm-udfs.md

@@ -13,35 +13,43 @@ such as strings in arguments and return values.
 
 # Motivation
 
+One of PostgreSQL's most important advantages lies in the rich embedded language support, allowing stored procedures to be authored in javascript, lua, python, etc in addition to the natively support pl/pgsql.
+
+Since Seafowl is meant to be run close to users (perhaps eventually as close as within their browsers), any extension mechanism supported by Seafowl should be portable across environments. With the [widespread adoption](https://caniuse.com/wasm) of WebAssembly, it seems like a [natural](https://sqlite.org/wasm/doc/tip/about.md) [choice](https://duckdb.org/2021/10/29/duckdb-wasm.html) for [user-defined functions](https://www.scylladb.com/2022/04/14/wasmtime/).
+
 Seafowl already has the ability to introduce
-[custom UDFs](https://seafowl.io/docs/guides/custom-udf-wasm). These functions
-are capable of receiving a tuple of primitive values and returning a primitive.
-The currently available WASM primitives are integer and floating point numbers:
+[custom UDFs](https://seafowl.io/docs/guides/custom-udf-wasm) using WebAssembly. These functions
+are capable of receiving a tuple of WebAssembly primitives and returning a primitive.
+The currently available data types are thus currently limited to integer and floating point numbers:
 `i32`, `i64`, `f32` and `f64`.
 
-While the opportunity to extend Seafowl is exciting, the current implementation
-falls short in several important ways:
+The WASM virtual machine is re-created for each invocation, isolating UDF invocations between rows.
+
+Assuming Seafowl's users will build upon the UDF interface, it will be difficult to change in a backward-incompatible way without breaking users' applications. As a result, any significant investment in the UDF logic should address near-term future requirements in addition to currently experienced pain points, specifically:
 
-1. Seafowl's [data types](https://seafowl.io/docs/reference/types) are are much
-   richer than what can be trivially represented with scalar numeric primitives
-   (eg: `TEXT` and `DATE`). Most data types cannot currently be used as UDF
+1. Seafowl's [data types](https://seafowl.io/docs/reference/types) are much
+   richer than what can be trivially represented with numbers 
+   (consider `TEXT` and `DATE`). Most data types cannot currently be used as UDF
    arguments or return values.
-1. Currently unsupported, future user-defined aggregate functions may receive an
-   array of tuples, which are also not easily represented with the numeric types
-   currently available (even if the tuples themselves contain only numbers).
-1. Aso currently unsupported user-defined table functions (the opposite of
-   aggregate functions) may return a list of tuples, which is also not possible
-   with the current limitations.
-1. Not a priority on the short term, but scalar functions with variable number
-   of arguments, eg: `CONCAT('prefix', table1.col1, 'suffix')` and those which
-   accept multiple types, eg: `MAX(0.5, 1)` need more flexibility than what can
-   be currently described in the `CREATE FUNCTION` expression.
+1. In the long term, UDFs should be able to emit and receive Arrow data types like lists or [Structs](https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html#variant.Struct).
+1. Planned support for UDF window and aggregate functions require UDFs to be capable of retaining persistent state while processing the tuples of a single batch.
+1. Planned support for user-defined aggregate functions (UDAFs) require support for emitting a scalar result from input consisting of multiple tuples (e.g.: a table).
+1. Aggregate functions require Seafowl to implement [an interface](https://docs.rs/datafusion/latest/datafusion/physical_plan/udaf/struct.AggregateUDF.html) with several methods (currently registering a UDF allows only a single entrypoint into the WASM module).
+1. Planned support for UDF table functions require support for returning one or more tuples.
+1. Planned support for UDFs with variable number of arguments, e.g.: `CONCAT('prefix', table1.col1, 'suffix', ...)`.
+1. Planned support for UDFs with multiple support argument types, e.g.: `MAX(2, 1)`, `MAX(0.5, 0.7)` and `MAX(0.2, 8)`.
 
 # Terminology
 
-- UDF - A _User Defined Function_ which can be used in `SELECT` statements just
+- UDF: A _User Defined Function_ which can be used in SQL queries just
   like built-in functions.
 
+- Aggregate function: a function which returns a scalar value based on a set of tuples, e.g.: [`avg()`](https://github.com/apache/arrow-datafusion/blob/8dcef91806443f9b9b512bf6d819dc20961b29c8/datafusion/physical-expr/src/aggregate/average.rs).
+
+- [Window function](https://www.postgresql.org/docs/current/functions-window.html): a function with some persistent state which is shared between invocations. An example is `lag()`.
+
+- [Table function](https://docs.snowflake.com/en/sql-reference/functions-table.html#what-are-table-functions): A function returning potentially multiple tuples (a table) for each invocation.
+
 - WASI - WebAssembly System Interface - a standard set of functions which
   essentially act as a `libc` for `wasm` programs.
 
@@ -50,20 +58,9 @@ falls short in several important ways:
 A calling convention refers to the rules of invoking a function. In this case,
 it describes the following:
 
-1. _The semantics of function arguments and return values_. In order to receive
-   or return complex data types such as strings or arrays of tuples, the WASM
-   function must use its linear memory to communicate with the host process.
-   There are lots well-known calling conventions, for example, writing the
-   arguments to a segment of allocated WASM memory and passing the starting
-   pointer and length in bytes of the segment to the function as arguments.
-   Unfortunately, only a single value may be returned by functions, so this
-   method only works for passing input.
-1. _Data serialization format_. Complex data types such as `DATE` or `TEXT`
-   values, tuples or sets must be serialized into a bytestream before they can
-   be written to the WASM function's linear memory. `JSON`, `MessagePack`,
-   `Protocol Buffers`, etc. are all potential solutions.
-1. _Error messages and status codes_. In addition to function results, a UDF may
-   need to communicate warnings or error messages in case of failure.
+1. A method of exchanging arguments and return values between Seafowl and the UDF.
+1. A data serialization format.
+1. A mechanism for signaling errors.
 
 Ideally the calling convention for UDFs should be:
 
@@ -72,9 +69,7 @@ Ideally the calling convention for UDFs should be:
 - _simple to implement_, since UDFs may be written in a number of languages
   which compile to WASM and each will need its own implementation of the calling
   convention logic.
-- _flexible_ scalar UDFs' argument and return value data types are not known in
-  advance when Seafowl is compiled, the calling convention shouldn't care about
-  the specific types involved.
+- _flexible_ enough to allow addressing the issues listed in the "Motivation" section.
 
 ## Serialization format
 
@@ -85,10 +80,7 @@ much richer, including `SMALLINT` (aka `i16`), `CHAR`, `VARCHAR`, `TEXT`,
 `DECIMAL`, `BOOLEAN` and `TIMESTAMP`. Representing integers with floats may lead
 to unexpected rounding / representational issues. For this reason projects like
 [`node-pg`](https://node-postgres.com/) default to encoding 64-bit integers as
-strings. That said,
-[this proof-of-concept](https://github.com/splitgraph/rust-wasm-udf-demo/tree/main/experiments/wasi-streams-json-2)
-demonstrates that JSON can be used as a serialization format if representational
-issues are addressed.
+strings.
 
 Serialization formats with an explicit schema such as
 [Google's Protocol Buffers](https://developers.google.com/protocol-buffers)
@@ -110,18 +102,18 @@ this approach:
 - Only languages for which an Apache Arrow Flight binding exists can be used to
   write UDFs.
 - The WASM module must include Apache Arrow Flight with all its dependencies.
-- The data Seafowl Receives is columnar. For each column, Seafowl gets an array
+- The data Seafowl receives from DataFusion is columnar. For each column, Seafowl gets an array
   of column values, one for each row. UDFs expect row-like tuples. This
   transformation prevents us from using the original serialized Arrow data as
-  originally received. Theoretically, it could be possible to transpose the
+  originally received. While it could be possible to transpose the
   columnar input to row-tuples but keep the serialization of individual scalar
-  values, but this seems a lot more complex than what its worth.
+  values, this seems a lot more complex than what its worth.
 
 [MessagePack](https://msgpack.org/index.html) is an efficient binary format
 similar to JSON with support for many programming languages. MessagePack has
 first class support for the numeric types offered by Seafowl as well as strings,
 maps and arrays. Any tuple which can be currently `SELECT`-ed in Seafowl has a
-trivial mapping to a MessagePack-encoded object. WASM MessagePack
+trivial mapping to a MessagePack-encodable object. WASM MessagePack
 implementations include:
 
 - [msgpack-rust](https://github.com/3Hren/msgpack-rust) for Rust.
@@ -130,84 +122,37 @@ implementations include:
   [waPC project](https://wapc.io/).
 - [TinyGo](https://github.com/wapc/tinygo-msgpack) support also by waPC.
 
-## WASI as a standard for invocation
-
-While it's certainly
-[possible](https://github.com/splitgraph/rust-wasm-udf-demo/blob/b6f5ea574e0221af0f34d0f495bcaf052e99c591/experiments/raw-string-passing/src/main.rs#L90-L123)
-to define our own calling convention, it requires an implementation for any
-language a UDF may be written in. Using an existing solution which already
-supports a variety of languages gives Seafowl users options and saves us from
-reimplementing the wheel. If during development UDF authors need a way to
-execute their functions outside of Seafowl, we need to create this harness as
-well.
-
-The proclaimed "ultimate solution" for WASM function invocation is WebAssembly
-Interface Types (WIT), which includes its own IDL to describe function
-signatures. Unfortunately, WIT is still half-baked, using it in production is
-currently not recommended. Another drawback of WIT is that we can't use the WIT
-code generation tooling as intended, since Seafowl doesn't know the UDFs'
-signatures at compile time. If UDF input and output is MessagePack-encoded, all
-that needs to be passed is a stream of bytes. For that something like WIT seems
-like overkill: we just need to read and write bytestreams, the language-specific
-MessagePack library takes care of the rest.
-
-While Webassembly as an instruction set is a stable standard, supported by a
-number of runtimes both in browsers and on servers, it doesn't offer any builtin
-stream IO functions. To write to stdout, interact with the filesystem, read
-environment variables, etc, a host function exposing these capabilities must be
-registered with the WASM runtime which can be called by WASM functions.
-
-The need for such functions led to the creation of WASI, which provides a
-standard implementation of these functions often mimicking the POSIX C standard
-library's function names. Unix concepts such as `stdin`, `stdout` and `stderr`,
-exit codes and file handles may be used in programs which compile to WASM as
-long as the runtime provides a WASI implementation. WASI's goal is to make WASM
-a viable platform for server-side applications.
-[wasmer-js](https://github.com/wasmerio/wasmer-js) provides a solution for
-running WASI-consuming WASM programs in the browser, making a WASI a promising
-"run anywhere" standard.
-
-A very convient way to pass the streams to / from our WASM UDF is to read from
-`stdin` and write to `stdout`. This too is is implemented by storing the
-bytestream in WASM linear memory, but the interface hides these details behind
-the standard posix APIs.
-
-Unfortunately, unlinke WASM, WASI is still in it's infancy. Currently, there are
-two versions, `wasi_unstable` and `wasi_snapshot_preview1`. The version names
-are the name of modules housing WASI functions. Due to this namespacing, both
-WASI version modules may be present in the runtime simultaneously.
-
-## WASI module types
-
-First and foremost, WASI is a WASM module containing a set of
-[standard functions](https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/witx/wasi_snapshot_preview1.witx)
-providing a POSIX-like API for server-side WASM applications, but a basic notion
-of application lifecycle is necessary in many situations.
-
-There are two major types of WASI programs:
-
-- `Commands` Programs like commandline apps which are invoked, do some
-  processing, and release all resources when the process terminates. When
-  running commands, the runtime tries to execute the `_start` function (if no
-  other export is implicitly specified).
-- `Reactors` Programs like FaaS endpoints on Lambda / GCP Cloud Functions, etc,
-  where a function is initialized and then is invoked over and over. The
-  `_initialize` export is invoked when the module is loaded, after which any
-  export may be invoked any number of times. AFAIK, there is no standard
-  'cleanup' or destructor function, although Seafowl can certainly designate an
-  optional cleanup export name for UDFs.
-
-For more information on `Commands` vs `Reactors`, see:
-
-- https://github.com/WebAssembly/WASI/issues/13
-- https://github.com/bytecodealliance/wasmtime/issues/3474#issuecomment-951734116
-
-UDFs definitely seem like they should be `Reactors`, although the semantics of
-the `_initialize` function are not trivial. Once option is to ignore
-`_initialize` / require it to be a `noop`. If UDFs need such an initializer, it
-would be logical to run `_initialize` once for each transaction (not sure how
-hard it is to implement in DataFusion) or each `SELECT` statement (this is more
-in line with the current implementation).
+## Existing WASM function call solutions
+
+### Shared linear memory
+
+Conceptually, the simplest approach is to write UDF input arguments to a section of WASM linear memory and pass a pointer to it as the UDF argument. In a similar vein, the result can be written to linear memory, with a pointer to the result buffer returned by the UDF. Spoiler: this is the recommended approach. Advantages include low overhead, minimal dependencies and maximal flexibility.
+
+Drawbacks:
+- Requires reimplementing the lightweight host-guest protocol for reading intput, writing output and memory management in each supported language.
+- No type safety (UDF must validate input, host must validate returned values).
+
+### WIT
+
+In the long run, WebAssembly Interface Types (`WIT`) promise to provide an elegant solution to the problem of passing complex data between webassembly functions written in various high-level languages and the host. WIT includes an IDL, also called "wit" which can be used [for code generation](https://bytecodealliance.github.io/wit-bindgen/).
+
+There exists a very early pre-alpha [WIT implementation for rust](https://github.com/bytecodealliance/wit-bindgen) supporting both rust hosts and WASM guests. The developers urge everyone interested in using this in production to hold their horses and look for other alternatives while the WIT standard is finalized, I'd guess somewhere between 12 - 18 months from now.
+
+### WaPC
+
+The [waPC](https://github.com/wapc) project attempts to simplify wasm host-guest RPC. They provide a rust host and a number of supported guest languages. WaPC has its own GraphQL-inspired IDL language (WIDL). Based on GitHub activity, it seems to be an active project but lacks significant backing (written and used mostly by 3 guys at a startup formerly called Vino). WaPC uses MessagePack to serialize data by default.
+
+### WASM-bindgen
+[wasm-bindgen](https://crates.io/crates/wasm-bindgen) is an important project in the Rust WASM community. Its a mature solution for WASM RPC, but unfortunately limited to JavaScript host -> Rust WASM module guest calls. There [was](https://github.com/bytecodealliance/wasmtime/issues/677) experimental support for WIT, but its not longer supported. In a future where WIT support returns, `wasm-bindgen` could be an ergonomic route to UDFs with complex inputs / outputs. Currently the [guide on using it with rust hosts](https://github.com/bytecodealliance/wasmtime/blob/main/docs/wasm-rust.md) does not work as advertised.
+
+### WASI-based communication
+
+The WebAssembly System Interface is an extension to WASM providing an interface to module functions for interacting with the host filesystem, command line arguments, environment variables, etc.
+Like most things WASM-related, WASI itself is still in it's infancy and subject to change. Currently, there are two versions, `wasi_unstable` and `wasi_snapshot_preview1`. These version names are also the names of modules housing WASI functions. Thanks to this namespacing, both WASI version modules may be present in the runtime simultaneously.
+
+Using WASI's `stdin` stream to pass function arguments and `stdout` to receive return values seemed like a promising solution. Unfortuantely, I didn't find a convenient way to reuse these streams for multiple invocations, and recreating the entire WASM instance resulted in too much overhead.
+
+Being able to print to `stderr` was a huge help in debugging UDFs during development. For this, I think we should continue to use WASI. Although initially meant to be used on the backend, [wasmer-js](https://github.com/wasmerio/wasmer-js) provides a solution for running WASI-consuming WASM programs in the browser, making a WASI a promising "run anywhere" standard.
 
 ## UDF function definition
 
@@ -240,9 +185,9 @@ Fields:
 
 ## UDF languages
 
-Proof-of-concept WASM+WASI+MessagePack implementations of a
-`concat(string, string) -> string` function exist for several languages, with
-module sizes differing by orders of magnitude:
+Proof-of-concept implementations of a
+`concat(string, string) -> string` UDF using linear memory to share data between Seafowl and the WASM module exist for several languages with
+significant differences in module size and performance.
 
 - Rust: currently 2Mb, but
   [this may be improved](https://rustwasm.github.io/docs/book/reference/code-size.html).
@@ -252,8 +197,8 @@ module sizes differing by orders of magnitude:
 Fortunately, Seafowl has no problem registering even the multi-Mb Rust module
 UDF, no query size limit prevented this so far.
 
-The interface between Seafowl and UDF is the programming language's MessagePack
-implementation (plus a thin wrapper which reads `stdin` and writes `stdout`).
+The interface between Seafowl and UDF is the mostly the programming language's MessagePack
+implementation.
 
 ## Error messages, status codes
 
@@ -346,3 +291,9 @@ UDFs, but would go a long way for the overall UDF experience.
 - How do Seafowl users receive warning messages emitted by UDFs if a result
   value can be computed? Some options are leaving such warning unsupported (any
   warning is an error), or displaying them only in the seafowl access log.
+
+# References
+
+* [Intro to WASM linear memory](https://radu-matei.com/blog/practical-guide-to-wasm-memory/) - well written, but includes some bugs in the code samples.
+* [Rust UDF code sample](https://wasmedge.org/book/en/sdk/go/memory.html)
+* [Blog post on using WASI to pass args to WASM functions](https://blog.scottlogic.com/2022/04/16/wasm-faas.html)