Proposal: WASI Observability API

[chrisdickinson]

Hi folks! I've been working with my colleagues at Dylibso to port our observe-sdk bindings from core modules to the component model. We've started defining an interface that captures the observe-sdk's capabilities from a component perspective. The relevant WIT section from that PR follows:

package dylibso:observe;

interface api {
  enum log-level {
    error,
    warn,
    info,
    debug,
    trace
  }

  enum metric-format {
    statsd,
  }

  metric: func(format: metric-format, name: list<u8>);
  log: func(level: log-level, msg: list<u8>);
  span-enter: func(name: string);
  span-tags: func(tags: string);
  span-exit: func();
}

We're interested in starting a conversation about what observability bindings look like at the WASI level. The observe-sdk takes the position of capturing high-level author intent (span creation, logging) while allowing the host to direct those intents to backing providers (via otel, datadog, honeycomb, zipkin, or lightstep.) We have the additional constraint that we'd like to continue to provide direct core module support for this interface, which others may or may not share.

We're interested in hearing what others think about this approach!

[pchickey]

This sounds like a proposal that is in scope for WASI, so the next step is to make it a phase 1 proposal: https://github.com/WebAssembly/WASI/blob/main/Contributing.md#1-feature-proposal-wasi-subgroup

To meet those entry requirements, please schedule a presentation and vote at a WASI subgroup meeting after Jan 11, by making a PR to https://github.com/webassembly/meetings

[oovm]

I have some disagreements with log-level, if you use enum to lock these levels, then other applications will have to force matching to these levels.

As far as I know, many frameworks and applications use six levels.

• For example, log4j uses (FATAL, ERROR, WARN, INFO, DEBUG, TRACE),

• Microsoft.Extensions.Logging uses (Critical, Error, Warning, Information, Debug, Trace)

there are more

• Syslog uses seven levels
• Apache Httpd uses ten levels

---

I'm not saying that enum should be expanded to more, what I'm saying is that for compatibility reasons, and to support various rural frameworks, log-level should be a number.

This number is ordered, and the user can choose to ignore all events less than a certain number.

And the authors of the framework can agree on several key levels, using specific numbers.

For service providers, there is actually no need for so many levels, and the logs can be reduced to the minimum requirements.

[oovm]

Consider a scenario where someone has built a game bug summary platform, and now there are two roles, the sender and the receiver.

• Sender: players, game developers, developing various games in various languages.
• Recipient: Development server using a single language, which uses five levels.

---

Suppose Game A uses a language with 3 levels

enum GameA {
    Error = 255, 
    Warning = 100, 
    Debug = 0, 
}

Game B uses a language with 6 levels.

enum GameB {
    FATAL = 255, 
    ERROR = 200, 
    WARN = 150, 
    INFO = 100, 
    DEBUG = 50, 
    TRACE = 0
}

The framework used for service providers has 5 levels.

enum Server {
    error = 255,
    warn  = 200,
    info  = 100,
    debug = 50,
    trace = 0
}

Then the following protocol cast will actually occur:

GameA::Error    => Server::error
GameA::Warning  => Server::warn
GameA::Debug    => Server::trace
GameB::FATAL    => Server::error
GameB::ERROR    => Server::warn
GameB::WARN     => Server::warn
GameB::INFO     => Server::info
GameB::DEBUG    => Server::debug
GameB::TRACE    => Server::trace

Then the service provider can use the specification levels to generate various reports without knowing what language, what framework, and how many levels the sender uses.

---

Considering the most extreme case, where the game is compiled into the browser and the statistical functions are also run on the cloud, then such a set of facilities may be needed:

type log-level = u8;

record logging {
    level: log-level,
    message: string
}

/// There may be multiple different recipients, so it is recommended to use resource
/// For example, send a copy to the local file system, game developers and monitoring platforms.
resource log-outgoing {
    send-log(level: logging);
}


resource log-incoming {
    /// Register a monitoring level and listen to all logs above this level and below the next level
    /// Then write to database or write to filesystem or others 
    register(level: log-level, poll: pollable)
}

[pchickey]

@oovm thanks for this feedback, the scope of this issue was to introduce wasi-observe which now has its own repository https://github.com/webassembly/wasi-observe, would you mind redirecting these comments to an issue there, or to https://github.com/webassembly/wasi-logging as appropriate?

[oovm]

@pchickey Can you move this issue to wasi-observe? I don't know who has permission to move issues in org.

---

My suggestion is to focus on wasi-observe, not wasi-logging.

The two are not quite the same, as proposed here #7.

Although some languages confuse the two, they actually have different focuses.

So I think a distinction should be made.