Spring Cloud Stream Dapr Binder Reference Guide

1. Usage

To use Dapr Binder, you need to clone this project and deploy it into your local maven repository.

git clone [email protected]:microsoft/spring-cloud-stream-binder-dapr.git
./mvnw clean install

Then you need add spring-cloud-stream-binder-dapr as a dependency to your Spring Cloud Stream application, as shown in the following example for Maven:

<dependency>
   <groupId>com.azure.spring</groupId>
   <artifactId>spring-cloud-stream-binder-dapr</artifactId>
   <version>1.0.0-SNAPSHOT</version>
</dependency>

Alternatively, you can use the spring-cloud-starter-stream-dapr, as follows:

<dependency>
   <groupId>com.azure.spring</groupId>
   <artifactId>spring-cloud-starter-stream-dapr</artifactId>
   <version>1.0.0-SNAPSHOT</version>
</dependency>

2. Spring Cloud Stream Dapr Binder Overview

Dapr is an event-driven, portable distributed application runtime, which provides the execution environment that distributed applications need to rely on.

As a runtime, Dapr can build our microservices on the cloud or edge computing and provides a unified API interface call for applications upward, making Dapr supporting different programming languages and frameworks.

Dapr also provides our developers with standard APIs and pluggable and replaceable components to organize various distributed capabilities for our applications, and encapsulate these distributed capabilities as runtimes to simplify distributed application development.

Spring Cloud Stream establishes a connection between the application and the middleware, separates the logic of sending and receiving messages with specific middleware from the application, and is finally provided by Binder. Binder shields the details of the underlying MQ message middleware and decouples our application from the specific message middleware.

In addition, Spring Cloud Stream abstracts the MQ in the Spring Cloud system and provides a unified API for sending and receiving messages, in order to simplify the development of messages in Spring Cloud applications.

Combining Spring Cloud Stream and Dapr, on the one hand Dapr can make up the dependent on a specific middleware library for Spring Cloud Stream, on the other hand, Spring Cloud Stream can help application and Dapr client decoupling. Therefore, the combination of the two is conducive to the maximum decoupling and enhanced features of both sides.

The ultimate goal and vision of Spring Cloud Stream Dapr Binder(Dapr Binder) is to strip all non-business logic parts from the application, so that developers can pay more attention to the development of business logic.

In Dapr Binder, we regard the entire Dapr Sidecar as an MQ. When sending messages, we call Grpc’s Client, and when receiving messages, we create a Grpc’s Server and open two interfaces to monitor Dapr’s calls. One is to tell Dapr, which topics have I subscribed to, and the other is to receive messages from the subscribed topics. This is what we at Dapr Binder do.

3. Configuration Options

This section contains the configuration options used by the Dapr Binder.

For common configuration options and properties pertaining to the binder, see the binding properties in core documentation.

3.1. Dapr Binder Properties

The following properties are available for Dapr Binder only and must be prefixed with spring.cloud.stream.dapr.binder..

spring.cloud.stream.dapr.binder.daprIp

The parameter of the channel, indicating the IP address of the dapr sidecar to which the message is finally sent through the channel.

Default: 127.0.0.1

spring.cloud.stream.dapr.binder.daprPort

The parameter of channel, indicating the port that the dapr sidecar to which the message is finally sent through the channel listens.

Default: 50001

spring.cloud.stream.dapr.binder.negotiationType

NegotiationType decide which connection method to use. TLS and PLAINTEXT are currently available.

• PLAINTEXT: Use of a plaintext connection to the server. By default a secure connection mechanism such as TLS will be used. Should only be used for testing or for APIs where the use of such API or the data exchanged is not sensitive. This assumes prior knowledge that the target of this channel is using plaintext. It will not perform HTTP/1.1 upgrades.

• TLS: Makes the client use TLS.

  Default: PLAINTEXT

spring.cloud.stream.dapr.binder.authority

Overrides the authority used with TLS and HTTP virtual hosting. It does not change what host is actually connected to. Is commonly in the form host:port.

This method is intended for testing, but may safely be used outside of tests as an alternative to DNS overrides.

spring.cloud.stream.dapr.binder.defaultLoadBalancingPolicy

Sets the default load-balancing policy that will be used if the service config doesn’t specify one.

This method is implemented by all stock channel builders that are shipped with gRPC, but may not be implemented by custom channel builders, in which case this method will throw.

spring.cloud.stream.dapr.binder.idleTimeout

Set the duration without ongoing RPCs before going to idle mode.

In idle mode the channel shuts down all connections, the NameResolver and the LoadBalancer. A new RPC would take the channel out of idle mode. A channel starts in idle mode. Defaults to 30 minutes.

This is an advisory option. Do not rely on any specific behavior related to this option.

TimeUnit: minutes

spring.cloud.stream.dapr.binder.keepAliveTime

Sets the time without read activity before sending a keepalive ping. An unreasonably small value might be increased, and Long.MAX_VALUE nano seconds or an unreasonably large value will disable keepalive. Defaults to infinite.

Clients must receive permission from the service owner before enabling this option. Keepalives can increase the load on services and are commonly "invisible" making it hard to notice when they are causing excessive load. Clients are strongly encouraged to use only as small of a value as necessary.

TimeUnit: minutes

spring.cloud.stream.dapr.binder.keepAliveTimeout

Sets the time waiting for read activity after sending a keepalive ping. If the time expires without any read activity on the connection, the connection is considered dead. An unreasonably small value might be increased. Defaults to 20 seconds.

This value should be at least multiple times the RTT to allow for lost packets.

TimeUnit: seconds

spring.cloud.stream.dapr.binder.perRpcBufferLimit

Sets the per RPC buffer limit in bytes used for retry. The RPC is not retriable if its buffer limit is exceeded. The implementation may only estimate the buffer size being used rather than count the exact physical memory allocated. It does not have any effect if retry is disabled by the client.

This method may not work as expected for the current release because retry is not fully implemented yet.

spring.cloud.stream.dapr.binder.retryBufferSize

Sets the retry buffer size in bytes. If the buffer limit is exceeded, no RPC could retry at the moment, and in hedging case all hedges but one of the same RPC will cancel. The implementation may only estimate the buffer size being used rather than count the exact physical memory allocated. The method does not have any effect if retry is disabled by the client.

This method may not work as expected for the current release because retry is not fully implemented yet.

spring.cloud.stream.dapr.binder.keepAliveWithoutCalls

Sets whether keepalive will be performed when there are no outstanding RPC on a connection. Defaults to false.

Clients must receive permission from the service owner before enabling this option. Keepalives on unused connections can easilly accidentally consume a considerable amount of bandwidth and CPU. idleTimeout() should generally be used instead of this option.

spring.cloud.stream.dapr.binder.maxInboundMessageSize

Sets the maximum message size allowed to be received on the channel. If not called, defaults to 4 MiB. The default provides protection to clients who haven’t considered the possibility of receiving large messages while trying to be large enough to not be hit in normal usage.

This method is advisory, and implementations may decide to not enforce this. Currently, the only known transport to not enforce this is InProcessTransport.

spring.cloud.stream.dapr.binder.maxInboundMetadataSize

Sets the maximum size of metadata allowed to be received. Integer.MAX_VALUE disables the enforcement. The default is implementation-dependent, but is not generally less than 8 KiB and may be unlimited.

This is cumulative size of the metadata. The precise calculation is implementation-dependent, but implementations are encouraged to follow the calculation used for HTTP/2’s SETTINGS_MAX_HEADER_LIST_SIZE . It sums the bytes from each entry’s key and value, plus 32 bytes of overhead per entry.

spring.cloud.stream.dapr.binder.maxRetryAttempts

Sets the maximum number of retry attempts that may be configured by the service config. If the service config specifies a larger value it will be reduced to this value. Setting this number to zero is not effectively the same as disableRetry() because the former does not disable transparent retry .

This method may not work as expected for the current release because retry is not fully implemented yet.

spring.cloud.stream.dapr.binder.maxHedgedAttempts

Sets the maximum number of hedged attempts that may be configured by the service config. If the service config specifies a larger value it will be reduced to this value.

This method may not work as expected for the current release because retry is not fully implemented yet.

spring.cloud.stream.dapr.binder.maxTraceEvents

Sets the maximum number of channel trace events to keep in the tracer for each channel or subchannel. If set to 0, channel tracing is effectively disabled.

3.2. Dapr Producer Properties

The following properties are available for Dapr producers only and must be prefixed with spring.cloud.stream.dapr.bindings.<bindingTarget>.producer..

pubsubName

Specifies the name of the Pub/Sub component.

Note	PubsubName must be specified and has no default value.

3.3. Dapr Consumer Properties

The following properties are available for Dapr consumers only and must be prefixed with spring.cloud.stream.dapr.bindings.<bindingTarget>.consumer..

pubsubName

Specifies the name of the Pub/Sub component.

Note	PubsubName must be specified and has no default value.

3.4. Dapr Grpc Service Properties

Dapr Binder receives messages by starting a Grpc Server, and Grpc Server depends on the grpc-spring-boot-starter dependency. The relevant parameter configuration can be referred to Configuration of Grpc Spring Boot Starter.

3.5. Dapr Message Headers

The following table illustrates how Dapr message properties are mapped to Spring message headers.

Dapr Message Properties	Spring Message Header Constants	Type	Description
contentType	DaprHeaders#CONTENT_TYPE	String	The contentType tells Dapr which content type your data adheres to when constructing a CloudEvent envelope.
ttlInSeconds	DaprHeaders#TTL_IN_SECONDS	Long	The number of seconds for the message to expire.
rawPayload	DaprHeaders#RAW_PAY_LOAD	Boolean	Determine if Dapr should publish the event without wrapping it as CloudEvent. Not using CloudEvents disables support for tracing, event deduplication per messageId, content-type metadata, and any other features built using the CloudEvent schema.
specifiedBrokerMetadata	DaprHeaders#SPECIFIED_Broker_METADATA	Map<String, String>	Some metadata parameters are available based on each pubsub broker component.

4. Migration Guide

4.1 Migration from Spring Cloud Azure Stream Binder Event Hubs

1. Update dependency

Remove spring-cloud-azure-stream-binder-eventhubs dependencies.

<dependency>
  <groupId>com.azure.spring</groupId>
  <artifactId>spring-cloud-azure-stream-binder-eventhubs</artifactId>
</dependency>

Add spring-cloud-stream-binder-dapr dependencies.

<dependency>
  <groupId>com.azure.spring</groupId>
  <artifactId>spring-cloud-stream-binder-dapr</artifactId>
</dependency>

2. Update application.yaml

• Remove all configurations prefixed with spring.cloud.azure.eventhubs..

• Remove all configurations prefixed with spring.cloud.stream.eventhubs..

• Add configurations prefixed with spring.cloud.stream.dapr. and specify pubsubName.

The final pubsub.yaml is as follows:

• spring.cloud.stream.bindings.<binding>.destination is configured the topic specified for sending messages.

• spring.cloud.stream.dapr.bindings.<binding>.producer.pubsubName is configured the name of the Pub/Sub component specified for sending messages.

• spring.cloud.stream.dapr.bindings.<binding>.consumer.pubsubName is configured the name of the Pub/Sub component specified for receiving messages.

spring:
  cloud:
    stream:
      function:
        definition: supply;consume
      bindings:
        supply-out-0:
          destination: <AZURE_EVENTHUB_NAME>
        consume-in-0:
          destination: <AZURE_EVENTHUB_NAME>
      dapr:
        bindings:
          supply-out-0:
            producer:
              pubsubName: eventhubs-pubsub
          consume-in-0:
            consumer:
              pubsubName: eventhubs-pubsub

3. Configure Azure Event Hubs component

Dapr integrates with Pub/Sub message buses to provide applications with the ability to create event-driven, loosely coupled architectures where producers send events to consumers via topics.

Dapr supports the configuration of multiple, named, Pub/Sub components per application. Each Pub/Sub component has a name and this name is used when publishing a message topic.

Pub/Sub components are extensible. A list of support Pub/Sub components is here and the implementations can be found in the components-contrib repo.

In this example, we configure the Azure Event Hubs Pub/Sub component described using the pubsub.yaml file:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: eventhubs-pubsub
spec:
  type: pubsub.azure.eventhubs
  version: v1
  metadata:
    - name: connectionString
      value: "<AZURE_CONNECTION_STRING>"
    - name: storageAccountName
      value: "<AZURE_STORAGE_ACCOUNT_NAME>"
    - name: storageAccountKey
      value: "<AZURE_STORAGE_ACCOUNT_KEY>"
    - name: storageContainerName
      value: "<AZURE_STORAGE_CONTAINER_NAME>"

Follow the instructions here to manage the storage account access keys. See here on how to get the Event Hubs connection string.

4. Run application with Dapr sidecar

dapr run --app-id dapr-app --app-port 9090 --components-path ${componentsPath}  --app-protocol grpc --dapr-grpc-port ${daprPort} mvn spring-boot:run

This command specifies:

• the id for your application with --app-id dapr-app, used for service discovery.

• the port your application is listening on (default -1) with --app-port 9090.

• the path for components directory with --components-path ./components.

• the protocol (gRPC or HTTP) Dapr with --app-protocol grpc uses to talk to the application.

• the gRPC port for Dapr to listen on (default -1) with --dapr-grpc-port 50001

5. Clean Up

To stop your services from running, simply stop the dapr run process. Alternatively, you can spin down your services with the Dapr CLI dapr stop command.

dapr stop --app-id dapr-app

Appendices

In the maven of the project we use Spring Cloud Stream as parent, thus to build it locally, you can refer to Spring Cloud Stream.