Consider whether Digest requirement is MUST, MAY, SHOULD or unnecessary

[alexdutton]

The original spec was RFC 3230 from 2002, and there's a draft update in progress as draft-ietf-httpbis-digest-headers.

As most/all implementations will/should make SWORD available over HTTPS, which presumably handles transport integrity. A client ensuring that it transmits the file it intends to transmit is not solved by hashing the file immediately before transmission, as the file may have been corrupted before then.

Is this therefore a necessary or useful part of the spec?

[richard-jones]

The new draft for Digest says:

The core specification of HTTP does not define a means to protect the
   integrity of resources.  When HTTP messages are transferred between
   endpoints, the protocol might choose to make use of features of the
   lower layer in order to provide some integrity protection; for
   instance TCP checksums or TLS records [RFC2818].

   However, there are cases where relying on this alone is insufficient.
   An HTTP-level integrity mechanism that operates independent of
   transfer can be used to detect programming errors and/or corruption
   of data at rest, be used across multiple hops in order to provide
   end-to-end integrity guarantees, aid fault diagnosis across hops and
   system boundaries, and can be used to validate integrity when
   reconstructing a resource fetched using different HTTP connections.

   This document defines a mechanism that acts on HTTP representation-
   data.  It can be combined with other mechanisms that protect
   representation-metadata, such as digital signatures, in order to
   protect the desired parts of an HTTP exchange in whole or in part.

There are a couple of points here that are relevant to SWORD:

• HTTPS does provide transfer integrity, HTTP does not. SWORD does not require HTTPS (though it RECOMMENDS). If we decide to remove the Digest header, we need to make the point about transfer integrity somewhere.

• Digests are very common in the kinds of systems being used here. It is common for a repo to store and check content digests regularly to monitor bit-rot, etc. The text from the rfc above alludes to this use case specifically - it's not about transfer, it's about maintaining integrity across the whole process. From that perspective, Digests are worthwhile for SWORD, because it may involve the transfer of data that has already been checksummed. Is this a case for keeping the Digest, and in which case to what level of requirement?

• I'm cautious about introducing an optional feature. The more optional features, the less interoperable two systems will be. If we make this optional it will have to be a choice for the client to provide the Digest, and the server shouldn't have to require a digest; that will eliminate the opportunity for a mismatch in compliance between servers and clients.

@alexsdutton do you recall during implementation what else we discussed regarding this header?