storage.get(key, { type })

[pi0]

Describe the feature

Currently, unstorage has two ways to access items:

• getItem(key): Returns a json parsed value if parsable or fallback to text
• `getItemRaw(key): Returns raw storage value (Typically arrayBuffer but unpredictable)

This is an unpredictable behavior and also introduced two sets of methods.

We could support storage.get(key, { type: "text" | "json" | "bytes" | "stream" | "blob" }) API to unify this and also make behavior more predictable.

await storage.get("key") // Defaults to json > text as before using destr
await storage.get("key", { type: "text" )  // Avoid serialization
await storage.get("key", { type: "json" )  // Strictly use JSON.parse
await storage.get("key", { type: "bytes" ) // Return Buffer or Unit8Array
await storage.get("key", { type: "stream" ) // Return a readable stream

undio can potentially be integrated or we might end-up directly adding small utils in unstorage core to convert

Additional information

• Would you be willing to help implement this feature?

[cjpearson]

This looks like a nice improvement to me. I suppose the typing would also be improved by looking at the type option? Does it make sense to also include blob as a type?

[lishaduck]

Instead of using type, what about integrating with standard-schema? That'd let you use zod, valibot, etc, parsers without a) pulling them in, b) picking a specific one, but allows users to do complex validations.

[pi0]

Validation is another topic (for object/json), these are data types.

[sandros94]

Oh, I didn't notice this!

What is the current plan for the actual data type within the storage itself? Personally I think this should be customizable depending on the actual project's requirement. But what default?

[sandros94]

Also, could this be the correct time to add optional compression?

Recently, as I was experimenting with custom drivers with built-in compression, I've noticed that I could directly return the compressed data (1:1 from the storage) as the response from Nitro event handlers (while also adding correct headers). This became particularly useful for me because I could cache event handlers and Nuxt pages more efficiently (about 2/3 of server memory saved. No benchmark yet, just watching the resource monitor every few hours)

[cjpearson]

I don't know if you've already seen it, but there is an open issue for compression: #94

I've also been using a custom driver to implement compression for unstorage and using it as the backing store for Nitro's page cache.

But I think in this case it's not the best approach. Compressing the response in Nitro and storing it as-is in the cache storage makes it straightforward to directly send the compressed response without decompressing. To do this with an unstorage driver would require an asymmetric behavior where you compress on set, but do not decompress on get.

[sandros94]

To do this with an unstorage driver would require an asymmetric behavior where you compress on set, but do not decompress on get.

indeed I don't use get but getRaw in my current approach. With this rewrite it could be simplified to type: 'compressed'