Allow compression of url

[Akeboshiwind]

I've found some examples hit the "URI too long" warning on safari.
Compressing the URL could help with this:

• http://nodeca.github.io/pako/
• https://pieroxy.net/blog/pages/lz-string/index.html

[Akeboshiwind]

Did a rough check with pako (picked at random) and a large set of txs + a query was compressed to ~10% of it's original size.
Roughly what I expected so definitely worth it for at least working with design partners.

[Akeboshiwind]

Hit into a need for this today had a quite long URL just get rejected by Cloudfront

[Akeboshiwind]

Apparently CloudFront has a hard limit of Maximum length of a URL: 8,192 bytes

[Akeboshiwind]

Just wanted to record a local test I did with a large test-string:

Code

(ns test
  (:require [clojure.string :as str]
            ["lz-string" :as lz-string]
            ["fflate" :as fflate]))

(defn time+out [f]
  (let [out (atom nil)
        t (with-out-str
            (time (reset! out (f))))]
    (str (str/trim t) ": " (count @out))))

(def common-alphabet "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")
(def base64-alphabet (str common-alphabet "+/"))
(def base64url-alphabet (str common-alphabet "-_"))

;; GPT translated from: https://github.com/zloirock/core-js/blob/167136f479d3b8519953f2e4c534ecdd1031d3cf/packages/core-js/modules/esnext.uint8-array.to-base64.js
(defn to-base64
  ([uint8-array] (to-base64 uint8-array {}))
  ([uint8-array options]
   (let [alphabet (if (= (:alphabet options) "base64") 
                   base64-alphabet 
                   base64url-alphabet)
         omit-padding? (:omit-padding options)
         length (.-length uint8-array)
         
         get-char (fn [triplet shift]
                   (nth alphabet (bit-and (bit-shift-right triplet (* 6 shift)) 63)))]

     (loop [i 0
            result ""]
       (cond
         (< (+ i 2) length)
         (let [triplet (+ (bit-shift-left (aget uint8-array i) 16)
                         (bit-shift-left (aget uint8-array (+ i 1)) 8)
                         (aget uint8-array (+ i 2)))]
           (recur (+ i 3)
                  (str result 
                       (get-char triplet 3)
                       (get-char triplet 2)
                       (get-char triplet 1)
                       (get-char triplet 0))))

         (= (+ i 2) length)
         (let [triplet (+ (bit-shift-left (aget uint8-array i) 16)
                         (bit-shift-left (aget uint8-array (+ i 1)) 8))]
           (str result 
                (get-char triplet 3)
                (get-char triplet 2)
                (get-char triplet 1)
                (if omit-padding? "" "=")))

         (= (+ i 1) length)
         (let [triplet (bit-shift-left (aget uint8-array i) 16)]
           (str result 
                (get-char triplet 3)
                (get-char triplet 2)
                (if omit-padding? "" "==")))

         :else result)))))

(comment
  (println
    (str "base: " (count test-string)
         "\nbtoa: " (time+out #(js/btoa test-string))
         "\nlz-string: " (time+out #(lz-string/compressToEncodedURIComponent test-string))
         "\ngzip: " (time+out #(to-base64 (fflate/gzipSync test-string)
                                          {:alphabet "base64url"
                                           :omit-padding true}))
         "\nzlibSync: " (time+out #(to-base64 (fflate/zlibSync test-string)
                                              {:alphabet "base64url"
                                               :omit-padding true}))
         "\nzlibSync 9: " (time+out #(to-base64 (fflate/zlibSync test-string #js{:level 9})
                                                {:alphabet "base64url"
                                                 :omit-padding true})))))

Results

; (out) base: 11026
; (out) btoa: "Elapsed time: 0.000000 msecs": 14704
; (out) lz-string: "Elapsed time: 19.000000 msecs": 6064
; (out) gzip: "Elapsed time: 119.000000 msecs": 4175
; (out) zlibSync: "Elapsed time: 508.000000 msecs": 4159
; (out) zlibSync 9: "Elapsed time: 1366.000000 msecs": 2318

compression	time (ms)	final size (char count)	ratio
base	0	11026	1
btoa	0	14704	1.33
lz-string	19	6064	0.55
gzip	119	4175	0.38
zlibSync	508	4159	0.38
zlibSync 9	1366	2318	0.21