Add Open Graph link unfurler service

unfurler/.editorconfig

@@ -0,0 +1,17 @@
+# Editor configuration, see https://editorconfig.org
+root = true
+
+[*]
+charset = utf-8
+indent_style = space
+indent_size = 2
+insert_final_newline = true
+trim_trailing_whitespace = true
+
+[*.ts]
+quote_type = single
+
+[*.md]
+max_line_length = off
+trim_trailing_whitespace = false
+

unfurler/.eslintignore

@@ -0,0 +1,2 @@
+node_modules
+dist

unfurler/.eslintrc

@@ -0,0 +1,33 @@
+{
+  "root": true,
+  "parser": "@typescript-eslint/parser",
+  "plugins": [
+    "@typescript-eslint"
+  ],
+  "extends": [
+    "eslint:recommended",
+    "plugin:@typescript-eslint/eslint-recommended",
+    "plugin:@typescript-eslint/recommended",
+    "prettier"
+  ],
+  "rules": {
+    "@typescript-eslint/ban-ts-comment": 1,
+    "@typescript-eslint/ban-types": 1,
+    "@typescript-eslint/no-empty-function": 1,
+    "@typescript-eslint/no-explicit-any": 1,
+    "@typescript-eslint/no-inferrable-types": 1,
+    "@typescript-eslint/no-namespace": 1,
+    "@typescript-eslint/no-this-alias": 1,
+    "@typescript-eslint/no-var-requires": 1,
+    "no-console": 1,
+    "no-constant-condition": 1,
+    "no-dupe-else-if": 1,
+    "no-empty": 1,
+    "no-prototype-builtins": 1,
+    "no-self-assign": 1,
+    "no-useless-catch": 1,
+    "no-var": 1,
+    "prefer-const": 1,
+    "prefer-rest-params": 1
+  }
+}

unfurler/.gitignore

@@ -0,0 +1,38 @@
+# See http://help.github.com/ignore-files/ for more about ignoring files.
+
+# production config and external assets
+config.json
+
+# compiled output
+/dist
+/tmp
+
+# dependencies
+/node_modules
+
+# IDEs and editors
+/.idea
+.project
+.classpath
+.c9/
+*.launch
+.settings/
+
+# IDE - VSCode
+.vscode/*
+!.vscode/settings.json
+!.vscode/tasks.json
+!.vscode/launch.json
+!.vscode/extensions.json
+
+# misc
+/.sass-cache
+/connect.lock
+/libpeerconnection.log
+npm-debug.log
+testem.log
+/typings
+
+#System Files
+.DS_Store
+Thumbs.db

unfurler/.prettierignore

@@ -0,0 +1,2 @@
+node_modules
+package-lock.json

unfurler/.prettierrc

@@ -0,0 +1,6 @@
+{
+  "endOfLine": "lf",
+  "printWidth": 80,
+  "tabWidth": 2,
+  "trailingComma": "es5"
+}

unfurler/README.md

@@ -0,0 +1,91 @@
+# Mempool Link Unfurler Service
+
+This is a standalone nodejs service which implements the [Open Graph protocol](https://ogp.me/) for Mempool instances. It performs two main tasks:
+
+1. Serving Open Graph html meta tags to social media link crawler bots.
+2. Rendering link preview images for social media sharing.
+
+Some additional server configuration is required to properly route requests (see section 4 below).
+
+## Setup
+
+### 1. Clone Mempool Repository
+
+Get the latest Mempool code:
+
+```
+git clone https://github.com/mempool/mempool
+cd mempool
+```
+
+Check out the latest release:
+
+```
+latestrelease=$(curl -s https://api.github.com/repos/mempool/mempool/releases/latest|grep tag_name|head -1|cut -d '"' -f4)
+git checkout $latestrelease
+```
+
+### 2. Prepare the Mempool Unfurler
+
+#### Install
+
+Install dependencies with `npm` and build the backend:
+
+```
+cd unfurler
+npm install
+```
+
+The npm install may fail if your system does not support automatic installation of Chromium for Puppeteer. In that case, manually install Puppeteer without Chromium first:
+```
+PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true npm install puppeteer
+npm install
+```
+
+#### Configure
+
+In the `unfurler` folder, make a copy of the sample config file:
+
+```
+cp config.sample.json config.json
+```
+
+Edit `config.json` as needed:
+
+| variable | usage |
+|---|---|
+| SERVER.HOST | the host where **this** service will be served |
+| SERVER.HTTP_PORT | the port on which **this** service should run |
+| MEMPOOL.HTTP_HOST | the host where **the Mempool frontend** is being served |
+| MEMPOOL.HTTP_PORT | the port on which **the Mempool frontend** is running (or `null`) |
+| PUPPETEER.CLUSTER_SIZE | the maximum number of Chromium browser instances to run in parallel, for rendering link previews |
+| PUPPETEER.EXEC_PATH | (optional) an absolute path to the Chromium browser executable, e.g. `/usr/local/bin/chrome`. Only required when using a manual installation of Chromium |
+
+#### Build
+
+```
+npm run build
+```
+
+### 3. Run the Mempool Unfurler
+
+```
+npm run start
+```
+
+### 4. Server configuration
+
+To enable social media link previews, the system serving the Mempool frontend should detect requests from social media crawler bots and proxy those requests to this service instead.
+
+Precise implementation is left as an exercise to the reader, but the following snippet may be of some help for Nginx users:
+```Nginx
+map $http_user_agent $crawler {
+    default 0;
+    ~*facebookexternalhit 1;
+    ~*twitterbot 1;
+    ~*slackbot 1;
+    ~*redditbot 1;
+    ~*linkedinbot 1;
+    ~*pinterestbot 1;
+}
+```

unfurler/config.sample.json

@@ -0,0 +1,14 @@
+{
+  "SERVER": {
+    "HOST": "http://localhost",
+    "HTTP_PORT": 4201
+  },
+  "MEMPOOL": {
+    "HTTP_HOST": "http://localhost",
+    "HTTP_PORT": 4200
+  },
+  "PUPPETEER": {
+    "CLUSTER_SIZE": 2,
+    "EXEC_PATH": "/usr/local/bin/chrome" // optional
+  }
+}