Skip to content
This repository has been archived by the owner on Jul 4, 2024. It is now read-only.

Commit

Permalink
refactor proxy
Browse files Browse the repository at this point in the history
  • Loading branch information
@tomoyane
tomoyane committed Apr 1, 2024
1 parent be6fd01 commit c216a27
Show file tree
Hide file tree
Showing 5 changed files with 57 additions and 39 deletions.
4 changes: 3 additions & 1 deletion Dockerfile.amd64
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
FROM node:18-slim

# See: https://github.com/puppeteer/puppeteer/tree/main/docker
# Install chrome on amd64
RUN apt-get update \
&& apt-get install -y wget gnupg python3 make g++ ca-certificates \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | gpg --dearmor -o /usr/share/keyrings/googlechrome-linux-keyring.gpg \
Expand All @@ -20,4 +22,4 @@ COPY src ./src
RUN npm ci
RUN pkg src/expressApp.js -o /notion-proxy/app

CMD ["/notion-proxy/app"]
CMD ["/notion-proxy/app"]
58 changes: 35 additions & 23 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,34 +1,46 @@
## Notion proxy
# Notion proxy

Deliver notion pages via your domain.
Reference [fruitionsite](https://github.com/stephenou/fruitionsite) for rewrite html processing.
This proxy does not depend on Cloudflare and launches on express server.

### Node Version
## Node Version

`<20.5.1`

If using version 21.x or above, it will generate a deprecation warning indicating the use of the deprecated punycode module.

### Environment variable

| ProxyConfig | Description | Default |
|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------|
| PROXY_PORT | Proxy port number | "3456" |
| DOMAIN | Proxy domain for rewrite | "localhost:3456" |
| IS_TLS | Proxy tls(http/https) for rewrite | "false" |
| NOTION_PAGE_ID | Notion public page id | "f1db0cfbe246475784c67f279289abea" |
| CUSTOM_SCRIPT | Custom script | "" |
| CONTENT_CACHE_SEC | Cache time for loaded content (sec) | "300" |
| GOOGLE_FONT | See: `https://developers.google.com/fonts` | "" |
| AUTO_SET_OGP | The server automatically extracts Open Graph Protocol (OGP) data from your NotionId upon startup. <br/>When this feature is enabled, the values of `OG_TAG_TITLE` and `OG_TAG_IMAGE_URL` are utilized for automatic configuration.<br/>If you prefer to wait until the OGP tags are fetched automatically, you can use the `/readyz` command.<br/><br/>Requirement<br/>- Headless chrome<br/>- CPU is always allocated<br/>- At least 512MB of memory for better | "false" |
| OG_TAG_TITLE | Title for og tag | "" |
| OG_TAG_DESC | Description for og tag | "" |
| OG_TAG_IMAGE_URL | Image url for og tag | "" |
| OG_TAG_TYPE | Type for og tag | "website" |
| TWITTER_CARD | Twitter card for og tag | "summary_large_image" |

### Getting started
## Environment variable

| ProxyConfig | Description | Default |
|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------|
| PROXY_PORT | Proxy port number | "3456" |
| DOMAIN | Proxy domain for rewrite | "localhost:3456" |
| IS_TLS | Proxy tls(http/https) for rewrite | "false" |
| NOTION_PAGE_ID | Notion public page id | "f1db0cfbe246475784c67f279289abea" |
| CUSTOM_SCRIPT | Custom script | "" |
| CONTENT_CACHE_SEC | Cache time for loaded content (sec) | "300" |
| GOOGLE_FONT | See: `https://developers.google.com/fonts` | "" |
| AUTO_SET_OGP | The server automatically extracts Open Graph Protocol (OGP) data from your NotionId upon startup. <br/>When this feature is enabled, the values of `OG_TAG_TITLE` and `OG_TAG_IMAGE_URL` are utilized for automatic configuration.<br/>If you prefer to wait until the OGP tags are fetched automatically, you can use the `/readyz` command.<br/><br/>Requirements<br/>- Headless chrome<br/>- CPU is always allocated<br/>- At least 512MB of memory for better | "false" |
| OG_TAG_TITLE | Title for og tag | "" |
| OG_TAG_DESC | Description for og tag | "" |
| OG_TAG_IMAGE_URL | Image url for og tag | "" |
| OG_TAG_TYPE | Type for og tag | "website" |
| TWITTER_CARD | Twitter card for og tag | "summary_large_image" |

### Note for AUTO_SET_OGP variable

**OgTag setting priority**

Environment variables with OG_xxx in prefix are set with the highest priority.
So, if AUTO_SET_OGP is enabled but the OG_xxx environment variable is set, OG_xxx will have priority.

**Headless chrome Requirements**

At startup, we are extracting og tags from the NotionId page using Chrome Headless.
So, CPU allocation is necessary. Please be cautious when using request allocation in services like Cloud Functions or Cloud Run or Other.

## Getting started

Start proxy for debug on local.

Expand All @@ -38,7 +50,7 @@ $ npm test
$ npm start_proxy
> [email protected] start
> node src/index.js
NotionProxy listening at localhost:3456, NotionId: f1db0cfbe246475784c67f279289abea
Proxy listening at localhost:3456, NotionId: f1db0cfbe246475784c67f279289abea
```

Start proxy binary.
Expand All @@ -47,5 +59,5 @@ Start proxy binary.
$ npm install -g pkg
$ npm run build
$ ./notion-proxy
NotionProxy listening at localhost:3456, NotionId: f1db0cfbe246475784c67f279289abea
Proxy listening at localhost:3456, NotionId: f1db0cfbe246475784c67f279289abea
```
2 changes: 1 addition & 1 deletion src/cloudFunctionApp.js
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const functions = require('@google-cloud/functions-framework');
const EnvConfig = require("./config/proxyConfig");
const NotionProxy = require("./proxy/notionProxy");
const NotionProxy = require("./proxy/proxy");

let finishInitialize = false

Expand Down
4 changes: 2 additions & 2 deletions src/expressApp.js
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
const ProxyConfig = require("./config/proxyConfig");
const NotionProxy = require("./proxy/notionProxy");
const Proxy = require("./proxy/proxy");
const compression = require('compression')
const express = require('express')

function main() {
const proxyConfig = new ProxyConfig();
const proxy = new NotionProxy(proxyConfig);
const proxy = new Proxy(proxyConfig);

const app = express()
app.use(compression())
Expand Down
28 changes: 16 additions & 12 deletions src/proxy/notionProxy.js → src/proxy/proxy.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ const ContentCache = require("./contentCache");
const mime = require("mime-types");
const {fetchUrl: fetch} = require("fetch");

class NotionProxy {
class Proxy {

/**
* Constructor.
Expand Down Expand Up @@ -67,20 +67,24 @@ class NotionProxy {
return;
}
const html = await this.autoOgpExtractor.fetchHtmlAfterExecutedJs();
const title = this.autoOgpExtractor.extractOgTitle(html);
const image = this.autoOgpExtractor.extractOgImage(html);
if (title !== null) {
this.proxyConfig.ogTag.replaceTitle(title)
this.proxyConfig.twitterTag.replaceTitle(title)
const fetchedTitle = this.autoOgpExtractor.extractOgTitle(html);
const fetchedImage = this.autoOgpExtractor.extractOgImage(html);
if (fetchedTitle !== null && this.proxyConfig.ogTag.title === '') {
this.proxyConfig.ogTag.replaceTitle(fetchedTitle)
this.proxyConfig.twitterTag.replaceTitle(fetchedTitle)
}
if (image !== null) {
this.proxyConfig.ogTag.replaceImage(image)
this.proxyConfig.twitterTag.replaceImage(image)
if (fetchedImage !== null && this.proxyConfig.ogTag.image === '') {
this.proxyConfig.ogTag.replaceImage(fetchedImage)
this.proxyConfig.twitterTag.replaceImage(fetchedImage)
}
if (title === null && image === null) {
if (fetchedTitle === null && fetchedImage === null) {
console.log('[WARN] Failed to fetch OGP tag automatically');
} else {
console.log(`Successful automatic fetched of OGP tag. Title: ${title}, Image: ${image.substring(0, 30)}...`);
const imgMsg = this.proxyConfig.ogTag.image.length > 30 ?
`${this.proxyConfig.ogTag.image.substring(0, 30)}...` : this.proxyConfig.ogTag.image;
console.log('Successful automatic fetched of OGP tag.' +
` Title: ${this.proxyConfig.ogTag.title}` +
` Image: ${imgMsg}`);
}
this.initialize(this.proxyConfig, true);
}
Expand Down Expand Up @@ -250,4 +254,4 @@ class NotionProxy {
}
}

module.exports = NotionProxy;
module.exports = Proxy;

0 comments on commit c216a27

Please sign in to comment.