Etag^ #101

.github/issue_template.md

@@ -0,0 +1,9 @@
+If you are reporting a bug, then providing the following is helpful:
+* browser name and version
+* privacy possum version
+
+If a website is broken, please provide the URL and describe the problem.
+
+If you would like to be extra helpful, and know how to access the Javascript console for the extension, please provide a debug log. The debug log may contain information about your browsing, such as URLS. Please read through it and redact information as you see fit.
+
+You can access the debug log from the popup console by running `popup.debug()`, or from the extension's background page by running `possum.prettyLog()`.*=/

.travis.yml

@@ -0,0 +1,5 @@
+language: node_js
+node_js:
+  - "10"
+install: make npm_install
+script: make test

Makefile

@@ -1,7 +1,13 @@
+test:
+	./scripts/test.sh
+
+npm_install:
+	./scripts/npm_install.sh
+
 psl: 
 	./scripts/getpsl.py > src/js/domains/psl.js
 
 release:
 	./scripts/release.sh
 
-.PHONY: psl release
+.PHONY: test npm_install psl release

PRIVACY_POLICY.md

@@ -0,0 +1,3 @@
+## Privacy Policy
+
+Privacy Possum does not collect or send data to any server.

README.md

@@ -1,3 +1,5 @@
+[![Build Status](https://travis-ci.org/cowlicks/privacypossum.svg?branch=master)](https://travis-ci.org/cowlicks/privacypossum)
+
 ![logo](/src/media/logo-med256.png)
 
 Install for [Chrome](https://chrome.google.com/webstore/detail/privacy-possum/ommfjecdpepadiafbnidoiggfpbnkfbj) and for [Firefox](https://addons.mozilla.org/en-US/firefox/addon/privacy-possum/).
@@ -7,14 +9,14 @@ Companies gobble up data about you to create an asymmetry of information that th
 Their profit comes from your informational disadvantage.
 Privacy Possum monkey wrenches common commercial tracking methods by reducing and falsifying the data gathered by tracking companies.
 
-# Current features
+# Current Features
 
-* blocks cookies that let trackers uniquely identify you across websites
-* blocks `refer` headers that reveal your browsing location
-* blocks `etag` tracking which leverages browser caching to uniquely identify you
-* blocks browser fingerprinting which tracks the inherent uniqueness of your browser
+* Blocks cookies that let trackers uniquely identify you across websites
+* Blocks `refer` headers that reveal your browsing location
+* Blocks `etag` tracking which leverages browser caching to uniquely identify you
+* Blocks browser fingerprinting which tracks the inherent uniqueness of your browser
 
-# Threat model
+# Threat Model
 
 __Privacy Possum does not have a threat model.__ Weird huh? We prioritize costing tracking companies money over protecting you. When considering some anti-tracking measure we do not ask "Is it possible to circumvent this?". Instead we ask "Is it cost-effective for a tracking company to circumvent this?". If the answer is "yes, no" we accept it.
 
@@ -23,20 +25,20 @@ Tracking companies are growing, they own more infrastructure, and make more mone
 We think tackling the problem from an economic angle is extremely important, and will ultimately help shift the internet into more private place.
 
 
-# Related projects
+# Related Projects
 
 ## Why not Privacy Badger?
 
-[Privacy Badger](https://github.com/EFForg/privacybadger) is another privacy focussed browser extension maintained by the Electronic Frontier Foundation.
+[Privacy Badger](https://github.com/EFForg/privacybadger) is another privacy focused browser extension maintained by the Electronic Frontier Foundation.
 I worked for the EFF on the project full time for 6 months, and found that it's current privacy benefits to be limited.
 Adding new privacy protections was difficult, or impossible with the current architecture.
 And the project maintainers were not interested in fixing these issues.
 
 Comparisons with other anti-tracking projects can be found [here](docs/).
 
-# Tracker blocking
+# Tracker Blocking
 
-## browser fingerprinting
+## Browser Fingerprinting
 
 Sites can inspect aspects of your browser itself to determine its uniqueness, and therefore track you. This tracking technique is widely used.
 
@@ -46,15 +48,15 @@ For example [many sites](https://publicwww.com/websites/cdn.jsdelivr.net%2Fnpm%2
 
 Fingerprinting usually aggregates information across many esoteric browser API's, so we watch for this behavior. When we detect it, we block it.
 
-However many sites load first party fingerprinting code alongside other neccessary code, like on reddit.com, so we can't simply block the script, or it will break the page. Instead when we see first party fingerprinting, we inject random data to spoil the fingerprint. Visit [valve.github.io/fingerprintjs2](https://valve.github.io/fingerprintjs2/) to see this. "get your fingerprint" multiple times, and see it change each time.
+However many sites load first party fingerprinting code alongside other necessary code, like on reddit.com, so we can't simply block the script, or it will break the page. Instead when we see first party fingerprinting, we inject random data to spoil the fingerprint. Visit [valve.github.io/fingerprintjs2](https://valve.github.io/fingerprintjs2/) to see this. "get your fingerprint" multiple times, and see it change each time.
 
-## cookie tracking
+## Cookie Tracking
 
 Most online tracking happens through cookies.
 
 Privacy Possum blocks all 3rd party cookies.
 
-## etag tracking
+## Etag Tracking
 
 Etags are a well known tracking vector, commonly used in lieu of cookies. 
 
@@ -65,10 +67,15 @@ We detect and block third party etags as follows:
      - If they are different, do not allow etags for this url now or in the future.
 
 Chrome withholds the `if-none-match` headers from `onBeforeSendHeaders` (https://developer.chrome.com/extensions/webRequest#Life_cycle_of_requests).
-So we can't prevent the browser from revealing some data via sending cache information, we are only able to intercept incomming etags from sources that are not already cached.
+So we can't prevent the browser from revealing some data via sending cache information, we are only able to intercept incoming etags from sources that are not already cached.
+
+## Referer headers
 
+Referer headers are not exactly used for tracking themselves. But they are used in conjunction with other methods to track you. So we block them, and use a simple algorithm to unblock them when this causes problems.
+* Block `referer` headers to 3rd party sources
+* If the source responds with bad status code, we retry with the header added back in
 
-## 301 moved permanent redirect tracking
+## 301 Moved Permanent Redirect Tracking
 
 If you visit a site, it might load a resource that has a 301 redirect. The resource can redirect you to url that is *unique* to you. Then, the next time you see the original resource, your browser will load the unique url from the cache, and fetch the resource from there. Making you uniquely identified.
 
@@ -79,25 +86,25 @@ One solution to this would be to use cached 301 redirects from 3rd party sources
 However we have not found a way to disable the cache like this in chrome's extension api. It is possible to intercept the redirect, but if you redirect back to the original url, you fetch from the cache again.
 One hack to disable the cache is to append a dummy query parameter, like `?` or `&`. With this you can re-try the url redirect to determine if it is unique per request.
 
-If this tests positive for a tracking redirect, we can't simply bust the cache everytime by appending dummy query parameters because we'd end up with urls like `https://foo.com/?&&&&&&&&&&&&&&&....`.
+If this tests positive for a tracking redirect, we can't simply bust the cache every time by appending dummy query parameters because we'd end up with urls like `https://foo.com/?&&&&&&&&&&&&&&&....`.
 
 The next best solution would be to just block the request. This is yet to be implemented.
 
-# developing
+# Development
 
-## dependencies
+## Dependencies
 
-The packaged extension contains *no* external dependencies. However we maintain our own copy of Mozilla's Public Suffix List, and Privacy Badger's Multi-domain First Parties list. These are used to determine if a given domain is "first party" or "third party".
+The packaged extension contains *no* external dependencies. However we have several local dependencies we manually update for development purposes. First, wemaintain our own copy of Mozilla's Public Suffix List, and Privacy Badger's Multi-domain First Parties list. These are used to determine if a given domain is "first party" or "third party". We also have a copy of React and ReactDOM.
 
 There are dependencies for development. These are all installed by running `npm install` inside `src/js`.
 
-## module system
+## Module System
 
 We use a lightweight implementation of nodes `require` function to implement modules without requiring a compilation step between running in the browser, and running in node.
 
 For this to work, we wrap each module in code like this (from `src/js/reasons/utils.js`):
 
-```
+```js
 "use strict";
 
 [(function(exports) {
@@ -111,20 +118,20 @@ Object.assign(exports, {sendUrlDeactivate, ...});
 Note the path to this module must be passed in as a string to the `define` function.
 Exported stuff is assigned to properties on `exports` just like in node.
 
-## releasing
+## Releases
 
 * edit the manifest.json version number to the form year.month.day with no leading zeros.
 * save and commit
-* run release.sh, this tags the repo with the manifest version and builds a zip file
+* run `make release`, this tags the repo with the manifest version and builds a zip file
 * test the zip file in a fresh instances of supported browsers.
     - for chrome run `google-chrome --user-data-dir=$(mktemp -d)` install the zip by dragging it to the chrome://extensions/ page.
     - for firefox run `firefox --profile $(mktemp -d) --no-remote --new-instance`. Go to `about:debugging` and click load temporary addon. Navigate to the zip file.
-    - visit https://valve.github.io/fingerprintjs2/ https://reddit.com/
+    - Do some basic Q&A tests, visit https://valve.github.io/fingerprintjs2/ https://reddit.com/ https://twitch.tv/ https://duckduckgo.com/
 * upload the zip.
     - for chrome visit https://chrome.google.com/webstore/developer/edit/ommfjecdpepadiafbnidoiggfpbnkfbj record any other edits to the chrome store profile in this repo
     - for firefox visit https://addons.mozilla.org/en-US/developers/addon/privacy-possum/edit
 * notify users
 
-## testing
+## Testing
 
 From inside `src/js/` run `npm test`. To check coverage run `npm run cover`.

docs/CNAME

@@ -0,0 +1 @@
+gisticon.privacypossum.io

scripts/npm_install.sh

@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+source $(git rev-parse --show-toplevel)/scripts/source_me.sh
+
+pushd ${js_dir} > /dev/null
+trap "popd > /dev/null" EXIT
+
+npm install

scripts/release.sh

@@ -1,5 +1,5 @@
-toplevel=$(git rev-parse --show-toplevel)
-src_dir=${toplevel}/src
+#!/usr/bin/env bash
+source $(git rev-parse --show-toplevel)/scripts/source_me.sh
 manifest=${src_dir}/manifest.json
 today=$(date '+%Y.%-m.%-d')
 out_file=${toplevel}/possum.zip
@@ -13,8 +13,7 @@ fi
 echo "tagging version: \"${today}\""
 git tag ${today}
 
-pushd ${toplevel}/src > /dev/null
-
+pushd ${src_dir} > /dev/null
 trap "popd > /dev/null" EXIT
 
 echo "packaging extension to: ${out_file}"

scripts/source_me.sh

@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+# Load this by running
+# `source $(git rev-parse --show-toplevel)/scripts/source_me.sh`
+# in a script.
+toplevel=$(git rev-parse --show-toplevel)
+src_dir=${toplevel}/src
+js_dir=${src_dir}/js

scripts/test.sh

@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+source $(git rev-parse --show-toplevel)/scripts/source_me.sh
+
+pushd ${js_dir} > /dev/null
+trap "popd > /dev/null" EXIT
+
+npm test

src/js/constants.js

@@ -55,7 +55,11 @@ const FINGERPRINTING = 'fingerprinting',
   BLOCK = 'block',
   HEADER_DEACTIVATE_ON_HOST = 'header_deactivate_on_host';
 
-const FINGERPRINTING_PATH = '/js/contentscripts/injector.js';
+const CONTENTSCRIPTS = new Set([
+  '/js/bootstrap.js',
+  '/js/contentscripts/fingercounting.js',
+  '/js/initialize_contentscripts.js',
+]);
 
 const etag = {
   ETAG_TRACKING: 'etag_tracking',
@@ -72,6 +76,8 @@ const POPUP = 'popup';
 
 const REMOVE_ACTION = 'remove_action';
 
+const GET_DEBUG_LOG = 'get_debug_log';
+
 Object.assign(exports, {
   DISK_NAME,
   responses,
@@ -85,7 +91,7 @@ Object.assign(exports, {
   request_methods,
   header_methods,
   FINGERPRINTING,
-  FINGERPRINTING_PATH,
+  CONTENTSCRIPTS,
   USER_HOST_DEACTIVATE,
   USER_URL_DEACTIVATE,
   BLOCK,
@@ -95,6 +101,7 @@ Object.assign(exports, {
   TAB_DEACTIVATE_HEADERS,
   POPUP,
   REMOVE_ACTION,
+  GET_DEBUG_LOG,
 });
 
 })].map(func => typeof exports == 'undefined' ? define('/constants', func) : func(exports));