Summary

April 5, 2018 ยท View on GitHub

Querying detectors is expensive. Once a regex has been queried once, there's no need to query it again. If we can save the result (cache/memoize it), we'll get vast speedups.

In addition, the same regexes often appear in multiple projects. So if two projects use the same regex, this regex can be checked once and the results used in both places.

Interacting with the cache

  • The server side is server/cache-server.js. Its API is defined here.
  • The client sides are in client/: cli/, npm/, and eslint/.

Configuration

The cache configuration is stored here: $VULN_REGEX_DETECTOR_ROOT/src/cache/.config.json. All parameters are included. This used by the server as well as the client CLI.

Note: The anonymity parameter is not implemented, open an issue if you want it.

Setting up your own cache

If you want to set up your own cache:

  1. Set up server/cache-server.js to run forever.
  2. Configure server/validate-uploads.js as a flock-guarded cron job.

Here's a sample crontab entry:

* * * * * jamie PATH=$NODE_BIN:$PATH VULN_REGEX_DETECTOR_ROOT=$VULN_REGEX_DETECTOR_ROOT /usr/bin/flock -w 0 /tmp/VULN_REGEX_DETECTOR-server.lock $NODE_BIN/node $VULN_REGEX_DETECTOR_ROOT/src/cache/server/cache-server.js >> /tmp/VULN_REGEX_DETECTOR-server.log 2>&1
* * * * * jamie PATH=$NODE_BIN:$PATH VULN_REGEX_DETECTOR_ROOT=$VULN_REGEX_DETECTOR_ROOT /usr/bin/flock -w 0 /tmp/VULN_REGEX_DETECTOR-validate.lock $NODE_BIN/node $VULN_REGEX_DETECTOR_ROOT/src/cache/server/validate-uploads.js >> /tmp/VULN_REGEX_DETECTOR-validate.log 2>&1
  1. Tweak .config.json with your server and DB details.
  2. The server accepts HTTPS connections. See here and here for advice. Store your key and cert in VULN_REGEX_DETECTOR_ROOT/src/cache/server/keys/: privkey.pem and fullchain.pem.

Default

This is the default cache configuration:

  • The cache is enabled.
  • It queries our server.
  • You give us permission to disclose your anonymized queries.