HTTP Proxying

September 1, 2024 ยท View on GitHub

Client Apps Using Proxies

Use Cases:

  • egress traffic to the internet from corporate networks for security controls and auditing - typical in banking (I've worked for several large well known banks)
  • reverse proxies like Squid
  • used by package managers and build tools to download programming language libraries eg. mvn for Java to pull from Maven Central, pip for Python to pull from PyPI etc.
  • SSH tunnels in GCP used by kubectl to access protected GKE clusters master control plane API

Many client apps use the following environment variables.

Environment Variables - https_proxy / no_proxy

  • https_proxy - tells applications using libraries like libwww to send the packets to this addresses to be forwarded on.

  • no_proxy - comma separated list of domains / FQDNs to not use the proxy

You can use HTTPS_PROXY and NO_PROXY too but the lowercase variables have higher precedence for most applications, except for Golang. See here for the rules various apps and languages follow.

Set both uppercase and lowercase to be the same to avoid any nasty surprises.

export https_proxy=http://localhost:8080
export no_proxy="domain1.com,host.domain2.com"

export HTTPS_PROXY="$https_proxy"
export NO_PROXY="$no_proxy"

HTTP Toolkit

Intercepting HTTP debugger.

https://httptoolkit.com/docs/

Compile Apache HTTPd with Proxying

wget http://www.mirrorservice.org/sites/ftp.apache.org/httpd/httpd-2.2.14.tar.gz &&
wget http://www.apache.org/dist/httpd/httpd-2.2.14.tar.gz.md5 &&
md5sum -c httpd-2.2.14.tar.gz.md5 &&
tar zxvf httpd-2.2.14.tar.gz &&
cd httpd-2.2.14 &&
./configure --enable-proxy --enable-proxy-http --with-mpm=worker --prefix=/usr/local/apache-proxy &&
make &&
make install

Ported from private Knowledge Base pages 2010+