Reverse Log Proxy (RLP) Gateway

December 21, 2018 ยท View on GitHub

The Reverse Log Proxy Gateway provides an HTTP API to access the Reverse Log Proxy. The HTTP endpoint is by default available to traffic outside the deployment (configurable through GATEWAY_ADDR environment variable). The RLP Gateway endpoint is authenticated and verifies provided access tokens against a UAA (configurable through the LOG_ACCESS* and LOG_ADMIN* environment variables.

HTTP API

GET /v2/read

Provides a stream to read envelope batches as Server Sent Events (SSE) in JSON format.

Parameters

  • shard_id - Set the shard ID. Envelopes will be split between clients using the same shard ID.
  • source_id - One or more source IDs.
  • log - Request log envelopes.
  • counter - Request counter envelopes.
  • gauge - Request gauge envelopes.
  • timer - Request timer envelopes.
  • event - Request event envelopes.
  • counter.name - Request counter envelopes and filter on counter name.
  • gauge.name - Request gauge envelopes and filter on gauge name. For a gauge that has multiple metrics, use comma separated list (e.g., gauge.name=x,y,z).
  • deterministic_name - Enable deterministic routing.

A 400 Bad Request is returned when no envelope types are passed into the query string.

Example Requests

Request log envelopes

curl -H "Authorization: $(cf oauth-token)" https://localhost:8088/v2/read?log

Request log envelopes with sharding:

curl -H "Authorization: $(cf oauth-token)" https://localhost:8088/v2/read?log&shard_id=shard-id-for-all-clients

Request all envelopes for a given source:

curl -H "Authorization: $(cf oauth-token)" https://localhost:8088/v2/read?log&counter&gauge&timer&event&source_id=SOURCE-ID

Request counter metrics with a given name:

curl -H "Authorization: $(cf oauth-token)" https://localhost:8088/v2/read?counter.name=request_count

Example SSE Response

data: {"batch":[{"timestamp":"1532030745755909241","sourceId":"doppler","tags":{"deployment":"loggregator","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"gauge":{"metrics":{"subscriptions":{"unit":"subscriptions","value":1}}}}]}

data: {"batch":[{"timestamp":"1532030745755669593","sourceId":"doppler","tags":{"deployment":"loggregator","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"counter":{"name":"egress","delta":"9","total":"1462"}},{"timestamp":"1532030745755852038","sourceId":"doppler","tags":{"deployment":"loggregator","direction":"ingress","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"counter":{"name":"dropped"}},{"timestamp":"1532030745756105677","sourceId":"doppler","tags":{"deployment":"loggregator","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"gauge":{"metrics":{"dump_sinks":{"unit":"sinks","value":1}}}},{"timestamp":"1532030745755954729","sourceId":"doppler","tags":{"deployment":"loggregator","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"counter":{"name":"ingress","delta":"9","total":"20260"}},{"timestamp":"1532030745755989588","sourceId":"doppler","tags":{"deployment":"loggregator","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"counter":{"name":"egress","total":"1462"}},{"timestamp":"1532030745756023906","sourceId":"doppler","tags":{"deployment":"loggregator","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"counter":{"name":"sinks.errors.dropped"}},{"timestamp":"1532030745756066632","sourceId":"doppler","tags":{"deployment":"loggregator","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"counter":{"name":"sinks.dropped"}},{"timestamp":"1532030745755810461","sourceId":"doppler","tags":{"deployment":"loggregator","index":"43a85aeb-89d1-4d36-9258-d781b571fe32","ip":"10.244.0.128","job":"doppler","metric_version":"2.0","origin":"loggregator.doppler"},"gauge":{"metrics":{"container_metric_sinks":{"unit":"sinks","value":1}}}}]}