Starbelly

April 1, 2026 ยท View on GitHub

Starbelly is a user-friendly web crawler that is easy to deploy and configure.

Build Status Coverage Read the Docs

Security

For information about security considerations and best practices for deploying Starbelly, please see docs/SECURITY.md.

If you discover a security vulnerability, please email acaceres@hyperiongray.com rather than opening a public issue.

  • Policy-Based Crawling: Define custom crawl policies to control crawler behavior
  • Graphical User Interface: Easy-to-use web interface for managing crawls
  • WebSocket API: Real-time communication and streaming results
  • Docker Deployment: Simple deployment using Docker containers
  • RethinkDB Backend: Scalable database for storing crawl data
  • Asynchronous I/O: Built on Trio for high-performance concurrent crawling

Installation

Starbelly is deployed using Docker and Docker Compose. See the Installation Guide for detailed instructions.

Quick start:

git clone https://github.com/hyperiongray/starbelly-docker.git
cd starbelly-docker/starbelly
docker-compose up -d

Usage

After installation, navigate to your server's address in a web browser. The default credentials are:

  • Username: admin
  • Password: admin

For detailed usage instructions, see the documentation.

Documentation

Complete documentation is available at starbelly.readthedocs.io.

Examples

Example notebooks and scripts are available in the notebooks and examples directories.

API

Starbelly provides a WebSocket API for programmatic access. See the WebSocket API documentation for details.

Python client library: starbelly-python-client

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

License

Starbelly is under the MIT License. See LICENSE for details.

For commercial support or inquiries, please contact Hyperion Gray at acaceres@hyperiongray.com

define hyperion gray