First, build the front-end assets
July 10, 2024 ยท View on GitHub
- RSS Funnel
/This project is in the early stage of development. Please expect bugs and frequent breaking changes./
RSS Funnel is a modular RSS processing pipeline system. It can manipulate RSS/Atom sources in various ways.
- Fetch full-text content
- Generate an RSS feed from an HTML page
- Remove unwanted elements or texts
- Filter articles matching keywords
- Highlight keywords
- Split aggregation (e.g. top 10 links of day) into individual articles
- Combine multiple feeds into one
- Convert between feed formats
- Run arbitrary JavaScript transformation
- Apply image proxy
- [[https://github.com/shouya/rss-funnel/wiki/Filter-config][And more!]]
The tool comes with a web UI for inspecting the feed and the result of each filter, all within a single statically-built binary with no external dependencies.
[[https://rss-funnel-demo.fly.dev/][Try out the live demo!]]
** Installation
You can use the Docker image ([[https://github.com/shouya/rss-funnel/pkgs/container/rss-funnel][latest version]]) in your =docker-compose.yaml=:
#+begin_src yaml version: "3.8" services: rss-funnel: image: ghcr.io/shouya/rss-funnel:latest ports: - 4080:4080 volumes: - ./funnel.yaml:/funnel.yaml environment: RSS_FUNNEL_CONFIG: /funnel.yaml RSS_FUNNEL_BIND: 0.0.0.0:4080 #+end_src
The web UI will be available at =http://localhost:4080=.
Alternatively, you can build it directly from source:
#+begin_src bash git clone https://github.com/shouya/rss-funnel.git cd rss-funnel
First, build the front-end assets
cd inspector && npm i && npm run build && cd ..
Then, build the binary
cargo build --release #+end_src
If you prefer not to build from source, you can download the pre-built artifacts from the [[https://github.com/shouya/rss-funnel/releases][release page]].
** Usage
To use =rss-funnel=, you need to supply a configuration file in YAML format. Here's an example configuration:
#+begin_src yaml endpoints:
-
path: /tokio-blog.xml note: Full text of Tokio blog source: https://tokio.rs/_next/static/feed.xml filters:
- full_text: {}
- simplify_html: {}
-
path: /hackernews.xml note: Full text of Hacker News source: https://news.ycombinator.com/rss filters:
- full_text: simplify: true append_mode: true #+end_src
Save the above file as =/path/to/funnel.yaml= and run the following command:
#+begin_src rss-funnel -c /path/to/funnel.yaml server #+end_src
You can optionally specify the bind address and port (default =127.0.0.1:4080=). Detailed usage can be found in the =--help= output.
The endpoints like =http://127.0.0.1:4080/tokio-blog.xml= should serve the filtered feeds.
** Documentation
- [[https://github.com/shouya/rss-funnel/wiki/Config-syntax][Config syntax]] contains information about each component in the configuration file.
- [[https://github.com/shouya/rss-funnel/wiki/Filter-config][Filter config]] details the configuration for all filters.
- [[https://github.com/shouya/rss-funnel/wiki/Cookbook][Cookbook]] contains snippets on how to use =rss-funnel= for several practical applications.
- [[https://github.com/shouya/rss-funnel/wiki/Deployment][Deployment]] includes examples of docker-compose and kubernetes definitions.