Media Scrapers

A collection of applications for the finding and organizing of media from the internet to our NAS drive.

The overall gist is this :

  • a Servarr application uses Jackett to search for torrents
  • Upon finding a torrent, it sends a magnet link to transmission (and category to place it in)
  • transmission starts the download process saving progress in the incomplete folder of the NAS drive
  • on completion, it moves the file to the complete folder of the NAS drive
  • the servarr application standardizes the name and moves it to the final collection

General notes on deployments :

  • Servarr applications should not use a NAS system for storing config, they run on sqlite databases and those get corrupted quickly
  • On startup you need to break into each Servarr pod and edit the file config/config.xml. The <UrlBase></UrlBase> needs to be set to whatever the path on the ingress definition is including the trailing slash.
  • Jackett will need the ServerConfig.json updated at BasePathOverride in the same way.

Debugging notes:

  • Getting these running at first was a PITA, running everything in just docker containers as a starting point was helpful.
  • Got lots of networking errors as well running portforwarding to the bastion node made it possible to debug a bit and figure out what was going on.

overall-layout

  • Nas

    • A shared persistant volume and claim for the movement of data between services.

  • Transmission

    • A torrent client that runs over an Open VPN client.

  • servARR stack*

    • Media applications from the servARR stack.

  • Postgres

    • Database to back serveARR applications

  • Flaresolverr

    • Proxy for dealing with cloudflare protection on some sites