Does torrust-tracker have inspection abilities?

[pfdint]

I am unable to discern if this program has the feature I need.
My comprehension is that a user-torrent-client simply informing a tracker that you have a particular torrent adds it to the tracker.
I want to run a tracker, and be able to look at all the torrents that have been registered to the tracker.
(Ideally it would have a usage log: what address added that torrent, what addresses later asked for it, when, etc).
(It would also be able to delete and ban certain torrents or addresses. So replace 'look at' with 'manage'.)
I don't care if it's a desktop program, served as a local webpage, or a command line tool.
Does torrust have this ability?

Investigations into the question:
I see on the demo for torrust-index that you can make an account and add a torrent by file.
Could this be what they mean by 'the index needs the tracker to run'? Are they saying the index is the inspection tool for the tracker?
In which case, without uploading any torrent files, the index will display the lodged torrents?

[josecelano]

Hi @pfdint,

Thank you for your question!

Understanding Trackers and Indexes in BitTorrent

In the BitTorrent ecosystem, the terms "tracker" and "index" often get intertwined, but they serve distinct purposes:

• Tracker: A tracker is a service (usually over HTTP or UDP) that coordinates the communication between peers. Given an infohash (a unique identifier for a torrent), a tracker provides a list of peers who are participating in that torrent (seeders and leechers). Essentially, the tracker helps peers find each other.

• Index: An index is a searchable catalog of torrent files, typically presented via a website. It allows users to search for torrents based on metadata (like name, category, size), download the torrent files, and thus discover content.

Your Requirements

You want to:

1. Run a tracker.
2. View all torrents registered with the tracker.
3. Manage those torrents (e.g., delete or ban certain torrents or addresses).
4. See usage logs (e.g., which addresses added or requested torrents and when).

Does Torrust Have This Ability?

Short Answer: Partially. Torrust provides tools that can help you achieve some of your goals, but not all features are available out-of-the-box.

Components of Torrust

Torrust consists of two main components:

1. Torrust-Tracker: A BitTorrent tracker written in Rust.
2. Torrust-Index: An index (website) for managing and displaying torrents.

Tracker Capabilities

• Peer Coordination: Torrust-Tracker effectively manages peer lists for torrents.
• Whitelist Feature: You can configure the tracker to only allow specific torrents (by their infohash) through a whitelist.
• API Access: The tracker provides an API, but it primarily focuses on operational metrics and basic management.

Limitations:

• No Built-in Torrent Listing: By default, the tracker does not provide a way to list all the torrents it is tracking.
• Limited Logging: Detailed logs of what address added a torrent or requested it are not maintained due to privacy and performance considerations.

Index Capabilities

• Torrent Management: Torrust-Index allows you to upload torrents, view them, and manage them through a web interface.
• User Accounts: You can create accounts and control who can upload or download torrents.
• Integration with Tracker: The index works alongside the tracker to provide a unified experience.

Important Note:

• Torrents Added via Peers: If a torrent is added to the tracker directly by a peer (and not through the index), it will not automatically appear in the index.

Achieving Your Goals with Torrust

Viewing and Managing Torrents

To view and manage all torrents registered with the tracker, you'll need to use Torrust-Index. Here's how:

1. Upload Torrents via the Index:

   • Use the index to upload any torrents you want to track.
   • Torrents uploaded through the index are visible and manageable via the web interface.

2. Control Uploads:

   • Configure user permissions to control who can upload torrents.
   • Disable public registrations if you want to keep the index private.

Deleting and Banning Torrents or Addresses

• Within the Index:

  • You can delete torrents that were uploaded via the index.
  • Manage user accounts and permissions.

• At the Tracker Level:

  • Use the whitelist feature to allow only specific torrents.
  • Though more manual, you can prevent certain torrents from being tracked.

Viewing Usage Logs

• Tracker Limitations:

  • The tracker does not keep detailed logs of which IP addresses added or requested torrents.
  • This is due to privacy concerns and the high volume of requests a tracker handles.

• Possible Solutions:

  • Custom Logging: Modify the tracker code to add logging capabilities. Be mindful of legal and privacy implications.
  • Use the Index for Audit Trails: The index can keep track of user activities like uploads and downloads, but only for actions performed through it.

Addressing Your Investigations

"Could this be what they mean by 'the index needs the tracker to run'? Are they saying the index is the inspection tool for the tracker?"

Yes, the Torrust-Index acts as a management and inspection tool for torrents that are intended to be tracked. It provides a user-friendly interface to:

• Upload and manage torrents.
• View torrent details and statistics.
• Control user access and permissions. WIP.

However, it only reflects torrents that have been added via the index itself. Torrents that peers add directly to the tracker (by having your tracker URL in their torrent files) won't appear in the index.

Making the Index Reflect All Tracked Torrents

If you want the index to display all torrents currently being tracked, including those added directly by peers:

• Custom Development:

  • Modify the index to import torrents from the tracker's active infohash list.
  • Implement periodic synchronization between the tracker and index.

• Considerations:

  • Be cautious about performance impacts when dealing with a large number of torrents.
  • Ensure compliance with privacy laws, as displaying torrents added by users without their consent may have legal implications.

Recommendations

1. Define Your Use Case:

   • Private Tracker/Index: If you want full control, set up both the tracker and index in private mode, allowing only authorized users to access and upload torrents.
   • Public Tracker with Management: This is more challenging, as you can't control which torrents peers might attempt to track unless you enforce strict whitelisting.

2. Use the Index for Management:

   • Rely on the index to upload and manage torrents.
   • Control user access to maintain a curated list of torrents.

3. Extended Logging and Monitoring:

   • If detailed logs are crucial, consider extending the tracker's functionality or setting up network-level monitoring.
   • Always ensure you're respecting user privacy and complying with relevant regulations.

4. Community and Support:

   • Engage with the Torrust community or contributors for assistance.
   • Consider contributing to the project if you develop features that others might find useful.

Conclusion

While Torrust provides powerful tools for running a tracker and managing torrents through an index, there are limitations regarding:

• Viewing and managing torrents added directly by peers (outside the index).
• Detailed usage logging at the tracker level. I'm working on improving the data the tracker can export. The idea is let users collect raw data and build their database according to their needs.

To fully meet your requirements, you might need to:

• Leverage the Index: Use it as your primary tool for torrent management.
• Customize the Tracker/Index: Modify the existing tools to fit your specific needs.
• Implement Additional Tools: Develop or integrate third-party solutions for logging and monitoring.

I hope this clarifies how Torrust can help you achieve your objectives. If you have further questions or need assistance with specific configurations, feel free to ask!

Additional Resources:

• Torrust-Tracker Documentation
• Torrust-Index Documentation
• Understanding BitTorrent Trackers

Let me know if there's anything else I can help you with!

[pfdint]

Modify the index to import torrents from the tracker's active infohash list.

Ok, I understand it will take development effort to have the feature in question.

Also, is it possible to make torrents on the index only viewable to logged-in users? I can't tell what torrust/torrust-index#431 concludes.

[josecelano]

Hi @pfdint, it's not possible right now. The list of torrents page and the details page are always public in the current version. The only thing the guest user cannot do is to download the torrent.

I guess the only creative and simplest solution for a fully-private tracker now would be:

• To create a collection of torrents with the metadata in a different file. For example, a folder with the torrents and another CSV file with the contents, categories, etc.
• You can share that collection with other users as you want.
• You can use the tracker API from console to generate a key for a new user. The tracker alone can work as a fully private tracker (it's the Index the one working semi-private).
• You send the lye to the user.
• The user can re-create the torrents adding the key to the tracker url.

It's a very tedious task, but depending of your needs it can be also very efficient and easy to mantain.

In fact the worst part it's to embed the key in the tracker URL for all torrents, but users are generally interested only in some torrents so they only have to do it for the torrents they want to download. That's very simple to do. I guess all bittorrent clients allow you to edit the tracker URLs.

So basically:

• You create the collection in any format (Excel, CSV, JSON, Word, private GitHub repo).
• You share the collection with your users (email, access to a repo, folder, etc).
• You create a new key and share that key with the user.
• The user adds that key to the tracker url after adding the torrent.

You don't even need to put the torrent files in your collection if you are seeding the torrents and you use DHT (see this example https://github.com/torrust/torrust-hash2torrent). In that case a simple Excel list including the infohash would work.

User's clients can use DHT to get the torrent from your client.

cc @da2ce7 @cgbosse