Include non standards field in the info dictionary for popular BitTorrent clients?

[josecelano]

Hi, I've been testing using BiglyBT. I wanted to add some test torrents to the demo and seed them. BiglyBT includes a non-standard field in the torrent info dict. The field is name.utf-8. For example:

{
   "announce": "udp://tracker.torrust-demo.com:6969/announce",
   "azureus_properties": {
      "dht_backup_enable": 1
   },
   "comment": "",
   "comment.utf-8": "",
   "created by": "BiglyBT/3.5.0.0",
   "creation date": 1713894490,
   "encoding": "UTF-8",
   "info": {
      "length": 195636,
      "name": "2404.12237.pdf",
      "name.utf-8": "2404.12237.pdf",
      "piece length": 32768,
      "pieces": "<hex>51 88 43 18 49 B4 61 31 52 FD 7B DB A6 A3 FF 0A 4F D6 42 4B 51 88 43 18 49 B4 61 31 52 FD 7B DB A6 A3 FF 0A 4F D6 42 4B 51 88 43 18 49 B4 61 31 52 FD 7B DB A6 A3 FF 0A 4F D6 42 4B 51 88 43 18 49 B4 61 31 52 FD 7B DB A6 A3 FF 0A 4F D6 42 4B 51 88 43 18 49 B4 61 31 52 FD 7B DB A6 A3 FF 0A 4F D6 42 4B C7 03 14 B3 46 6B B0 42 50 1F CC A1 E2 FA 5A B0 61 3A E2 FC</hex>"
   }
}

Since it's not a standard field, the Index removes it, which changes the torrent infohash (hash for the info dictionary). If you want to seed the torrent, the process would be:

1. Generate the torrent from sources with BiglyBT.
2. Upload the torrent to the Index. That would generate a torrent with a new infohash.
3. Download the new torrent from the Index.
4. Remove the previous torrent from the BiglyBT.
5. Add the new torrent.
6. BiglyBT starts downloading it, but nobody has it since you are the first seeder.
7. You'll need to tell BiglyBT you already have the sources for that torrent. However, that does not work for me. In general, that is a tricky task for most clients. I guess the option to synchronize the client with the folder storing the sources is the option Check Files Exist.

I think that process is excruciating, but maybe it's just me. I recommend adding non-standard fields in the info dict to the Index to avoid changing the infohash.

• Transmission
• KTorrent
• BiglyBT
• Deluge
• qBittorrent

I have not checked all of them yet; it seems Transmission does not include any non-standard field.

cc @torrust/maintainers

[josecelano]

@da2ce7 should we include non-standard fields like this? or only fields included in official BEPs?

[da2ce7]

@josecelano I think that we should have a set of "community" info dictionary schemas; some sort of whitelist format. In general if we find the format unofficial format somehow useful and not abusive; then I think that we should be quick to accept a new schema into our whitelist.

This white-list could be controlled by the configuration of the index.

[josecelano]

@josecelano I think that we should have a set of "community" info dictionary schemas; some sort of whitelist format. In general if we find the format unofficial format somehow useful and not abusive; then I think that we should be quick to accept a new schema into our whitelist.

This white-list could be controlled by the configuration of the index.

Hi @da2ce7 do you mean the white-list should be dynamically injected as configuration at run time? We have to change a lot of things to allow that:

• The way we parse the torrents. It's a hardcoded Rust struct.
• The way to store them in the database. We also use hardcoded table columns.

For now, I would just add it, and add a schema with what we consider a "valid" torrent.

In the future, we could implement those changes. I think those changes could also be good for other reasons:

• A more generic way to parse the torrent would be good for giving the user better errors.
• A more flexible way (maybe a NoSQL database) to store the torrents could also allow us to easily change the database structure for torrents, allowing at the same time old torrents without a field and new ones with the new field. Although, it could be implemented with an extra table for dynamic extra fields (rows not columns). However, I would keep it simple if there are only a couple of cases for torrents using custom fields out there. It would be nice to have stats.