IPFS multi-sources pull issue / scaling issue

[jad-darrous]

Version information:

go-ipfs version: 0.4.6-
Repo version: 5
System version: amd64/linux
Golang version: go1.8

Type:

Probably Bug

Priority:

P2

Description:

I was playing with IPFS and I feel that I discovered a strange behavior! I have multiple machines, let's says 4, on the first one, I generated a random binary file and I added it to IPFS. On the second node, I pulled this file by its multihash. When the data is completely transferred to the second node, I repeated the same steps again on the third node, fourth node...

What I was excepting is that the time needed to pull the image for each node to be less than the time need by the previous ones, because the last started node has more sources that can supply the data and the node can pull the data in parallel. But what I experienced was different. Each new node takes more time than the previous ones! The reason is caused by the increasing amount of data sent over the network. It seems that the file is being pulled from all the other nodes. I verified that by observing the data size passing through the network interface.

To replicate the experiment or to see the results, please refer to this repository https://github.com/jad-darrous/IPFS-multi-sources-pull-issue

[whyrusleeping]

I've observed this in scenarios where the bandwidth between machines is high, and the latency above 50ms, the wantlist updates end up not being able to propogate before blocks get sent around.

This is a flaw in the current implementation of bitswap, and should be greatly improved by: #3786

[rddaz2013]

"It seems that the file is being pulled from all the other nodes. I verified that by observing the data size passing through the network interface."

i hope it is a bug not a design issue. Have the dev's on https://github.com/ipfs/ipfs-cluster seen this behavior?

[whyrusleeping]

definitely a bug, see the issue i linked. It is step one towards resolving the problem