Severe difference in block put performance depending on network state

[mib-kd743naq]

Version information:

go-ipfs version: 0.4.5-dev-bf9927f
Repo version: 4
System version: amd64/linux
Golang version: go1.7.1

Type:

Bug

Priority:

P3

Description:

When performing large amounts of block put operations ( ~2000 puts / second ) via the REST api, I am noticing marked difference in behavior depending whether the daemon is started in normal or --offline mode.

When the daemon is connected normally - it runs 1~2 threads at 100% cpu while the puts are coming in, generating a sustained packet storm as illustrated by this dstat -n --net-packets output (numbers are the limit of what my network can handle)

-net/total- -pkt/total-
 recv  send|#recv #send
3501k  655k|4458  2469 
3375k  702k|4714  2464 
2849k  727k|3994  2366 
2671k  832k|3906  2247 
3050k  753k|4695  2422 
4645k  685k|5604  2530 
4648k  748k|5770  2737 
4416k  779k|5289  2784 
4098k  742k|5086  2602 

The storm persisted for at least 10 minutes after I stopped performing API calls, at which point I restarted the daemon. After restart the network remained quiet.

If I add all my blocks while the daemon is in --offline mode - no storm is observed on restart with normal connectivity.

I suspect there is a missing rate limiter for DHT updates or something like that...

This is likely related to https://github.com/ipfs/go-ipfs/milestone/24

[mib-kd743naq]

@whyrusleeping does needs_refining mean you require extra input from myself or...?

[Kubuxu]

I think not, it means that the task needs to be researched and isn't ready to be just fixed.

[mib-kd743naq]

I think this ticket is a direct duplicate of #2828. Please close it if that is the case.

[Kubuxu]

Yes it is.