Skip to content
This repository has been archived by the owner on Aug 19, 2022. It is now read-only.

include current limits in debug messages #42

Merged
merged 6 commits into from
Jun 19, 2022
Merged

Conversation

marten-seemann
Copy link
Contributor

Fixes #26.

Potentially controversial: also removes the stat from the debug output, to focus on the thing that is actually getting blocked. We can add it back if preferred.

@BigLep
Copy link
Contributor

BigLep commented Jun 6, 2022

@marten-seemann : can you please post an example error message before vs now?

Copy link
Collaborator

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep the stat, its very useful.

@marten-seemann marten-seemann force-pushed the rework-errors branch 2 times, most recently from 59c71ed to bf5dd82 Compare June 7, 2022 10:10
@marten-seemann
Copy link
Contributor Author

@vyzo Re-added the stat.

@BigLep
before:

2022-06-07T12:11:15.719+0200	DEBUG	rcmgr	go-libp2p-resource-manager/scope.go:370	blocked stream from constraining edge	{"scope": "stream-16", "edge": "transient", "direction": "Outbound", "stat": {"NumStreamsInbound":0,"NumStreamsOutbound":2,"NumConnsInbound":0,"NumConnsOutbound":0,"NumFD":0,"Memory":0}, "error": "transient: cannot reserve stream: resource limit exceeded"}

after:

2022-06-07T12:04:27.543+0200	DEBUG	rcmgr	go-libp2p-resource-manager/scope.go:488	blocked stream from constraining edge	{"scope": "stream-117", "edge": "transient", "direction": "Outbound", "current": 2, "attempted": 1, "limit": 2, "stat": {"NumStreamsInbound":0,"NumStreamsOutbound":2,"NumConnsInbound":0,"NumConnsOutbound":4,"NumFD":4,"Memory":0}, "error": "transient: cannot reserve outbound stream: resource limit exceeded"}

@marten-seemann
Copy link
Contributor Author

Actually, we shouldn't include the stat here. It's racy.

@marten-seemann
Copy link
Contributor Author

Included the stat in a non-racy way (by collecting it at the same time we generate the error, while holding the mutex). This should be ready for review now.

@marten-seemann marten-seemann requested a review from vyzo June 19, 2022 08:09
Copy link
Collaborator

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets keep the stat, i find it useful for context.

Copy link
Collaborator

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too much noise, func the loggables bussiness pls.

scope.go Outdated
@@ -239,7 +329,14 @@ func (s *resourceScope) ReserveMemory(size int, prio uint8) error {
}

if err := s.rc.reserveMemory(int64(size), prio); err != nil {
log.Debugw("blocked memory reservation", "scope", s.name, "size", size, "priority", prio, "stat", s.rc.stat(), "error", err)
logValues := make([]interface{}, 0, 7)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we abstract this in a function? It adds noise in the logic.

scope.go Outdated
var stat network.ScopeStat
stat, err = e.ReserveMemoryForChild(int64(size), prio)
if err != nil {
logValues := make([]interface{}, 0, 7)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here too

scope.go Outdated
@@ -344,7 +450,14 @@ func (s *resourceScope) AddStream(dir network.Direction) error {
}

if err := s.rc.addStream(dir); err != nil {
log.Debugw("blocked stream", "scope", s.name, "direction", dir, "stat", s.rc.stat(), "error", err)
logValues := make([]interface{}, 0, 7)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here

scope.go Outdated
var stat network.ScopeStat
stat, err = e.AddStreamForChild(dir)
if err != nil {
logValues := make([]interface{}, 0, 8)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here

scope.go Outdated
@@ -444,7 +566,14 @@ func (s *resourceScope) AddConn(dir network.Direction, usefd bool) error {
}

if err := s.rc.addConn(dir, usefd); err != nil {
log.Debugw("blocked connection", "scope", s.name, "direction", dir, "usefd", usefd, "stat", s.rc.stat(), "error", err)
logValues := make([]interface{}, 0, 8)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

booh

scope.go Outdated
var stat network.ScopeStat
stat, err = e.AddConnForChild(dir, usefd)
if err != nil {
logValues := make([]interface{}, 0, 8)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and again!

@marten-seemann
Copy link
Contributor Author

@vyzo Moved to separate functions. The reason I chose to have separate functions for stream, conn and memory errors is that I want to avoid allocations: The logValues slice is preallocated such that we only do a single allocation per log message.

@marten-seemann marten-seemann requested a review from vyzo June 19, 2022 09:42
Copy link
Collaborator

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, just a cosmetic issue.

scope.go Outdated
@@ -18,6 +19,80 @@ type resources struct {
memory int64
}

type errStreamOrConnLimitExceeded struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we put all that stuff at the bottom of the file?
New readers shouldnt be welcomed with an incomprehensible blob of logging arcana.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved it to a separate file.

@marten-seemann marten-seemann merged commit 3f28daf into master Jun 19, 2022
@marten-seemann marten-seemann deleted the rework-errors branch June 19, 2022 13:55
@MarcoPolo MarcoPolo mentioned this pull request Jul 7, 2022
41 tasks
@ajnavarro ajnavarro mentioned this pull request Aug 24, 2022
72 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"resource limit exceeded" error message makes clear what limit and its value were exceeded
3 participants