-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: Add waker to StreamResource #4293
Conversation
@ry @bartlomieju (review) |
cli/ops/io.rs
Outdated
let nread = match resource_holder | ||
.resource | ||
.poll_read(cx, &mut buf.as_mut()[..]) | ||
{ | ||
Poll::Ready(t) => { | ||
resource_holder.untrack_task(); | ||
t | ||
} | ||
Poll::Pending => { | ||
resource_holder.track_task(cx)?; | ||
return Poll::Pending; | ||
} | ||
}?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ry, @piscisaureus and I had a discussion about this solution; it looks good on the surface; but it effectively prevents having multiple ops reading from single resource. Although in most cases we only want to have a single op, there are some cases where having multiple ops reading from single resource is desirable.
Ideally we would track multiple tasks for single resource, but Waker
is very simple struct that has no PartialEq
or any other trait implementation that would allow to discriminate wakers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll discuss it more tonight
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bartlomieju I pushed some small changes to the StreamResourceWrapper
to make it capable of waking multiple tasks. It just a suggestion to help the discussion.
@ry I’ll add some tests ofc. I just wanted to know what you guys thought about this solution first. |
Another test case (in |
We should get a test that involves only TCP code (not the http server) and just does listen, accept, read. @jsouto18 Thanks for the patch - but we want to look into this in more detail. |
@jsouto18 can you incorporate one of tests mentioned above? |
@bartlomieju I've implemented #4284 example as a test. I use a new timeout arg to test if the process is hanging. |
That's a bit too complex; can you revert those changes and used this boiled down reproduction (put it in unitTest(
{
perms: { net: true }
},
async function netHangsOnClose() {
let acceptedConn: Deno.Conn;
const resolvable = createResolvable();
async function iteratorReq(listener: Deno.Listener): Promise<void> {
const p = new Uint8Array(10);
const conn = await listener.accept();
acceptedConn = conn;
while (true) {
const nread = await conn.read(p);
console.log("read", nread, "bytes");
if (nread === Deno.EOF) {
break;
}
const nwritten = await conn.write(new Uint8Array([1, 2, 3]));
console.log("written", nwritten, "bytes");
}
resolvable.resolve();
}
const addr = { hostname: "127.0.0.1", port: 4500 };
const listener = Deno.listen(addr);
iteratorReq(listener);
const conn = await Deno.connect(addr);
await conn.write(new Uint8Array([1, 2, 3, 4]));
const buf = new Uint8Array(10);
const nread = await conn.read(buf);
console.log("main read", nread);
console.log("accepted", acceptedConn!);
acceptedConn!.close();
listener.close();
await resolvable;
}
); It does hang on master for me |
@bartlomieju the changes are reverted and I've added the new test. 😃 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jsouto18 Thank you for this patch and thank you @bartlomieju for the succinct test.
I think there might be a deeper design issue here - but the solution isn't clear to me now - maybe we should be tracking in a more general way which ops operate on which resources. This makes me hesitate on this patch, because it introduces infrastructure for only stream resources.
That said, it fixes bugs and we have a new test that we can refactor around, so I will land it now.
* denoland/master: Remove doc strings from cli/js TS files (denoland#4329) upgrade: Rust 1.42.0 (denoland#4331) Enable std tests in debug mode (denoland#4332) fix: Node polyfill fsAppend rework (denoland#4322) v0.36.0 Add waker to StreamResource to fix hang on close bugs (denoland#4293) reorg: Deno global initialization (denoland#4317) move compiler API tests to integration tests (denoland#4319) support permission mode in mkdir (denoland#4286) Stricter permissions for Deno.makeTemp* (denoland#4318) reorg: remove dispatch.ts, move signals, factor out web utils (denoland#4316) reorg: cli/js/compiler/, move more API to cli/js/web/ (denoland#4310) Improve dprint config (denoland#4314) doc(cli/flags): Reduce empty lines in help messages (denoland#4312)
This Pull Request serves to present a WIP solution for the issues #4284 and #3390.
I've traced the cause of #4284 to the
poll_read
task that was getting stuck onPoll::Pending
. It was never being woken up due to the already closed underlying connection.This WIP solution implements the same waking solution used on
TcpListenerResource
under//cli/ops/net.rs
for theStreamResourceHolder
struct.Currently only thenet.rs
tcp resources function properly.