NS_ERROR_NET_HTTP2_SENT_GOAWAY errors

[cw789]

With #19 released, within a Phoenix app we are getting some page load network failures in the dev env, probably caused by NS_ERROR_NET_HTTP2_SENT_GOAWAY, as long as the app is using HTTPS with a self-signed certificate. Could this be related? I am not deep into this stuff.

[dustinaleksiuk]

We use a self-signed certificate with HTTPS in development. When I'm on 1.0.1 I see a ERR_HTTP2_COMPRESSION_ERROR in the browser when our SSO provider redirects back to our app after logging in.

If I manually change the URL to hit our app, neither our fonts nor our CSS load, with ERR_HTTP2_COMPRESSION_ERROR in the browser's network tab.

[krainboltgreene]

In chrome this represents as a net::ERR_HTTP2_COMPRESSION_ERROR exception, where as the NS exception above is from the Gecko engine in Firefox.

[whatyouhide]

Interesting. It totally could be related. @mtrudel have you seen this?

[mtrudel]

It's likely related; I'm setting up a repro now

[mtrudel]

I've pushed up a basic phoenix install here with a self-serve cert generated per mix phx.gen.cert and I'm unable to repro using it in the following environment:

• macOS 14.2.1 (ARM)
• OpenSSL 1.1.1w
• Chrome 131.0.6778.141
• Firefox 133.0.3

I am attempting to repro by loading https://localhost:4001/assets/app.js in either Chrome or Firefox. In all cases (I've tried hundreds of reloads) they both load without issue.

If folks are able to either try the above repo and see if they can repro the issue, it would be greatly appreciated. For bonus points, if you're able to share a minimal case that DOES reproduce this that would be super as well.

Data points that would be interesting:

• What % of requests (approximately) are failing
• If there are particular requests that fail / succeed
• If you are accessing via localhost or a different hostname
• x86 or ARM
• Exact Elixir and OTP versions (if you're not using the asdf versions pinned in the above repo)
• The output of running :crypto.info_lib() in iex (mine is [{"OpenSSL", 269488511, "OpenSSL 1.1.1w 11 Sep 2023"}])

[dustinaleksiuk]

I've pushed up a basic phoenix install here with a self-serve cert generated per mix phx.gen.cert and I'm unable to repro using it in the following environment:

I pulled this repo. It seems to be using a much older version of hpax. The version that immediately causes the issue for is is 1.0.1. 1.0.0 is okay.

In our app that has the issue:

* What % of requests (approximately) are failing

All requests to our local js and css assets, from what I can tell.

* If you are accessing via `localhost` or a different hostname

A different hostname. We use an internal hostname that maps to localhost.

* x86 or ARM

Mac M1

* Exact Elixir and OTP versions (if you're not using the asdf versions pinned in the above repo)

Erlang/OTP 27 [erts-15.2] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit] [dtrace]

Elixir 1.17.3 (compiled with Erlang/OTP 27)

* The output of running `:crypto.info_lib()` in iex (mine is `[{"OpenSSL", 269488511, "OpenSSL 1.1.1w  11 Sep 2023"}]`)

[{"OpenSSL", 809500672, "OpenSSL 3.4.0 22 Oct 2024"}]

I'm hoping to get a chance to replicate this in a repo if I can find some time later today.

[mtrudel]

I pulled this repo. It seems to be using a much older version of hpax. The version that immediately causes the issue for is is 1.0.1. 1.0.0 is okay.

My bad - I didn't notice that my push failed because I already had a repo named test. Fixed now.

[mtrudel]

A different hostname. We use an internal hostname that maps to localhost.

You're accessing it directly though? Like, there's no intermediate proxy or anything?

[dustinaleksiuk]

My bad - I didn't notice that my push failed because I already had a repo named test. Fixed now.

Nice - I'll grab that new repo at some point today and try it.

You're accessing it directly though? Like, there's no intermediate proxy or anything?

Correct, no intermediate proxy.

[mtrudel]

@dustinaleksiuk I've dialled in all of those settings you had above (using https://localtest.me:4001/assets/app.js, recompiled erlang 27.1 against openssl 3.4.0) and I still cannot reproduce the issue. Hopefully you have better luck on your side

[krainboltgreene]

@mtrudel I can absolutely reproduce it locally on my machine and anytime in the next 42h I can be available to screenshare and walk you through it to debug it directly if it'd help.

[mtrudel]

@krainboltgreene Great! Reaching out on Elixir slack (or somewhere else if that's easier)

[whatyouhide]

Thanks for taking a look at this folks! Let me know if I can help. 💟

[mtrudel]

With @krainboltgreene's help I've got an explanation - the remote end is closing with a COMPRESSION_ERROR GOAWAY frame with an informational message of Framer error: 19 (HPACK_INVALID_INDEX).. Now to build a test repro ferda fix it.

[mtrudel]

Fix is up if anyone wants to try and confirm it fixes the issue for you (it does for me locally):

{:hpax, "~> 1.0", github: "mtrudel/hpax", branch: "table_size_fixes", override: true}

[dustinaleksiuk]

Fix is up if anyone wants to try and confirm it fixes the issue for you (it does for me locally):

This works! It resolved the ERR_HTTP2_COMPRESSION_ERROR I was seeing in Chrome. Thank you for the fix.

[whatyouhide]

v1.0.2 is out.