Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

postgres_backend: don't print error stack traces #10307

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

erikgrinaker
Copy link
Contributor

Problem

The use of :? when printing errors will dump stack traces, which can be overly dramatic and noisy for benign errors. For example:

2025-01-08T09:00:12.184800Z ERROR {cid=1168202 ttid=foo/bar application_name=Some("walreceiver")}: query handler for 'IDENTIFY_SYSTEM' failed: Timeline foo/bar was not found in global map
Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.94/src/backtrace.rs:27:14
   1: <T as core::convert::Into<U>>::into
             at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/convert/mod.rs:759:9
   2: safekeeper::handler::SafekeeperPostgresHandler::handle_identify_system::{{closure}}::{{closure}}
             at /home/nonroot/safekeeper/src/handler.rs:403:46
   3: core::result::Result<T,E>::map_err
             at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/result.rs:856:27
   4: safekeeper::handler::SafekeeperPostgresHandler::handle_identify_system::{{closure}}
             at /home/nonroot/safekeeper/src/handler.rs:400:19
   5: <safekeeper::handler::SafekeeperPostgresHandler as postgres_backend::Handler<IO>>::process_query::{{closure}}
             at /home/nonroot/safekeeper/src/handler.rs:306:95
   6: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/future/future.rs:123:9
[50 more lines of stack]

Summary of changes

Don't print stack traces for errors.

@erikgrinaker erikgrinaker requested review from a team and arpad-m and removed request for a team January 8, 2025 11:24
@erikgrinaker erikgrinaker enabled auto-merge January 8, 2025 11:25
Copy link

github-actions bot commented Jan 8, 2025

7264 tests run: 6905 passed, 0 failed, 359 skipped (full report)


Flaky tests (3)

Postgres 17

Postgres 15

  • test_physical_replication_config_mismatch_max_locks_per_transaction: release-arm64

Postgres 14

Code coverage* (full report)

  • functions: 31.2% (8411 of 26962 functions)
  • lines: 48.0% (66779 of 139239 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
2d36a1d at 2025-01-08T13:20:29.984Z :recycle:

@arpad-m
Copy link
Member

arpad-m commented Jan 8, 2025

I have found backtraces to be useful, not in the walreceiver, but in other areas, to quickly identify where an error is coming from. Given that this is a place that logs at the ERROR log level, we should reach it rarely.

If the error is benign, we should catch it earlier and downgrade it to warning or info level. We've done so in the past for many other errors.

@erikgrinaker
Copy link
Contributor Author

erikgrinaker commented Jan 8, 2025

The Safekeeper logs are full of these kinds of errors, happens all the time due to various races. I think the error type is opaque here (it's just an anyhow::Error). But sure, I'll see if I can tweak it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants