Panics on failed to send a message: SendError

[ConradIrwin]

We've noticed when using rust analyzer on the zed codebase that it crashes every 10-30 minutes:

stderr: thread '<unnamed>' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsp-server-0.7.6/src/stdio.rs:28:37:
stderr: receiver was dropped, failed to send a message: "SendError(..)"
stderr: stack backtrace:
stderr: 0: _rust_begin_unwind
stderr: 1: core::panicking::panic_fmt
stderr: 2: core::result::unwrap_failed
stderr: note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stderr: thread 'LspServer' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsp-server-0.7.6/src/stdio.rs:59:17:
stderr: Box<dyn Any>
stderr: stack backtrace:
stderr: 0: std::panicking::begin_panic
stderr: 1: lsp_server::stdio::IoThreads::join
stderr: 2: rust_analyzer::run_server
stderr: note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stderr: thread 'main' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/jod-thread-0.1.2/src/lib.rs:33:22:
stderr: called `Result::unwrap()` on an `Err` value: Any { .. }
stderr: stack backtrace:
stderr: 0: _rust_begin_unwind
stderr: 1: core::panicking::panic_fmt
stderr: 2: core::result::unwrap_failed
stderr: 3: stdx::thread::JoinHandle<T>::join
stderr: 4: rust_analyzer::with_extra_thread
stderr: 5: rust_analyzer::main
stderr: note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

We have also seen a few users reporting this (presumably on their own projects) at zed-industries/zed#17376.

[Veykril]

All of those are follow up errors, the original cause isn't in that log (something panicked outside a catch_unwind)

[Veykril]

#18055 should remove some of the noise

[Oppen]

What I see in the code is that IoThreads::join always panics on error.
From the backtrace I believe the crash happens either here or here. Both are error recovery paths, so I believe they could safely ignore panics with catch_unwind.
Of course, I may be missing context due to inlining or be plainly wrong in my assessment, but unless that's the case, would it be reasonable to add catch_unwind there?

[Veykril]

IoThreads::join panicking is just extra noise only occuring when something else panicked (we have changed the behavior for IoThreads::join last week though because its so noisy). catch_unwind won't do anything, as stated above, all the panic messages in the posted backtrace are follow up panics, the actual cause for the issue is missing in the backtrace.

[ConradIrwin]

This is the error I get now:

Interestingly the file does seem to exist, it's just not a rust file?

stderr: Error: file not found: /Users/conrad/0/zed/zed/a.txt
stderr: sending on a disconnected channel
stderr:
stderr: Stack backtrace:
stderr: 0: std::backtrace::Backtrace::create
stderr: 1: anyhow::error::<impl anyhow::Error>::msg
stderr: 2: anyhow::__private::format_err
stderr: 3: rust_analyzer::run_server
stderr: 4: std::sys::backtrace::__rust_begin_short_backtrace
stderr: 5: core::ops::function::FnOnce::call_once{{vtable.shim}}
stderr: 6: std::sys::pal::unix::thread::Thread::new::thread_start
stderr: 7: __pthread_joiner_wake

[Oppen]

It doesn't look for the file but whether it has been intended from what I've seen, so I think you're onto something there.

[Veykril]

Ah okay, we are returning an error in a notification handler (which exits the main loop) hence all the other panics due to dropping channels unexpectedly

[Veykril]

The only handler exerting this behavior is for textDocument/didSave. Zed incorrectly tells r-a that a file was saved that r-a did not register the event for

rust-analyzer/crates/rust-analyzer/src/main_loop.rs

Lines 211 to 242 in 77e1969

	let mut selectors = vec![
	lsp_types::DocumentFilter {
	language: None,
	scheme: None,
	pattern: Some("**/*.rs".into()),
	},
	lsp_types::DocumentFilter {
	language: None,
	scheme: None,
	pattern: Some("**/Cargo.toml".into()),
	},
	lsp_types::DocumentFilter {
	language: None,
	scheme: None,
	pattern: Some("**/Cargo.lock".into()),
	},
	];
	selectors.extend(additional_filters);
	let save_registration_options = lsp_types::TextDocumentSaveRegistrationOptions {
	include_text: Some(false),
	text_document_registration_options: lsp_types::TextDocumentRegistrationOptions {
	document_selector: Some(selectors),
	},
	};
	let registration = lsp_types::Registration {
	id: "textDocument/didSave".to_owned(),
	method: "textDocument/didSave".to_owned(),
	register_options: Some(serde_json::to_value(save_registration_options).unwrap()),
	};

https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didSave

(now we still shouldnt exit here because of this either)

[ConradIrwin]

Sorry about that, and thanks for the fix! Let me see if we can figure out why that regressed on our end.