-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unexpected response exception #2577
Comments
That seems odd; how are you targeting the replica here - via the optional enum? Or...? And is there anything unusual in your usage? Does this happen reliably / always? Do you have code that shows such? And: are you talking to an unusual server in any way? (For example, "not actually redis, but redis compatible") The short version here is "this should just work", so: I'm trying to see how we can reproduce this, as a precursor to identifying and fixing it. |
@mgravell thanks very much for the quick reply
What does it mean The Redis we're using is the master-slave mode, and the slave is The API we use likes follows: await redis.StreamReadAsync(_typedCacheConfig.Subscription, latestMsgId, count: _typedCacheConfig.PrefetchCount)
await redis.StreamRangeAsync(_typedCacheConfig.Subscription, maxId: msgId, count: 1,
messageOrder: Order.Descending)
await redis.KeyTypeAsync(name) This exception happened recently, no experience with this before, think it may relate to the Redis server, but no errors from other services using the same Redis server. |
replica and slave are synonyms in this context (with replica being the terminology redis uses more recently); my question is: "what does your setup/code look like, for targeting the replica?". Based on the comment above, it sounds like your config string only mentions the replica endpoint, is that correct? (the library can connect to both, and use different nodes for different requests). This sounds like a de-sync between client and server, hence me trying to find whether it happens routinely / all the time. And: what is the server here? Is this on-prem redis vanilla, or some hosted cloud offering? Or something redis-like (I think AWS offer a redis-like server that isn't redis, for example). I'm asking because the setup and server change a few things - for example, hosted offerings often use a proxy on the connection, so I'm wondering if the proxy might have de-sync'd. Since redis doesn't have a correlation identifier, this is hard to investigate. |
We're using a self-hosted contained-based Redis deployed in k8s
Yeah, we only have the replica host address in the connection config From the Redis metrics, it seems the replica is up with the master |
And again: does this happen repeatedly / reliably? Trying to think how we can investigate this. |
Yes, it happens repeatedly these days from 10 days ago, may need more research and testing |
ok, allow me to rephrase: is there code, that I can run, that would show it happening reliably, so that I can investigate? without a repro, I don't have much of a route in here. Oh, and : what server version and library version is this? |
We are seeming something similar to this. It happens on one container every week or so. The heartbeat option has been tried but it takes > 20 mins to correct it. |
I'm seeing a similar issue. Many occurrences of StackExchange.Redis.RedisConnectionException: Unexpected response to ZADD: MultiBulk: 26 items. Could many of these occurrences lead to OutOfMemoryExceptions? Would low memory possibly cause these kinds of RedisConnectionExceptions? Other occurrences of StackExchange.Redis.RedisConnectionException that happen around the same time include Unexpected response to HGETALL: Integer: 0 |
When read from redis slave, get unexpected response exception, while the redis slave is read-only and there's no error when from redis master writes, how could we handle or avoid this kind of error? thanks very much
The text was updated successfully, but these errors were encountered: