Skip to content

Commit

Permalink
row: set TargetBytes for kvfetcher
Browse files Browse the repository at this point in the history
Fixes #33660.

Release note (bug fix): Significantly reduce the amount of memory
allocated while scanning tables with a large average row size.
  • Loading branch information
tbg committed Feb 28, 2020
1 parent 9e11347 commit ef8ec0f
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions pkg/sql/row/kv_batch_fetcher.go
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,16 @@ func makeKVBatchFetcherWithSendFunc(
func (f *txnKVFetcher) fetch(ctx context.Context) error {
var ba roachpb.BatchRequest
ba.Header.MaxSpanRequestKeys = f.getBatchSize()
if ba.Header.MaxSpanRequestKeys > 0 {
// If this kvfetcher limits the number of rows returned, also use
// target bytes to guard against the case in which the average row
// is very large.
// If no limit is set, the assumption is that SQL *knows* that there
// is only a "small" amount of data to be read, and wants to preserve
// concurrency for this request inside of DistSender, which setting
// TargetBytes would interfere with.
ba.Header.TargetBytes = 10 * (1 << 20)
}
ba.Header.ReturnRangeInfo = f.returnRangeInfo
ba.Requests = make([]roachpb.RequestUnion, len(f.spans))
if f.reverse {
Expand Down

0 comments on commit ef8ec0f

Please sign in to comment.