Skip to content

Commit

Permalink
changefeedccl: pick up TargetBytes for backfill ScanRequest
Browse files Browse the repository at this point in the history
Now that cockroachdb#44925 has merged, we can use a byte limit rather than a row limit.

Release note: None
  • Loading branch information
ajwerner committed Mar 2, 2020
1 parent a75396a commit 3433fbd
Showing 1 changed file with 2 additions and 4 deletions.
6 changes: 2 additions & 4 deletions pkg/ccl/changefeedccl/kvfeed/scanner.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,15 +103,13 @@ func (p *scanRequestScanner) exportSpan(
txn.SetFixedTimestamp(ctx, ts)
stopwatchStart := timeutil.Now()
var scanDuration, bufferDuration time.Duration
// TODO(ajwerner): Adopt the byte limit here as soon as it merges. This must
// happen in 20.1!
const maxKeysPerScan = 1 << 18
const targetBytesPerScan = 16 << 20 // 16 MiB
for remaining := span; ; {
start := timeutil.Now()
b := txn.NewBatch()
r := roachpb.NewScan(remaining.Key, remaining.EndKey).(*roachpb.ScanRequest)
r.ScanFormat = roachpb.BATCH_RESPONSE
b.Header.MaxSpanRequestKeys = maxKeysPerScan
b.Header.TargetBytes = targetBytesPerScan
// NB: We use a raw request rather than the Scan() method because we want
// the MVCC timestamps which are encoded in the response but are filtered
// during result parsing.
Expand Down

0 comments on commit 3433fbd

Please sign in to comment.