perf: optimize ResultSet decoding

[olavloite]

Optimize ResultSet Decoding

ResultSet decoding went through a long if-elif-else construct for every row and every column to determine how to decode that specific cell. This caused large result sets to see a significantly higher decoding time than necessary, as determining how to decode a column only needs to be determined once for the entire ResultSet.

This change therefore collects the decoders once before starting to decode any rows. It does this by:

1. Iterating over the columns in the ResultSet and get a decoder for the specific type of that column.
2. Store those decoders as function references in an array.
3. Pick the appropriate function directly from this array each time a column needs to be decoded.

Selecting and decoding a query result with 100 rows consisting of 24 columns (one for each supported data type) takes ~35-40ms without this change, and ~15-20ms with this change. The following benchmarks were executed locally against an in-mem mock Spanner server running in Java. The latter was chosen because:

1. We have a random ResultSet generator in Java that can be used for this.
2. Having the mock Spanner server running in a separate process and in another programming language reduces the chance that the mock server itself has an impact on the differences that we see between the different runs.

Lazy Decoding

This change also introduces an option to lazily decode result sets. This option can be used if the application wants to decode (some of) the data manually, or access the raw underlying data. This can significantly reduce the decode time that is needed for large result sets, especially if the results contain structured data (e.g JSON), and the application just wants to access the underlying strings.

Benchmarks

Results without this change (100 iterations):

Elapsed:  43.5490608215332 ms
Elapsed:  39.53838348388672 ms
Elapsed:  38.68389129638672 ms
Elapsed:  38.26117515563965 ms
Elapsed:  38.28692436218262 ms
Elapsed:  38.12098503112793 ms
Elapsed:  39.016008377075195 ms
Elapsed:  38.15174102783203 ms
Elapsed:  38.3448600769043 ms
Elapsed:  38.00082206726074 ms
Elapsed:  38.0091667175293 ms
Elapsed:  38.02800178527832 ms
Elapsed:  38.03110122680664 ms
Elapsed:  38.42306137084961 ms
Elapsed:  38.535356521606445 ms
Elapsed:  38.86699676513672 ms
Elapsed:  38.702964782714844 ms
Elapsed:  38.881778717041016 ms
Elapsed:  38.08116912841797 ms
Elapsed:  38.084983825683594 ms
Elapsed:  38.04278373718262 ms
Elapsed:  38.74492645263672 ms
Elapsed:  38.57111930847168 ms
Elapsed:  38.17009925842285 ms
Elapsed:  38.64407539367676 ms
Elapsed:  38.00559043884277 ms
Elapsed:  38.06161880493164 ms
Elapsed:  38.233280181884766 ms
Elapsed:  38.48695755004883 ms
Elapsed:  38.71011734008789 ms
Elapsed:  37.92428970336914 ms
Elapsed:  38.8491153717041 ms
Elapsed:  38.90705108642578 ms
Elapsed:  38.20919990539551 ms
Elapsed:  38.07401657104492 ms
Elapsed:  38.30099105834961 ms
Elapsed:  38.07377815246582 ms
Elapsed:  38.61117362976074 ms
Elapsed:  39.58392143249512 ms
Elapsed:  39.69216346740723 ms
Elapsed:  38.27810287475586 ms
Elapsed:  37.88185119628906 ms
Elapsed:  38.763999938964844 ms
Elapsed:  39.05320167541504 ms
Elapsed:  38.82408142089844 ms
Elapsed:  38.47217559814453 ms
Elapsed:  38.024187088012695 ms
Elapsed:  38.07687759399414 ms
Elapsed:  38.11931610107422 ms
Elapsed:  37.9488468170166 ms
Elapsed:  38.04421424865723 ms
Elapsed:  38.57421875 ms
Elapsed:  39.543867111206055 ms
Elapsed:  38.4981632232666 ms
Elapsed:  37.89806365966797 ms
Elapsed:  38.0861759185791 ms
Elapsed:  38.72990608215332 ms
Elapsed:  38.47217559814453 ms
Elapsed:  38.71774673461914 ms
Elapsed:  38.27619552612305 ms
Elapsed:  38.08403015136719 ms
Elapsed:  38.6350154876709 ms
Elapsed:  38.03229331970215 ms
Elapsed:  39.01100158691406 ms
Elapsed:  38.4981632232666 ms
Elapsed:  38.25807571411133 ms
Elapsed:  38.59400749206543 ms
Elapsed:  38.83624076843262 ms
Elapsed:  38.584232330322266 ms
Elapsed:  39.54625129699707 ms
Elapsed:  38.268089294433594 ms
Elapsed:  39.3218994140625 ms
Elapsed:  37.9948616027832 ms
Elapsed:  38.05804252624512 ms
Elapsed:  38.88821601867676 ms
Elapsed:  38.08021545410156 ms
Elapsed:  38.22588920593262 ms
Elapsed:  37.97507286071777 ms
Elapsed:  38.03110122680664 ms
Elapsed:  37.91308403015137 ms
Elapsed:  38.00201416015625 ms
Elapsed:  38.529157638549805 ms
Elapsed:  38.44308853149414 ms
Elapsed:  38.87534141540527 ms
Elapsed:  38.85912895202637 ms
Elapsed:  38.48695755004883 ms
Elapsed:  38.41686248779297 ms
Elapsed:  38.10882568359375 ms
Elapsed:  37.98198699951172 ms
Elapsed:  38.50507736206055 ms
Elapsed:  38.16986083984375 ms
Elapsed:  38.07711601257324 ms
Elapsed:  37.92715072631836 ms
Elapsed:  37.93692588806152 ms
Elapsed:  38.04588317871094 ms
Elapsed:  38.62190246582031 ms
Elapsed:  38.5129451751709 ms
Elapsed:  37.960052490234375 ms
Elapsed:  37.99295425415039 ms
Elapsed:  38.45930099487305 ms

Results with this change:

Elapsed:  21.09503746032715 ms
Elapsed:  17.00878143310547 ms
Elapsed:  17.43626594543457 ms
Elapsed:  16.201019287109375 ms
Elapsed:  16.66712760925293 ms
Elapsed:  15.926837921142578 ms
Elapsed:  16.408205032348633 ms
Elapsed:  16.13783836364746 ms
Elapsed:  16.27206802368164 ms
Elapsed:  17.15087890625 ms
Elapsed:  16.06607437133789 ms
Elapsed:  16.852855682373047 ms
Elapsed:  23.713111877441406 ms
Elapsed:  17.20905303955078 ms
Elapsed:  16.60609245300293 ms
Elapsed:  16.30997657775879 ms
Elapsed:  15.933990478515625 ms
Elapsed:  15.688180923461914 ms
Elapsed:  16.228914260864258 ms
Elapsed:  16.252994537353516 ms
Elapsed:  16.33000373840332 ms
Elapsed:  15.842676162719727 ms
Elapsed:  16.328096389770508 ms
Elapsed:  16.4949893951416 ms
Elapsed:  16.47210121154785 ms
Elapsed:  16.674041748046875 ms
Elapsed:  15.768766403198242 ms
Elapsed:  16.48569107055664 ms
Elapsed:  15.876054763793945 ms
Elapsed:  16.852140426635742 ms
Elapsed:  16.035079956054688 ms
Elapsed:  16.407012939453125 ms
Elapsed:  15.882015228271484 ms
Elapsed:  16.71886444091797 ms
Elapsed:  15.86294174194336 ms
Elapsed:  16.566038131713867 ms
Elapsed:  15.904903411865234 ms
Elapsed:  16.289234161376953 ms
Elapsed:  16.14999771118164 ms
Elapsed:  16.31784439086914 ms
Elapsed:  16.106843948364258 ms
Elapsed:  16.581058502197266 ms
Elapsed:  16.435861587524414 ms
Elapsed:  15.904903411865234 ms
Elapsed:  16.408205032348633 ms
Elapsed:  16.062021255493164 ms
Elapsed:  16.256093978881836 ms
Elapsed:  15.87367057800293 ms
Elapsed:  16.23702049255371 ms
Elapsed:  16.745805740356445 ms
Elapsed:  15.92707633972168 ms
Elapsed:  16.142845153808594 ms
Elapsed:  16.492843627929688 ms
Elapsed:  21.553754806518555 ms
Elapsed:  17.05002784729004 ms
Elapsed:  16.932964324951172 ms
Elapsed:  16.810894012451172 ms
Elapsed:  16.577720642089844 ms
Elapsed:  15.714168548583984 ms
Elapsed:  16.2351131439209 ms
Elapsed:  16.072988510131836 ms
Elapsed:  16.038894653320312 ms
Elapsed:  16.055822372436523 ms
Elapsed:  16.378164291381836 ms
Elapsed:  15.806913375854492 ms
Elapsed:  15.5792236328125 ms
Elapsed:  15.954732894897461 ms
Elapsed:  15.566825866699219 ms
Elapsed:  15.707969665527344 ms
Elapsed:  15.514135360717773 ms
Elapsed:  15.43116569519043 ms
Elapsed:  15.332937240600586 ms
Elapsed:  15.470027923583984 ms
Elapsed:  15.269756317138672 ms
Elapsed:  15.250921249389648 ms
Elapsed:  15.47694206237793 ms
Elapsed:  15.306949615478516 ms
Elapsed:  15.72728157043457 ms
Elapsed:  15.938043594360352 ms
Elapsed:  16.324996948242188 ms
Elapsed:  16.198158264160156 ms
Elapsed:  15.982627868652344 ms
Elapsed:  16.308069229125977 ms
Elapsed:  17.843246459960938 ms
Elapsed:  15.820026397705078 ms
Elapsed:  16.428232192993164 ms
Elapsed:  15.978097915649414 ms
Elapsed:  16.347885131835938 ms
Elapsed:  16.026020050048828 ms
Elapsed:  16.362905502319336 ms
Elapsed:  16.900062561035156 ms
Elapsed:  17.3337459564209 ms
Elapsed:  17.65608787536621 ms
Elapsed:  20.101070404052734 ms
Elapsed:  18.137216567993164 ms
Elapsed:  16.952991485595703 ms
Elapsed:  16.7691707611084 ms
Elapsed:  16.71290397644043 ms
Elapsed:  16.3421630859375 ms
Elapsed:  16.36195182800293 ms

Benchmarking Lazy Decode

Reading the same result set with lazy decoding enabled (and no further decoding of the data):

Elapsed:  3.3600330352783203 ms
Elapsed:  3.0918121337890625 ms
Elapsed:  2.929210662841797 ms
Elapsed:  3.748178482055664 ms
Elapsed:  3.213167190551758 ms
Elapsed:  3.747224807739258 ms
Elapsed:  3.6122798919677734 ms
Elapsed:  3.6940574645996094 ms
Elapsed:  3.075838088989258 ms
Elapsed:  2.7778148651123047 ms
Elapsed:  2.5119781494140625 ms
Elapsed:  2.4051666259765625 ms
Elapsed:  2.3148059844970703 ms
Elapsed:  2.321958541870117 ms
Elapsed:  2.5272369384765625 ms
Elapsed:  2.5489330291748047 ms
Elapsed:  2.547025680541992 ms
Elapsed:  2.463102340698242 ms
Elapsed:  2.4421215057373047 ms
Elapsed:  2.454996109008789 ms
Elapsed:  2.4137496948242188 ms
Elapsed:  2.702951431274414 ms
Elapsed:  2.540111541748047 ms
Elapsed:  2.761363983154297 ms
Elapsed:  2.679109573364258 ms
Elapsed:  2.538919448852539 ms
Elapsed:  2.930879592895508 ms
Elapsed:  13.792037963867188 ms
Elapsed:  2.5997161865234375 ms
Elapsed:  2.32696533203125 ms
Elapsed:  2.341032028198242 ms
Elapsed:  2.5482177734375 ms
Elapsed:  3.1960010528564453 ms
Elapsed:  2.435922622680664 ms
Elapsed:  2.359151840209961 ms
Elapsed:  2.2530555725097656 ms
Elapsed:  2.424955368041992 ms
Elapsed:  2.4061203002929688 ms
Elapsed:  2.3589134216308594 ms
Elapsed:  2.7451515197753906 ms
Elapsed:  2.5119781494140625 ms
Elapsed:  2.5069713592529297 ms
Elapsed:  2.401113510131836 ms
Elapsed:  2.8276443481445312 ms
Elapsed:  2.7098655700683594 ms
Elapsed:  2.405881881713867 ms
Elapsed:  2.4399757385253906 ms
Elapsed:  2.5370121002197266 ms
Elapsed:  2.2652149200439453 ms
Elapsed:  2.3300647735595703 ms
Elapsed:  2.2721290588378906 ms
Elapsed:  2.1958351135253906 ms
Elapsed:  3.5200119018554688 ms
Elapsed:  2.151012420654297 ms
Elapsed:  2.110004425048828 ms
Elapsed:  2.0599365234375 ms
Elapsed:  2.0401477813720703 ms
Elapsed:  2.0842552185058594 ms
Elapsed:  2.029895782470703 ms
Elapsed:  2.0720958709716797 ms
Elapsed:  2.026081085205078 ms
Elapsed:  2.028226852416992 ms
Elapsed:  2.0978450775146484 ms
Elapsed:  1.9879341125488281 ms
Elapsed:  2.0449161529541016 ms
Elapsed:  2.5441646575927734 ms
Elapsed:  2.2101402282714844 ms
Elapsed:  2.2950172424316406 ms
Elapsed:  2.2792816162109375 ms
Elapsed:  2.3500919342041016 ms
Elapsed:  2.3832321166992188 ms
Elapsed:  2.3360252380371094 ms
Elapsed:  2.353191375732422 ms
Elapsed:  2.077817916870117 ms
Elapsed:  2.4199485778808594 ms
Elapsed:  2.312898635864258 ms
Elapsed:  2.1369457244873047 ms
Elapsed:  2.4871826171875 ms
Elapsed:  2.3207664489746094 ms
Elapsed:  2.3491382598876953 ms
Elapsed:  2.907991409301758 ms
Elapsed:  2.399921417236328 ms
Elapsed:  2.2487640380859375 ms
Elapsed:  2.343893051147461 ms
Elapsed:  2.3610591888427734 ms
Elapsed:  2.2449493408203125 ms
Elapsed:  2.474069595336914 ms
Elapsed:  2.414226531982422 ms
Elapsed:  2.2780895233154297 ms
Elapsed:  2.328157424926758 ms
Elapsed:  3.0121803283691406 ms
Elapsed:  3.253936767578125 ms
Elapsed:  2.691984176635742 ms
Elapsed:  2.5730133056640625 ms
Elapsed:  2.5200843811035156 ms
Elapsed:  2.295970916748047 ms
Elapsed:  2.4101734161376953 ms
Elapsed:  2.721071243286133 ms
Elapsed:  2.354145050048828 ms
Elapsed:  2.4611949920654297 ms

[sechava4]

Timings Looking better with this change:

Compared to what we use to have before: