Incorrect not in function behavior for rough set filter when one parameter is column type

[yibin87]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

mysql> create table t1 (a int unsigned);
mysql> insert into t1 values(300);
mysql> insert into t1 values(null);
mysql> insert into t1 values(981);
mysql> alter table t1 set tiflash replica 1;
mysql> set session tidb_isolation_read_engines='tiflash';
mysql> select * from t1 where not cast(a as signed) not in (a, 88);

2. What did you expect to see? (Required)

+------+
| a |
+------+
| 300 |
| 981 |
+------+

3. What did you see instead (Required)

Empty result set.

4. What is your TiFlash version? (Required)

[yibin87]

From execution summary, RoughSet filter bahaves incorrectly:

mysql> explain analyze  select * from t1 where not cast(a as signed) not in (a, 88);
+----------------------------+---------+---------+--------------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------+-----------+------+
| id                         | estRows | actRows | task         | access object | execution info                                                                                                                                                                                                                                                                                                                                                                                                          | operator info                                            | memory    | disk |
+----------------------------+---------+---------+--------------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------+-----------+------+
| TableReader_13             | 2.40    | 0       | root         |               | time:8.33ms, loops:1, RU:0.000000, cop_task: {num: 1, max: 0s, proc_keys: 0, copr_cache_hit_ratio: 0.00}                                                                                                                                                                                                                                                                                                                | MppVersion: 2, data:ExchangeSender_12                    | 305 Bytes | N/A  |
| └─ExchangeSender_12        | 2.40    | 0       | mpp[tiflash] |               | tiflash_task:{time:5.13ms, loops:0, threads:8}                                                                                                                                                                                                                                                                                                                                                                          | ExchangeType: PassThrough                                | N/A       | N/A  |
|   └─Selection_11           | 2.40    | 0       | mpp[tiflash] |               | tiflash_task:{time:4.92ms, loops:0, threads:8}                                                                                                                                                                                                                                                                                                                                                                          | in(cast(test.t1.a, bigint(22) BINARY), test.t1.a, 88)    | N/A       | N/A  |
|     └─TableFullScan_10     | 3.00    | 0       | mpp[tiflash] | table:t1      | tiflash_task:{time:4.92ms, loops:0, threads:8}, tiflash_scan:{dtfile:{total_scanned_packs:0, total_skipped_packs:1, total_scanned_rows:0, total_skipped_rows:3, total_rs_index_check_time: 0ms, total_read_time: 0ms, total_disagg_read_cache_hit_size: 0, total_disagg_read_cache_miss_size: 0}, total_create_snapshot_time: 0ms, total_local_region_num: 1, total_remote_region_num: 0, total_learner_read_time: 0ms} | pushed down filter:empty, keep order:false, stats:pseudo | N/A       | N/A  |
+----------------------------+---------+---------+--------------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------+-----------+------+

[Lloyd-Pottiger]

introduce by #8212

[JaySon-Huang]

Does not affects release-7.5