Extstore in parts for review#8
Open
artikell wants to merge 168 commits into
Open
Conversation
artikell
pushed a commit
that referenced
this pull request
Aug 4, 2025
…y-io#2257) **Current state** During `hashtableScanDefrag`, rehashing is paused to prevent entries from moving, but the scan callback can still delete entries which triggers `hashtableShrinkIfNeeded`. For example, the `expireScanCallback` can delete expired entries. **Issue** This can cause the table to be resized and the old memory to be freed while the scan is still accessing it, resulting in the following memory access violation: ``` [err]: Sanitizer error: ================================================================= ==46774==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000003100 at pc 0x0000004704d3 bp 0x7fffcb062000 sp 0x7fffcb061ff0 READ of size 1 at 0x611000003100 thread T0 #0 0x4704d2 in isPositionFilled /home/gusakovy/Projects/valkey/src/hashtable.c:422 #1 0x478b45 in hashtableScanDefrag /home/gusakovy/Projects/valkey/src/hashtable.c:1768 #2 0x4789c2 in hashtableScan /home/gusakovy/Projects/valkey/src/hashtable.c:1729 #3 0x47e3ca in kvstoreScan /home/gusakovy/Projects/valkey/src/kvstore.c:402 #4 0x6d9040 in activeExpireCycle /home/gusakovy/Projects/valkey/src/expire.c:297 #5 0x4859d2 in databasesCron /home/gusakovy/Projects/valkey/src/server.c:1269 #6 0x486e92 in serverCron /home/gusakovy/Projects/valkey/src/server.c:1577 #7 0x4637dd in processTimeEvents /home/gusakovy/Projects/valkey/src/ae.c:370 #8 0x4643e3 in aeProcessEvents /home/gusakovy/Projects/valkey/src/ae.c:513 #9 0x4647ea in aeMain /home/gusakovy/Projects/valkey/src/ae.c:543 valkey-io#10 0x4a61fc in main /home/gusakovy/Projects/valkey/src/server.c:7291 valkey-io#11 0x7f471957c139 in __libc_start_main (/lib64/libc.so.6+0x21139) valkey-io#12 0x452e39 in _start (/local/home/gusakovy/Projects/valkey/src/valkey-server+0x452e39) 0x611000003100 is located 0 bytes inside of 256-byte region [0x611000003100,0x611000003200) freed by thread T0 here: #0 0x7f471a34a1e5 in __interceptor_free (/lib64/libasan.so.4+0xd81e5) #1 0x4aefbc in zfree_internal /home/gusakovy/Projects/valkey/src/zmalloc.c:400 #2 0x4aeff5 in valkey_free /home/gusakovy/Projects/valkey/src/zmalloc.c:415 #3 0x4707d2 in rehashingCompleted /home/gusakovy/Projects/valkey/src/hashtable.c:456 #4 0x471b5b in resize /home/gusakovy/Projects/valkey/src/hashtable.c:656 #5 0x475bff in hashtableShrinkIfNeeded /home/gusakovy/Projects/valkey/src/hashtable.c:1272 #6 0x47704b in hashtablePop /home/gusakovy/Projects/valkey/src/hashtable.c:1448 #7 0x47716f in hashtableDelete /home/gusakovy/Projects/valkey/src/hashtable.c:1459 #8 0x480038 in kvstoreHashtableDelete /home/gusakovy/Projects/valkey/src/kvstore.c:847 #9 0x50c12c in dbGenericDeleteWithDictIndex /home/gusakovy/Projects/valkey/src/db.c:490 valkey-io#10 0x515f28 in deleteExpiredKeyAndPropagateWithDictIndex /home/gusakovy/Projects/valkey/src/db.c:1831 valkey-io#11 0x516103 in deleteExpiredKeyAndPropagate /home/gusakovy/Projects/valkey/src/db.c:1844 valkey-io#12 0x6d8642 in activeExpireCycleTryExpire /home/gusakovy/Projects/valkey/src/expire.c:70 valkey-io#13 0x6d8706 in expireScanCallback /home/gusakovy/Projects/valkey/src/expire.c:139 valkey-io#14 0x478bd8 in hashtableScanDefrag /home/gusakovy/Projects/valkey/src/hashtable.c:1770 valkey-io#15 0x4789c2 in hashtableScan /home/gusakovy/Projects/valkey/src/hashtable.c:1729 valkey-io#16 0x47e3ca in kvstoreScan /home/gusakovy/Projects/valkey/src/kvstore.c:402 valkey-io#17 0x6d9040 in activeExpireCycle /home/gusakovy/Projects/valkey/src/expire.c:297 valkey-io#18 0x4859d2 in databasesCron /home/gusakovy/Projects/valkey/src/server.c:1269 valkey-io#19 0x486e92 in serverCron /home/gusakovy/Projects/valkey/src/server.c:1577 valkey-io#20 0x4637dd in processTimeEvents /home/gusakovy/Projects/valkey/src/ae.c:370 valkey-io#21 0x4643e3 in aeProcessEvents /home/gusakovy/Projects/valkey/src/ae.c:513 valkey-io#22 0x4647ea in aeMain /home/gusakovy/Projects/valkey/src/ae.c:543 valkey-io#23 0x4a61fc in main /home/gusakovy/Projects/valkey/src/server.c:7291 valkey-io#24 0x7f471957c139 in __libc_start_main (/lib64/libc.so.6+0x21139) previously allocated by thread T0 here: #0 0x7f471a34a753 in __interceptor_calloc (/lib64/libasan.so.4+0xd8753) #1 0x4ae48c in ztrycalloc_usable_internal /home/gusakovy/Projects/valkey/src/zmalloc.c:214 #2 0x4ae757 in valkey_calloc /home/gusakovy/Projects/valkey/src/zmalloc.c:257 #3 0x4718fc in resize /home/gusakovy/Projects/valkey/src/hashtable.c:645 #4 0x475bff in hashtableShrinkIfNeeded /home/gusakovy/Projects/valkey/src/hashtable.c:1272 #5 0x47704b in hashtablePop /home/gusakovy/Projects/valkey/src/hashtable.c:1448 #6 0x47716f in hashtableDelete /home/gusakovy/Projects/valkey/src/hashtable.c:1459 #7 0x480038 in kvstoreHashtableDelete /home/gusakovy/Projects/valkey/src/kvstore.c:847 #8 0x50c12c in dbGenericDeleteWithDictIndex /home/gusakovy/Projects/valkey/src/db.c:490 #9 0x515f28 in deleteExpiredKeyAndPropagateWithDictIndex /home/gusakovy/Projects/valkey/src/db.c:1831 valkey-io#10 0x516103 in deleteExpiredKeyAndPropagate /home/gusakovy/Projects/valkey/src/db.c:1844 valkey-io#11 0x6d8642 in activeExpireCycleTryExpire /home/gusakovy/Projects/valkey/src/expire.c:70 valkey-io#12 0x6d8706 in expireScanCallback /home/gusakovy/Projects/valkey/src/expire.c:139 valkey-io#13 0x478bd8 in hashtableScanDefrag /home/gusakovy/Projects/valkey/src/hashtable.c:1770 valkey-io#14 0x4789c2 in hashtableScan /home/gusakovy/Projects/valkey/src/hashtable.c:1729 valkey-io#15 0x47e3ca in kvstoreScan /home/gusakovy/Projects/valkey/src/kvstore.c:402 valkey-io#16 0x6d9040 in activeExpireCycle /home/gusakovy/Projects/valkey/src/expire.c:297 valkey-io#17 0x4859d2 in databasesCron /home/gusakovy/Projects/valkey/src/server.c:1269 valkey-io#18 0x486e92 in serverCron /home/gusakovy/Projects/valkey/src/server.c:1577 valkey-io#19 0x4637dd in processTimeEvents /home/gusakovy/Projects/valkey/src/ae.c:370 valkey-io#20 0x4643e3 in aeProcessEvents /home/gusakovy/Projects/valkey/src/ae.c:513 valkey-io#21 0x4647ea in aeMain /home/gusakovy/Projects/valkey/src/ae.c:543 valkey-io#22 0x4a61fc in main /home/gusakovy/Projects/valkey/src/server.c:7291 valkey-io#23 0x7f471957c139 in __libc_start_main (/lib64/libc.so.6+0x21139) SUMMARY: AddressSanitizer: heap-use-after-free /home/gusakovy/Projects/valkey/src/hashtable.c:422 in isPositionFilled Shadow bytes around the buggy address: 0x0c227fff85d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c227fff85e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c227fff85f0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd 0x0c227fff8600: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c227fff8610: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa =>0x0c227fff8620:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c227fff8630: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c227fff8640: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c227fff8650: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c227fff8660: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c227fff8670: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==46774==ABORTING ``` **Solution** Suggested solution is to also pause auto shrinking during `hashtableScanDefrag`. I noticed that there was already a `hashtablePauseAutoShrink` method and `pause_auto_shrink` counter, but it wasn't actually used in `hashtableShrinkIfNeeded` so I fixed that. **Testing** I created a simple tcl test that (most of the times) triggers this error, but it's a little clunky so I didn't add it as part of the PR: ``` start_server {tags {"expire hashtable defrag"}} { test {hashtable scan defrag on expiry} { r config set hz 100 set num_keys 20 for {set i 0} {$i < $num_keys} {incr i} { r set "key_$i" "value_$i" } for {set j 0} {$j < 50} {incr j} { set expire_keys 100 for {set i 0} {$i < $expire_keys} {incr i} { # Short expiry time to ensure they expire quickly r psetex "expire_key_${i}_${j}" 100 "expire_value_${i}_${j}" } # Verify keys are set set initial_size [r dbsize] assert_equal $initial_size [expr $num_keys + $expire_keys] after 150 for {set i 0} {$i < 10} {incr i} { r get "expire_key_${i}_${j}" after 10 } } set remaining_keys [r dbsize] assert_equal $remaining_keys $num_keys # Verify server is still responsive assert_equal [r ping] {PONG} } {} } ``` Compiling with ASAN using `make noopt SANITIZER=address valkey-server` and running the test causes error above. Applying the fix resolves the issue. Signed-off-by: Yakov Gusakov <yaakov0015@gmail.com>
kronwerk
force-pushed
the
extstore_in_parts
branch
3 times, most recently
from
9fb94a7 to
783b796
Compare
artikell
pushed a commit
that referenced
this pull request
Oct 14, 2025
With valkey-io#1401, we introduced additional filters to CLIENT LIST/KILL subcommand. The intended behavior was to pick the last value of the filter. However, we introduced memory leak for all the preceding filters. Before this change: ``` > CLIENT LIST IP 127.0.0.1 IP 127.0.0.1 id=4 addr=127.0.0.1:37866 laddr=127.0.0.1:6379 fd=10 name= age=0 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=21 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=16989 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=49 tot-net-out=0 tot-cmds=0 ``` Leak: ``` Direct leak of 11 byte(s) in 1 object(s) allocated from: #0 0x7f2901aa557d in malloc (/lib64/libasan.so.4+0xd857d) #1 0x76db76 in ztrymalloc_usable_internal /workplace/harkrisp/valkey/src/zmalloc.c:156 #2 0x76db76 in zmalloc_usable /workplace/harkrisp/valkey/src/zmalloc.c:200 #3 0x4c4121 in _sdsnewlen.constprop.230 /workplace/harkrisp/valkey/src/sds.c:113 #4 0x4dc456 in parseClientFiltersOrReply.constprop.63 /workplace/harkrisp/valkey/src/networking.c:4264 #5 0x4bb9f7 in clientListCommand /workplace/harkrisp/valkey/src/networking.c:4600 #6 0x641159 in call /workplace/harkrisp/valkey/src/server.c:3772 #7 0x6431a6 in processCommand /workplace/harkrisp/valkey/src/server.c:4434 #8 0x4bfa9b in processCommandAndResetClient /workplace/harkrisp/valkey/src/networking.c:3571 #9 0x4bfa9b in processInputBuffer /workplace/harkrisp/valkey/src/networking.c:3702 valkey-io#10 0x4bffa3 in readQueryFromClient /workplace/harkrisp/valkey/src/networking.c:3812 valkey-io#11 0x481015 in callHandler /workplace/harkrisp/valkey/src/connhelpers.h:79 valkey-io#12 0x481015 in connSocketEventHandler.lto_priv.394 /workplace/harkrisp/valkey/src/socket.c:301 valkey-io#13 0x7d3fb3 in aeProcessEvents /workplace/harkrisp/valkey/src/ae.c:486 valkey-io#14 0x7d4d44 in aeMain /workplace/harkrisp/valkey/src/ae.c:543 valkey-io#15 0x453925 in main /workplace/harkrisp/valkey/src/server.c:7319 valkey-io#16 0x7f2900cd7139 in __libc_start_main (/lib64/libc.so.6+0x21139) ``` Note: For filter ID / NOT-ID we group all the option and perform filtering whereas for remaining filters we only pick the last filter option. --------- Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
kronwerk
force-pushed
the
extstore_in_parts
branch
2 times, most recently
from
5f7f960 to
7abf0a2
Compare
kronwerk
force-pushed
the
extstore_in_parts
branch
3 times, most recently
from
bf0ccfe to
8fd5791
Compare
kronwerk
force-pushed
the
extstore_in_parts
branch
5 times, most recently
from
113aff2 to
409423f
Compare
In `dbSetValue()` the `old` pointer may be reassigned to point to the incoming value object which was created without an embedded key, so calling `dbUntrackKeyWithVolatileItems()` would call `objectGetKey()` which returns NULL, causing a crash in `hashtableSdsHash()` when trying to hash the NULL key. Idea is to assign `old_was_hash_with_volatile` before the swap and use `new` instead of `old` for untracking when theres no embedded key. Introduced in valkey-io#3003 Run with NULL ptr dereference: https://github.com/valkey-io/valkey/actions/runs/20701343184/job/59424029880 --------- Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com> Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
In valkey-io#2876 we added cluster info stats to the info command. However these statistics where added to the "cluster" section which is part of the default info sections. Since the calculation of the cluster info is done "by-demand" it can lead to high server side CPU consumption when many clients are using the "info" command on a large cluster. In this PR we separated the cluster info to a dedicated "cluster_stats" section, which is not enabled by default. Alternatives considered: 1. Revert valkey-io#2876 - this is possible as the information is available via `CLUSTER INFO` command. The only downside is that some "control plane" adaptations need to be placed in order to gather this information in addition to the "already collected" server wide `INFO` data. 2. Optimize cluster info status collection - We could dynamically update and maintain these statistics during cluster topology state changes. The main "expensive" statistics which are needed to be collected are: slots_assigned, slots_ok, slots_pfail, slots_fail, nodes_pfail, nodes_fail, voting_nodes_pfail, voting_nodes_fail We could track and constantly update these during cluster pings. The main downside is potential bugs and bad accounting which might be caused by additional complexity and unhandled edge cases. --------- Signed-off-by: Ran Shidlansik <ranshid@amazon.com> Co-authored-by: Harkrishn Patro <bunty.hari@gmail.com>
Use the `lm_asprintf `method to solve the warm problem：
```
debug_lua.c: In function ‘ldbEval’:
debug_lua.c:470:13: warning: ignoring return value of ‘asprintf’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
470 | asprintf(&err_msg, "Error compiling code: %s", lua_tostring(lua, -1));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
---------
Signed-off-by: skyfirelee <739609084@qq.com>
) Introduced `clusterMsgHeader` and `toClusterMsg()` `toClusterMsgLight()` helpers to unify clusterMsg parsing and make casting explicit. switched packet read/validation/processing paths to use the common 16-byte header first, then cast to clusterMsg/clusterMsgLight only after determining message type. this is a refactor to improve readability and avoid accidental miscasts between full vs light headers. issue : valkey-io#2837 --------- Signed-off-by: Su Ko <rhtn1128@gmail.com>
…ey-io#2516) Achieved 20-30% reduction in overhead. This was accomplished by reusing the 8B robj->ptr memory to store embedded data. For 16B key and 16B value, this reduces the per-item memory overhead by ~20%. Overall performance was not measurably changed. Raising the threshold for embedding strings produces ~30% reduction in per-item overhead for affected value sizes. Here's what I did: 1. Modify all `robj->ptr` access to use get/set methods. Most of this was done with a script (code listed below), with a few manual touches. Manual and programmatic changes are in separate commits to make review easier. 2. Next I changed `objectGetVal()` to calculate the location instead of using `robj->ptr`, and modified the embedding code to start writing embedded data 8B earlier, overwriting the ptr location. I changed `objectSetVal()` to assert that no value is embedded - all of the code that assigned `o->ptr` would have been incorrect for embedded strings anyway. (Except for one instance in module.c which I made a separate method for.) --------- Signed-off-by: Rain Valentine <rsg000@gmail.com>
…-io#3006) The HFE uses EXPIRY_NONE(-1) for fields without a TTL. A bug exists in `HSETEX` and RDB loading where `expiry > 0` is used to check for an expiration. This is problematic because `0` might be treated as no expiry in `import-mode`, instead of an already expired timestamp, leading to incorrect behavior. --------- Signed-off-by: cjx-zar <jxchenczar@foxmail.com>
Introduce a json log format that can be invoked by passing `log-format json` in your config. We also refactor `escapeJsonString` to `util` since it is now a shared function. Closes: valkey-io#1006 --------- Signed-off-by: Johan Bergström <bugs@bergstroem.nu> Signed-off-by: Binbin <binloveplay1314@qq.com> Co-authored-by: Binbin <binloveplay1314@qq.com> Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Resolves: valkey-io#2228 Visualization: https://github.com/sarthakaggarwal97/valkey/actions/runs/19113712295 Currently, there are no tests running on the already released branches. We often do backport for bug fixes and CVEs in these older versions, and end up with multiple CI tests failures on these branches. The PR adds support for running weekly tests on already released versions `>= 7.2`. The workflow will execute the "daily" test workflow for each of these branches on `Sunday 06:00 UTC`. The idea is to continuously monitor our released versions through weekly test runs (during the time when is lesser activity on github runners). --------- Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
…ey-io#2945) Previously, only changes to argv[0] were considered when deciding whether to re-prepare the command and recompute the cluster slot. This caused issues where changes to the argument count (argc) would not trigger the necessary prepare in case additional keys were injected to the command This fix adds a check to ensure that changes to argc are properly considered. Signed-off-by: Patrik Hermansson <phermansson@gmail.com>
… explicitly set (valkey-io#2777) By default, servers in a cluster don't have human-readable nodename unless explicitly set by user. Most logging statements in the cluster are in the form of `serverLog(LL_NOTICE, "Node %.40s (%s) did something..", node->name, node->human_nodename, ...);` , and thus this would always result in an empty `()` like the following: ``` * Node dd4f43d9e4a1a2875a514733c08d4870c21db682 () is now a replica of node 1b5d2a4865c04aaa1ccf06db66f22a043411ded7 () in shard d3fc557d12ab3c5a8850c7b0c9afb252b473cad1 * Clear FAIL state for node dd4f43d9e4a1a2875a514733c08d4870c21db682 (): replica is reachable again. * NODE 524978a9924b5841c4c93f83581d202cfbce1443 () possibly failing. * FAIL message received from 747e0f97a58cd5a7ac4c2130417fc47a807802ed () about 1b5d2a4865c04aaa1ccf06db66f22a043411ded7 () ``` **This PR adds a helper function `humanNodename` specifically for logging purpose that can print the node's ip:port when its nodename is not explicitly set.** Now the logs can greatly help developers to quickly recognize which node is doing stuff: ``` * NODE 03b3811da7b33451f6e4988295b0158f410d2c00 (127.0.0.1:7001) possibly failing. * Marking node 03b3811da7b33451f6e4988295b0158f410d2c00 (127.0.0.1:7001) as failing (quorum reached). * NODE b2f0fc93632d12c06039335b025365d484ff9049 (127.0.0.1:7002) possibly failing. * Marking node b2f0fc93632d12c06039335b025365d484ff9049 (127.0.0.1:7002) as failing (quorum reached). # Cluster state changed: fail ``` There’s no behavior change in cluster state, gossip, failover, etc, and **the human nodename of each node is not changed**. This PR is for logging only and is safe: - For developers who never cared about the empty (), this is just a small quality-of-life improvement: suddenly the logs tell them which node is which, without extra node ID lookups. - For developers who rely on custom names they prefer, nothing changes – their explicit nodename takes precedence. Signed-off-by: Zhijun <dszhijun@gmail.com>
…n and data corruption (valkey-io#3004) When loading AOF in cluster mode, keys inside a MULTI/EXEC block could be inserted into wrong hash slots, causing key duplication and data corruption. The root cause was the slot caching optimization in getKeySlot(). This optimization reuses a cached slot value to avoid recalculating the hash for every key operation. However, when replaying AOF, a transaction may contain commands affecting keys in different slots. The cached slot from a previous command (e.g., SET k1) would incorrectly be used for subsequent commands in the transaction (e.g., SET k0), causing k0 to be stored in k1's slot. The existing code already skipped this optimization for replicated clients (commands from primary) using isReplicatedClient(). This change extends that to also skip for AOF clients by using mustObeyClient() instead, which covers both replicated clients and the AOF client. Fixes valkey-io#2995, introduced in valkey-io#1949. Signed-off-by: aditya.teltia <teltia.aditya22@gmail.com>
…on is in the past (valkey-io#3023) if an expired time is used, the condition is ignored, and it directly becomes the effect of the `hdel` command. This is mainly important for cases where the user would like to protect deleting the data in case the `HEXPIREAT` command is used and the user would like to protect delay execution of this command to delete fields. fixes: valkey-io#3021 --------- Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
**The problem/use-case that the feature addresses** I teach Valkey to people frequently. The general process is to start on standalone Valkey (or 'try me' Valkey) to get started. However, in the more advanced sections, I introduce the concepts of required to develop for a Valkey Cluster. The key learning here is to understand slots (e.g. multi-key operations, using hash tags, etc.). Standing up a cluster is a hassle for someone just learning, and, while I usually demo `CLUSTER KEYSLOT` for the users, they can't run that command on their single instance since all of the `CLUSTER` commands are blocked. I think Valkey students would learn a lot more quickly if they could experiment with `CLUSTER KEYSLOT` themselves in standalone mode. In this case, `CLUSTER KEYSLOT` doesn't require anything in the cluster - it just does the math on the key to determine the slot number. **Description of the feature** This feature unblocks `CLUSTER KEYSLOT` on single instances. **Alternatives you've considered** - Having a cluster on try-me valkey (too many resources) - Having single instance clusters (discussed before, but far more complex than this trivial change) **Additional information** Their might be less repetitious logic, but the entire bits that generates `CLUSTER KEYSLOT` result is just 3 lines, so repetitious feels fine? --------- Signed-off-by: Kyle J. Davis <kyledvs@amazon.com>
…ion (valkey-io#3030) Before valkey-io#2858, we has this check: ``` static int scriptVerifyOOM(scriptRunCtx *run_ctx, char **err) { if (run_ctx->flags & SCRIPT_ALLOW_OOM) { /* Allow running any command even if OOM reached */ return C_OK; } /* If we reached the memory limit configured via maxmemory, commands that * could enlarge the memory usage are not allowed, but only if this is the * first write in the context of this script, otherwise we can't stop * in the middle. */ if (server.maxmemory && /* Maxmemory is actually enabled. */ !mustObeyClient(run_ctx->original_client) && /* Don't care about mem for replicas or AOF. */ !(run_ctx->flags & SCRIPT_WRITE_DIRTY) && /* Script had no side effects so far. */ server.pre_command_oom_state && /* Detected OOM when script start. */ (run_ctx->c->cmd->flags & CMD_DENYOOM)) { *err = sdsdup(shared.oomerr->ptr); return C_ERR; } return C_OK; } ``` If we reached the memory limit configured via maxmemory, commands that could enlarge the memory usage are not allowed, but only if this is the first write in the context of this script, otherwise we can't stop in the middle. --------- Signed-off-by: Binbin <binloveplay1314@qq.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
…nc external data dump: Non-blocking behavior' Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
kronwerk
force-pushed
the
extstore_in_parts
branch
from
8c3d57c to
fcac7ec
Compare
madolson
pushed a commit
that referenced
this pull request
Mar 9, 2026
Fix new unittest networking use-after-free error
```
==96611==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000075e00 at pc 0x55e52cbe1495 bp 0x7ffd9e1fc690 sp 0x7ffd9e1fc688
READ of size 8 at 0x503000075e00 thread T0
#0 0x55e52cbe[149](https://github.com/valkey-io/valkey/actions/runs/13230922385/job/36927929457#step:10:150)4 in freeReplicaReferencedReplBuffer /home/runner/work/valkey/valkey/src/replication.c:401:27
#1 0x55e52cbe7abf in freeClientReplicationData /home/runner/work/valkey/valkey/src/replication.c:1261:5
#2 0x55e52cb17a44 in test_writeToReplica /home/runner/work/valkey/valkey/src/unit/test_networking.c:188:5
#3 0x55e52cac976b in runTestSuite /home/runner/work/valkey/valkey/src/unit/test_main.c:26:28
#4 0x55e52cac9bae in main /home/runner/work/valkey/valkey/src/unit/test_main.c:61:14
#5 0x7fded4c2a1c9 (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
#6 0x7fded4c2a28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
#7 0x55e52c9b5ec4 in _start (/home/runner/work/valkey/valkey/src/valkey-unit-tests+0x177ec4) (BuildId: 587aaf0e86abaf104cbb714f290b1436f8ddf614)
0x503000075e00 is located 16 bytes inside of 24-byte region [0x503000075df0,0x503000075e08)
freed by thread T0 here:
#0 0x55e52ca50a7a in free (/home/runner/work/valkey/valkey/src/valkey-unit-tests+0x212a7a) (BuildId: 587aaf0e86abaf104cbb714f290b1436f8ddf614)
#1 0x55e52cb905ba in listEmpty /home/runner/work/valkey/valkey/src/adlist.c:64:9
#2 0x55e52cb179e5 in test_writeToReplica /home/runner/work/valkey/valkey/src/unit/test_networking.c:179:9
#3 0x55e52cac976b in runTestSuite /home/runner/work/valkey/valkey/src/unit/test_main.c:26:28
#4 0x55e52cac9bae in main /home/runner/work/valkey/valkey/src/unit/test_main.c:61:14
#5 0x7fded4c2a1c9 (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
#6 0x7fded4c2a28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
#7 0x55e52c9b5ec4 in _start (/home/runner/work/valkey/valkey/src/valkey-unit-tests+0x177ec4) (BuildId: 587aaf0e86abaf104cbb714f290b1436f8ddf614)
previously allocated by thread T0 here:
#0 0x55e52ca50d13 in malloc (/home/runner/work/valkey/valkey/src/valkey-unit-tests+0x212d13) (BuildId: 587aaf0e86abaf104cbb714f290b1436f8ddf614)
#1 0x55e52cbb844f in ztrymalloc_usable_internal /home/runner/work/valkey/valkey/src/zmalloc.c:[155](https://github.com/valkey-io/valkey/actions/runs/13230922385/job/36927929457#step:10:156):17
#2 0x55e52cbb844f in valkey_malloc /home/runner/work/valkey/valkey/src/zmalloc.c:184:17
#3 0x55e52cb90be6 in listAddNodeTail /home/runner/work/valkey/valkey/src/adlist.c:126:17
#4 0x55e52cb17873 in test_writeToReplica /home/runner/work/valkey/valkey/src/unit/test_networking.c:167:9
#5 0x55e52cac976b in runTestSuite /home/runner/work/valkey/valkey/src/unit/test_main.c:26:28
#6 0x55e52cac9bae in main /home/runner/work/valkey/valkey/src/unit/test_main.c:61:14
#7 0x7fded4c2a1c9 (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
#8 0x7fded4c2a28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
#9 0x55e52c9b5ec4 in _start (/home/runner/work/valkey/valkey/src/valkey-unit-tests+0x177ec4) (BuildId: 587aaf0e86abaf104cbb714f290b1436f8ddf614)
```
https://github.com/valkey-io/valkey/actions/runs/13230922385/job/36927929457
Signed-off-by: Uri Yagelnik <uriy@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.