Jason Volk
|
66231676f1
|
gracefully ignore unknown columns; add dropped flag in descriptor
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-10 22:29:05 +00:00 |
|
Jason Volk
|
16fa2eca87
|
add conf item for write buffer size
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-10 07:03:34 +00:00 |
|
Jason Volk
|
6a0f9add0c
|
refactor database engine/options; add column descriptors
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-10 07:03:34 +00:00 |
|
Jason Volk
|
02f19cf951
|
tweak tracing spans; inlines
db deserializer tracing instrument cover
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-10 07:03:34 +00:00 |
|
Jason Volk
|
685b127f99
|
simplify iterator state constructor arguments
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-10 07:03:34 +00:00 |
|
Jason Volk
|
44e6b1af3c
|
fixes for tests to be run in release-mode
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-09 19:56:42 +00:00 |
|
Jason Volk
|
94c8683836
|
improve db pool topology configuration
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-09 18:14:23 +00:00 |
|
Jason Volk
|
d36167ab64
|
partially revert 9a9c071e82; use std threads for db pool.
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-09 18:14:23 +00:00 |
|
strawberry
|
7a349fdc58
|
use latest sccache, bump nix flake lock, cache everything in rust-cache
Signed-off-by: strawberry <strawberry@puppygock.gay>
|
2025-01-02 19:13:27 -05:00 |
|
Jason Volk
|
77d7e8a3ad
|
add json array to serde_json::Value db deserialization without Raw
|
2025-01-02 06:52:32 +00:00 |
|
Jason Volk
|
2259e2c82f
|
batch queries to maximize throughput
query-side streams for first level of callsites
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
1792711d09
|
support core affinity mask
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
c5b94be5b8
|
remove dev_release_log_level cfg
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
2bbb28bb88
|
get device info with libc using major/minor
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
b56e480b3c
|
integrate some std io error kinds with our status code abstraction
associate rocksdb error kinds with std io error kinds
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
74eb30c106
|
add LIFO scheduling for database frontend pool workers
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
7a6d657558
|
configurable dynamic stream concurrency scalar
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
b195107053
|
optimize for multi-queue storage topologies with affinity
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
a3d87be22f
|
enable atomic_flush for database columns
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
98e6c81e49
|
offload remaining db iterator initial seeks on cache miss
consume task budget on cache hit
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
14341bb906
|
simplify and improve db iter State init interface
Signed-off-by: Jason Volk <jason@zemos.net>
|
2025-01-01 23:28:01 -05:00 |
|
Jason Volk
|
4d46df2af5
|
abort tasks for non-async pool shudown
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-18 14:16:23 -05:00 |
|
Jason Volk
|
5f7636f177
|
set dbpool thread name (gated by tokio_unstable)
Signed-off-by: Jason Volk <jason@zemos.net>
Signed-off-by: strawberry <strawberry@puppygock.gay>
|
2024-12-18 13:31:41 -05:00 |
|
Jason Volk
|
ad8cbcaac1
|
check cache prior to offloading iterator seek
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-18 13:29:43 -05:00 |
|
strawberry
|
77e0b76408
|
apply new rustfmt.toml changes, fix some clippy lints
Signed-off-by: strawberry <strawberry@puppygock.gay>
|
2024-12-15 01:00:41 -05:00 |
|
strawberry
|
0317cc8cc5
|
rename conduit to conduwuit finally
Signed-off-by: strawberry <strawberry@puppygock.gay>
|
2024-12-14 22:24:45 -05:00 |
|
Jason Volk
|
aba88ccead
|
misc tracing span tweaks
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-10 06:20:30 +00:00 |
|
strawberry
|
faf48405aa
|
bump rocksdb to v9.8.4, enable track_and_verify_wals_in_manifest by default
Signed-off-by: strawberry <strawberry@puppygock.gay>
|
2024-12-09 12:52:06 -05:00 |
|
strawberry
|
61670370ed
|
add rocksdb paranoid_file_checks config option, add some more config checks
Signed-off-by: strawberry <strawberry@puppygock.gay>
|
2024-12-09 12:28:14 -05:00 |
|
Jason Volk
|
ff56ee7413
|
add back the consume_budget on cache hits
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-08 07:12:27 +00:00 |
|
Jason Volk
|
6aa35260e6
|
misc database options tweaks
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-08 07:12:27 +00:00 |
|
Jason Volk
|
65a370836c
|
derive specific ReadOptions for iterators
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-08 07:12:27 +00:00 |
|
Jason Volk
|
48703173bc
|
split get_batch from get.rs; add aqry_batch
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-04 00:51:57 +00:00 |
|
Jason Volk
|
3109c0daba
|
perform async shutdown for database pool after services stop
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-04 00:51:57 +00:00 |
|
Jason Volk
|
b5006a4c41
|
offload initial iterator seeks to threadpool
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-03 13:25:33 +00:00 |
|
Jason Volk
|
9a9c071e82
|
use tokio for threadpool mgmt
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-12-03 07:39:02 +00:00 |
|
Jason Volk
|
2a9bb1ce11
|
add configurables for frontend pool options
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-28 07:20:43 +00:00 |
|
Jason Volk
|
3ad6aa59f9
|
use smallvec for db query buffering
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-28 06:03:33 +00:00 |
|
Jason Volk
|
c7ae951676
|
add frontend threadpool to database
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-27 10:53:44 +00:00 |
|
Jason Volk
|
527494a34b
|
fix oversized tracing span arguments; lints
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-27 06:30:20 +00:00 |
|
morguldir
|
63d1fcf213
|
add queued transactions rocksdb cf cache
Signed-off-by: strawberry <strawberry@puppygock.gay>
|
2024-11-25 16:08:30 -05:00 |
|
Jason Volk
|
62d560e2fb
|
improve tracing instruments on database::map
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-25 07:00:55 +00:00 |
|
Jason Volk
|
343ec59a8b
|
use arrayvec for integer deserialization buffer
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-25 02:54:29 +00:00 |
|
Jason Volk
|
3789d60b6a
|
refactor to iterator inputs for auth_chain/short batch functions
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-24 21:30:32 +00:00 |
|
strawberry
|
9100af9974
|
add eventid_pdu database cf cache
Signed-off-by: strawberry <strawberry@puppygock.gay>
|
2024-11-21 23:45:16 -05:00 |
|
Jason Volk
|
7680d1bd5e
|
replace yields point with consume_budget
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-19 08:37:25 +00:00 |
|
Jason Volk
|
90106c4c33
|
streamline batch insertions
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-19 08:37:25 +00:00 |
|
Jason Volk
|
14e3b242df
|
add database get_batch stream wrapper
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-16 00:33:40 +00:00 |
|
Jason Volk
|
20836cc3db
|
flush=false for database-backup in read-only/secondary modes; improve error
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-16 00:33:40 +00:00 |
|
Jason Volk
|
59834a4b05
|
add is_read_only()/is_secondary() to Engine
Signed-off-by: Jason Volk <jason@zemos.net>
|
2024-11-16 00:33:40 +00:00 |
|