Commit graph

35 commits

Author SHA1 Message Date
Jason Volk
98f9570547 add option to disable rocksdb checksums
reference runtime state for default option initialization

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-28 18:38:47 +00:00
Jason Volk
d0b4a619af furnish batch interface with trait
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-28 18:36:00 +00:00
Jason Volk
94f2384fb0 fix missing iteration-optimized read options on several stream types
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-25 01:35:18 +00:00
Jason Volk
dda27ffcb1 add some compaction related interfaces
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-18 12:05:07 +00:00
Jason Volk
96e85adc32 use cache builder for row and table cache options
add cache check using multi-get path

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk
6a0f9add0c refactor database engine/options; add column descriptors
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 07:03:34 +00:00
Jason Volk
685b127f99 simplify iterator state constructor arguments
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 07:03:34 +00:00
Jason Volk
2259e2c82f batch queries to maximize throughput
query-side streams for first level of callsites

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk
7a6d657558 configurable dynamic stream concurrency scalar
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk
98e6c81e49 offload remaining db iterator initial seeks on cache miss
consume task budget on cache hit

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk
ad8cbcaac1 check cache prior to offloading iterator seek
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 13:29:43 -05:00
strawberry
77e0b76408
apply new rustfmt.toml changes, fix some clippy lints
Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-12-15 01:00:41 -05:00
strawberry
0317cc8cc5
rename conduit to conduwuit finally
Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-12-14 22:24:45 -05:00
Jason Volk
aba88ccead misc tracing span tweaks
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-10 06:20:30 +00:00
Jason Volk
ff56ee7413 add back the consume_budget on cache hits
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:27 +00:00
Jason Volk
65a370836c derive specific ReadOptions for iterators
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:27 +00:00
Jason Volk
48703173bc split get_batch from get.rs; add aqry_batch
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk
b5006a4c41 offload initial iterator seeks to threadpool
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 13:25:33 +00:00
Jason Volk
9a9c071e82 use tokio for threadpool mgmt
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk
3ad6aa59f9 use smallvec for db query buffering
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-28 06:03:33 +00:00
Jason Volk
c7ae951676 add frontend threadpool to database
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-27 10:53:44 +00:00
Jason Volk
62d560e2fb improve tracing instruments on database::map
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-25 07:00:55 +00:00
Jason Volk
3789d60b6a refactor to iterator inputs for auth_chain/short batch functions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 21:30:32 +00:00
Jason Volk
7680d1bd5e replace yields point with consume_budget
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 08:37:25 +00:00
Jason Volk
90106c4c33 streamline batch insertions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 08:37:25 +00:00
Jason Volk
14e3b242df add database get_batch stream wrapper
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-16 00:33:40 +00:00
Jason Volk
ed76797b55 add raw_ overloads for prefix/from counting
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-03 07:32:09 +00:00
Jason Volk
efb28c1a99 add a Map::contains suite to db
Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-27 00:09:05 +00:00
Jason Volk
8258d16a94 re-scheme naming of stream iterator overloads
Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-26 18:50:29 -04:00
Jason Volk
d3d11356ee add serialized insert interface
Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-26 18:50:29 -04:00
Jason Volk
814b9e28b6 fix unnecessary re-serializations
Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-26 18:50:28 -04:00
Jason Volk
ab06701ed0 refactor multi-get to handle result type
Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-25 00:36:30 -04:00
Jason Volk
0e8ae1e13e add ArrayVec-backed serialized query overload; doc comments
Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-25 00:32:33 -04:00
Jason Volk
5192927a53 split remaining map suites
Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-25 00:32:33 -04:00
Jason Volk
946ca364e0 Database Refactor
combine service/users data w/ mod unit

split sliding sync related out of service/users

instrument database entry points

remove increment crap from database interface

de-wrap all database get() calls

de-wrap all database insert() calls

de-wrap all database remove() calls

refactor database interface for async streaming

add query key serializer for database

implement Debug for result handle

add query deserializer for database

add deserialization trait for option handle

start a stream utils suite

de-wrap/asyncify/type-query count_one_time_keys()

de-wrap/asyncify users count

add admin query users command suite

de-wrap/asyncify users exists

de-wrap/partially asyncify user filter related

asyncify/de-wrap users device/keys related

asyncify/de-wrap user auth/misc related

asyncify/de-wrap users blurhash

asyncify/de-wrap account_data get; merge Data into Service

partial asyncify/de-wrap uiaa; merge Data into Service

partially asyncify/de-wrap transaction_ids get; merge Data into Service

partially asyncify/de-wrap key_backups; merge Data into Service

asyncify/de-wrap pusher service getters; merge Data into Service

asyncify/de-wrap rooms alias getters/some iterators

asyncify/de-wrap rooms directory getters/iterator

partially asyncify/de-wrap rooms lazy-loading

partially asyncify/de-wrap rooms metadata

asyncify/dewrap rooms outlier

asyncify/dewrap rooms pdu_metadata

dewrap/partially asyncify rooms read receipt

de-wrap rooms search service

de-wrap/partially asyncify rooms user service

partial de-wrap rooms state_compressor

de-wrap rooms state_cache

de-wrap room state et al

de-wrap rooms timeline service

additional users device/keys related

de-wrap/asyncify sender

asyncify services

refactor database to TryFuture/TryStream

refactor services for TryFuture/TryStream

asyncify api handlers

additional asyncification for admin module

abstract stream related; support reverse streams

additional stream conversions

asyncify state-res related

Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-25 00:32:30 -04:00