magmaus3/continuwuity
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
4562 commits 37 branches 19 tags 14 MiB
Commit graph

862 commits

Author SHA1 Message Date
Jason Volk
3ad6aa59f9 use smallvec for db query buffering 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-28 06:03:33 +00:00
Jason Volk
c7ae951676 add frontend threadpool to database 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-27 10:53:44 +00:00
Jason Volk
527494a34b fix oversized tracing span arguments; lints 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-27 06:30:20 +00:00
morguldir
63d1fcf213
add queued transactions rocksdb cf cache 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-11-25 16:08:30 -05:00
Jason Volk
62d560e2fb improve tracing instruments on database::map 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-25 07:00:55 +00:00
Jason Volk
343ec59a8b use arrayvec for integer deserialization buffer 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-25 02:54:29 +00:00
Jason Volk
3789d60b6a refactor to iterator inputs for auth_chain/short batch functions 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-24 21:30:32 +00:00
strawberry
9100af9974
add eventid_pdu database cf cache 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-11-21 23:45:16 -05:00
Jason Volk
7680d1bd5e replace yields point with consume_budget 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-19 08:37:25 +00:00
Jason Volk
90106c4c33 streamline batch insertions 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-19 08:37:25 +00:00
Jason Volk
14e3b242df add database get_batch stream wrapper 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-16 00:33:40 +00:00
Jason Volk
20836cc3db flush=false for database-backup in read-only/secondary modes; improve error 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-16 00:33:40 +00:00
Jason Volk
59834a4b05 add is_read_only()/is_secondary() to Engine 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-16 00:33:40 +00:00
Jason Volk
004be3bf00 prepare utf-8 check bypass for database deserializer 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-13 22:59:28 +00:00
Jason Volk
8742437036 wrap unimplemented ser/de branches with internal macro 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-03 08:03:25 +00:00
Jason Volk
9775694423 inline database stream interface functions lt 64B 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-03 07:32:09 +00:00
Jason Volk
ed76797b55 add raw_ overloads for prefix/from counting 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-11-03 07:32:09 +00:00
Jason Volk
354dc9e703 add map accessor to Database; move cork interface 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-28 20:52:52 -04:00
Jason Volk
0426f92ac0 unify database record separator constants 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-27 00:11:50 +00:00
Jason Volk
efb28c1a99 add a Map::contains suite to db 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-27 00:09:05 +00:00
Jason Volk
c06f560913 add some additional database::de test cases 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-26 18:50:29 -04:00
Jason Volk
8258d16a94 re-scheme naming of stream iterator overloads 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-26 18:50:29 -04:00
Jason Volk
19880ce12b add IgnoreAll directive to deserializer 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-26 18:50:29 -04:00
Jason Volk
d3d11356ee add serialized insert interface 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-26 18:50:29 -04:00
Jason Volk
2f24d7117a further develop serializer for insertions 
add JSON delegator to db serializer

consolidate writes through memfun; simplifications

Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-26 18:50:29 -04:00
Jason Volk
814b9e28b6 fix unnecessary re-serializations 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-26 18:50:28 -04:00
Jason Volk
c6b7c24e99 consume all bytes for top-level Ignore; add comments/tweaks 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-25 00:38:45 -04:00
Jason Volk
ab06701ed0 refactor multi-get to handle result type 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-25 00:36:30 -04:00
Jason Volk
96fcf7f94d add rocksdb secondary; fix read_only mode. 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-25 00:36:30 -04:00
Jason Volk
a8d5cf9651 Add rocksdb logging integration with tracing. 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-25 00:36:30 -04:00
Jason Volk
0e8ae1e13e add ArrayVec-backed serialized query overload; doc comments 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-25 00:32:33 -04:00
Jason Volk
5192927a53 split remaining map suites 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-25 00:32:33 -04:00
Jason Volk
4776fe66c4 handle serde_json for deserialized() 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-25 00:32:33 -04:00
Jason Volk
946ca364e0 Database Refactor 
combine service/users data w/ mod unit

split sliding sync related out of service/users

instrument database entry points

remove increment crap from database interface

de-wrap all database get() calls

de-wrap all database insert() calls

de-wrap all database remove() calls

refactor database interface for async streaming

add query key serializer for database

implement Debug for result handle

add query deserializer for database

add deserialization trait for option handle

start a stream utils suite

de-wrap/asyncify/type-query count_one_time_keys()

de-wrap/asyncify users count

add admin query users command suite

de-wrap/asyncify users exists

de-wrap/partially asyncify user filter related

asyncify/de-wrap users device/keys related

asyncify/de-wrap user auth/misc related

asyncify/de-wrap users blurhash

asyncify/de-wrap account_data get; merge Data into Service

partial asyncify/de-wrap uiaa; merge Data into Service

partially asyncify/de-wrap transaction_ids get; merge Data into Service

partially asyncify/de-wrap key_backups; merge Data into Service

asyncify/de-wrap pusher service getters; merge Data into Service

asyncify/de-wrap rooms alias getters/some iterators

asyncify/de-wrap rooms directory getters/iterator

partially asyncify/de-wrap rooms lazy-loading

partially asyncify/de-wrap rooms metadata

asyncify/dewrap rooms outlier

asyncify/dewrap rooms pdu_metadata

dewrap/partially asyncify rooms read receipt

de-wrap rooms search service

de-wrap/partially asyncify rooms user service

partial de-wrap rooms state_compressor

de-wrap rooms state_cache

de-wrap room state et al

de-wrap rooms timeline service

additional users device/keys related

de-wrap/asyncify sender

asyncify services

refactor database to TryFuture/TryStream

refactor services for TryFuture/TryStream

asyncify api handlers

additional asyncification for admin module

abstract stream related; support reverse streams

additional stream conversions

asyncify state-res related

Signed-off-by: Jason Volk <jason@zemos.net>
 2024-10-25 00:32:30 -04:00
strawberry
f163ebf3bb implement MSC4133 only with MSC4175 for GET/PUT/DELETE 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-09-07 09:27:35 -04:00
Jason Volk
f540bed61e propagate additional errors from db options 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-08-29 19:19:20 +00:00
Jason Volk
f69f4c479c propagate error from database options building 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-08-29 19:19:20 +00:00
Jason Volk
ac941a801a add interface to query rocksdb properties w/ admin cmd 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-08-08 18:09:21 +00:00
Jason Volk
f261d44edb add method to iterate opened database columns 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-08-08 18:09:21 +00:00
Jason Volk
849cfdcdfa add rocksdb stats level option with conf item 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-08-08 18:09:21 +00:00
Jason Volk
a74613be96 improve database repair/shutdown log messages 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-28 05:59:12 -07:00
Jason Volk
2f85a5c1ac de-global services 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-27 09:08:51 +00:00
Jason Volk
2100618d47 add rustc build flags reflection 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-27 08:26:42 +00:00
Jason Volk
f014231644 add conf item to disable rocksdb compaction 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-27 08:26:42 +00:00
strawberry
bacffd6174 rename conduit_cache_capacity_modifier to cache_capacity_modifier 
this prefix causes you to require setting the environment variable
to `CONDUWUIT_CONDUIT_CACHE_CAPACITY_MODIFIER`

alias this so we dont break any configs

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-14 12:08:15 -04:00
Jason Volk
05efd9b044 elaborate error macro and apply at various callsites 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-14 11:31:39 +00:00
Jason Volk
b3f2288d07 add constant-expression string utils 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-14 11:31:39 +00:00
Jason Volk
100c6f572b trim unused dependencies in member crates 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-08 22:02:10 +00:00
Jason Volk
eb5556e74e additional tracing spans / log cleanup. 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-08 22:02:10 +00:00
Jason Volk
5af880e4f4 fix reference count on punned ColumnFamily Arc 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-07-08 22:02:09 +00:00