Commit graph

880 commits

Author SHA1 Message Date
Jason Volk
4776fe66c4 handle serde_json for deserialized()
Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-25 00:32:33 -04:00
Jason Volk
946ca364e0 Database Refactor
combine service/users data w/ mod unit

split sliding sync related out of service/users

instrument database entry points

remove increment crap from database interface

de-wrap all database get() calls

de-wrap all database insert() calls

de-wrap all database remove() calls

refactor database interface for async streaming

add query key serializer for database

implement Debug for result handle

add query deserializer for database

add deserialization trait for option handle

start a stream utils suite

de-wrap/asyncify/type-query count_one_time_keys()

de-wrap/asyncify users count

add admin query users command suite

de-wrap/asyncify users exists

de-wrap/partially asyncify user filter related

asyncify/de-wrap users device/keys related

asyncify/de-wrap user auth/misc related

asyncify/de-wrap users blurhash

asyncify/de-wrap account_data get; merge Data into Service

partial asyncify/de-wrap uiaa; merge Data into Service

partially asyncify/de-wrap transaction_ids get; merge Data into Service

partially asyncify/de-wrap key_backups; merge Data into Service

asyncify/de-wrap pusher service getters; merge Data into Service

asyncify/de-wrap rooms alias getters/some iterators

asyncify/de-wrap rooms directory getters/iterator

partially asyncify/de-wrap rooms lazy-loading

partially asyncify/de-wrap rooms metadata

asyncify/dewrap rooms outlier

asyncify/dewrap rooms pdu_metadata

dewrap/partially asyncify rooms read receipt

de-wrap rooms search service

de-wrap/partially asyncify rooms user service

partial de-wrap rooms state_compressor

de-wrap rooms state_cache

de-wrap room state et al

de-wrap rooms timeline service

additional users device/keys related

de-wrap/asyncify sender

asyncify services

refactor database to TryFuture/TryStream

refactor services for TryFuture/TryStream

asyncify api handlers

additional asyncification for admin module

abstract stream related; support reverse streams

additional stream conversions

asyncify state-res related

Signed-off-by: Jason Volk <jason@zemos.net>
2024-10-25 00:32:30 -04:00
strawberry
f163ebf3bb implement MSC4133 only with MSC4175 for GET/PUT/DELETE
Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-09-07 09:27:35 -04:00
Jason Volk
f540bed61e propagate additional errors from db options
Signed-off-by: Jason Volk <jason@zemos.net>
2024-08-29 19:19:20 +00:00
Jason Volk
f69f4c479c propagate error from database options building
Signed-off-by: Jason Volk <jason@zemos.net>
2024-08-29 19:19:20 +00:00
Jason Volk
ac941a801a add interface to query rocksdb properties w/ admin cmd
Signed-off-by: Jason Volk <jason@zemos.net>
2024-08-08 18:09:21 +00:00
Jason Volk
f261d44edb add method to iterate opened database columns
Signed-off-by: Jason Volk <jason@zemos.net>
2024-08-08 18:09:21 +00:00
Jason Volk
849cfdcdfa add rocksdb stats level option with conf item
Signed-off-by: Jason Volk <jason@zemos.net>
2024-08-08 18:09:21 +00:00
Jason Volk
a74613be96 improve database repair/shutdown log messages
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-28 05:59:12 -07:00
Jason Volk
2f85a5c1ac de-global services
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-27 09:08:51 +00:00
Jason Volk
2100618d47 add rustc build flags reflection
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-27 08:26:42 +00:00
Jason Volk
f014231644 add conf item to disable rocksdb compaction
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-27 08:26:42 +00:00
strawberry
bacffd6174 rename conduit_cache_capacity_modifier to cache_capacity_modifier
this prefix causes you to require setting the environment variable
to `CONDUWUIT_CONDUIT_CACHE_CAPACITY_MODIFIER`

alias this so we dont break any configs

Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-07-14 12:08:15 -04:00
Jason Volk
05efd9b044 elaborate error macro and apply at various callsites
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-14 11:31:39 +00:00
Jason Volk
b3f2288d07 add constant-expression string utils
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-14 11:31:39 +00:00
Jason Volk
100c6f572b trim unused dependencies in member crates
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-08 22:02:10 +00:00
Jason Volk
eb5556e74e additional tracing spans / log cleanup.
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-08 22:02:10 +00:00
Jason Volk
5af880e4f4 fix reference count on punned ColumnFamily Arc
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-08 22:02:09 +00:00
Jason Volk
dfd13780df mitigate additional cast lints
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-08 22:02:09 +00:00
Jason Volk
eeda96d94a inline analysis and symbol reduction; emits smaller than 64 bytes marked inline
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-04 21:06:35 +00:00
strawberry
1a64e42cfe add /openid/request_token and /openid/userinfo routes
heavily changed and improved by me

Co-authored-by: mikoto <avdb@keemail.me>
Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-07-03 11:30:11 -04:00
Jason Volk
be2d1c722b encap dep:chrono in time utils
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-03 06:34:16 +00:00
Jason Volk
473b29d524 move remaining runtime caches into their respective service
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-03 06:34:16 +00:00
Jason Volk
a2d25215a3 consolidate key/value types; consistent interface arguments
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-03 06:34:16 +00:00
Jason Volk
2e2cf08bb2 optimize increment / increment batch
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-03 06:34:16 +00:00
Jason Volk
ee64fb149c optimize get w/ zero-copy ref handle
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-03 06:34:16 +00:00
Jason Volk
0522fe7d92 reimplement iterator from lowlevel
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-03 06:34:16 +00:00
Jason Volk
6dd6e4bfaf simplify cork interface related
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-03 06:34:16 +00:00
Jacob Taylor
0171262581 make pduid_pdu/eventid_outlierpdu scale with conduit_cache_capacity_modifier via cache_size(), and fix lint 2024-07-03 06:34:16 +00:00
Jason Volk
972037dcd9 optimize column family handles
Signed-off-by: Jason Volk <jason@zemos.net>
2024-07-03 06:34:16 +00:00
Jason Volk
96a16190c5 mute dev_release_log_level from --all-features for now
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-30 18:02:16 +00:00
Jason Volk
7638bbc49c keep column list lexically sorted
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-30 18:02:16 +00:00
Jason Volk
35049c94de simplify db ThreadMode typedef
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-30 18:02:16 +00:00
Jason Volk
7db8478dc0 hoist iterator modes; group fns
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-30 18:02:16 +00:00
Jason Volk
dc633100da hoist read_options/write_options instances
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-30 18:02:16 +00:00
Jason Volk
30b5ad3870 de-feature rocksdb 2024-06-30 18:02:16 +00:00
Jason Volk
de21f7442a devirtualize database
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-30 18:02:14 +00:00
Jason Volk
01f4455ceb convert rocksdb errors locally; remove from Error.
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-30 18:01:20 +00:00
Jason Volk
c1712d4d8b optimize increment
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-30 18:01:20 +00:00
Paul Robertson
19d8f0b27e
dont use any for a single feature check 2024-06-30 10:40:54 -04:00
Paul Robertson
73b85372d6
drop parking_lot feature 2024-06-29 11:20:48 -04:00
Paul Robertson
5e6b9ec1cf
drop thread_local feature 2024-06-29 11:20:48 -04:00
Paul Robertson
325f1c0bd2
drop the sqlite feature from database 2024-06-29 11:20:45 -04:00
Paul Robertson
624cd2acfa
remove sqlite code 2024-06-29 11:05:33 -04:00
Jason Volk
1ccdba8921 use markdown for list-database-files command
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-17 02:10:27 +00:00
Matthias Ahouansou
f712c0cefb fix: restrict who can remove aliases
Previously, anyone could remove any local alias, meaning that someone could re-route a popular alias elsewhere
Now, only the creator of the alias, users who can set canonical aliases for the room, server admins and the server user can delete aliases

added some additional changes/fixes to adapt to our codebase

Co-authored-by: strawberry <strawberry@puppygock.gay>
Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-06-12 14:04:47 -04:00
Jason Volk
eae41fc411 Fix use-self
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-09 20:06:50 +00:00
Jason Volk
c3c91e9d80 Fix suboptimal flops
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-09 20:06:50 +00:00
Jason Volk
67f4285504 Fix branches sharing code
Signed-off-by: Jason Volk <jason@zemos.net>
2024-06-09 20:06:50 +00:00
strawberry
c738c119f8 delete unnecessary real_users_cache, fix overwriting push_target iter, add proper function for getting local active users in room
this `real_users_cache` cache seems weird, and i have no idea what
prompted its creation upstream. perhaps they did this because
sqlite was very slow and their rocksdb setup is very poor, so
a "solution" was to stick member counts in memory.
slow iterators, scanning, etc do not apply to conduwuit where
our rocksdb is extremely tuned, and i seriously doubt something
like this would have any real world net-positive performance impact.

also for some reason, there is suspicious logic where we
overwrite the entire push target collection.

both of these things could be a potential cause for receiving
notifications in rooms we've left.

Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-06-05 18:18:08 -04:00