various misc documentation improvements

Signed-off-by: strawberry <strawberry@puppygock.gay>

CONTRIBUTING.md

@@ -131,7 +131,8 @@ allowed to be licenced under the Apache-2.0 licence and all of your conduct is
 in line with the Contributor's Covenant, and conduwuit's Code of Conduct.
 
 Contribution by users who violate either of these code of conducts will not have
-their contributions accepted.
+their contributions accepted. This includes users who have been banned from
+conduwuit Matrix rooms for Code of Conduct violations.
 
 [issues]: https://github.com/girlbossceo/conduwuit/issues
 [conduwuit-matrix]: https://matrix.to/#/#conduwuit:puppygock.gay

README.md

@@ -1,7 +1,6 @@
 # conduwuit
 
-`main`: [![CI and
+[![conduwuit main room](https://img.shields.io/matrix/conduwuit%3Apuppygock.gay?server_fqdn=matrix.transfem.dev&style=flat&logo=matrix&logoColor=%23f5b3ff&label=%23conduwuit%3Apuppygock.gay&color=%23f652ff)](https://matrix.to/#/#conduwuit:puppygock.gay) [![conduwuit space](https://img.shields.io/matrix/conduwuit-space%3Apuppygock.gay?server_fqdn=matrix.transfem.dev&style=flat&logo=matrix&logoColor=%23f5b3ff&label=%23conduwuit-space%3Apuppygock.gay&color=%23f652ff)](https://matrix.to/#/#conduwuit-space:puppygock.gay) [![CI and Artifacts](https://github.com/girlbossceo/conduwuit/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/girlbossceo/conduwuit/actions/workflows/ci.yml)
-Artifacts](https://github.com/girlbossceo/conduwuit/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/girlbossceo/conduwuit/actions/workflows/ci.yml)
 
 <!-- ANCHOR: catchphrase -->
 

debian/README.md

@@ -1,17 +1,22 @@
 # conduwuit for Debian
 
-Information about downloading and deploying the Debian package. This may also be referenced for other `apt`-based distros such as Ubuntu.
+Information about downloading and deploying the Debian package. This may also be
+referenced for other `apt`-based distros such as Ubuntu.
 
 ### Installation
 
-It is recommended to see the [generic deployment guide](../deploying/generic.md) for further information if needed as usage of the Debian package is generally related.
+It is recommended to see the [generic deployment guide](../deploying/generic.md)
+for further information if needed as usage of the Debian package is generally
+related.
 
 ### Configuration
 
-When installed, the example config is placed at `/etc/conduwuit/conduwuit.toml` as the default config. At the minimum, you will need to change your `server_name` here.
+When installed, the example config is placed at `/etc/conduwuit/conduwuit.toml`
+as the default config. The config mentions things required to be changed before
+starting.
 
-You can tweak more detailed settings by uncommenting and setting the config options
+You can tweak more detailed settings by uncommenting and setting the config
-in `/etc/conduwuit/conduwuit.toml`.
+options in `/etc/conduwuit/conduwuit.toml`.
 
 ### Running
 

docs/SUMMARY.md

@@ -8,6 +8,7 @@
   - [Generic](deploying/generic.md)
   - [NixOS](deploying/nixos.md)
   - [Docker](deploying/docker.md)
+  - [Kubernetes](deploying/kubernetes.md)
   - [Arch Linux](deploying/arch-linux.md)
   - [Debian](deploying/debian.md)
   - [FreeBSD](deploying/freebsd.md)

docs/deploying/docker.md

@@ -11,9 +11,9 @@ OCI images for conduwuit are available in the registries listed below.
 
 | Registry        | Image                                                           | Size                          | Notes                  |
 | --------------- | --------------------------------------------------------------- | ----------------------------- | ---------------------- |
-| GitHub Registry | [ghcr.io/girlbossceo/conduwuit:latest][gh] | ![Image Size][shield-latest]  | Stable tagged image.          |
+| GitHub Registry | [ghcr.io/girlbossceo/conduwuit:latest][gh] | ![Image Size][shield-latest]  | Stable latest tagged image.          |
-| GitLab Registry | [registry.gitlab.com/conduwuit/conduwuit:latest][gl] | ![Image Size][shield-latest]  | Stable tagged image.          |
+| GitLab Registry | [registry.gitlab.com/conduwuit/conduwuit:latest][gl] | ![Image Size][shield-latest]  | Stable latest tagged image.          |
-| Docker Hub      | [docker.io/girlbossceo/conduwuit:latest][dh]             | ![Image Size][shield-latest]  | Stable tagged image.          |
+| Docker Hub      | [docker.io/girlbossceo/conduwuit:latest][dh]             | ![Image Size][shield-latest]  | Stable latest tagged image.          |
 | GitHub Registry | [ghcr.io/girlbossceo/conduwuit:main][gh]   | ![Image Size][shield-main]    | Stable main branch.   |
 | GitLab Registry | [registry.gitlab.com/conduwuit/conduwuit:main][gl]   | ![Image Size][shield-main]    | Stable main branch.   |
 | Docker Hub      | [docker.io/girlbossceo/conduwuit:main][dh]               | ![Image Size][shield-main]    | Stable main branch.   |
@@ -92,16 +92,28 @@ Additional info about deploying conduwuit can be found [here](generic.md).
 
 ### Build
 
-To build the conduwuit image with docker-compose, you first need to open and
+Official conduwuit images are built using Nix's
-modify the `docker-compose.yml` file. There you need to comment the `image:`
+[`buildLayeredImage`][nix-buildlayeredimage]. This ensures all OCI images are
-option and uncomment the `build:` option. Then call docker compose with:
+repeatable and reproducible by anyone, keeps the images lightweight, and can be
+built offline.
 
-```bash
+This also ensures portability of our images because `buildLayeredImage` builds
-docker compose up
+OCI images, not Docker images, and works with other container software.
-```
 
-This will also start the container right afterwards, so if want it to run in
+The OCI images are OS-less with only a very minimal environment of the `tini`
-detached mode, you also should use the `-d` flag.
+init system, CA certificates, and the conduwuit binary. This does mean there is
+not a shell, but in theory you can get a shell by adding the necessary layers
+to the layered image. However it's very unlikely you will need a shell for any
+real troubleshooting.
+
+The flake file for the OCI image definition is at [`nix/pkgs/oci-image/default.nix`][oci-image-def].
+
+To build an OCI image using Nix, the following outputs can be built:
+- `nix build -L .#oci-image` (default features, x86_64 glibc)
+- `nix build -L .#oci-image-x86_64-linux-musl` (default features, x86_64 musl)
+- `nix build -L .#oci-image-aarch64-linux-musl` (default features, aarch64 musl)
+- `nix build -L .#oci-image-x86_64-linux-musl-all-features` (all features, x86_64 musl)
+- `nix build -L .#oci-image-aarch64-linux-musl-all-features` (all features, aarch64 musl)
 
 ### Run
 
@@ -136,3 +148,6 @@ those two files.
 ## Voice communication
 
 See the [TURN](../turn.md) page.
+
+[nix-buildlayeredimage]: https://ryantm.github.io/nixpkgs/builders/images/dockertools/#ssec-pkgs-dockerTools-buildLayeredImage
+[oci-image-def]: https://github.com/girlbossceo/conduwuit/blob/main/nix/pkgs/oci-image/default.nix

docs/deploying/kubernetes.md

@@ -1,4 +1,8 @@
 # conduwuit for Kubernetes
 
-conduwuit doesn't support horizontal scalability or distributed loading natively, however a community maintained Helm Chart is available here to run conduwuit on Kubernetes:
+conduwuit doesn't support horizontal scalability or distributed loading
-<https://gitlab.cronce.io/charts/conduwuit>
+natively, however a community maintained Helm Chart is available here to run
+conduwuit on Kubernetes: <https://gitlab.cronce.io/charts/conduwuit>
+
+Should changes need to be made, please reach out to the maintainer in our
+Matrix room as this is not maintained/controlled by the conduwuit maintainers.

docs/development.md

@@ -75,21 +75,21 @@ development (unresponsive or slow upstream), conduwuit-specific usecases, or
 lack of time to upstream some things.
 
 - [ruma/ruma][1]: <https://github.com/girlbossceo/ruwuma> - various performance
-improvements, more features, faster-paced development, client/server interop
+improvements, more features, faster-paced development, better client/server interop
 hacks upstream won't accept, etc
 - [facebook/rocksdb][2]: <https://github.com/girlbossceo/rocksdb> - liburing
-build fixes, GCC build fix, and logging callback C API for Rust tracing
+build fixes and GCC debug build fix
-integration
 - [tikv/jemallocator][3]: <https://github.com/girlbossceo/jemallocator> - musl
-builds seem to be broken on upstream
+builds seem to be broken on upstream, fixes some broken/suspicious code in
+places, additional safety measures, and support redzones for Valgrind
 - [zyansheep/rustyline-async][4]:
 <https://github.com/girlbossceo/rustyline-async> - tab completion callback and
-`CTRL+\` signal quit event for CLI
+`CTRL+\` signal quit event for conduwuit console CLI
 - [rust-rocksdb/rust-rocksdb][5]:
-<https://github.com/girlbossceo/rust-rocksdb-zaidoon1> - [`@zaidoon1`'s][8] fork
+<https://github.com/girlbossceo/rust-rocksdb-zaidoon1> - [`@zaidoon1`][8]'s fork
-has quicker updates, more up to date dependencies. Our changes fix musl build
+has quicker updates, more up to date dependencies, etc. Our fork fixes musl build
-issues, Rust part of the logging callback C API, removes unnecessary `gtest`
+issues, removes unnecessary `gtest` include, and uses our RocksDB and jemallocator
-include, and uses our RocksDB and jemallocator
+forks.
 - [tokio-rs/tracing][6]: <https://github.com/girlbossceo/tracing> - Implements
 `Clone` for `EnvFilter` to support dynamically changing tracing envfilter's
 alongside other logging/metrics things
@@ -103,12 +103,16 @@ tokio_unstable` flag to enable experimental tokio APIs. A build might look like
 this:
 
 ```bash
-RUSTFLAGS="--cfg tokio_unstable" cargo build \
+RUSTFLAGS="--cfg tokio_unstable" cargo +nightly build \
     --release \
     --no-default-features \
     --features=systemd,element_hacks,gzip_compression,brotli_compression,zstd_compression,tokio_console
 ```
 
+You will also need to enable the `tokio_console` config option in conduwuit when
+starting it. This was due to tokio-console causing gradual memory leak/usage
+if left enabled.
+
 [1]: https://github.com/ruma/ruma/
 [2]: https://github.com/facebook/rocksdb/
 [3]: https://github.com/tikv/jemallocator/

docs/development/testing.md

@@ -5,8 +5,8 @@
 Have a look at [Complement's repository][complement] for an explanation of what
 it is.
 
-To test against Complement, with Nix (or [Lix](https://lix.systems) and direnv installed
+To test against Complement, with Nix (or [Lix](https://lix.systems) and direnv
-and set up, you can:
+installed and set up, you can:
 
 * Run `./bin/complement "$COMPLEMENT_SRC" ./path/to/logs.jsonl
 ./path/to/results.jsonl` to build a Complement image, run the tests, and output

docs/troubleshooting.md

@@ -106,24 +106,46 @@ Various debug commands can be found in `!admin debug`.
 
 #### Debug/Trace log level
 
-conduwuit builds without debug or trace log levels by default for at least
+conduwuit builds without debug or trace log levels at compile time by default
-performance reasons. This may change in the future and/or binaries providing
+for substantial performance gains in CPU usage and improved compile times. If
-such configurations may be provided. If you need to access debug/trace log
+you need to access debug/trace log levels, you will need to build without the
-levels, you will need to build without the `release_max_log_level` feature.
+`release_max_log_level` feature or use our provided static debug binaries.
 
 #### Changing log level dynamically
 
 conduwuit supports changing the tracing log environment filter on-the-fly using
-the admin command `!admin debug change-log-level`. This accepts a string
+the admin command `!admin debug change-log-level <log env filter>`. This accepts
-**without quotes** the same format as the `log` config option.
+a string **without quotes** the same format as the `log` config option.
+
+Example: `!admin debug change-log-level debug`
+
+This can also accept complex filters such as:
+`!admin debug change-log-level info,conduit_service[{dest="example.com"}]=trace,ruma_state_res=trace`
+`!admin debug change-log-level info,conduit_service[{dest="example.com"}]=trace,conduit_service[send{dest="example.org"}]=trace`
+
+And to reset the log level to the one that was set at startup / last config
+load, simply pass the `--reset` flag.
+
+`!admin debug change-log-level --reset`
 
 #### Pinging servers
 
-conduwuit can ping other servers using `!admin debug ping`. This takes a server
+conduwuit can ping other servers using `!admin debug ping <server>`. This takes
-name and goes through the server discovery process and queries
+a server name and goes through the server discovery process and queries
 `/_matrix/federation/v1/version`. Errors are outputted.
 
+While it does measure the latency of the request, it is not indicative of
+server performance on either side as that endpoint is completely unauthenticated
+and simply fetches a string on a static JSON endpoint. It is very low cost both
+bandwidth and computationally.
+
 #### Allocator memory stats
 
-When using jemalloc with jemallocator's `stats` feature, you can see conduwuit's
+When using jemalloc with jemallocator's `stats` feature (`--enable-stats`), you
-jemalloc memory stats by using `!admin debug memory-stats`
+can see conduwuit's high-level allocator stats by using
+`!admin server memory-usage` at the bottom.
+
+If you are a developer, you can also view the raw jemalloc statistics with
+`!admin debug memory-stats`. Please note that this output is extremely large
+which may only be visible in the conduwuit console CLI due to PDU size limits,
+and is not easy for non-developers to understand.