diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/API/admin_api.md | 65 | ||||
| -rw-r--r-- | docs/API/differences_in_mastoapi_responses.md | 50 | ||||
| -rw-r--r-- | docs/administration/CLI_tasks/release_environments.md | 9 | ||||
| -rw-r--r-- | docs/configuration/cheatsheet.md | 13 | ||||
| -rw-r--r-- | docs/configuration/howto_ejabberd.md | 136 | ||||
| -rw-r--r-- | docs/configuration/optimizing_beam.md | 66 | ||||
| -rw-r--r-- | docs/dev.md | 23 | ||||
| -rw-r--r-- | docs/installation/debian_based_en.md | 3 | ||||
| -rw-r--r-- | docs/installation/otp_en.md | 3 |
9 files changed, 335 insertions, 33 deletions
diff --git a/docs/API/admin_api.md b/docs/API/admin_api.md index f7b5bcae7..266f8cef8 100644 --- a/docs/API/admin_api.md +++ b/docs/API/admin_api.md @@ -554,7 +554,7 @@ Response: * `show_role` * `skip_thread_containment` * `fields` - * `discoverable` + * `is_discoverable` * `actor_type` * Responses: @@ -1499,3 +1499,66 @@ Returns the content of the document "url": "https://example.com/instance/panel.html" } ``` + +## `GET /api/pleroma/admin/frontends + +### List available frontends + +- Response: + +```json +[ + { + "build_url": "https://git.pleroma.social/pleroma/fedi-fe/-/jobs/artifacts/${ref}/download?job=build", + "git": "https://git.pleroma.social/pleroma/fedi-fe", + "installed": true, + "name": "fedi-fe", + "ref": "master" + }, + { + "build_url": "https://git.pleroma.social/lambadalambda/kenoma/-/jobs/artifacts/${ref}/download?job=build", + "git": "https://git.pleroma.social/lambadalambda/kenoma", + "installed": false, + "name": "kenoma", + "ref": "master" + } +] +``` + +## `POST /api/pleroma/admin/frontends/install` + +### Install a frontend + +- Params: + - `name`: frontend name, required + - `ref`: frontend ref + - `file`: path to a frontend zip file + - `build_url`: build URL + - `build_dir`: build directory + +- Response: + +```json +[ + { + "build_url": "https://git.pleroma.social/pleroma/fedi-fe/-/jobs/artifacts/${ref}/download?job=build", + "git": "https://git.pleroma.social/pleroma/fedi-fe", + "installed": true, + "name": "fedi-fe", + "ref": "master" + }, + { + "build_url": "https://git.pleroma.social/lambadalambda/kenoma/-/jobs/artifacts/${ref}/download?job=build", + "git": "https://git.pleroma.social/lambadalambda/kenoma", + "installed": false, + "name": "kenoma", + "ref": "master" + } +] +``` + +```json +{ + "error": "Could not install frontend" +} +``` diff --git a/docs/API/differences_in_mastoapi_responses.md b/docs/API/differences_in_mastoapi_responses.md index 38865dc68..6b0ad85d1 100644 --- a/docs/API/differences_in_mastoapi_responses.md +++ b/docs/API/differences_in_mastoapi_responses.md @@ -9,12 +9,16 @@ Pleroma uses 128-bit ids as opposed to Mastodon's 64 bits. However just like Mas ## Timelines Adding the parameter `with_muted=true` to the timeline queries will also return activities by muted (not by blocked!) users. + Adding the parameter `exclude_visibilities` to the timeline queries will exclude the statuses with the given visibilities. The parameter accepts an array of visibility types (`public`, `unlisted`, `private`, `direct`), e.g., `exclude_visibilities[]=direct&exclude_visibilities[]=private`. + Adding the parameter `reply_visibility` to the public and home timelines queries will filter replies. Possible values: without parameter (default) shows all replies, `following` - replies directed to you or users you follow, `self` - replies directed to you. +Adding the parameter `instance=lain.com` to the public timeline will show only statuses originating from `lain.com` (or any remote instance). + ## Statuses -- `visibility`: has an additional possible value `list` +- `visibility`: has additional possible values `list` and `local` (for local-only statuses) Has these additional fields under the `pleroma` object: @@ -80,7 +84,7 @@ Has these additional fields under the `pleroma` object: - `show_role`: boolean, nullable, true when the user wants his role (e.g admin, moderator) to be shown - `no_rich_text` - boolean, nullable, true when html tags are stripped from all statuses requested from the API -- `discoverable`: boolean, true when the user allows discovery of the account in search results and other services. +- `discoverable`: boolean, true when the user allows external services (search bots) etc. to index / list the account (regardless of this setting, user will still appear in regular search results) - `actor_type`: string, the type of this account. ## Conversations @@ -125,12 +129,30 @@ The `type` value is `pleroma:emoji_reaction`. Has these fields: - `account`: The account of the user who reacted - `status`: The status that was reacted on +### ChatMention Notification (not default) + +This notification has to be requested explicitly. + +The `type` value is `pleroma:chat_mention` + +- `account`: The account who sent the message +- `chat_message`: The chat message + +### Report Notification (not default) + +This notification has to be requested explicitly. + +The `type` value is `pleroma:report` + +- `account`: The account who reported +- `report`: The report + ## GET `/api/v1/notifications` Accepts additional parameters: - `exclude_visibilities`: will exclude the notifications for activities with the given visibilities. The parameter accepts an array of visibility types (`public`, `unlisted`, `private`, `direct`). Usage example: `GET /api/v1/notifications?exclude_visibilities[]=direct&exclude_visibilities[]=private`. -- `include_types`: will include the notifications for activities with the given types. The parameter accepts an array of types (`mention`, `follow`, `reblog`, `favourite`, `move`, `pleroma:emoji_reaction`). Usage example: `GET /api/v1/notifications?include_types[]=mention&include_types[]=reblog`. +- `include_types`: will include the notifications for activities with the given types. The parameter accepts an array of types (`mention`, `follow`, `reblog`, `favourite`, `move`, `pleroma:emoji_reaction`, `pleroma:chat_mention`, `pleroma:report`). Usage example: `GET /api/v1/notifications?include_types[]=mention&include_types[]=reblog`. ## DELETE `/api/v1/notifications/destroy_multiple` @@ -151,7 +173,7 @@ Additional parameters can be added to the JSON body/Form data: - `preview`: boolean, if set to `true` the post won't be actually posted, but the status entitiy would still be rendered back. This could be useful for previewing rich text/custom emoji, for example. - `content_type`: string, contain the MIME type of the status, it is transformed into HTML by the backend. You can get the list of the supported MIME types with the nodeinfo endpoint. - `to`: A list of nicknames (like `lain@soykaf.club` or `lain` on the local server) that will be used to determine who is going to be addressed by this post. Using this will disable the implicit addressing by mentioned names in the `status` body, only the people in the `to` list will be addressed. The normal rules for for post visibility are not affected by this and will still apply. -- `visibility`: string, besides standard MastoAPI values (`direct`, `private`, `unlisted` or `public`) it can be used to address a List by setting it to `list:LIST_ID`. +- `visibility`: string, besides standard MastoAPI values (`direct`, `private`, `unlisted`, `local` or `public`) it can be used to address a List by setting it to `list:LIST_ID`. - `expires_in`: The number of seconds the posted activity should expire in. When a posted activity expires it will be deleted from the server, and a delete request for it will be federated. This needs to be longer than an hour. - `in_reply_to_conversation_id`: Will reply to a given conversation, addressing only the people who are part of the recipient set of that conversation. Sets the visibility to `direct`. @@ -185,7 +207,7 @@ Additional parameters can be added to the JSON body/Form data: - `skip_thread_containment` - if true, skip filtering out broken threads - `allow_following_move` - if true, allows automatically follow moved following accounts - `pleroma_background_image` - sets the background image of the user. Can be set to "" (an empty string) to reset. -- `discoverable` - if true, discovery of this account in search results and other services is allowed. +- `discoverable` - if true, external services (search bots) etc. are allowed to index / list the account (regardless of this setting, user will still appear in regular search results). - `actor_type` - the type of this account. - `accepts_chat_messages` - if false, this account will reject all chat messages. @@ -211,7 +233,7 @@ Post here request with `grant_type=refresh_token` to obtain new access token. Re `POST /api/v1/accounts` -Has theses additional parameters (which are the same as in Pleroma-API): +Has these additional parameters (which are the same as in Pleroma-API): - `fullname`: optional - `bio`: optional @@ -239,6 +261,16 @@ Has theses additional parameters (which are the same as in Pleroma-API): - `pleroma.metadata.post_formats`: A list of the allowed post format types - `vapid_public_key`: The public key needed for push messages +## Push Subscription + +`POST /api/v1/push/subscription` +`PUT /api/v1/push/subscription` + +Permits these additional alert types: + +- pleroma:chat_mention +- pleroma:emoji_reaction + ## Markers Has these additional fields under the `pleroma` object: @@ -249,6 +281,12 @@ Has these additional fields under the `pleroma` object: There is an additional `user:pleroma_chat` stream. Incoming chat messages will make the current chat be sent to this `user` stream. The `event` of an incoming chat message is `pleroma:chat_update`. The payload is the updated chat with the incoming chat message in the `last_message` field. +For viewing remote server timelines, there are `public:remote` and `public:remote:media` streams. Each of these accept a parameter like `?instance=lain.com`. + +## User muting and thread muting + +Both user muting and thread muting can be done for only a certain time by adding an `expires_in` parameter to the API calls and giving the expiration time in seconds. + ## Not implemented Pleroma is generally compatible with the Mastodon 2.7.2 API, but some newer features and non-essential features are omitted. These features usually return an HTTP 200 status code, but with an empty response. While they may be added in the future, they are considered low priority. diff --git a/docs/administration/CLI_tasks/release_environments.md b/docs/administration/CLI_tasks/release_environments.md deleted file mode 100644 index 36ab43864..000000000 --- a/docs/administration/CLI_tasks/release_environments.md +++ /dev/null @@ -1,9 +0,0 @@ -# Generate release environment file - -```sh tab="OTP" - ./bin/pleroma_ctl release_env gen -``` - -```sh tab="From Source" -mix pleroma.release_env gen -``` diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index ebf95ebc9..85551362c 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -63,6 +63,7 @@ To add configuration to your config file, you can copy it from the base config. * `external_user_synchronization`: Enabling following/followers counters synchronization for external users. * `cleanup_attachments`: Remove attachments along with statuses. Does not affect duplicate files and attachments without status. Enabling this will increase load to database when deleting statuses on larger instances. * `show_reactions`: Let favourites and emoji reactions be viewed through the API (default: `true`). +* `password_reset_token_validity`: The time after which reset tokens aren't accepted anymore, in seconds (default: one day). ## Welcome * `direct_message`: - welcome message sent as a direct message. @@ -220,18 +221,6 @@ config :pleroma, :mrf_user_allowlist, %{ * `total_user_limit`: the number of scheduled activities a user is allowed to create in total (Default: `300`) * `enabled`: whether scheduled activities are sent to the job queue to be executed -## FedSockets -FedSockets is an experimental feature allowing for Pleroma backends to federate using a persistant websocket connection as opposed to making each federation a seperate http connection. This feature is currently off by default. It is configurable throught he following options. - -### :fedsockets -* `enabled`: Enables FedSockets for this instance. `false` by default. -* `connection_duration`: Time an idle websocket is kept open. -* `rejection_duration`: Failures to connect via FedSockets will not be retried for this period of time. -* `fed_socket_fetches` and `fed_socket_rejections`: Settings passed to `cachex` for the fetch registry, and rejection stacks. See `Pleroma.Web.FedSockets` for more details. - - -## Frontends - ### :frontend_configurations This can be used to configure a keyword list that keeps the configuration data for any kind of frontend. By default, settings for `pleroma_fe` and `masto_fe` are configured. You can find the documentation for `pleroma_fe` configuration into [Pleroma-FE configuration and customization for instance administrators](/frontend/CONFIGURATION/#options). diff --git a/docs/configuration/howto_ejabberd.md b/docs/configuration/howto_ejabberd.md new file mode 100644 index 000000000..520a0acbc --- /dev/null +++ b/docs/configuration/howto_ejabberd.md @@ -0,0 +1,136 @@ +# Configuring Ejabberd (XMPP Server) to use Pleroma for authentication + +If you want to give your Pleroma users an XMPP (chat) account, you can configure [Ejabberd](https://github.com/processone/ejabberd) to use your Pleroma server for user authentication, automatically giving every local user an XMPP account. + +In general, you just have to follow the configuration described at [https://docs.ejabberd.im/admin/configuration/authentication/#external-script](https://docs.ejabberd.im/admin/configuration/authentication/#external-script). Please read this section carefully. + +Copy the script below to suitable path on your system and set owner and permissions. Also do not forget adjusting `PLEROMA_HOST` and `PLEROMA_PORT`, if necessary. + +```bash +cp pleroma_ejabberd_auth.py /etc/ejabberd/pleroma_ejabberd_auth.py +chown ejabberd /etc/ejabberd/pleroma_ejabberd_auth.py +chmod 700 /etc/ejabberd/pleroma_ejabberd_auth.py +``` + +Set external auth params in ejabberd.yaml file: + +```bash +auth_method: [external] +extauth_program: "python3 /etc/ejabberd/pleroma_ejabberd_auth.py" +extauth_instances: 3 +auth_use_cache: false +``` + +Restart / reload your ejabberd service. + +After restarting your Ejabberd server, your users should now be able to connect with their Pleroma credentials. + + +```python +import sys +import struct +import http.client +from base64 import b64encode +import logging + + +PLEROMA_HOST = "127.0.0.1" +PLEROMA_PORT = "4000" +AUTH_ENDPOINT = "/api/v1/accounts/verify_credentials" +USER_ENDPOINT = "/api/v1/accounts" +LOGFILE = "/var/log/ejabberd/pleroma_auth.log" + +logging.basicConfig(filename=LOGFILE, level=logging.INFO) + + +# Pleroma functions +def create_connection(): + return http.client.HTTPConnection(PLEROMA_HOST, PLEROMA_PORT) + + +def verify_credentials(user: str, password: str) -> bool: + user_pass_b64 = b64encode("{}:{}".format( + user, password).encode('utf-8')).decode("ascii") + params = {} + headers = { + "Authorization": "Basic {}".format(user_pass_b64) + } + + try: + conn = create_connection() + conn.request("GET", AUTH_ENDPOINT, params, headers) + + response = conn.getresponse() + if response.status == 200: + return True + + return False + except Exception as e: + logging.info("Can not connect: %s", str(e)) + return False + + +def does_user_exist(user: str) -> bool: + conn = create_connection() + conn.request("GET", "{}/{}".format(USER_ENDPOINT, user)) + + response = conn.getresponse() + if response.status == 200: + return True + + return False + + +def auth(username: str, server: str, password: str) -> bool: + return verify_credentials(username, password) + + +def isuser(username, server): + return does_user_exist(username) + + +def read(): + (pkt_size,) = struct.unpack('>H', bytes(sys.stdin.read(2), encoding='utf8')) + pkt = sys.stdin.read(pkt_size) + cmd = pkt.split(':')[0] + if cmd == 'auth': + username, server, password = pkt.split(':', 3)[1:] + write(auth(username, server, password)) + elif cmd == 'isuser': + username, server = pkt.split(':', 2)[1:] + write(isuser(username, server)) + elif cmd == 'setpass': + # u, s, p = pkt.split(':', 3)[1:] + write(False) + elif cmd == 'tryregister': + # u, s, p = pkt.split(':', 3)[1:] + write(False) + elif cmd == 'removeuser': + # u, s = pkt.split(':', 2)[1:] + write(False) + elif cmd == 'removeuser3': + # u, s, p = pkt.split(':', 3)[1:] + write(False) + else: + write(False) + + +def write(result): + if result: + sys.stdout.write('\x00\x02\x00\x01') + else: + sys.stdout.write('\x00\x02\x00\x00') + sys.stdout.flush() + + +if __name__ == "__main__": + logging.info("Starting pleroma ejabberd auth daemon...") + while True: + try: + read() + except Exception as e: + logging.info( + "Error while processing data from ejabberd %s", str(e)) + pass + +```
\ No newline at end of file diff --git a/docs/configuration/optimizing_beam.md b/docs/configuration/optimizing_beam.md new file mode 100644 index 000000000..e336bd36c --- /dev/null +++ b/docs/configuration/optimizing_beam.md @@ -0,0 +1,66 @@ +# Optimizing the BEAM + +Pleroma is built upon the Erlang/OTP VM known as BEAM. The BEAM VM is highly optimized for latency, but this has drawbacks in environments without dedicated hardware. One of the tricks used by the BEAM VM is [busy waiting](https://en.wikipedia.org/wiki/Busy_waiting). This allows the application to pretend to be busy working so the OS kernel does not pause the application process and switch to another process waiting for the CPU to execute its workload. It does this by spinning for a period of time which inflates the apparent CPU usage of the application so it is immediately ready to execute another task. This can be observed with utilities like **top(1)** which will show consistently high CPU usage for the process. Switching between procesess is a rather expensive operation and also clears CPU caches further affecting latency and performance. The goal of busy waiting is to avoid this penalty. + +This strategy is very successful in making a performant and responsive application, but is not desirable on Virtual Machines or hardware with few CPU cores. Pleroma instances are often deployed on the same server as the required PostgreSQL database which can lead to situations where the Pleroma application is holding the CPU in a busy-wait loop and as a result the database cannot process requests in a timely manner. The fewer CPUs available, the more this problem is exacerbated. The latency is further amplified by the OS being installed on a Virtual Machine as the Hypervisor uses CPU time-slicing to pause the entire OS and switch between other tasks. + +More adventurous admins can be creative with CPU affinity (e.g., *taskset* for Linux and *cpuset* on FreeBSD) to pin processes to specific CPUs and eliminate much of this contention. The most important advice is to run as few processes as possible on your server to achieve the best performance. Even idle background processes can occasionally create [software interrupts](https://en.wikipedia.org/wiki/Interrupt) and take attention away from the executing process creating latency spikes and invalidation of the CPU caches as they must be cleared when switching between processes for security. + +Please only change these settings if you are experiencing issues or really know what you are doing. In general, there's no need to change these settings. + +## VPS Provider Recommendations + +### Good + +* Hetzner Cloud + +### Bad + +* AWS (known to use burst scheduling) + + +## Example configurations + +Tuning the BEAM requires you provide a config file normally called [vm.args](http://erlang.org/doc/man/erl.html#emulator-flags). If you are using systemd to manage the service you can modify the unit file as such: + +`ExecStart=/usr/bin/elixir --erl '-args_file /opt/pleroma/config/vm.args' -S /usr/bin/mix phx.server` + +Check your OS documentation to adopt a similar strategy on other platforms. + +### Virtual Machine and/or few CPU cores + +Disable the busy-waiting. This should generally only be done if you're on a platform that does burst scheduling, like AWS. + +**vm.args:** + +``` ++sbwt none ++sbwtdcpu none ++sbwtdio none +``` + +### Dedicated Hardware + +Enable more busy waiting, increase the internal maximum limit of BEAM processes and ports. You can use this if you run on dedicated hardware, but it is not necessary. + +**vm.args:** + +``` ++P 16777216 ++Q 16777216 ++K true ++A 128 ++sbt db ++sbwt very_long ++swt very_low ++sub true ++Mulmbcs 32767 ++Mumbcgs 1 ++Musmbcs 2047 +``` + +## Additional Reading + +* [WhatsApp: Scaling to Millions of Simultaneous Connections](https://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf) +* [Preemptive Scheduling and Spinlocks](https://www.uio.no/studier/emner/matnat/ifi/nedlagte-emner/INF3150/h03/annet/slides/preemptive.pdf) +* [The Curious Case of BEAM CPU Usage](https://stressgrid.com/blog/beam_cpu_usage/) diff --git a/docs/dev.md b/docs/dev.md index ba2718673..765380a58 100644 --- a/docs/dev.md +++ b/docs/dev.md @@ -21,3 +21,26 @@ This document contains notes and guidelines for Pleroma developers. ## Auth-related configuration, OAuth consumer mode etc. See `Authentication` section of [the configuration cheatsheet](configuration/cheatsheet.md#authentication). + +## MRF policies descriptions + +If MRF policy depends on config, it can be added into MRF tab to adminFE by adding `config_description/0` method, which returns map with special structure. + +Example: + +```elixir +%{ + key: :mrf_activity_expiration, + related_policy: "Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy", + label: "MRF Activity Expiration Policy", + description: "Adds automatic expiration to all local activities", + children: [ + %{ + key: :days, + type: :integer, + description: "Default global expiration time for all local activities (in days)", + suggestions: [90, 365] + } + ] + } +``` diff --git a/docs/installation/debian_based_en.md b/docs/installation/debian_based_en.md index b9fc4e112..2b1c7406f 100644 --- a/docs/installation/debian_based_en.md +++ b/docs/installation/debian_based_en.md @@ -35,7 +35,7 @@ sudo apt full-upgrade * Install some of the above mentioned programs: ```shell -sudo apt install git build-essential postgresql postgresql-contrib cmake libmagic-devel +sudo apt install git build-essential postgresql postgresql-contrib cmake libmagic-dev ``` ### Install Elixir and Erlang @@ -182,7 +182,6 @@ sudo cp /opt/pleroma/installation/pleroma.service /etc/systemd/system/pleroma.se ``` * Edit the service file and make sure that all paths fit your installation -* Check that `EnvironmentFile` contains the correct path to the env file. Or generate the env file: `sudo -Hu pleroma mix pleroma.release_env gen` * Enable and start `pleroma.service`: ```shell diff --git a/docs/installation/otp_en.md b/docs/installation/otp_en.md index 98360bcf7..63eda63ca 100644 --- a/docs/installation/otp_en.md +++ b/docs/installation/otp_en.md @@ -149,9 +149,6 @@ chown -R pleroma /etc/pleroma # Run the config generator su pleroma -s $SHELL -lc "./bin/pleroma_ctl instance gen --output /etc/pleroma/config.exs --output-psql /tmp/setup_db.psql" -# Run the environment file generator. -su pleroma -s $SHELL -lc "./bin/pleroma_ctl release_env gen" - # Create the postgres database su postgres -s $SHELL -lc "psql -f /tmp/setup_db.psql" |