PSSP troubleshooting
Operations and diagnostics

Troubleshoot PSSP connections

Use this guide when a client cannot connect, authenticate, subscribe, publish, bind UDP, or resume data. Work from top to bottom: a credential change cannot fix a broker that is not listening, and an ACL change cannot fix a failed login.

If the shell cannot find pssp-broker

The executable is not automatically installed into PATH by checking out this repository. Its source is in broker/; local builds are written to target/debug/pssp-broker or target/release/pssp-broker (with .exe on Windows). From the repository root, either run it through Cargo or use the built path directly:

cargo run -p pssp-broker -- --help
./target/debug/pssp-broker --help                 # macOS/Linux
.\target\debug\pssp-broker.exe --help             # Windows PowerShell
Bare command convention

Examples beginning with pssp-broker describe an installed deployment whose executable is in PATH. For repository development, replace that prefix with cargo run -p pssp-broker -- or the appropriate path under target/. See Where to find and how to run the Broker for the complete path table and examples.

Use this diagnostic order

  1. Start the Broker. Confirm that it remains running instead of exiting with a configuration, certificate, key, or database error.
  2. Confirm TCP reachability. Verify the expected process is listening on TCP port 6688 and that the client can reach that host and port.
  3. Match the security mode. TLS, PSSP AES, the TLS checkbox, certificate trust, and the AES-only public-key pin must agree.
  4. Verify authentication. The exact clientId + username record must exist, be enabled, and match the current password.
  5. Verify authorization. A successful login does not grant publish or subscribe access. Check the principal's topic ACLs.
  6. Test data flow. Test TCP ping, subscribe/publish, UDP bind/ping, then replay and retention behaviour.
Fast interpretation

Connection refused normally means no listener or the wrong address. A TLS or AES pin error happens before AUTH. authentication_failed comes from the credential database. publish_not_authorized and subscribe_not_authorized mean authentication succeeded but ACL evaluation failed.

Case study: pssp-admin / admin could not connect

The following issues were found while investigating a client using client ID pssp-admin, username admin, and a supplied password. They are documented here because the same combination of failures is easy to reproduce during development.

FindingWhy it blocked the connectionResolution
No process was listening on TCP or UDP 6688.The client could not reach a Broker, so credential validation never began.Start the Broker and confirm the listener before changing credentials.
The active config's storage.sqlite_path selected a protected production database owned by root with mode 0600.A Broker started as a normal development user exited because that user could not traverse the database directory or open SQLite.Use the repository development config for a user-run Broker, or configure the database owner to match the installed service account and rerun setup. Never solve this with chmod 777.
TLS was off and PSSP AES was on.The desktop client requires the exact Broker X25519 public-key pin before it will send credentials in AES-only mode. Its pin field is empty by default.Provision the public key using the AES-only steps below and paste it into the client.
The configured private key was the all-zero format example.It is syntactically valid but publicly known and therefore unsuitable as a Broker identity key.Generate a random private key, protect it, update the config, and redistribute the new public-key pin.
pssp-admin and admin appeared only in setup examples.PSSP has no built-in default account or default password. The password is whatever the administrator entered during setup or a later rotation.List the principal, enable it if necessary, or rotate its password. Password hashes cannot be reversed or displayed.
No installed macOS launch service, installed binary, or /etc/pssp/pssp.toml was present.The repository config and database existed in a partial installation state, but nothing automatically started the Broker.Either run from the repository for development or complete the platform service installation.
Summary

The first root cause was service availability: the Broker process identity did not match the owner selected for the database in the active config. AES-only pinning was the next blocker. Only after those are fixed can the supplied password be meaningfully tested.

Start the desktop client correctly

  1. Install dependencies once.
    cd client
    npm install
  2. Start the Tauri application, including its Rust backend.
    npm run tauri:dev
  3. Do not use npm run dev as the complete client. That command starts only the Vite web interface; native Tauri commands such as connect require the desktop runtime.

If the application does not build, confirm Node.js 20+, a current Rust toolchain, and the operating-system packages required by Tauri 2. Build the Rust workspace separately to expose backend compiler errors:

cargo check --workspace

Confirm that the Broker is running and listening

Development start

cargo run -p pssp-broker -- serve \
  --config broker/config/pssp.example.toml

Keep the terminal open and read the first error. A healthy startup logs TCP and UDP listener addresses and the selected TLS/AES mode.

Listener checks

PlatformCommand
macOSlsof -nP -iTCP:6688 -sTCP:LISTEN
Linuxss -lntup | grep 6688
Windows PowerShellGet-NetTCPConnection -LocalPort 6688 -State Listen
Remote TCP checknc -vz BROKER_HOST 6688 or Test-NetConnection BROKER_HOST -Port 6688

If the Broker listens on 127.0.0.1, only local clients can connect. Use an appropriate interface such as 0.0.0.0:6688 only when firewall policy restricts access to approved networks. Confirm that another process is not already using the port.

Service checks

PlatformUseful checks
Linux/systemdsystemctl status pssp-broker
journalctl -u pssp-broker -n 100 --no-pager
macOS/launchdlaunchctl print system/io.oxrecorder.pssp-broker
Inspect the configured standard-output and standard-error logs.
Windows Task SchedulerGet-ScheduledTask -TaskName 'PSSP Broker'
Review task history and the configured executable/config paths.

Initialize SQLite and fix ownership safely

The Broker will not start against an absent, uninitialized, or unreadable database. SQLite stores principals, Argon2id password hashes, enabled state, ACLs, and credential audit events. Topic payloads and session state remain in memory.

The config selects the database location

The Broker does not have one fixed database directory. storage.sqlite_path in the exact config passed to setup, every administration command, and serve is the source of truth. A relative path is resolved from the process working directory. The repository development config uses data/pssp-broker.sqlite3, which is <repository-root>/data/pssp-broker.sqlite3 when the documented Cargo commands are run from the repository root. Installed services should use an absolute protected path chosen in their deployment config.

  1. Open the active TOML config and verify storage.sqlite_path. Do not diagnose one config and start the Broker with another.
  2. Choose the identity that will run the Broker service. On Unix, set both storage.unix_owner_user and storage.unix_owner_group to that existing service account, or omit both for a user-owned development database. The two settings must be supplied together.
  3. Initialize the schema and bootstrap administrator with that same config. The command creates missing parent directories and the SQLite file, securely prompts for a password of at least 12 characters, and on Unix restricts the database and existing sidecars to 0600. When the Unix owner pair is configured, it also restricts the parent directory to 0700 and applies the configured ownership; run setup as root or as that service identity so those changes are permitted.
    pssp-broker setup --config /etc/pssp/pssp.toml \
      --bootstrap-client-id pssp-admin \
      --bootstrap-username admin
  4. Verify metadata without exposing password hashes.
    pssp-broker list-principals --config /etc/pssp/pssp.toml
    pssp-broker list-acls --config /etc/pssp/pssp.toml \
      --client-id pssp-admin --username admin
  5. Start the service under the same identity and confirm the listener.

Filesystem access and topic grants are different

Permission layerConfigured withWhat it controls
Operating-system filesystem accessstorage.sqlite_path, unix_owner_user, unix_owner_group, and setupWhether the Broker process can open the database and create SQLite WAL/SHM sidecars.
PSSP authenticationsetup, create-principal, rotate-password, and account enable/disable commandsWhether an exact clientId + username + password can authenticate.
PSSP topic authorizationgrant-acl and revoke-aclWhich concrete topics a principal may publish and which filters it may subscribe to. A newly created principal has no topic access until ACLs are granted.

Run every principal and ACL command with the same --config used by serve. Mutating CLI commands preserve the configured Unix owner and secure modes; manually changing a file to be world-readable or world-writable grants no PSSP topic access.

Local repository alternative

For local development, use broker/config/pssp.example.toml. Run these commands from the repository root; the configured relative path then creates data/pssp-broker.sqlite3 inside the workspace and avoids mixing an installed-service database with a user-run Broker:

cargo run -p pssp-broker -- setup \
  --config broker/config/pssp.example.toml \
  --bootstrap-client-id pssp-admin --bootstrap-username admin

cargo run -p pssp-broker -- serve \
  --config broker/config/pssp.example.toml
Do not weaken the whole directory

A database error should be fixed by selecting the correct service identity and ownership. Avoid world-writable permissions, and do not run an untrusted desktop client with direct access to the Broker database. Clients authenticate over PSSP only.

Resolve authentication_failed

The wire login contains three independent values: clientId, username, and password. The first two select one exact SQLite record; the third is checked against its Argon2id hash.

  1. List principals and confirm the exact spelling, case, and enabled state.
    pssp-broker list-principals --config /etc/pssp/pssp.toml
  2. If the password is unknown, rotate it; it cannot be recovered from the hash.
    pssp-broker rotate-password --config /etc/pssp/pssp.toml \
      --client-id pssp-admin --username admin
  3. Enable a disabled principal.
    pssp-broker set-principal-enabled --config /etc/pssp/pssp.toml \
      --client-id pssp-admin --username admin --enabled
  4. Wait for storage.auth_cache_seconds—30 seconds by default—or restart the Broker before retesting a recently failed or changed credential.
  5. Enter the values without leading/trailing spaces. The client sends the values as entered; it does not create accounts or substitute a default password.
Bootstrap setup is not a password reset

If the principal already exists, running setup verifies its administrator ACLs but does not replace its existing password. Use rotate-password when the password must change.

Set up an AES-only connection step by step

AES-only means TLS is disabled while PSSP AES-256-GCM is enabled. The Broker has a long-lived X25519 identity key; every session derives a fresh AES key using the pinned Broker public key, an ephemeral client key, and HKDF-SHA-256. The AES session key is never transmitted.

Where to find the Broker public key

The Broker does not store a separate AES-only public-key file. It derives the X25519 public key from encryption.static_private_key. Run pssp-broker aes-only-public-key and enter the private key at the secure prompt, or run pssp-broker aes-only-public-key --private-key-file /etc/pssp/aes-only-private-key. The printed base64url value is the public key to paste into the client's Broker AES-only X25519 public key field. It is also advertised as HELLO_ACK.aesOnlyKeyAgreement.brokerStaticPublicKey, but clients must pin an independently derived and trusted copy rather than trusting the value received over the same connection.

  1. Copy the AES-only template to the protected deployment config.
    cp broker/config/pssp.aes-only.example.toml /etc/pssp/pssp.toml
  2. Generate a random private key and save it in a protected operator-only location.
    umask 077
    pssp-broker generate-aes-only-private-key \
      > /etc/pssp/aes-only-private-key
  3. Copy that generated value into encryption.static_private_key in /etc/pssp/pssp.toml. Do not use AAAAAAAA...AAA; it is only a format example.
  4. Derive the public key that clients must pin.
    pssp-broker aes-only-public-key \
      --private-key-file /etc/pssp/aes-only-private-key
  5. Confirm the security configuration.
    [tls]
    enabled = false
    
    [encryption]
    enabled = true
    static_private_key = "GENERATED_PRIVATE_KEY"
    
    [transport]
    allow_insecure_tcp_udp = false
  6. Initialize SQLite with this exact config, then start the Broker.
    pssp-broker setup --config /etc/pssp/pssp.toml \
      --bootstrap-client-id pssp-admin --bootstrap-username admin
    pssp-broker serve --config /etc/pssp/pssp.toml
  7. In the desktop client, set the Broker host and port, leave TLS 1.3 transport off, paste the derived public key into Broker AES-only X25519 public key, enter the exact credentials, and select Connect.
  8. Verify that the client reports AES session: active. Then test TCP Ping, subscribe/publish, Bind UDP, and UDP Ping.
AES-only symptomCause and correction
TLS-off AES broker requires a pinned X25519 public keyThe pin field is empty. Provision and paste the derived Broker public key.
broker AES-only public key does not match the pinned keyThe Broker key changed, the wrong environment was selected, or the pin was copied incorrectly. Verify out of band before replacing a pin.
Broker rejects config or key agreementConfirm the private key is base64url without padding and decodes to exactly 32 bytes.
Existing clients fail after key rotationRotation changes the public pin. Securely redistribute the new public key to every client before or during the planned cutover.
The public key is not secret, but it must be authentic.

Distribute the public pin through a trusted channel such as managed device configuration, a signed package, or a separately verified administrator channel. Accepting an unverified key shown by the same network endpoint defeats pinning.

Set up TLS + AES

  1. Start from broker/config/pssp.production.example.toml.
  2. Install a TLS certificate and private key readable by the Broker service identity. The certificate must include the DNS name clients use.
  3. Configure both layers.
    [tls]
    enabled = true
    certificate_file = "/etc/pssp/tls/fullchain.pem"
    private_key_file = "/etc/pssp/tls/privkey.pem"
    
    [encryption]
    enabled = true
    
    [transport]
    allow_insecure_tcp_udp = false
  4. Initialize SQLite, start the Broker, and confirm it logs tls=true and encryption=true.
  5. In the client, connect using the certificate's DNS hostname and turn TLS 1.3 transport on. The AES-only public-key field is not used.

A TLS handshake error normally means the TLS checkbox does not match the Broker, the certificate is expired or untrusted, the hostname does not match, or an intermediate certificate is missing. Connecting by IP address will fail unless that IP address is present in the certificate's Subject Alternative Name.

Use plaintext only for isolated local testing

[tls]
enabled = false

[encryption]
enabled = false

[transport]
allow_insecure_tcp_udp = true

Leave the client's TLS checkbox off and leave the AES-only pin empty. The Broker intentionally rejects a no-TLS/no-AES configuration unless allow_insecure_tcp_udp = true; it also rejects that opt-in when either TLS or AES is enabled.

Credentials and payloads are visible on the network.

Use this mode only on an isolated developer machine or an explicitly trusted private boundary. Do not expose it to Wi-Fi, the public Internet, or an untrusted LAN.

Fix publish and subscribe authorization

Authentication and authorization are separate. A principal can log in successfully while having no ACLs.

pssp-broker list-acls --config /etc/pssp/pssp.toml \
  --client-id pssp-admin --username admin

Grant an administrator test account access to all topics:

pssp-broker grant-acl --config /etc/pssp/pssp.toml \
  --client-id pssp-admin --username admin \
  --action publish --topic-filter '#'

pssp-broker grant-acl --config /etc/pssp/pssp.toml \
  --client-id pssp-admin --username admin \
  --action subscribe --topic-filter '#'
IssueResolution
publish_not_authorizedGrant a matching publish ACL. Publish to a concrete topic such as demo/topic; do not publish to # or a topic containing +.
subscribe_not_authorizedGrant a subscribe ACL that contains the requested filter. ACL and subscription filters may use + and a final #.
Small payload works; large payload failsCheck the Broker's global limits.max_payload_bytes and the principal's optional per-account maximum.
ACL change seems ignoredWait for the authentication cache TTL or reconnect/restart as appropriate.

Fix UDP bind and ping failures

  1. Complete the authenticated TCP connection first. UDP is associated with a live TCP session.
  2. Select Bind UDP before UDP Ping.
  3. Keep the TCP connection open; closing it invalidates the UDP token, AES material, and endpoint association.
  4. Allow UDP port 6688 through the host firewall, cloud firewall, VPN policy, NAT, and container/VM forwarding.
  5. Confirm TCP and UDP point to the same Broker host and configured port.
  6. If TCP works but UDP times out, capture Broker logs while binding. A network policy can permit TCP while silently dropping UDP.

UDP is QoS 0: it has no replay, receipt guarantee, or retry. A lost heartbeat is not proof that the TCP session is dead; use TCP Ping to test the reliable control path.

Replay, GAP, disconnects, and slow clients

ObservationMeaning and action
GAPThe requested sequence was already evicted from the bounded in-memory topic ring. Recover from application storage or increase retention after sizing memory.
resume returns no old messagesReuse the same workerID, reconnect before subscriber_resume_ttl_seconds expires, and confirm the Broker did not restart.
Data disappeared after Broker restartExpected: topic rings, session keys, subscriptions, UDP associations, deduplication state, and resume cursors are volatile.
Slow subscriber misses live deliveryPer-session outbound queues are bounded so a slow client cannot block the Broker. Reconnect and replay retained data, or fix the consumer.
Duplicate client ID has multiple connectionsExpected. PSSP does not evict an existing connection merely because another session uses the same client ID.
Idle connection closesSend periodic TCP PING before limits.idle_timeout_seconds expires.

Common error and symptom reference

Error or symptomLikely layerFirst action
connect TCP, connection refusedProcess/networkStart the Broker and verify the listener address and port.
Connection timeoutRouting/firewallCheck host, route, firewall, VPN, NAT, and security groups.
open SQLite database / error code 14Storage permissionsAlign the Broker process identity with database directory/file ownership.
database is not initializedStorage schemaRun pssp-broker setup with the same config used by serve.
TLS and AES disabled without explicit opt-inConfiguration validationFor isolated plaintext testing only, set transport.allow_insecure_tcp_udp = true.
TLS requires certificate_file and private_key_fileConfiguration/TLSSet both protected file paths, verify service-account read access, and confirm the private key matches the certificate.
TLS-off PSSP AES requires encryption.static_private_keyConfiguration/AESGenerate and configure a 32-byte base64url X25519 private key.
Broker requires TLSSecurity mismatchEnable TLS in the client and use a trusted matching hostname.
TLS handshake failureCertificate/TLSCheck trust chain, validity, hostname, and TLS mode.
AES-only pin missing/mismatchKey pinningVerify and provision the Broker X25519 public key out of band.
authentication_failedIdentityCheck exact principal, enabled state, password, and auth-cache timing.
publish_not_authorizedACL/topicCheck publish ACL and use a concrete topic.
subscribe_not_authorizedACL/filterCheck the subscribe ACL and filter syntax.
PSSP payload exceeds limitLimitsReduce/chunk the payload or deliberately raise compatible limits.
PSSP request timed outSession/BrokerInspect Broker logs, TCP liveness, encryption state, and firewall behaviour.
unsupported PSSP version, invalid magic, or unknown record type/flagsProtocol compatibilityConfirm both applications use compatible PSSP v1 builds and that the endpoint is really a PSSP Broker, not an HTTP/TLS service on the same port.
connection limit reached or outbound queue fullCapacity/backpressureFind leaked or slow sessions, review max_connections and queue sizing, and scale only after measuring resource use.
UDP bind/ping timeoutUDP pathKeep TCP alive, bind first, and permit UDP 6688.
GAPRetentionRecover externally or review topic ring limits.

Collect a useful troubleshooting report

Before asking another developer for help, record the following without including passwords, private keys, raw session keys, or sensitive payloads:

Redact secrets

Never paste encryption.static_private_key, TLS private keys, passwords, database files, or packet captures containing plaintext credentials into an issue report. The AES-only public key is safe to identify, but publishing it may still reveal environment relationships.