Audit and rebuild of a fault-tolerant Nginx+PHP-FPM+PostgreSQL stack on 2 VPS with strict SLOs

Initial situation: there are 2 VPS in Hetzner (Ubuntu 22.04) for B2B SaaS (peak load up to 800 simultaneous users, ~120 req/s). Now everything runs on one server: Nginx, PHP-FPM, PostgreSQL 14, Redis, cron, sending mail. At peaks, 502/504 appear, the disk gets clogged periodically, backups are made irregularly, updates are installed manually. The second VPS is empty, we want to use it for fault tolerance and secure backups.

Technical environment and limitations:
- Access: SSH root to both servers, domain and DNS from Cloudflare
- You cannot change the application code and database schema, only changes to the OS configuration and services are allowed
- Allowable degradation/switching window at night - up to 10 minutes, complete unavailability - no more than 2 minutes
- You cannot buy additional managed services, only the capabilities of two VPS and Cloudflare
- It is necessary to leave the ability to maintain the system without Docker/Kubernetes (it is allowed to use systemd and standard packages)

What needs to be done in stages:
1) Diagnosis of performance and reliability problems on the current server - analysis of Nginx/PHP-FPM/PostgreSQL logs, load bottlenecks, disk subsystem, systemd limits, network, timeouts, keepalive settings, buffers, slow queries
2) Reassignment of roles on 2 VPS:
- VPS-1 - frontend: Nginx, PHP-FPM, Redis (if needed locally), cron
- VPS-2 - database and backups: PostgreSQL + replication/reservation, storage of backups, minimal services
3) Setting up PostgreSQL with an emphasis on stability - correct memory parameters/autovacuum, WAL archiving, growth control, basic hardening, latency monitoring
4) Backup and recovery testing - daily base backup + WAL (or other reasonable approach), rotation, encryption, test restore to a separate database/directory, documentation of the recovery procedure
5) Safety and operation:
- firewall (ufw/nftables) with the minimum required ports, restricting access to PostgreSQL
- SSH hardening (keys, disabling passwords, fail2ban or similar)
- automatic security updates (unattended-upgrades) with controlled reboot
6) Observability:
- metrics and alerts: CPU/RAM/disk load, I/O wait, disk fullness, 5xx count, response time, replication/lag, backup jobs
- Prometheus node_exporter + simple alerting (for example, via Alertmanager/telegram) or another stack is allowed, but without heavy platforms
7) Failover plan - step-by-step instructions on how to switch an application to a backup server/standby database within time limits

Specific result:
- Two servers with separated roles, a documented scheme, configured backups and proven recovery
- Nginx/PHP-FPM/PostgreSQL configs and systemd units are brought into a state where they can be reproduced (ideally through Ansible, but neat scripts + README are also acceptable)
- A separate file with a map of ports/accesses and a list of changes made

Measurable acceptance criteria:
- On the Nginx side, the share is 5xx in 24 hours under typical load - no more than 0.1%
- P95 response time based on Nginx access logs - no worse than the current value and a target improvement of at least 20% after tuning (we fix the baseline at the beginning of work)
- The backup chain runs daily, at least 7 restore points are stored, and at least 1 test restore with a report is completed
- In the event of artificial unavailability of VPS-1 or VPS-2, there is a switching operating procedure that fits within the limitations (up to 2 minutes of complete unavailability, up to 10 minutes of degradation at night)
- All services are automatically restored after rebooting the servers without manual actions

Important: the work includes analysis and responsibility for changes in production. Before making risky changes, agree on an action plan and rollback.