Every repository points at a single external remote: git@git.devteam.space:scrivas/<repo>.git —
the incumbent contractor's self-hosted GitLab, which Scrivas does not control. The on-instance checkouts are
therefore the client's only independent leverage over their own source.
/home/admin/| Repository | Commits | Last commit |
|---|---|---|
| scrivas_backend | 1380 | 2026-08-28 · Vasilii |
| scrivas_gate | 183 | 2026-08-14 · Vasilii |
| scrivas_search | 96 | 2026-06-24 · Azamat |
| patient | 59 | 2026-08-13 · vturtugeshev |
/srv/| Repository | Commits | Last commit |
|---|---|---|
| post_processor | 137 | 2026-06-30 |
| sai_suggestions | 93 | 2026-06-16 |
| ml_monitoring | 62 | 2026-07-20 |
| patient-summary-service | 49 | 2026-06-12 |
| patient_document_parser | 41 | 2026-06-16 |
| soniox_transcriber | 25 | 2026-06-16 |
/var/www/scrivas_frontend and /var/www/Scrivas_admin_frontend are
deployed build output with no .git. Frontend source cannot be recovered from EC2 and must be
obtained from the contractor's GitLab.
Why this is urgent: scrivas_backend received a commit on the day of this assessment.
Development is active on infrastructure the contractor controls. Mirror all 10 repos
(git clone --mirror) to Scrivas-controlled storage before any contract conversation changes
access posture.
23 containers on a single host — encounter_api,
gate_api, patient_api, 8 Celery workers, celery_beat,
3 × Postgres 17.5, Redis, Kafka, Debezium CDC, OpenSearch, autoheal.
Deployed by docker-compose directly from git checkouts.
No orchestration, no image registry, no immutable artifact.
Deployed from ECR
(…dkr.ecr.us-east-2…/scrivas/*) with commit-SHA image tags and
blue/green slots (dev-green-*, stage-blue-*).
Health checks defined and passing across every service. This is a materially stronger delivery pipeline — and a template the application tier can adopt.
| # | Finding | Evidence | Risk |
|---|---|---|---|
| 1 | No memory limit on any prod container | HostConfig.Memory=0 on all 23 | High |
| 2 | No swap on any instance | swapon --show empty, all 7 | High |
| 3 | Prod at 73% memory at rest | 11 GiB used of 15 GiB | High |
| 4 | Whole stack on one host, incl. 3 databases | docker ps — 23 containers | High |
| 5 | 3 prod containers set to never restart | Kafka, OpenSearch, search-api — policy no | High |
| 6 | Prod OS differs from dev/stage | Ubuntu 26.04 vs 24.04 elsewhere | Medium |
| 7 | autoheal deployed in production | willfarrell/autoheal:1.2.0 | Medium |
| 8 | Uncommitted config drift in prod checkouts | docker-compose.yml.bkp, .env.save | Medium |
| 9 | 47 interactive login sessions on prod | uptime (32 on Ml_prod) | Medium |
| 10 | sai-suggestions-redis.service failed | ML_stage systemctl --state=failed | Low |
Production memory, at rest — before any workload spike:
That yields precisely the profile a client reports as "random reliability issues": unpredictable, uncorrelated with the triggering workload, and difficult to reproduce.
Finding 5 compounds recovery. Kafka, OpenSearch and search-api carry restart policy no — if
they die or the host reboots, they stay down until someone intervenes. Everything else is
unless-stopped, so a reboot yields a partially-recovered stack that looks healthy from the outside.
Finding 7 corroborates. Someone deliberately deployed autoheal, a daemon whose only job is
restarting containers that fail health checks. That treats a symptom — and indicates health-check failures were
frequent enough to be worth automating around.
· No OOM kill was captured. dmesg and journalctl on prod show no OOM events in
the retained window; State.OOMKilled is false on all 23 containers.
· No crash-looping. RestartCount is 0 on every prod container. The Celery workers
showing "Up 6 hours" were redeployed at 13:49 that day, not restarted by failure.
· last is not installed on prod, so reboot history could not be read there.
Findings 1–5 are therefore a demonstrated structural exposure, not an observed root cause. Confirming the hypothesis requires history these instances do not currently retain — which is itself a finding.
unless-stopped on Kafka, OpenSearch and search-api — removes the silent partial-recovery failure.EC2-CloudWatchAgent-Role is already attached.mem_limit per service, sized from observed peaks; add swap as an OOM buffer.mem_used_percent to date the onset of instability..env.save, docker-compose.yml.bkp) into version control.