EC2 source-code discovery and reliability triage #2

Merged
adelvalle merged 1 commits from ec2-code-discovery into main 2026-09-02 14:36:28 -06:00
Owner

Second discovery pass over the Scrivas EC2 estate (716468089330, us-east-2, 7 instances). Prompted by two client signals: they do not hold the source code for their own platform (built under contract by the incumbent vendor), and they are reporting production reliability issues.

All access was read-only — no writes, restarts or config changes on any instance.

Source code is recoverable

All 10 application repositories exist as complete git checkouts on Scrivas-owned instances, with full commit history rather than deployed artifacts.

Tier Location Repos
App Scrivas_dev_env:/home/admin/ scrivas_backend (1380 commits), scrivas_gate (183), scrivas_search (96), patient (59)
ML ML_dev:/srv/ post_processor (137), sai_suggestions (93), ml_monitoring (62), patient-summary-service (49), patient_document_parser (41), soniox_transcriber (25)

Every remote points at git@git.devteam.space:scrivas/* — the contractor's self-hosted GitLab, which Scrivas does not control. The on-instance checkouts are the client's only independent leverage over their own source.

Gap: both /var/www frontends are build output with no .git. Frontend source is not recoverable from EC2.

Time-sensitive: scrivas_backend received a commit on the assessment date. Development is active on infrastructure the contractor controls. Mirroring the repos to Scrivas-controlled storage is the recommended immediate action and is not included in this PR — it needs a scope decision first (it involves the contractor's GitLab credentials held on the dev box).

Reliability triage

Production runs 23 containers on a single 15 GiB host — including 3 Postgres instances, Kafka and OpenSearch — at 73% memory at rest, with no per-container memory limits and no swap on any of the 7 instances. With no limits the OOM killer selects by resident size, so it would typically kill a database rather than the worker that caused the pressure. That matches the "random, unreproducible" symptom profile.

Kafka, OpenSearch and search-api additionally carry restart policy no, so a host reboot yields a partially-recovered stack that looks healthy from outside.

Recorded as a structural exposure, not an observed root cause. No OOM event appears in retained logs, State.OOMKilled is false on all 23 containers, and RestartCount is 0 on every one — the Celery workers showing "Up 6 hours" were redeployed, not crash-restarted. Confirming the hypothesis needs CloudWatch history these boxes do not retain, which is itself a finding and the basis for recommendations 3–5 in the report.

One encouraging contrast: the ML tier is well-built — ECR images tagged by commit SHA, blue/green slots, passing health checks. The application tier is compose-from-git-checkout with uncommitted .env.save and docker-compose.yml.bkp files in the prod working tree. The better pattern already exists in-house.

Contents

File
scripts/ec2_code_discovery.py EC2 inventory — describe + user data
scripts/ec2_code_inspect.py read-only SSM probe set; commands reviewable in PROBES
findings/ec2_code_discovery_report.md narrative writeup
findings/code_dashboard.html client-facing dashboard, matching the existing design system
findings/ec2_code_inspect*.json raw probe output
index.html links the new dashboard and evidence

Review notes

  • Probe output was scanned for credentials before commit ��� no AWS keys, passwords, tokens or private key blocks. The probe set reads manifests and VCS metadata, never file contents.
  • Git metadata was read as the owning user (sudo -u) rather than by writing a safe.directory entry into root's gitconfig, to preserve the read-only guarantee.
  • Several prod probes initially returned empty and were re-run with stderr visible; the empties were artifacts (last is not installed on prod), not clean health. Worth knowing when reading the JSON.
  • The dashboard keeps the "what the evidence does not show" section in the client-facing version. If CloudWatch later points elsewhere, overclaiming a root cause would cost more credibility than the softer framing gains.

https://claude.ai/code/session_01YMxVaHXJsqpqKwncNQ9b1e

Second discovery pass over the Scrivas EC2 estate (`716468089330`, `us-east-2`, 7 instances). Prompted by two client signals: they do not hold the source code for their own platform (built under contract by the incumbent vendor), and they are reporting production reliability issues. All access was **read-only** — no writes, restarts or config changes on any instance. ## Source code is recoverable **All 10 application repositories exist as complete git checkouts on Scrivas-owned instances**, with full commit history rather than deployed artifacts. | Tier | Location | Repos | |---|---|---| | App | `Scrivas_dev_env:/home/admin/` | `scrivas_backend` (1380 commits), `scrivas_gate` (183), `scrivas_search` (96), `patient` (59) | | ML | `ML_dev:/srv/` | `post_processor` (137), `sai_suggestions` (93), `ml_monitoring` (62), `patient-summary-service` (49), `patient_document_parser` (41), `soniox_transcriber` (25) | Every remote points at `git@git.devteam.space:scrivas/*` — the contractor's self-hosted GitLab, which Scrivas does **not** control. The on-instance checkouts are the client's only independent leverage over their own source. **Gap:** both `/var/www` frontends are build output with no `.git`. Frontend source is not recoverable from EC2. **Time-sensitive:** `scrivas_backend` received a commit on the assessment date. Development is active on infrastructure the contractor controls. Mirroring the repos to Scrivas-controlled storage is the recommended immediate action and is **not** included in this PR — it needs a scope decision first (it involves the contractor's GitLab credentials held on the dev box). ## Reliability triage Production runs **23 containers on a single 15 GiB host** — including 3 Postgres instances, Kafka and OpenSearch — at **73% memory at rest**, with **no per-container memory limits** and **no swap** on any of the 7 instances. With no limits the OOM killer selects by resident size, so it would typically kill a database rather than the worker that caused the pressure. That matches the "random, unreproducible" symptom profile. Kafka, OpenSearch and search-api additionally carry restart policy `no`, so a host reboot yields a partially-recovered stack that looks healthy from outside. **Recorded as a structural exposure, not an observed root cause.** No OOM event appears in retained logs, `State.OOMKilled` is false on all 23 containers, and `RestartCount` is **0** on every one — the Celery workers showing "Up 6 hours" were redeployed, not crash-restarted. Confirming the hypothesis needs CloudWatch history these boxes do not retain, which is itself a finding and the basis for recommendations 3–5 in the report. One encouraging contrast: the **ML tier is well-built** — ECR images tagged by commit SHA, blue/green slots, passing health checks. The application tier is compose-from-git-checkout with uncommitted `.env.save` and `docker-compose.yml.bkp` files in the prod working tree. The better pattern already exists in-house. ## Contents | File | | |---|---| | `scripts/ec2_code_discovery.py` | EC2 inventory — describe + user data | | `scripts/ec2_code_inspect.py` | read-only SSM probe set; commands reviewable in `PROBES` | | `findings/ec2_code_discovery_report.md` | narrative writeup | | `findings/code_dashboard.html` | client-facing dashboard, matching the existing design system | | `findings/ec2_code_inspect*.json` | raw probe output | | `index.html` | links the new dashboard and evidence | ## Review notes - Probe output was **scanned for credentials before commit** ��� no AWS keys, passwords, tokens or private key blocks. The probe set reads manifests and VCS metadata, never file contents. - Git metadata was read **as the owning user** (`sudo -u`) rather than by writing a `safe.directory` entry into root's gitconfig, to preserve the read-only guarantee. - Several prod probes initially returned empty and were re-run with stderr visible; the empties were artifacts (`last` is not installed on prod), not clean health. Worth knowing when reading the JSON. - The dashboard keeps the "what the evidence does not show" section in the **client-facing** version. If CloudWatch later points elsewhere, overclaiming a root cause would cost more credibility than the softer framing gains. https://claude.ai/code/session_01YMxVaHXJsqpqKwncNQ9b1e
adelvalle added 1 commit 2026-08-28 14:25:27 -06:00
Second discovery pass over the Scrivas EC2 estate (716468089330, us-east-2,
7 instances), prompted by the client reporting reliability issues and by their
lack of access to source code held under contract by the incumbent vendor.

Source code recovery
- All 10 application repositories exist as complete git checkouts on
  Scrivas-owned instances, with full history rather than deployed artifacts:
  4 app repos on Scrivas_dev_env, 6 ML repos on ML_dev.
- Every remote points at git@git.devteam.space (the contractor's self-hosted
  GitLab), which Scrivas does not control. The on-instance checkouts are the
  client's only independent leverage over their own source.
- Gap: both /var/www frontends are build output with no .git, so frontend
  source is not recoverable from EC2.
- Time-sensitive: scrivas_backend received a commit on the assessment date.

Reliability triage
- Production runs 23 containers on a single 15 GiB host, including 3 Postgres
  instances, Kafka and OpenSearch, at 73% memory at rest with no per-container
  memory limits and no swap on any of the 7 instances.
- Kafka, OpenSearch and search-api carry restart policy `no`, so a host reboot
  yields a partially-recovered stack.
- Recorded as a structural exposure, not an observed root cause: no OOM event
  is present in retained logs and RestartCount is 0 on every prod container.
  Confirming the hypothesis needs CloudWatch history the boxes do not retain.

Contents
- scripts/ec2_code_discovery.py  EC2 inventory (describe + user data)
- scripts/ec2_code_inspect.py    read-only SSM probe set, reviewable in PROBES
- findings/ec2_code_discovery_report.md   narrative writeup
- findings/code_dashboard.html            client-facing dashboard
- findings/ec2_code_inspect*.json         raw probe output
- index.html                              links the new dashboard and evidence

All access was read-only: no writes, restarts or config changes on any
instance. Probe output was scanned for credentials before commit; git metadata
was read as the owning user rather than by writing a safe.directory entry.

Claude-Session: https://claude.ai/code/session_01YMxVaHXJsqpqKwncNQ9b1e
adelvalle merged commit c987ed07c5 into main 2026-09-02 14:36:28 -06:00
adelvalle deleted branch ec2-code-discovery 2026-09-02 14:36:28 -06:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: adelvalle/scrivas#2