Security Incident Triage

When something looks wrong, you bring order: one log, one picture of what is known, clear decisions for the owner, and evidence kept intact for later. You move fast on the reversible things and slow on the irreversible ones. The standard: at any moment the owner can read one page and know what happened, what has been done, what is still unknown, and what they need to decide.

When to use

• "I think we've been hacked."
• "Someone logged into the admin account from another country."
• "An employee clicked a phishing link and entered their password."
• "Our API key was posted publicly." / "Files are encrypted and there's a ransom note."
• A security alert, a customer report or a vendor breach notice that may affect the company.

What you need

• Whatever the reporter saw: screenshots, alert text, email headers, times, account names.
• Read access to the logs and consoles involved: the identity provider, email, cloud console, servers. Either the owner signs your browser in, connects the app, or exports logs to the workspace. Prefer read-only access.
• Who the owner is (incident commander unless they delegate), who holds admin rights for each affected system, and contact details for legal counsel and the cyber insurer if there is a policy.
• Write access to a single incident log file in the workspace.

Method

1. Open the incident log now. Use references/incident-log-and-sitrep.md. Every entry gets a UTC timestamp, who, what was observed or done, and the source. Record the "time of awareness": legal clocks often start there.
2. Assign roles, even if one person holds several: incident commander (the owner or their delegate; makes decisions), scribe (you, keeping the log), technical lead (whoever administers the affected system), communications (internal and external updates). The commander is the single source of truth about what happens next.
3. Establish facts. Separate what is observed from what is assumed. Answer: what systems, accounts and data are involved; when it started (earliest evidence, not first report); whether it is ongoing; how it was detected.
4. Set severity. If unsure between two levels, choose the higher one and reassess later; never spend the first hour debating it.

   Level	Meaning	Update cadence
   SEV1	Active compromise of production or admin accounts, ransomware, confirmed exposure of personal or payment data, ongoing data theft	Owner engaged now; updates hourly
   SEV2	Confirmed compromise of one user account or host, contained or limited scope	Owner informed within the hour; updates every few hours
   SEV3	Suspicious, not yet confirmed	Investigate today; update at milestones
   SEV4	Policy breach or near miss, no impact	Log and fix normally

5. Preserve evidence before changing anything you do not have to change. Collect the most volatile first:
   • from a suspect Linux host, read-only: running processes, network connections, logged-in users, recent logins, crontabs and systemd timers, recently modified files in system and web directories, authorized_keys files
   • export logs before rotation deletes them: auth logs, web server logs, cloud audit logs, identity provider and email audit logs, firewall logs
   • in the cloud, ask the owner to approve disk snapshots of affected instances before any change Hash every export with sha256sum, store copies where they cannot be edited, and record who collected what, when and from where (chain of custody table in the template). Do not power off a suspect machine unless damage is ongoing and cannot be stopped another way; isolating it from the network usually preserves more.
6. Propose containment; the owner approves each action. Order options from most reversible to least and state the side effects:
   • compromised user account: revoke all sessions, reset the password, re-enrol MFA, remove mail forwarding rules and unknown OAuth app grants
   • leaked key or token: rotate or revoke at the provider, then check its audit log for use since exposure
   • suspect host: restrict its security group or firewall to an isolation rule, keep it running for evidence
   • malicious IPs or domains: block at the firewall, DNS filter or email gateway
   • affected integration: disable the token or webhook Never delete a compromised account, instance or mailbox during the incident: it is evidence. Never contact the attacker.
7. Scope the incident. Search for the same indicators across other systems: the same source IPs, user agents, file hashes, newly created users, new SSH keys, new API keys, OAuth grants, forwarding rules, unusual admin role changes. Ask: how did they get in, what did they touch, are they still here? Record each answer with its evidence and confidence.
8. Flag notification clocks to the owner for a legal decision. Examples you must check with web_search for the owner's jurisdiction and cite: GDPR and UK GDPR Article 33 (notify the supervisory authority without undue delay and, where feasible, within 72 hours of awareness when there is a risk to individuals) and Article 34 (tell individuals when the risk is high); NIS2 (early warning within 24 hours for in-scope entities); US state breach laws, which vary; SEC Form 8-K Item 1.05 for US public companies (four business days after determining materiality); card-brand and PCI rules for payment data; contractual notice terms in customer contracts; the cyber insurance policy's notice requirement. You draft; a lawyer decides; the owner sends. You never notify a regulator, customer or law enforcement yourself.
9. Ransomware specifics: isolate affected systems from the network, protect backups (disconnect or confirm immutability), do not pay or negotiate, and bring in legal counsel, the insurer and law enforcement through the owner. Payment can breach sanctions law; that decision belongs to lawyers.
10. Recovery plan, once contained: rebuild compromised hosts from known-good images rather than cleaning them; restore data from backups dated before the earliest evidence of compromise; rotate every credential the attacker could have seen; keep heightened monitoring for an agreed period.
11. Communicate on cadence using the situation report in the template: what we know, what we do not know, actions taken, decisions needed, next update time.
12. Hand over when contained: the log, evidence register and open questions go to the security-post-incident-review process.

Communication rules during an incident

• If email or chat may be compromised, move the response to a separate channel the attacker cannot see (phone, a fresh group on a different service) and say so in the log.
• One person speaks for the incident; everyone else routes questions to them.
• Updates state facts and unknowns; no guesses about who did it or how much data was taken.
• Keep the circle small until the owner decides who else needs to know; tell staff not to discuss it publicly.
• Never confirm or deny details to outsiders, including journalists and customers, without the owner's approved wording.

First moves for common incidents

Each action still needs the owner's approval, and evidence is exported first.

• Email account takeover: export the sign-in and mailbox audit logs; revoke sessions; reset password and MFA; remove forwarding and inbox rules and unknown app grants; search sent items for phishing sent to customers or staff; warn recipients through the owner.
• Leaked cloud key: record where and when it leaked; deactivate the key; search the cloud audit log for every call made with it since the leak; look for new users, keys, instances or policy changes; check billing for unexpected spend.
• Phishing click with password entered: reset that password and any reused elsewhere; revoke sessions; check sign-ins after the click; block the sender and URL; check whether others received the same message.
• Ransomware or mass encryption: isolate affected machines from the network; protect backups; identify the first affected machine and time; do not reboot or wipe; engage the owner, insurer and legal at once.
• Lost or stolen laptop: confirm disk encryption status; remotely lock or wipe through device management if the owner approves; revoke the device's sessions and tokens; rotate credentials stored on it.
• Suspicious admin change: confirm with the named admin whether they made it; if not, treat their account as compromised and look for other changes in the same window.

Output

• The live incident log and evidence register (references/incident-log-and-sitrep.md).
• A situation report on a show_card at each update: severity, status, known, unknown, decisions needed, next update.
• Draft notifications only when the owner asks, clearly marked DRAFT for legal review.

Checks before you finish

• Every action in the log has a time, an actor and the owner's approval where needed.
• Evidence exports are hashed and the chain of custody is complete.
• Nothing that is evidence was deleted.
• All notification clocks relevant to the owner's jurisdiction were flagged with sources and the time of awareness.
• Every credential the attacker could have seen is listed with its rotation status.

Pitfalls

• Cleaning up before collecting. Rebooting, deleting the rogue user or wiping the server destroys the evidence that tells you how far it went.
• Containing one account and stopping. Attackers create persistence: forwarding rules, extra keys, new admins. Scope before declaring it over.
• Working from chat memory. Without the log, timelines and legal clocks fall apart. Write everything down as you go.
• Optimistic severity. Downgrade later if the facts allow; upgrading late costs hours.
• Saying "no data was accessed" without logs to prove it. Say "no evidence of access in the logs available, which cover X to Y".
• Sign-off. The owner makes containment and business decisions. Legal counsel decides on notification. For SEV1, recommend engaging a qualified incident response firm, ideally the insurer's panel firm if insured.

Credits for adapted material: references/CREDITS.md.