feat(profiles): security-pentest, recon and scanning inside an engagement (step 5, PR 5)

NEWS

@@ -75,6 +75,16 @@ a safety policy you can read and change.
   Imaging tools work on image files in the project only, hashed and
   logged (dc3dd, ewfacquire), and exiftool may only read.
 
+  security-pentest, reconnaissance and scanning inside an engagement:
+  it runs no command until an engagement is open, and the engagement's
+  scope guard reads every target of dig, host, whois, nmap, masscan,
+  nc, curl, wget, whatweb, nikto, gobuster, ffuf, feroxbuster, sslscan
+  and testssl.sh -- out of scope is refused.  nmap scripts are judged by
+  nmap's own script database (no exploit, brute, dos, intrusive or
+  external ones); redirects are never followed; no listeners, no
+  exploitation tools, no files sent to a target.  A profile requires an
+  engagement with engagement = true.
+
   A profile can ask for observer isolation (isolation = "observer"):
   command steps see the host's processes through a read-only /proc,
   for live triage, but still cannot signal them or reach the network.

docs/chapters/architecture.texi

@@ -638,8 +638,8 @@ src/codebud/                   the package
 src/codebud_sec/               the security layer (authorized security work)
 ├── engagement.py      engagement file: authorized scope
 ├── scope.py           scope guard: every network target read from the
-│                      command line and checked (DNS tools, scanners,
-│                      nc, web clients, fuzzers, TLS scanners);
+│                      command line and checked (DNS tools, whois,
+│                      scanners, nc, web clients, fuzzers, TLS scanners);
 │                      registered with the core's safety.add_step_check()
 └── evidence.py        hash-chained, append-only evidence log,
                        encrypted at rest (appends need no passphrase)

docs/chapters/configuration.texi

@@ -488,6 +488,50 @@ done by you, outside Codebud, with a write-blocker where it applies
 (NIST SP 800-86 advises against imaging a whole device from a live
 system); Codebud analyses the images, and re-images or converts them
 (raw to E01) only as files inside the project.
+
+@item security-pentest
+Reconnaissance and scanning of hosts an engagement authorizes, the
+first two tiers of a penetration test (NIST SP 800-115): no
+exploitation, no password guessing, no denial of service.  Its
+settings: mode @code{ask}; privacy @code{private} (@code{locked} would
+refuse every network step); encryption at rest on; an integrity
+manifest required; and @code{engagement = true}, so it runs no command
+and no web search until an engagement is open.  The engagement's scope
+guard then reads every host a network step will contact --- targets,
+proxies, DNS servers, addresses given to @command{curl --resolve} ---
+and refuses the step when one is out of scope, excluded, or the
+engagement window is closed.  A command it cannot read (an unknown
+option, a target list outside the project, an ambiguous URL) is refused
+too.
+
+Its allow-list: reading results runs once the plan is approved
+(@command{cat}, @command{grep}, @command{rg}, @command{jq},
+@command{diff}, hashes, @dots{}); every network tool is powerful and
+asked for every time --- @command{dig}, @command{host}, @command{whois},
+@command{nmap}, @command{masscan}, @command{nc}, @command{ncat},
+@command{curl}, @command{wget}, @command{whatweb}, @command{nikto},
+@command{gobuster}, @command{ffuf}, @command{feroxbuster},
+@command{sslscan} and @command{testssl.sh}.
+
+What the scope guard refuses beyond out-of-scope targets: nmap scripts
+that nmap's own database marks exploit, brute, dos, intrusive, fuzzer,
+broadcast or external, worked out from the whole @option{--script}
+expression --- nmap's @code{default} category holds two external
+scripts, so @option{-sC} and @option{-A} are refused; use
+@code{--script "default and not external"} (@option{-sV} is allowed,
+though nmap marks one version script, @code{iec61850-mms}, intrusive);
+following redirects (@command{curl -L}; @command{wget} needs
+@option{--max-redirect=0} and @command{whatweb}
+@option{--follow-redirect=never}); a fuzzing keyword anywhere but the
+path; @command{masscan} faster than 1000 packets a second;
+@command{nikto} without @option{-Tuning} leaving out class 6, its
+denial-of-service tests (@code{-Tuning x6}); @command{nmap
+--datadir}, which would load scripts from elsewhere; sending a file to
+a target (@command{curl -T}, @code{-d @@file}, @command{wget
+--post-file}, @command{feroxbuster --data @@file}); environment
+assignments in front of a network tool (@code{env HTTPS_PROXY=@dots{}}).  The profile itself refuses
+listeners (@command{nc -l}, @command{ncat --listen}), which catch
+shells.
 @end table
 
 @subheading Your own profiles
@@ -507,6 +551,7 @@ prompt = "Never modify files under tests/."
 encrypt = true                # turn encryption at rest on
 integrity = true              # refuse to start without a pinned manifest
 isolation = "observer"        # steps see the host's processes (read-only)
+engagement = true             # commands only inside an open engagement
 @end example
 
 @noindent
@@ -523,6 +568,11 @@ or level that does not exist, a missing policy directory --- is refused
 with a message rather than half applied.  A project cannot define or
 choose a profile, since a profile can lower the privacy level.  @code{isolation = "observer"} runs command steps with the host's
 processes in view, for live triage (@pxref{Isolated Steps}).
+@code{engagement = true} refuses every command and web search step
+unless an engagement is open, so that the security layer's scope check
+reads the hosts each step reaches; reading and writing files still
+works.  It is for profiles whose tools reach other hosts, such as
+@code{security-pentest}.
 
 
 @node Isolated Steps

docs/chapters/development.texi

@@ -163,11 +163,15 @@ tests/
 │                                egress rule, bad entries, sources
 ├── test_security_ir.py          security-ir: settings, what triage may run,
 │                                what would change the host is refused
+├── test_security_pentest.py     security-pentest: nothing without an
+│                                engagement; recon in scope runs, beyond
+│                                scanning is refused
 ├── test_shipped_profiles.py     profiles of codebud_sec: found by name
 │                                beside codebud, not imported, not shadowed
 ├── test_scope.py                engagement scope: targets read per tool,
 │                                third parties, ranges, active refused,
-│                                passive asked
+│                                passive asked; nmap scripts judged by
+│                                nmap's script.db
 ├── test_scope_web.py            engagement scope for web and TLS tools:
 │                                URL hosts read strictly, proxies, redirects
 │                                and fuzz words in the host refused

docs/chapters/safety-and-sandbox.texi

@@ -226,7 +226,11 @@ registers a function that may refuse a step, and
 engine asks them after its own rules, for every step and again right
 before it runs, without knowing what they are; a check that fails with
 an error refuses the step.  The security layer's engagement scope guard
-is registered this way, so the core never imports that layer.
+is registered this way, so the core never imports that layer.  A check
+can say what it @emph{provides} (@code{add_step_check(check,
+provides="engagement")}), and a profile with @code{engagement = true}
+refuses command and web search steps while no registered check provides
+an engagement (@pxref{Profiles}).
 
 @code{describe_rules()} returns a compact summary of the three policy
 lists, which is embedded in the planner prompt so the model can avoid

src/Makefile.am

@@ -75,7 +75,12 @@ nobase_dist_python_DATA = \
 	codebud_sec/data/profiles/security-ir/harmless_commands.json \
 	codebud_sec/data/profiles/security-ir/powerful_commands.json \
 	codebud_sec/data/profiles/security-ir/refused_arguments.json \
-	codebud_sec/data/profiles/security-ir/trusted_sequences.json
+	codebud_sec/data/profiles/security-ir/trusted_sequences.json \
+	codebud_sec/data/profiles/security-pentest.toml \
+	codebud_sec/data/profiles/security-pentest/harmless_commands.json \
+	codebud_sec/data/profiles/security-pentest/powerful_commands.json \
+	codebud_sec/data/profiles/security-pentest/refused_arguments.json \
+	codebud_sec/data/profiles/security-pentest/trusted_sequences.json
 
 # A minimal dist-info directory, so importlib.metadata (and therefore
 # `codebud version`) knows the installed version without pip.

src/codebud/profiles.py

@@ -18,6 +18,9 @@
             network namespaces -- for live triage (isolation.py)
 - integrity true refuses to start without a pinned integrity manifest
             (integrity.py); with one, it is checked whatever the profile
+- engagement true refuses command and web search steps unless a step
+            check providing "engagement" is registered (safety.py) -- the
+            security layer's scope guard, when an engagement is open
 
 Built in: "default" (ask, private) and "unattended" (full-auto, trusted).
 The security layer, codebud_sec, ships more in its data/profiles/: they
@@ -52,7 +55,7 @@
 _NAME = re.compile(r"^[a-z0-9][a-z0-9_-]{0,63}$")
 _KEYS = ("description", "mode", "privacy", "policy", "prompt", "isolation")
 ISOLATIONS = ("default", "observer")
-_BOOL_KEYS = ("encrypt", "integrity")
+_BOOL_KEYS = ("encrypt", "integrity", "engagement")
 
 
 class ProfileError(Exception):
@@ -71,10 +74,12 @@ class Profile:
     encrypt: bool = False  # turn encryption at rest on
     integrity: bool = False  # refuse to start without an integrity manifest
     isolation: str = "default"  # or "observer" (isolation.py)
+    engagement: bool = False  # steps need an open engagement (safety.py)
 
     def summary(self) -> str:
         extras = (", encrypted" if self.encrypt else "") + (", pinned" if self.integrity else "")
         extras += ", observer" if self.isolation == "observer" else ""
+        extras += ", engagement" if self.engagement else ""
         return f"{self.mode}, privacy {self.privacy}" + extras
 
 
@@ -164,6 +169,7 @@ def _load_file(name: str, path: Path) -> Profile:
         data.get("encrypt", False),
         data.get("integrity", False),
         kind,
+        data.get("engagement", False),
     )
 
 

src/codebud/safety.py

@@ -478,15 +478,43 @@ def egress_reason(step: dict[str, Any]) -> str | None:
 # a notice returns why it must be confirmed (or None).
 _STEP_CHECKS: list = []
 _STEP_NOTICES: list = []
+# What a registered check stands for ("engagement": the scope guard of an
+# open engagement), so that a profile can require it (profiles.py).
+_PROVIDES: dict = {}
 
 
-def add_step_check(check) -> None:
+def add_step_check(check, provides: str | None = None) -> None:
     _STEP_CHECKS.append(check)
+    if provides:
+        _PROVIDES[check] = provides
 
 
 def remove_step_check(check) -> None:
     if check in _STEP_CHECKS:
         _STEP_CHECKS.remove(check)
+    _PROVIDES.pop(check, None)
+
+
+def provided(what: str) -> bool:
+    """Whether a registered step check provides ``what``."""
+    return any(_PROVIDES.get(check) == what for check in _STEP_CHECKS)
+
+
+def _engagement_refusal(step: dict[str, Any]) -> str | None:
+    """A profile with ``engagement = true`` runs no command or web search
+    unless an engagement's scope check is registered: its tools reach
+    other hosts, and only that check reads which."""
+    from . import profiles
+
+    profile = profiles.active()
+    if not profile.engagement or step.get("tool") not in ("command", "web_search"):
+        return None
+    if provided("engagement"):
+        return None
+    return (
+        f"profile {profile.name} works only inside an open engagement, whose scope "
+        "check reads every target; open the engagement first"
+    )
 
 
 def add_step_notice(notice) -> None:
@@ -555,6 +583,9 @@ def refusal_reason(self, step: dict[str, Any]) -> str | None:
         tool = step.get("tool")
         if not step_allowed(self.mode, step):
             return f"{tool} is not allowed in {self.mode} mode"
+        engagement = _engagement_refusal(step)
+        if engagement:
+            return engagement
         if tool == "command" and step.get("args", {}).get("network"):
             if privacy.level() == "locked":
                 return "steps that need the network are not allowed at privacy level locked"

src/codebud_sec/data/profiles/security-pentest.toml

@@ -0,0 +1,30 @@
+# security-pentest: reconnaissance and scanning (tiers T0 and T1 of the
+# step 5 plan) of hosts an engagement authorizes. Every command runs only
+# while an engagement is open, and its scope guard reads every target a
+# network step names: out of scope, excluded or outside the window is
+# refused. Exploitation (T2) is not part of this profile. See "Security
+# profiles" in the manual and, for the reasoning, NIST SP 800-115 and
+# the PTES rules of engagement.
+description = "Reconnaissance and scanning inside an engagement's scope; no exploitation"
+mode = "ask"
+privacy = "private"
+encrypt = true
+integrity = true
+engagement = true
+policy = "security-pentest"
+prompt = """
+This is an authorized penetration test. Every target must be in the
+engagement's scope; a step naming any other host is refused, so never guess
+or widen targets. Declare the network ("network": true) on every step that
+contacts a host.
+Reconnaissance and scanning only: no exploitation, no password guessing, no
+denial of service. For nmap use -sV and named scripts, or --script
+"default and not external" (plain -sC is refused). masscan at --rate 1000
+or less. Never follow redirects: curl without -L, wget --max-redirect=0,
+whatweb --follow-redirect=never. Fuzz only the path (https://host/FUZZ).
+nikto always with -Tuning x6 (its denial-of-service tests left out).
+Never send files to a target.
+Write every output into the project (for example scans/ and notes/) with
+the tool's own output option (-oA, -o, --jsonfile) so the work can be
+reported, and hash what you keep with sha256sum.
+"""

src/codebud_sec/data/profiles/security-pentest/harmless_commands.json

@@ -0,0 +1,5 @@
+[
+  "cat", "head", "tail", "ls", "stat", "file", "wc", "sort", "uniq", "cut",
+  "diff", "cmp", "tr", "grep", "rg", "jq", "xxd", "strings", "base64",
+  "sha256sum", "sha1sum", "md5sum", "find", "pwd", "echo", "date"
+]

src/codebud_sec/data/profiles/security-pentest/powerful_commands.json

@@ -0,0 +1,5 @@
+[
+  "dig", "host", "whois", "nmap", "masscan", "nc", "ncat", "curl", "wget",
+  "whatweb", "nikto", "gobuster", "ffuf", "feroxbuster", "sslscan",
+  "testssl.sh", "testssl", "timeout", "mkdir"
+]

src/codebud_sec/data/profiles/security-pentest/refused_arguments.json

@@ -0,0 +1,23 @@
+{
+  "nc": [
+    "-l"
+  ],
+  "ncat": [
+    "-l",
+    "--listen",
+    "--broker",
+    "--chat"
+  ],
+  "sort": [
+    "-o",
+    "--output"
+  ],
+  "rg": [
+    "--pre",
+    "--pre-glob"
+  ],
+  "date": [
+    "-s",
+    "--set"
+  ]
+}

src/codebud_sec/data/profiles/security-pentest/trusted_sequences.json

@@ -0,0 +1 @@
+[]