feat: the scope fence, pasta + nftables per step (step 5, PR 6a)

docs/chapters/architecture.texi

@@ -569,7 +569,8 @@ src/codebud/                   the package
 ├── executor.py        Executor (step-by-step runner)
 ├── privacy.py         privacy level: locked / private / trusted
 ├── profiles.py        named profiles: mode + privacy (+ policy, prompt)
-├── isolation.py       bubblewrap / unshare wrapping for command steps
+├── isolation.py       bubblewrap / unshare wrapping for command steps;
+│                      the network fence (pasta + nftables)
 ├── confine.py         where file tools may read and write
 ├── redact.py          mask secrets before sessions/history are saved
 ├── safety.py          SafetyEngine (policy validation)
@@ -641,6 +642,8 @@ src/codebud_sec/               the security layer (authorized security work)
 │                      command line and checked (DNS tools, whois,
 │                      scanners, nc, web clients, fuzzers, TLS scanners);
 │                      registered with the core's safety.add_step_check()
+├── fence.py           network fence policy: which addresses a step
+│                      may reach (its targets, names resolved per step)
 └── evidence.py        hash-chained, append-only evidence log,
                        encrypted at rest (appends need no passphrase)
 

docs/chapters/configuration.texi

@@ -502,7 +502,15 @@ proxies, DNS servers, addresses given to @command{curl --resolve} ---
 and refuses the step when one is out of scope, excluded, or the
 engagement window is closed.  A command it cannot read (an unknown
 option, a target list outside the project, an ambiguous URL) is refused
-too.
+too.  Scanning steps also run behind the network fence (@pxref{Isolated
+Steps}), which lets through only the addresses the step names --- its
+names resolved just before it runs --- and never an excluded one, so a
+tool that reaches further than its command line says meets a closed
+door.  Behind the fence nmap needs @option{--unprivileged} (connect
+scans), and SYN, UDP and OS scans and @command{masscan} are refused; an
+engagement file with @code{fence_scanning = false} gives them back, with
+the scope guard as their only check.  Lookups (@command{dig},
+@command{host}, @command{whois}) are not fenced.
 
 Its allow-list: reading results runs once the plan is approved
 (@command{cat}, @command{grep}, @command{rg}, @command{jq},
@@ -697,6 +705,41 @@ Install bubblewrap with your package manager (@command{pacman -S
 bubblewrap}, @command{apt install bubblewrap}, @command{dnf install
 bubblewrap}).
 
+@subheading The network fence
+@cindex network fence
+@cindex pasta
+@cindex nftables
+
+A step that declares the network normally shares the machine's network.
+An optional layer --- the security layer, while an engagement is open
+--- can instead have a network step @dfn{fenced}: it then runs in a
+network namespace of its own whose only way out is @command{pasta}
+(from the passt package), and @command{nft} rules in that namespace let
+through only the addresses the layer allows for that step, refusing
+excluded ranges first.
+
+@itemize
+@item Nothing of the machine itself is reachable: not its loopback, not
+through the gateway address, and no port is forwarded in either
+direction (pasta runs with @option{--no-map-gw -t none -u none -T none -U
+none}; its defaults would reach the machine's own services).
+@item The step runs without capabilities (@code{bwrap --cap-drop ALL}),
+in a user namespace of its own, so it cannot change the rules --- and
+has no raw sockets, whose packets would pass them by.  Scans that need
+raw sockets (nmap's SYN, UDP and OS scans, masscan) cannot run fenced;
+nmap needs @option{--unprivileged}.
+@item There is no DNS: the names the step uses are resolved before it
+runs and written into an @file{/etc/hosts} of its own.  Lookups through
+a resolver would carry data to whoever serves a name.
+@end itemize
+
+A step that must be fenced is refused when the fence cannot be built:
+without bubblewrap, @command{pasta} or @command{nft}, or at privacy
+@code{trusted}.  @command{codebud integrity pin} pins @command{pasta},
+@command{nft} and @command{sh} too when the security layer is installed.
+Install them with @command{pacman -S passt nftables}, @command{apt
+install passt nftables} or @command{dnf install passt nftables}.
+
 
 @node Feature Flags
 @section Feature Flags

docs/chapters/development.texi

@@ -72,6 +72,8 @@ tests/
 ├── test_evidence.py             evidence log: hash chain, tamper
 │                                detection, artefact hashes, encrypted
 │                                entries (append while locked)
+├── test_fence.py                the network fence: rules, pasta options,
+│                                no DNS, refusals (live where available)
 ├── test_forensic_rules.py       dc3dd/ewfacquire on images in the project,
 │                                hashed and logged; exiftool only reads
 ├── test_fsutil.py               atomic_write_text() / _bytes()
@@ -172,6 +174,9 @@ tests/
 │                                third parties, ranges, active refused,
 │                                passive asked; nmap scripts judged by
 │                                nmap's script.db
+├── test_scope_fence.py          engagement fence: a scan reaches exactly
+│                                its targets, names resolved per step, raw
+│                                scans refused, fence_scanning = false
 ├── test_scope_web.py            engagement scope for web and TLS tools:
 │                                URL hosts read strictly, proxies, redirects
 │                                and fuzz words in the host refused

src/Makefile.am

@@ -63,6 +63,7 @@ nobase_python_PYTHON = \
 	codebud_sec/__init__.py \
 	codebud_sec/engagement.py \
 	codebud_sec/evidence.py \
+	codebud_sec/fence.py \
 	codebud_sec/scope.py
 
 nobase_dist_python_DATA = \

src/codebud/cli/integrity.py

@@ -76,13 +76,15 @@ def _shipped_programs() -> set[str]:
 
 
 def _programs(extra) -> set[str]:
-    from codebud import isolation, policy
+    from codebud import isolation, policy, profiles
 
     programs = integrity.allow_listed_programs(policy.effective()) | set(extra)
     programs |= _shipped_programs()
     tool = isolation.available()
     if tool:
         programs.add(tool)  # it runs every isolated step
+    if profiles.shipped_profile_dir() is not None and isolation.fence_available():
+        programs |= set(isolation.FENCE_PROGRAMS)  # the security layer's network fence
     return programs
 
 

src/codebud/isolation.py

@@ -21,12 +21,33 @@
 but nothing else. Where neither works (macOS, BSD, containers, blocked
 user namespaces), steps are not isolated: privacy level "private" warns
 once and runs them, "locked" refuses them. "trusted" never isolates.
+
+A network step can be fenced (phase 10, step 5 plan, PR 6): an optional
+layer registers a function (add_network_fence) that says which addresses
+the step may reach, and the step then runs in a network namespace of its
+own whose only way out is pasta (passt), with nftables letting through
+only those addresses:
+
+- pasta --no-map-gw -t none -u none -T none -U none: nothing of the host
+  is reachable -- not its loopback, not through the gateway address, no
+  port forwarded either way (pasta's defaults reach the host's loopback);
+- nft loads the rules in that namespace (pasta's user namespace gives
+  CAP_NET_ADMIN there) before bwrap starts the step with --cap-drop ALL in
+  a user namespace of its own, so the step cannot change them -- and has
+  no raw sockets, whose packets would pass by the rules;
+- no DNS: the names the step uses are resolved before, into an /etc/hosts
+  of its own, and resolv.conf is empty -- DNS through the host's resolver
+  would carry data to whoever serves a name.
 """
 
+import atexit
+import ipaddress
 import logging
 import os
 import shutil
 import subprocess
+import tempfile
+from dataclasses import dataclass
 from functools import lru_cache
 from pathlib import Path
 
@@ -138,12 +159,20 @@ def _contains(directory: Path, path: Path) -> bool:
 
 
 def wrap(
-    argv: list[str], project: Path, network: bool, kind: str, observer: bool = False
+    argv: list[str],
+    project: Path,
+    network: bool,
+    kind: str,
+    observer: bool = False,
+    files: tuple[tuple[str, str], ...] = (),
+    no_capabilities: bool = False,
 ) -> list[str]:
     """``argv`` wrapped for ``kind`` of isolation. ``observer`` (bwrap only):
     the host's /proc instead of a fresh one, so the step sees the host's
     processes and, through /proc/1/net, its sockets -- while its own process
-    namespace keeps it from signalling them and it still has no network."""
+    namespace keeps it from signalling them and it still has no network.
+    ``files`` (bwrap only): (host file, path in the step) bound read-only
+    over the step's view; ``no_capabilities``: --cap-drop ALL."""
     if kind == "unshare":
         if network:
             return argv  # unshare only cuts the network
@@ -153,6 +182,8 @@ def wrap(
     wrapped = ["bwrap", "--die-with-parent", "--new-session", "--unshare-all"]
     if network:
         wrapped.append("--share-net")
+    if no_capabilities:
+        wrapped += ["--cap-drop", "ALL"]
     proc = ["--ro-bind", "/proc", "/proc"] if observer else ["--proc", "/proc"]
     wrapped += ["--ro-bind", "/", "/", "--dev", "/dev", *proc, "--tmpfs", "/tmp"]
     # After the /tmp tmpfs, so a project under /tmp is still there.
@@ -163,49 +194,187 @@ def wrap(
     for name in HIDDEN_FILES:
         if (home / name).is_file():
             wrapped += ["--ro-bind", "/dev/null", str(home / name)]
+    for source, target in files:
+        wrapped += ["--ro-bind", source, target]
     wrapped += ["--chdir", str(cwd), "--", *argv]
     return wrapped
 
 
+# --- the network fence ---------------------------------------------------------------
+
+FENCE_PROGRAMS = ("pasta", "nft", "sh")
+_PASTA_OPTIONS = ["--config-net", "--quiet", "--no-map-gw", "-t", "none", "-u", "none",
+                  "-T", "none", "-U", "none"]  # fmt: skip
+# Run inside pasta's namespace: load the rules, then the step.
+_FENCE_SCRIPT = 'printf %s "$1" | "$2" -f - || exit 97; shift 2; exec "$@"'
+
+
+@dataclass(frozen=True)
+class Fence:
+    """What a fenced step may reach: ``allow`` networks, never ``deny``
+    networks (they win), and ``hosts``: (name, address) for its
+    /etc/hosts."""
+
+    allow: tuple = ()
+    deny: tuple = ()
+    hosts: tuple[tuple[str, str], ...] = ()
+
+
+def nft_ruleset(fence: Fence) -> str:
+    """The nftables rules for ``fence``: everything out is dropped but the
+    namespace's loopback, replies, and the allowed networks; denied
+    networks are dropped first, so they win over an allowed one around
+    them."""
+
+    def rule(network, verdict):
+        network = ipaddress.ip_network(network, strict=False)
+        family = "ip" if network.version == 4 else "ip6"
+        return f"    {family} daddr {network} {verdict}"
+
+    lines = [
+        "table inet codebud_fence {",
+        "  chain out {",
+        "    type filter hook output priority 0; policy drop;",
+        '    oif "lo" accept',
+        "    ct state established,related accept",
+        *(rule(n, "drop") for n in fence.deny),
+        *(rule(n, "accept") for n in fence.allow),
+        "  }",
+        "}",
+    ]
+    return "\n".join(lines) + "\n"
+
+
+def fence_available() -> bool:
+    return all(shutil.which(program) for program in FENCE_PROGRAMS)
+
+
+_FENCES: list = []
+_fence_dir: Path | None = None
+
+
+def add_network_fence(fence) -> None:
+    """Register ``fence(words)``: a Fence for a network step that must be
+    fenced, None for one that need not be, or a str: why it must not run."""
+    _FENCES.append(fence)
+
+
+def remove_network_fence(fence) -> None:
+    if fence in _FENCES:
+        _FENCES.remove(fence)
+
+
+def _fence_for(words: list[str]):
+    for hook in list(_FENCES):
+        try:
+            decision = hook(words)
+        except Exception as exc:  # noqa: BLE001 - a broken fence never lets a step out
+            return f"the network fence failed ({type(exc).__name__}: {exc})"
+        if decision is not None:
+            return decision
+    return None
+
+
+def _fence_files(fence: Fence) -> tuple[tuple[str, str], ...]:
+    """A hosts file and an empty resolv.conf for the step, in a private
+    directory removed when codebud exits."""
+    global _fence_dir
+    if _fence_dir is None:
+        _fence_dir = Path(tempfile.mkdtemp(prefix="codebud-fence-"))
+        atexit.register(shutil.rmtree, _fence_dir, True)
+    handle, hosts = tempfile.mkstemp(dir=_fence_dir, prefix="hosts-")
+    with os.fdopen(handle, "w") as f:
+        f.write("127.0.0.1 localhost\n::1 localhost\n")
+        f.writelines(f"{address} {name}\n" for name, address in fence.hosts)
+    resolv = _fence_dir / "resolv.conf"
+    resolv.write_text("# no DNS in a fenced step\n")
+    return ((hosts, "/etc/hosts"), (str(resolv), "/etc/resolv.conf"))
+
+
+def fenced(inner: list[str], fence: Fence) -> list[str]:
+    """``inner`` (a bwrap command line) run behind ``fence``."""
+    return ["pasta", *_PASTA_OPTIONS, "--", "sh", "-c", _FENCE_SCRIPT, "codebud-fence",
+            nft_ruleset(fence), "nft", *inner]  # fmt: skip
+
+
 def prepare(argv: list[str], network: bool = False) -> tuple[list[str] | None, str | None]:
     """(argv to run, None) -- or (None, why the step must not run).
 
     Follows the privacy level: trusted runs ``argv`` as it is; private and
     locked isolate it where possible; without isolation private warns once
-    and runs it, locked refuses. Network steps are refused when locked.
+    and runs it, locked refuses. Network steps are refused when locked, and
+    fenced when a registered fence says so (which needs bwrap and pasta).
     """
+    argv, refusal, _ = prepare_with_programs(argv, network)
+    return argv, refusal
+
+
+def prepare_with_programs(
+    argv: list[str], network: bool = False
+) -> tuple[list[str] | None, str | None, list[int]]:
+    """prepare(), and the positions in the result of the programs it added
+    (the isolation tool, the fence's), for the integrity check."""
     global _warned
     from codebud import privacy, profiles
     from codebud.rag.indexer import project_root
 
     level = privacy.level()
+    fence = _fence_for(argv) if network else None
+    if isinstance(fence, str):
+        return None, f"Refused: {fence}", []
+    if fence is not None:
+        if level == "trusted" or available() != "bwrap" or not fence_available():
+            return (
+                None,
+                (
+                    "this step must run behind the network fence, which needs bubblewrap, "
+                    "pasta (passt) and nft, at privacy private or locked; not running it."
+                ),
+                [],
+            )
+        if level == "locked":
+            return None, privacy.locked_refusal("a step that needs the network"), []
+        inner = wrap(argv, project_root(), True, "bwrap", files=_fence_files(fence),
+                     no_capabilities=True)  # fmt: skip
+        outer = fenced(inner, fence)
+        start = len(outer) - len(inner)
+        sh, nft = outer.index("sh"), start - 1
+        return outer, None, [0, sh, nft, start]
     if level == "trusted":
-        return argv, None
+        return argv, None, []
     if network and level == "locked":
-        return None, privacy.locked_refusal("a step that needs the network")
+        return None, privacy.locked_refusal("a step that needs the network"), []
     kind = available()
     # Observer isolation is bubblewrap's or nothing: anything less would
     # share the host's process namespace and could signal its processes.
     observer = profiles.active().isolation == "observer"
     if observer and kind != "bwrap":
-        return None, (
-            "this profile's observer isolation needs bubblewrap: without it a step "
-            "would share the host's process namespace and could signal (kill) its "
-            "processes; not running it."
+        return (
+            None,
+            (
+                "this profile's observer isolation needs bubblewrap: without it a step "
+                "would share the host's process namespace and could signal (kill) its "
+                "processes; not running it."
+            ),
+            [],
         )
     if kind is None:
         if level == "locked":
-            return None, (
-                "privacy is locked and steps cannot be isolated here (install bubblewrap, "
-                "or allow user namespaces); not running it."
+            return (
+                None,
+                (
+                    "privacy is locked and steps cannot be isolated here (install bubblewrap, "
+                    "or allow user namespaces); not running it."
+                ),
+                [],
             )
         if not _warned:
             _warned = True
             logger.warning(
                 "Steps are not isolated: bubblewrap / user namespaces are unavailable. "
                 "Install bubblewrap for network-free, project-only steps."
             )
-        return argv, None
+        return argv, None, []
     if kind == "unshare" and network:
         # unshare can only cut the network; a step that needs it gets no
         # isolation at all here, so say so, as above (security review, low).
@@ -217,4 +386,4 @@ def prepare(argv: list[str], network: bool = False) -> tuple[list[str] | None, s
                 "unshare is available, and it can only cut the network. Install "
                 "bubblewrap for project-only writes and hidden secrets."
             )
-    return wrap(argv, project_root(), network, kind, observer=observer), None
+    return wrap(argv, project_root(), network, kind, observer=observer), None, [0]