Imaging and metadata tools for security-ir, held by forensic argument rules

NEWS

@@ -72,6 +72,8 @@ a safety policy you can read and change.
   security-ir, the first security profile (in codebud_sec): incident
   response on hosts you own -- locked, observer isolation, encrypted,
   integrity required, a read-only triage and forensic allow-list.
+  Imaging tools work on image files in the project only, hashed and
+  logged (dc3dd, ewfacquire), and exiftool may only read.
 
   A profile can ask for observer isolation (isolation = "observer"):
   command steps see the host's processes through a read-only /proc,

docs/chapters/architecture.texi

@@ -573,6 +573,7 @@ src/codebud/                   the package
 ├── confine.py         where file tools may read and write
 ├── redact.py          mask secrets before sessions/history are saved
 ├── safety.py          SafetyEngine (policy validation)
+├── forensic.py        argument rules for dc3dd, ewfacquire, exiftool
 ├── modes.py           agent modes (ask, auto, full-auto, plan, edit)
 ├── sandbox.py         Sandbox (subprocess + timeout)
 ├── memory.py          MemoryStore (timeline persistence)

docs/chapters/configuration.texi

@@ -457,13 +457,16 @@ text and binary viewers (@command{cat}, @command{grep}, @command{rg},
 (@command{ps}, @command{pstree}, @command{lsof}, @command{who},
 @command{w}, @command{last}, @command{lastb}), @command{journalctl},
 @command{ausearch}, @command{aureport}, reading a capture
-(@command{tcpdump -r}, @command{tshark -r}), @command{find},
-@command{lsblk}, @command{date}.  Collecting copies and the forensic
+(@command{tcpdump -r}, @command{tshark -r}), reading images and
+metadata (@command{ewfverify}, @command{ewfinfo}, @command{exiftool}),
+@command{find}, @command{lsblk}, @command{date}.  Collecting copies and the forensic
 tools are powerful and asked for every time: @command{cp},
 @command{tar}, @command{yara}, Volatility (@command{vol},
 @command{vol3}), the Sleuth Kit (@command{fls}, @command{icat},
 @command{mmls}, @command{istat}, @command{fsstat}) and plaso
-(@command{log2timeline.py}, @command{psort.py}, @command{pinfo.py}).
+(@command{log2timeline.py}, @command{psort.py}, @command{pinfo.py}),
+and imaging image files that are in the project (@command{dc3dd},
+@command{ewfacquire}), hashed and logged (@pxref{The Safety Engine}).
 Nothing that deletes, moves, kills, changes permissions or installs is
 on it, and no trusted sequence.
 
@@ -483,7 +486,8 @@ isolation they would show the sandbox --- a falsely clean answer; use
 system bus, which steps cannot reach.  Disk and memory acquisition is
 done by you, outside Codebud, with a write-blocker where it applies
 (NIST SP 800-86 advises against imaging a whole device from a live
-system); Codebud analyses the images.
+system); Codebud analyses the images, and re-images or converts them
+(raw to E01) only as files inside the project.
 @end table
 
 @subheading Your own profiles

docs/chapters/development.texi

@@ -72,6 +72,8 @@ tests/
 ├── test_evidence.py             evidence log: hash chain, tamper
 │                                detection, artefact hashes, encrypted
 │                                entries (append while locked)
+├── test_forensic_rules.py       dc3dd/ewfacquire on images in the project,
+│                                hashed and logged; exiftool only reads
 ├── test_fsutil.py               atomic_write_text() / _bytes()
 ├── test_gitsafe.py              git runs none of the repository's configured
 │                                programs (fsmonitor, filters, textconv)

docs/chapters/safety-and-sandbox.texi

@@ -142,6 +142,52 @@ allowed.  A program with such rules run by @command{xargs} or
 file or from input, which cannot be checked (@code{xargs -a files.txt
 wc -l} is fine).
 
+@cindex imaging tools, rules
+@cindex dc3dd
+@cindex ewfacquire
+@cindex exiftool
+Three tools get rules a list of options cannot express.  They apply
+wherever a profile allow-lists the tool (@code{security-ir} does) and
+keep it to working on evidence as NIST SP 800-86 (section 4.2.2) and the
+ACPO principles ask: on copies, hashed, logged, never writing to a
+device or outside the project.
+
+@table @command
+@item dc3dd
+Its input (@option{if=}, @option{ifs=}) must be an image file inside the
+project, never a device; every output and log (@option{of=},
+@option{hof=}, @option{ofs=}, @option{hofs=}, @option{log=},
+@option{hlog=}, @option{mlog=}) inside the project; @option{hash=} and
+@option{log=} or @option{hlog=} required.  @option{wipe=},
+@option{hwipe=}, @option{pat=} and @option{tpat=} are refused, and so is
+any key not known to be safe.
+@item ewfacquire
+Unattended (@option{-u}), since nobody can answer its prompts in a step,
+and so with the case metadata on the command line: @option{-C} (case),
+@option{-e} (examiner), @option{-E} (evidence number), @option{-D}
+(description); @option{-d} including @code{sha256}; a log
+(@option{-l}); the target (@option{-t}) and any second target
+(@option{-2}) inside the project; the source one image file inside the
+project.  Options not known to be safe are refused.
+@item exiftool
+Only reading.  Refused: an argument with @samp{=}, @samp{<} or @samp{>}
+(writing, copying or renaming tags), one with @samp{$@{} (Perl code),
+and the options that write, run code or read arguments from a file
+(@option{-o}, @option{-overwrite_original}, @option{-tagsFromFile},
+@option{-geotag}, @option{-p}, @option{-if}, @option{-config},
+@option{-api}, @option{-@@}, @option{-w}, @option{-execute}, @dots{}).
+Read options and tag names (@code{exiftool -a -G1 -s},
+@code{exiftool -Artist photo.jpg}) are fine.
+@end table
+
+@noindent
+Every output, target and log must be a new file: one that already
+exists --- the input image, another piece of evidence, an earlier
+@file{.E01} --- is refused, so an imaging step can never overwrite
+evidence.  The paths are judged with symbolic links followed, so a link
+inside the project that points out of it does not count as inside.  These rules
+were written from the tools' manual pages.
+
 @cindex refused_arguments.json
 You can refuse more with @file{refused_arguments.json}, which maps a
 program to the options it may never be given:

src/Makefile.am

@@ -26,6 +26,7 @@ nobase_python_PYTHON = \
 	codebud/crypto.py \
 	codebud/encrypt_existing.py \
 	codebud/executor.py \
+	codebud/forensic.py \
 	codebud/fsutil.py \
 	codebud/gitsafe.py \
 	codebud/integrity.py \

src/codebud/forensic.py

@@ -0,0 +1,208 @@
+"""
+codebud/forensic.py
+
+Argument rules for imaging and metadata tools that a word list cannot
+express (phase 10, step 5 plan, PR 3b). They apply wherever a profile
+allow-lists the tool -- security-ir does -- and keep it to working on
+evidence the way NIST SP 800-86 (section 4.2.2) and ACPO principle 3
+ask: on copies, hashed, logged, never writing to a device or outside the
+project.
+
+- dc3dd: input an image file inside the project, never a device; every
+  output and log inside the project; hash= and a log required; wipe and
+  pattern writes refused; unknown keys refused.
+- ewfacquire: unattended (-u: in a step nobody can answer its prompts)
+  and so the case metadata given on the command line (-C -e -E -D),
+  -d with sha256, a log (-l), the target (-t) and any second target
+  inside the project, the source an image file inside the project;
+  options not known to be safe refused.
+- exiftool: only reading. Its format strings (${...}), -if conditions and
+  -config files run Perl code, and tag values are written with =, < and
+  >: those, and the options that write, run or read arguments from a
+  file, are refused. Read options and tag names (-Artist) are fine.
+
+Rules were written from the tools' manual pages (dc3dd(1),
+ewfacquire(1), exiftool(1)).
+"""
+
+import os
+import re
+
+# --- paths -------------------------------------------------------------------
+
+
+def _inside_project(path: str) -> bool:
+    """Whether ``path`` (symlinks followed) is inside the current project
+    directory and is not a device."""
+    real = os.path.realpath(path)
+    if real.startswith("/dev/") or real == "/dev":
+        return False
+    project = os.path.realpath(os.getcwd())
+    return os.path.commonpath([real, project]) == project
+
+
+def _new_output(path: str) -> str | None:
+    """Why ``path`` may not be written, or None: it must be inside the
+    project, not a device, and not exist yet -- so an output can never
+    overwrite the input or another piece of evidence."""
+    if not _inside_project(path):
+        return "must be inside the project, not a device"
+    if os.path.lexists(path):
+        return "exists already: outputs and logs must be new files"
+    return None
+
+
+def _image_file(path: str) -> bool:
+    """An existing regular file inside the project (an image to work on)."""
+    return _inside_project(path) and os.path.isfile(path)
+
+
+# --- dc3dd -------------------------------------------------------------------
+
+_DC3DD_INPUT = {"if", "ifs"}
+_DC3DD_OUTPUT = {"of", "hof", "ofs", "hofs", "log", "hlog", "mlog"}
+_DC3DD_REFUSED = {
+    "wipe": "wipes",
+    "hwipe": "wipes",
+    "pat": "writes a pattern",
+    "tpat": "writes a pattern",
+}
+_DC3DD_VALUES = {
+    "hash": re.compile(r"^(md5|sha1|sha256|sha512)(,(md5|sha1|sha256|sha512))*$"),
+    "cnt": re.compile(r"^\d+$"),
+    "iskip": re.compile(r"^\d+$"),
+    "oskip": re.compile(r"^\d+$"),
+    "bufsz": re.compile(r"^\d+[KMG]?$"),
+    "ssz": re.compile(r"^\d+$"),
+    "verb": re.compile(r"^on$"),
+    "nwspc": re.compile(r"^on$"),
+    "rec": re.compile(r"^off$"),
+}
+
+
+def _dc3dd(args: list[str]) -> str | None:
+    keys = {}
+    for arg in args:
+        key, sep, value = arg.partition("=")
+        if not sep:
+            return f"{arg}: dc3dd takes key=value arguments only"
+        if key in _DC3DD_REFUSED:
+            return f"{key}= {_DC3DD_REFUSED[key]}"
+        if key in _DC3DD_INPUT:
+            if not _image_file(value):
+                return (
+                    f"{key}={value}: input must be an image file inside the project, not a device"
+                )
+        elif key in _DC3DD_OUTPUT:
+            problem = _new_output(value)
+            if problem:
+                return f"{key}={value}: {problem}"
+        elif key in _DC3DD_VALUES:
+            if not _DC3DD_VALUES[key].match(value):
+                return f"{key}={value}: not an accepted value"
+        else:
+            return f"{key}=: not a known safe dc3dd option"
+        keys[key] = value
+    if not keys.keys() & _DC3DD_INPUT:
+        return "dc3dd needs an input image (if=)"
+    if "hash" not in keys:
+        return "dc3dd must hash what it copies (hash=sha256)"
+    if not keys.keys() & {"log", "hlog"}:
+        return "dc3dd must log what it does (log= or hlog=)"
+    return None
+
+
+# --- ewfacquire --------------------------------------------------------------
+
+# Options that take a value; paths among them must stay inside the project.
+_EWF_VALUE = {"-A", "-b", "-B", "-c", "-C", "-d", "-D", "-e", "-E", "-f", "-l", "-m",
+              "-M", "-N", "-o", "-p", "-P", "-r", "-S", "-t", "-T", "-2"}  # fmt: skip
+_EWF_PATHS = {"-l", "-t", "-2"}
+_EWF_SEGMENTS = (".E01", ".Ex01", ".e01", ".s01", ".L01", ".Lx01")
+_EWF_FLAGS = {"-u", "-q", "-v", "-R", "-s", "-x", "-h", "-V"}
+_EWF_REQUIRED = {
+    "-u": "unattended (-u): in a step nobody can answer its prompts",
+    "-C": "the case number (-C)",
+    "-e": "the examiner (-e)",
+    "-E": "the evidence number (-E)",
+    "-D": "a description (-D)",
+    "-l": "a log file (-l)",
+    "-t": "the target (-t)",
+}
+
+
+def _ewfacquire(args: list[str]) -> str | None:
+    given, sources = {}, []
+    i = 0
+    while i < len(args):
+        arg = args[i]
+        if arg in _EWF_VALUE:
+            if i + 1 >= len(args):
+                return f"{arg} needs a value"
+            value = args[i + 1]
+            if arg in _EWF_PATHS:
+                problem = _new_output(value)
+                if problem:
+                    return f"{arg} {value}: {problem}"
+                if arg != "-l":
+                    # A target is a base name; its segment files must be new too.
+                    existing = [value + e for e in _EWF_SEGMENTS if os.path.lexists(value + e)]
+                    if existing:
+                        return f"{arg} {value}: {existing[0]} exists already: images are never overwritten"
+            given[arg] = value
+            i += 2
+            continue
+        if arg in _EWF_FLAGS:
+            given[arg] = ""
+        elif arg.startswith("-"):
+            return f"{arg}: not a known safe ewfacquire option"
+        else:
+            sources.append(arg)
+        i += 1
+    for option, what in _EWF_REQUIRED.items():
+        if option not in given:
+            return f"ewfacquire needs {what}"
+    if "sha256" not in given.get("-d", "").split(","):
+        return "ewfacquire must compute a SHA-256 (-d sha256)"
+    if len(sources) != 1 or not _image_file(sources[0]):
+        return "ewfacquire's source must be one image file inside the project, not a device"
+    return None
+
+
+# --- exiftool ----------------------------------------------------------------
+
+# Options that write, run code, or take arguments from somewhere unseen
+# (compared lower-case, before any "=").
+_EXIF_REFUSED = {
+    "-o", "-out", "-overwrite_original", "-overwrite_original_in_place",
+    "-tagsfromfile", "-geotag", "-geosync", "-geotime", "-config", "-p",
+    "-printformat", "-api", "-@", "-execute", "-srcfile", "-delete_original",
+    "-delete_original!", "-restore_original", "-textout", "-tagout", "-k",
+    "-pause", "-stay_open", "-common_args", "-userparam", "-wm", "-writemode",
+}  # fmt: skip
+_EXIF_REFUSED_FORMS = re.compile(r"^(-if\d*|-w[!+]*|-execute\d*)$")
+
+
+def _exiftool(args: list[str]) -> str | None:
+    for arg in args:
+        if "${" in arg:
+            return f"{arg}: exiftool runs Perl code in ${{...}}"
+        if not arg.startswith("-"):
+            continue  # a file or a value
+        if any(c in arg for c in "=<>"):
+            return f"{arg}: writes, copies or renames tags"
+        name = arg.lower()
+        if name in _EXIF_REFUSED or _EXIF_REFUSED_FORMS.match(name):
+            return f"{arg}: writes, runs code or reads arguments from a file"
+    return None
+
+
+_RULES = {"dc3dd": _dc3dd, "ewfacquire": _ewfacquire, "exiftool": _exiftool}
+PROGRAMS = frozenset(_RULES)
+
+
+def argument_refusal(program: str, args: list[str]) -> str | None:
+    """Why ``program args`` is refused by these rules, or None (also for
+    programs without rules)."""
+    rule = _RULES.get(program)
+    return rule(args) if rule else None

src/codebud/safety.py

@@ -30,7 +30,7 @@
 import shlex
 from typing import Any
 
-from . import privacy
+from . import forensic, privacy
 from .modes import DEFAULT_MODE, step_allowed
 
 # The policy in effect -- yours, adjusted by the project's -- comes from
@@ -583,7 +583,7 @@ def _ruled_programs(self) -> set[str]:
         """Program names some refusal rule applies to (family names)."""
         refused = getattr(self, "refused", {})
         return {
-            "find", "python", "git", *_AWKS, *_SEDS, *_REFUSED_FLOOR,
+            "find", "python", "git", *_AWKS, *_SEDS, *_REFUSED_FLOOR, *forensic.PROGRAMS,
             *(_program_name(name) for name in refused),
         }  # fmt: skip
 
@@ -597,6 +597,9 @@ def _argument_refusal(self, words: list[str]) -> str | None:
         forbidden = _forbidden_argument([program, *args])
         if forbidden:
             return f"{shown} {forbidden} is never allowed (it can run or delete anything)"
+        evidence = forensic.argument_refusal(program, args)
+        if evidence:
+            return f"{shown} refused: {evidence}"
         if program == "tar" and args and not args[0].startswith("-"):
             # tar's old style: a dash-less first argument is a cluster of
             # options (tar cIf a.tar . is tar -c -I -f).

src/codebud_sec/data/profiles/security-ir/harmless_commands.json

@@ -4,5 +4,6 @@
   "readelf", "objdump", "nm", "sha256sum", "sha1sum", "md5sum", "b3sum",
   "ps", "pstree", "lsof", "who", "w", "last", "lastb", "lastlog", "id",
   "getent", "uname", "uptime", "date", "hostname", "lsblk", "journalctl",
-  "ausearch", "aureport", "tcpdump -r", "tshark -r", "find", "pwd", "echo"
+  "ausearch", "aureport", "tcpdump -r", "tshark -r", "find", "pwd", "echo",
+  "ewfverify", "ewfinfo", "exiftool"
 ]

src/codebud_sec/data/profiles/security-ir/powerful_commands.json

@@ -1,4 +1,4 @@
 [
   "cp", "tar", "yara", "vol", "vol3", "fls", "icat", "mmls", "istat",
-  "fsstat", "log2timeline.py", "psort.py", "pinfo.py"
+  "fsstat", "log2timeline.py", "psort.py", "pinfo.py", "dc3dd", "ewfacquire"
 ]