bashcap — the book

A transparent bash wrapper that writes the full state of a running shell at every BASHCAP call site, as one JSON object per line.

The core's book — sessions, rigs, joining, the wire — is bash-interop's book in its own repository; this book covers only what bashcap adds.

bashcap

A transparent bash wrapper that writes the full state of a running shell at every BASHCAP call site. It is the reference consumer of the rig, one subject per file:

instrument and Capture::of are the pair another tool reuses: the bash that produces a snapshot, and the code that reads one back. bash-interop-examples' snapshotting.rs is that reuse, bashcap expressed in the core, with typed captures for a session, instrument(Tracing::Calls) for the full stack, and no command line in between.

bashcap run   [--reach bash-env|by-hand] --into FILE [--verbose] [--trace-calls] [--] <command…>
bashcap serve --at DIR --into FILE [--verbose] [--trace-calls]
bashcap show FILE

The first two differ in who started the shells, and take the same options from the same Capture struct:

--verbose goes to stderr in both roles, and stdout is left to the subject. --trace-calls behaves differently by where it is sourced: through BASH_ENV it arms itself before the subject's first line, and into a shell already running, by hand or under serve, it installs a DEBUG trap there, replacing one the client had. run --help and serve --help end with every way a script joins, in this tool's words.

The words arrive with the laid files, or under run through BASH_ENV, so BASHCAP is defined from the moment rig.bash is sourced and joined from BASHCAP_INIT.

show renders a capture through Capture's Display, the same text a library caller gets from println!("{capture}").

The client's side

A script opts in by calling BASHCAP:

BASHCAP [-BCV:<var>]… [-BCS:<note>]…
WITH_BASHCAP [-BCV:<var>]… [-BCS:<note>]… <command> [args…]

-BCV: captures a variable by name and -BCS: attaches a note. Every variable named BASHCAP__CTX__* is captured automatically, which is how ambient context rides along without being named at each site. WITH_BASHCAP snapshots, runs the continuation, and returns the continuation's status.

A call site makes bashcap a dependency of the script that says it, since outside a session the word is a missing command. A script that must also run without the tool defines the word itself, in one guarded line.

The instrument

__bc_capture, whole — the tail of src/words.bash:

__bc_capture() {
    declare IFS=' '

    declare -a __bc_walk=()
    __bc_stack __bc_walk "$1"

    # What changes while a shell runs and nothing else says. The rest of what a
    # shell is — which bash, how it was started, which options it had on, how
    # deep a subshell it is — it said once when it joined.
    declare -a __bc_state=(
        seconds "$SECONDS"
    )

    declare -a __bc_rematch=("${BASH_REMATCH[@]}")

    declare -a __bc_declared=()
    declare __bc_name
    for __bc_name in "${__bc_vars[@]}" ${!BASHCAP__CTX__@}; do
        declare -p "$__bc_name" &>/dev/null || continue
        declare -n __bc_ref="$__bc_name"
        __bc_declared+=("${__bc_ref[*]@A}")
        unset -n __bc_ref
    done

    declare -- BC_SAY__ARG_LABEL=BASHCAP
    BC_SAY __BASHCAP__ \
        "${__bc_walk[@]}" \
        state   "(${__bc_state[*]@Q})" \
        rematch "(${__bc_rematch[*]@Q})" \
        vars    "(${__bc_declared[*]@Q})" \
        notes   "(${__bc_notes[*]@Q})"
}

The frame walk does not belong to bashcap. __bc_stack is shared with every tool that reports a stack and contributes six sections of its own (bash-interop: stack). Each section here is an array literal, read back with parse_array.

state holds what changes while a shell runs and nothing else records, which is $SECONDS. Which bash it is, how it was started and which options it had on were said once when the shell joined (bash-interop: shell), $SHLVL among them. A snapshot repeating any of those would be a second source for one fact.

Two details in the bash need explaining.

declare IFS=' '

Function-scoped, because the sections above are joined with [*], and a client that had set IFS would otherwise collapse each one into a single word. The envelope is safe without it, since __bc_send uses printf, but bashcap does these nested joins itself.

declare -n __bc_ref="$__bc_name"
__bc_declared+=("${__bc_ref[*]@A}")
unset -n __bc_ref

${ref[*]@A} through a nameref yields a complete self-describing declaration for every type — declare -i n='7', declare -A m=([k]="v") — attributes included. Indirect ${!name@A} cannot be used here: it collapses an array to its first element.

The decoder

#![allow(unused)]
fn main() {
/// Whether the subject's shells record what each call was passed.
pub enum Tracing { Off, Calls }

/// The bash a rig hands the subject, for any rig that wants what bashcap
/// harvests. One way to compose it; `BASH` and `TRACE` are not public.
pub fn instrument(tracing: Tracing) -> String;

pub struct Snapshot {
    pub stack: Stack,
    pub state: IndexMap<String, String>,   // what only this moment can say
    pub rematch: Vec<String>,
    pub vars: IndexMap<String, Variable>,
    pub notes: Vec<String>,
}

pub struct Frame {
    pub funcname: String,
    pub source: String,
    pub lineno: u32,

    /// The call's arguments, when the shell was recording them. `None` is
    /// "not recorded", never "called with none".
    pub args: Option<Vec<String>>,
}

pub struct Variable { pub attrs: String, pub value: Value }
pub enum Value { Scalar(String), Indexed(IndexMap<usize, String>), Assoc(IndexMap<String, String>) }

/// One snapshot under the provenance the wire gave it — the output format,
/// one per line, and what `show` reads back.
pub struct Capture {
    pub shell: Arc<Shell>,
    pub stamp: Stamp,
    pub snapshot: Snapshot,
}

impl Capture {
    /// `None` for a message that is not one of ours; `Some(Err)` for one that
    /// is and will not decode.
    pub fn of(line: &Line) -> Option<Result<Self, Failure>>;
}

/// Every capture in a file `BashCap` wrote: one JSON object per line. The one
/// way to read one back, used by `bashcap show` and by the tests alike.
pub fn captures(text: &str) -> Result<Vec<Capture>, Failure>;
}

The word a snapshot message begins with is __BASHCAP__, and Capture::of is the only thing that reads it, which lets several tools share one wire while a decode failure stays visible.

Call arguments

Bash records them only under extdebug, and bashcap never turns it on. Three things stand in the way, each of them enough on its own.

From BASH_ENV, the usual injection point, shopt -s extdebug means start the debugger. Bash warns on the subject's stderr, disables debugging mode and records nothing, and where bashdb is installed it would attach a debugger to the subject.

It implies errtrace and functrace, so a subject's own ERR and DEBUG traps become inherited by subshells and functions, changing the subject's behaviour.

Turning it on part-way leaves BASH_ARGC shorter than FUNCNAME, so the arguments that are there belong to the wrong frames. The reader detects that and carries them as absent, which is nothing to rely on.

--trace-calls, or BashCap::tracing_calls(), asks for them anyway. It puts nothing on the command line, argv reaching only the top-level shell, and instead injects trace.bash, which arms extdebug one command past startup from a DEBUG trap that removes itself. That handler has to return zero: under extdebug a non-zero DEBUG handler makes bash skip the command it fired for.

A subject that traces itself — shopt -s extdebug as its own first statement, or a bashdb session — gets them without the flag.

The stack math

__bc_capture does none. It calls __bc_stack, which ships bash's five arrays as they are, and every index is undone in Rust: which frames belong to the instrument, which line a frame is executing, and where a call's arguments sit in the flat stack and which way round they are. bash-interop: stack covers all of it, including why alignment rather than shopt -q decides whether a record is trustworthy.

__fixtures/bashcap_demo/child.bash traces itself, so one demo run shows both paths. About +45 µs on a six-deep stack when traced, against a ~480 µs snapshot — see bash-interop: measurements.

Every decoder in the crate recognises first and decodes second, and decoding mirrors the assembly: Columns::of takes the six sections the frame walk contributed, flat reads the rest with parse_array, and Declaration::read parses declare -aX name=rhs back into a name, its attribute letters and its value.

The snapshot carries no timestamp and no pid. The clocks are on the message, and everything about the shell is on the shell, which a reaction was handed at construction (bash-interop: shell).

The tool

#![allow(unused)]
fn main() {
type Sink = Rc<RefCell<BufWriter<File>>>;

pub struct BashCap   { into: PathBuf, sink: Sink, tracing: Tracing }
pub struct Capturing { shell: Arc<Shell>, into: PathBuf, sink: Sink, written: usize }

impl Rig for BashCap {
    type Reaction = Capturing;

    fn bash(&self, _at: &Layout) -> String {
        instrument(self.tracing)
    }

    async fn joined(&self, _at: &Layout, shell: Arc<Shell>) -> Result<Capturing, Failure> {
        Ok(Capturing { shell, into: self.into.clone(), sink: Rc::clone(&self.sink), written: 0 })
    }
}

impl Reacting for Capturing {
    type Kept = usize;   // how many this shell wrote; what they said is in the file

    async fn hear(&mut self, said: Message) -> Result<(), Failure> { /* decodes and writes */ }
    async fn answer(&mut self, asked: Message) -> Result<Answer, Failure> { /* hears it; unknown */ }
    async fn finish(self) -> Result<usize, Failure> { /* flushes */ }
}
}

bashcap only listens, so a shell that asks it something is heard and told the word is unknown.

One file, one reaction per shell. BashCap::writing opens the file, so a path that cannot be written fails before any shell has run, and each reaction holds a share of it. The shell is a member, because a walk is read against the shell it was taken in, and this reaction had it before its first message could arrive.

#![allow(unused)]
fn main() {
let ran = BashCap::writing(into)?
    .run(&argv, |at| {
        Ok(vec![session(at), at.bash_env(Provision::Joining(&joining(at)))?])
    })
    .await?
    .whole()?;
let written: usize = ran.shells.iter().map(|shell| shell.kept).sum();
}

The tally is a sum over the shells rather than a counter kept beside them.

The wrapped command carries its own program, so bashcap run --into out bash build.bash is the ordinary form and bashcap run --into out make test also works: every bash make starts reads the same BASH_ENV.

A failed flush in end ends the run rather than being lost in a Drop.

The output format belongs to the tool, since the core moves arglists and knows nothing about JSON. bashcap declares its own row, Capture, and serde(flatten) puts the provenance beside the snapshot's own fields rather than above them.

Lines are written as they arrive, each carrying the shell's own clock and the run's, so ordering downstream is exact and is sort's job. Writing in hear keeps resident memory independent of run length (bash-interop: measurements).

An indexed array travels as [index, value] pairs rather than as an object. A bash indexed array is sparse, so its indices are data, and JSON can spell an object key only as a string, which serde(flatten) cannot then read back as a number.

Rendering

Display on Capture, Frame, Variable and Value, and nothing else renders. A value prints as the bash that would declare it, which is what bash-strings' emitters already produce:

[3] pid 488092 at 1786796436346775 shlvl 7 subshell 0
    at    child_work@child.bash:12 ('a first argument' 'a second')
    at    main@child.bash:14 ()
    note  child process, own pid and SHLVL
    var   payload [a] ([0]='x' [1]='y' [2]='z')

Empty parentheses are a call with no arguments; no parentheses at all is a shell that was not recording them.

The program

src/bin/bashcap.rs is the whole of it: the two subcommands, and a capture that calls run. Four properties make it a transparent wrapper.

--into is required, with no default output location. stderr belongs to the subject unless --verbose is passed. The first plain word ends bashcap's options, so bashcap run --into out build.bash --into elsewhere passes --into elsewhere to the script; an unknown flag before that point is an error, and -- takes a wrapped command that starts with a dash. The subject's exit code is passed straight through, via ExitStatus::shell_code().

Playground

make demo [SCRIPT=path/to/your.bash]        # in this crate; make bash-demo at the workspace

Builds the debug binary, shows the words, runs __fixtures/bashcap_demo/demo.bash once with no tool and once under bashcap run, and renders the capture with bashcap show. The fixture exercises every facility in one file — typed variables, ambient context, BASH_REMATCH, nested frames with argv, the wrapping form, a subshell and a child process — and nothing asserts its line numbers, counts or variable names, so it can be edited freely.

Typical output is the block above. Its last two captures come from a subshell and a child process; the first reaches the wire by re-joining under its own $BASHPID, the second because the invocation runs there too, through BASH_ENV.

See also

• bash-interop: wire — how a rig's bash reaches every shell
• bash-interop: rigs — the trait it implements
• bash-interop-examples' snapshotting.rs — its instrument, reused without its CLI
• src/tests/ — its bash-level tests: one run covering every section