OVERVIEW

sigtree.pl is a tool for creating records of your filesystem structure
in order to detect unauthorized modifications to your system.  Its
records are called "specifications."  A specification is created for
a particular directory tree on your system, or, in some cases, for
a particular file.  Specifications are classified into sets, which
serve two functions.  First, you can choose which sets of trees you
want to create specifications for, which sets of trees you want to
check for changes, or which sets of trees' specifications you want
to update with detected changes.  Second, the primary set for each
tree determines a numeric priority level and a set of attributes
to be checked for changes.  The priority level is used for generating
summary reports of changes; higher priority items are listed first.

sigtree.pl uses pledge and unveil on OpenBSD, and can run with
privilege separation if you create a _sigtree user and group and
run it with -p or with "privsep: yes" in your config file.

INSTALLATION

sigtree.pl runs on OpenBSD, Linux (Debian/Proxmox), and macOS (at least),
and requires the following in order to work:

* Perl 5.10 or later.
* CPAN Perl modules Digest::SHA, Sys::Hostname, Getopt::Std, File::Basename,
  File::Temp (or OpenBSD::MkTemp on OpenBSD), Cwd, and
  Storable (all included with OpenBSD standard distribution).
* CPAN Perl module Digest::SHA3.
* CPAN Perl module Parallel::ForkManager (1.19 w/OpenBSD or later).
* On OpenBSD, uses OpenBSD::Pledge, OpenBSD::Unveil, and OpenBSD::MkTemp.
* Privilege separation requires IO::Socket, IO::Handle, IO::Select,
  IO::FDPass, MIME::Base64, Privileges::Drop, and either JSON::MaybeXS
  or JSON::PP.
  Note that Privileges::Drop fails on macOS with the base perl installation
  included with Tahoe 26, but works fine with the latest perl install with
  Homebrew (or you can make a small patch to Privileges::Drop to make it
  workaround a bug in perl 5.34.1).
  Note also that GPG2 signing with privsep requires use of provided
  gpg-noagent shell script.

If signify signing is used (which is recommended), the following is also
required:

* Signify.pm 1.4 or later (see below for where to obtain).
* signify (standard on OpenBSD) or signify-openbsd (Linux package).

If PGP or GPG signing is used, the following is also
required:

* PGP 5 or later, or GPG.
* CPAN Perl module PGP::Sign (security/p5-PGP-Sign port in OpenBSD).
* /bin/stty to turn off echoing when PGP passphrases are requested.

The default is no signing; signify is preferred. To sign, set
crypto_sigs: in your config file (see CONFIGURATION).

If immutable flags are to be used, sigtree.pl requires:

* An operating system that supports immutable files (e.g., *BSD, Linux).
* /usr/bin/chflags for setting/unsetting immutable flags.
* /sbin/sysctl (/usr/sbin/sysctl on macOS) to get the kernel securelevel.
OR (for Linux)
* /usr/bin/chattr for setting/unsetting the immutable file attribute.
* /usr/bin/lsattr for obtaining system file attributes.

To obtain CPAN Perl modules, it is recommended that you use the
standard CPAN module, obtained via OpenBSD ports tree, Homebrew,
apt, or manually as follows:

   perl -MCPAN -e shell;
   install Digest::SHA3
   install Parallel::ForkManager
   install PGP::Sign                    (only for PGP or GPG signing)
   install IO::FDPass Privileges::Drop  (only for privilege separation)

The Signify.pm perl module may be obtained from either
   https://github.com/lippard661/Signify
 or
   https://www.discord.org/lippard/software

The recommended location for sigtree.pl is in /usr/local/bin.  Sigtree
specifications go in /var/db/sigtree by default, but you can use any other
location, so long as you specify that location with the -r (root dir)
option to sigtree.pl. Sigtree-managed files are owned by root (even
with privilege separation; the unprivileged _sigtree user does not
have direct access).

The default config file is /etc/<hostname>.sigtree.conf, but a
different config file can be specified with the -c (config file)
option to sigtree.pl (a relative path is taken relative to /etc).
The config file must be owned by root and not writable by group or
others, and should not be readable by others (0600).  Modify the configuration file to
monitor directories and files that you want monitored, and prioritize
based on your own criteria.

The recommended mechanism for running sigtree tests is to use
a daily and weekly cron tab entry that emails the results to
another machine.  On *BSD machines, you can put the following
lines in daily.local and weekly.local, respectively:

(For daily but a full test on Wednesdays.)

    host=`/bin/hostname -s`
    dayofweek=`/bin/date +"%a"`
    case $dayofweek in
        Wed) sigtreetest="daily,weekly"
	     ;;
	*) sigtreetest="daily"
	   ;;
    esac
    echo "Running sigtree $sigtreetest."
    /usr/bin/perl /usr/local/bin/sigtree.pl -d secondary -s $sigtreetest check | /usr/bin/mail -s "$sigtreetest $host secondary sigtree report" email-address
    /usr/bin/perl /usr/local/bin/sigtree.pl -d secondary -s $sigtreetest changes >/dev/null && /usr/bin/perl /usr/local/bin/sigtree.pl -d secondary -s $sigtreetest update

(For a weekly full test.)

    host=`/bin/hostname -s`
    echo "Running sigtree weekly test."
    /usr/bin/perl /usr/local/bin/sigtree.pl check | mail -s "weekly $host sigtree report" email-address

Add the -v option if you want more verbose results that show the old
and new values of all file attributes that change.

The changes command exits 0 if changes are pending in the specified
sets, 1 if none are, and 2 on an error (with the message on stderr),
which is how the daily example runs update only when there is
something to update. From sigtree 1.30 the changed file (and the
secondary changed file) is kept even when nothing is pending, so its
modification time is the time of the last check; test for pending
changes with changes, not with "test -f" on the changed file as
earlier versions of this example did. (With "test -f" and 1.30,
update runs every day and says there is nothing to update.)

These will cause a daily report of all new changes that occur each day
in files in the "daily" set and a check of the "weekly" set each
Wednesday for against the secondary specs, and update them, along with
a weekly report of all changes against the primary specification that
occur in any files in your config. The checks against the secondary
specs are updated immediately afterward, so changes detected won't
be reported again (except via the primary specs test).

From 1.31, check records the spec's value and the value it found for
each changed attribute in the changed file, so the summary at the end of
check (which "changes -v" repeats) shows them, with digests cut to 12
characters:

   /etc/rc.conf.local (mode 0644 -> 0600; shadigest 3f2a9c4b1d0e... -> 8be01d7a2c44...)

Plain "changes" lists each changed path with the names of its changed
attributes, or (added) or (deleted). A changed file written by an
earlier version holds names only, and is shown that way until the next
check replaces it.

A check replaces the pending changes only for what it checked (1.31):
check -s for the trees in those sets, check_file for that path (and
everything beneath a directory), check_specs for the spec directory;
the rest are kept from the previous check. Earlier versions replaced
all of them, and check_file checked only the spec directory. From 1.32
check_file makes its path canonical as specs record paths (a trailing /,
//, /./ and .. are resolved; .. after a symbolic link is refused).

"changes etc" (a spec file's name: etc for /etc, usr.local for
/usr/local) or "changes /etc/rc.conf.local" (a file, or a directory for
everything pending at or beneath it) shows those paths one at a time
with full values, or with -v in check's long form, and looks at each
path again: sigtree reports a change undone since the check ("since
reverted: as in the spec again", an added file since removed, a deleted
one restored), or a further change, with the new values. -k with a list
of sigtree keywords (e.g. -k mode,uid,shadigest) limits changes, its
summary and these queries to those attributes. Options go before the
command: sigtree.pl -v changes /etc.

COMMAND OPTIONS (from "sigtree.pl -h")

Usage: sigtree.pl [options] command
Options:
-r root_dir
-c config_file
-d spec_dir (absolute path or relative to root_dir)
-s set list ("new" is synonymous with "uninitialized" and can be used with
             the "initialize" command; you can specify multiple sets
	     separated by commas)
-f number of child processes (check, initialize)
-m don't list changes inside macOS applications one by one (but still
   process them): check -v, the summary and changes count them per app,
   under "Inside macOS apps" (1.31); "changes /path" for an app or a
   path inside one lists them in full
-p use privilege separation
-v verbose
-k keyword list (changes and the verify_hosts detail query: the attributes to show)
-V show version (must be the only option, with no command)
-h help and version
Commands:
initialize: Initialize specifications for a set of trees.
initialize_specs: Initialize specification for the specification dir.
changes: Show non-updated or reinitialized changes found by check.
   changes spec | /path: show the pending changes in that spec's tree, or at or
   beneath that path, with values, and any since undone or changed again.
check: Check specifications for a set of trees.
check_file: Check an individual file against a specification.
   (The file name is specified on the command line after "check_file".)
check_specs: Check specification for the specification dir.
update: Update specifications with changes found by check.
verify_hosts: Check copies of other hosts' sigtree databases (on a central server).
   (-c names its own config file, default /etc/sigtree.hosts.conf.)
   verify_hosts host [spec | /path]: show what changed in that host's specs
   (or one spec, or one file, with a diff from the backups if both versions are there).

The commands initialize_specs, check_specs, and check_file do not
take the -s or -f options, and changes and update do not take -f.
-k is for changes and the verify_hosts detail query only. verify_hosts
takes only -c and -v, and -k for its detail query (see VERIFY_HOSTS
below).

By default, the root directory is /var/db/sigtree and the
specification directory for each host is specs/<hostname> relative
to that root directory.  This is designed to allow you to rsync
your specs directories across multiple machines without name
conflicts.

Giving the argument "secondary" to the -d option is special-cased
to mean <root dir>/secondary/<hostname>, and to not use PGP signing
or immutable flags--unless you are using BSD, in which case the
secondary files use user immutability (uchg) if the immutable option
is turned on.

CONFIGURATION

The configuration file is divided into three sections--the global
attributes section, the set definition section, and the trees list
section.  Any line in the configuration file may be blank or a
comment (a line beginning with a # is a comment).  The configuration
file must end with a newline.

The global attributes section consists of the following:

    crypto_sigs: none|GPG|GPG1|PGP|signify
    pgpkeyring: path (default /root/.gnupg)
    pgpkeyid: keyid
    signify_pubkey: path (default /etc/signify/sigtree.pub)
    signify_seckey: path (default /etc/signify/sigtree.sec)
    immutable-specs: yes|no|schg|uchg
    sha2_digest: 256|384|512
    sha3_digest: 224|256|384|512
    max_child_procs: [default 5]
    default_child_procs: [default 4]
    privsep: yes|no [default no]

All of these fields may be omitted. If the pgpkeyid: alone is used,
sigtree.pl will use GPG signatures on all specifications. It must
specify a PGP key ID which is in the keyring of the user running
sigtree.pl (usually root). The public key is required to do a check,
and access to the private key is required to do an initialize or
update.  Specify a crypto_sigs: field value to use GPG1, PGP, or
signify.  The pgpkeyring: field is optional and defaults to
/root/.gnupg.  The signify_pubkey: and signify_seckey: fields default
to /etc/signify/sigtree.pub and /etc/signify/sigtree.sec when
crypto_sigs: signify is used.  A value runs to the end of its line:
a "#" after a value is part of the value, not a comment.

Signatures are checked against specifications during check commands,
and are generated from specifications during initialize and update
commands.

The immutable-specs: field may be omitted, in which case the default
is "no."  If it is set to "yes," then system immutability flags are
set on specifications (and PGP signatures, if used) after their
creation, and initialize and update commands then require booting
to single user mode so that the system secure level will allow the
flags to be reset so that the files may be changed.  On *BSD systems,
immutable-specs may be set to "schg" (equivalent to "yes") or to
"uchg"--system or user immutability, respectively.  "schg" means the
system must be in single-user mode to reset the flag. On *BSD systems,
enabling immutability will also enable user immutability on secondary
files.

The pgpkeyid: field is ignored when secondary specifications are used,
but if immutable-specs: are turned on (yes, schg, or uchg), then
user immutability will be used for secondary specifications
(-d secondary).

The default value for SHA digests if none is specified is 256-bit
SHA3. SHA2 or SHA3 digests are only generated and comparisons are only
performed if shadigest is specified in the keywords for a set. If you
initialize specifications without any shadigest specified in the
keywords and subsequently add it to the keywords, the next time you do
a check the associated files will show as having changed from
<undefined> to the current actual value.

The set definition section consists of any number of the following
fields used to define sets:

   set: long name,short name
   description: Human-readable description of set.
   keywords: list of keywords
   priority: priority level

The set: and description: fields are required; the keywords: and
priority: keywords are optional.  Set names may consist of up to
twelve alpha-numeric characters.  Hyphens may also be used in
set names.  A set named "sigtree" must be defined; it is used for
sigtree's own specification directory.

Sets are used in two ways by sigtree.pl.  First, they are used
as a method of grouping trees, so that sigtree.pl's commands
may be used on different sets of trees.  The initialize, check,
and update operations work on entire trees.  The -s option to
sigtree.pl references sets used in this way; exceptions within
a tree are always members of the sets their tree is a member of.
The keywords: and priority: fields have no meaning for sets used
only in this way, and should not be defined for them.  The -s
option will complain about sets that are only used as primary
sets for exceptions, which have no trees as members.
  Second, sets are used to define groups of files that share
common characteristics, that need to have similar attributes
monitored for changes, and that are of a similar priority level.
The keywords: field determines which file attributes are checked
for changes, and the priority: field determines where changes
are reported in the check command's summary report--the higher
the priority, the higher it appears in the report.  Each tree or
exception may only have one set used in this way; this set is
known as its primary set and is the first one associated with
its name in the configuration file.
   Keywords available are

        type   type of file (file, dir, link, fifo, device, socket)
	linktarget_type  type of file that is the target of a link
        uid    user ID of owner
        gid    group ID of owner
        mode   file permissions
        size   size in bytes
        nlink  number of hard links to the file
        link   link target, if it's a symbolic link
        mtime  modification time of file
        mtimestasis-NN[smhd]  report if mtime doesn't change for
               more than NN seconds/minutes/hours/days
        ctime  inode change time of file 
        shadigest  SHA-3 or SHA-2 digest of file (the old sha2digest and
	       sha3digest keywords are now synonyms, the digests computed
	       are based on the global shaN_digest setting)
        flags  file system flags (those shown with ls -lo/set by chflags,
               or lsattr/chattr on Linux)
        rdev   device number of a block or character device (1.30 and
               later); use it in a set for device nodes, such as a
               chroot's dev directory. Specs made before 1.30 have no
               device numbers, so re-initialize such trees; versions
               before 1.30 reject this keyword.
        ignore overrides all other keywords, no file attributes are
                  checked

and must be separated by commas in the keywords list.  If no
keywords: field is specified, the default is to check all of these
attributes except linktarget_type, mtimestasis, rdev, and ignore
(type,uid,gid,mode,size,nlink,link,mtime,ctime,shadigest,flags).

Priority must be a decimal integer.

The trees list section of the configuration file lists the pathnames
of trees for which specifications are to be produced and a list of
the sets they belong to.  The first set listed is the tree or file's
primary set and determines its priority and keyword settings.
   For each tree, any number of exception files within the tree may
be specified with a different primary set.  The exact syntax is:

    tree: path:set list
    exception: path:primary set
    exception-tree: dirpath:primary set
    primary-exception: path:primary set
    secondary-exception: path:primary set
    primary-exception-tree: dirpath:primary set
    secondary-exception-tree: dirpath:primary set

The "exception" field specifies a single file or directory as an
exception, while the "exception-tree" field specifies a directory
subtree (a directory and everything it contains) as an exception.
Exception paths are relative to the tree (mtab, not /etc/mtab); an
absolute path is a configuration error.
If "." is specified as an exception, the primary set of the tree
directory itself (but not its contents) is changed.  This is useful
for directories that frequently change mtime and ctime (like /etc).
An exception-tree for "." (the whole tree) is an error; change the
tree's own sets instead.

The primary- and secondary- forms (1.28 and later) apply only to the
primary specifications (any run without -d secondary) or only to the
secondary specifications, so a path can be in a different set in
each: for example, a log that must change daily, checked for stasis
in the secondary specs but ignored in the primary specs because it
does not exist in single-user mode.  A path has either a plain
exception line or per-mode lines, not both.  Versions before 1.28
reject these fields.

Set names specified may be either long or short names of sets.

NOTES

Symbolic links are never followed by sigtree.pl, to avoid loops.
If you have a file you want monitored that is not under a specified
tree, but only the target of a symbolic link under a specified tree,
that file will not be monitored unless you add it or one of its
parent directories to the configuration file.

File flags are read only for paths whose set includes the flags
keyword, with ls (BSD and macOS) or lsattr (Linux), in batches.

Trees may not overlap--no tree in the configuration file may be a
subtree of another tree in the configuration file.

VERIFY_HOSTS

If each host's /var/db/sigtree is copied to a central server (and
backed up there with rsnapshot), "sigtree.pl verify_hosts" on that
server checks the copies, using its own config file
(/etc/sigtree.hosts.conf, or -c; see sigtree.hosts.conf.sample and
README.md for its fields).  Added in sigtree 1.29.  For each host it
reports how long ago its primary changed file was written, spec
files changed between two rsnapshot snapshots, and any spec put back
to an older version anywhere in between; and, for hosts whose signify
public key is pinned in the config, any spec whose signature does not
verify.  Among hosts marked as peers, it reports files under the
configured common paths that differ from what most peers have; for
each configured chroot (1.30), setuid files, misplaced device nodes,
wrong ownership, and system files that no longer match the host's
own copies; and for each reference file (1.30), hosts whose copy
differs from the server's.  It prints nothing when there is nothing
to report, so it can run from daily.local or weekly.local:

    /usr/bin/perl /usr/local/bin/sigtree.pl verify_hosts

On demand, "sigtree.pl verify_hosts host" shows what changed in that
host's specs between the two snapshots; "verify_hosts host spec" does
it for one spec, and "verify_hosts host /path" for one file, with a
diff from the backups when both versions are found there.  Specs are
loaded only after their signatures verify, and only as _sigtree.
Since a spec records times for every path whatever its set's keywords,
the detail query leaves out mtime and ctime changes unless -v is given
(1.31), saying how many paths changed only in those; -k with a list of
sigtree keywords (e.g. -k mode,uid,gid,rdev) shows just those
attributes.  If the oldest snapshot's copy of a spec cannot be
verified (a GPG signature from before the host moved to signify), the
changes are shown from the oldest later snapshot whose copy verifies.

TO REPORT BUGS AND SUGGESTIONS

Send bug reports and suggested enhancements to lippard-sigtree@discord.org.
