InterviewPrepKit

Home / Learn / Object-Oriented Design

How to design a file-search utility

In this lesson, we’ll design a file-search library, something like the Unix find command, and let it answer the queries that break a normal function signature. Ask for every .log file over 10 MB that is not under /tmp, and the moment a query mixes OR and NOT, keyword arguments cannot express it. A library like this is only as useful as the queries it can express, so we’ll make each query an object. By the end you’ll be able to explain why a parameter list caps out, build a small algebra of filters that combine with and, or, and not, and name what that composition costs.

This lesson covers:

  • why a function with optional keyword arguments cannot express queries that mix OR and NOT;
  • a small algebra of filter objects that combine with and, or, and not;
  • a directory tree where files and folders answer the same questions;
  • what the composed design costs, in three specific ways.

What goes in, and what comes out

We start at the boundary, because it settles most of the design. The library has one entry point, and its shape decides everything downstream.

IN   root     a Node -- the top of the directory tree to search
IN   where    a Filter -- one object whose matches(node) returns True or False
IN   descend  an optional Filter -- which directories are worth walking into

OUT  an iterator of Nodes: results handed back one at a time, as they are found,
     in pre-order (a directory is reported before the things inside it)

find(root, (LargerThan(10 * MB) | Extension(".log")) & ~UnderPath("/tmp"))
  -> /
  -> /usr
  -> ...
  -> /var/log/syslog.log
  -> /var/log/old.log

What is not in that signature matters: no name=, no size=, no extension=. The entire query is one argument, because the query is one object.

The two ideas that make this work are named patterns. The query side is the Specification pattern: each criterion is an object with a single yes/no method, and criteria combine into more criteria of the same kind. The tree side is the Composite pattern: a file (leaf) and a directory (branch) expose one interface, so the traversal never asks which it is holding. Both are worked out below.

Why a parameter list cannot express the query

A common first attempt is find(directory, name=None, size=None, extension=None). It is a reasonable v1, and it fails on one specific requirement.

from __future__ import annotations

from abc import ABC, abstractmethod
from fnmatch import fnmatch
from typing import Iterator, Protocol


def find_naive(root, name=None, min_size=None, extension=None, is_dir=None):
    """Every criterion is optional; supplying two means AND. There is no third option."""
    out = []
    for node in walk(root):
        if name is not None and not fnmatch(node.name, name):
            continue
        if min_size is not None and node.size() < min_size:
            continue
        if extension is not None and not node.name.endswith(extension):
            continue
        if is_dir is not None and isinstance(node, Directory) != is_dir:
            continue
        out.append(node)
    return out

Three imports there are used throughout the lesson:

  • from __future__ import annotations treats type annotations as plain text instead of evaluating them, which lets Filter | None (3.10 syntax) run on Python 3.9.
  • ABC is Python’s abstract base class: a class that refuses to be instantiated while any @abstractmethod is unimplemented, so a missing matches fails at construction, not at call time.
  • fnmatch matches a filename against a shell wildcard such as *.txt.

Each if ... continue guard silently means and. Supplying min_size and extension together can only ask for “over 10 MB and ending in .log”. The requirement was over 10 MB OR ending in .log, and not under /tmp, and no argument list says it. A parameter list is an implicit AND and nothing else.

The gap is not small. With five criteria, an AND-only flag list reaches 2^5 = 32 combinations. The full space of boolean questions over five yes/no facts is 2^32, about 4.3 billion. The flag list reaches 32 of them: exactly the conjunctions. Every OR, every NOT, and every parenthesised mixture is outside its range, and adding a criterion only widens the gap.

The fix turns each criterion into an object and lets those objects compose. Adding a criterion then stops being an edit to a function that already works and becomes a new class with no callers to break. That is the open-closed principle in practice: open for extension (add a class), closed for modification (touch no existing code).

The filter algebra

Filter declares one method, matches, and supplies three combinators. The key detail is that every combinator returns a Filter, so a combination is usable anywhere a single criterion is.

from __future__ import annotations

class Filter(ABC):
    @abstractmethod
    def matches(self, node: Node) -> bool: ...

    def __and__(self, other: "Filter") -> "Filter":
        return And(self, other)

    def __or__(self, other: "Filter") -> "Filter":
        return Or(self, other)

    def __invert__(self) -> "Filter":
        return Not(self)


class And(Filter):
    def __init__(self, *parts: Filter) -> None:
        self.parts = parts

    def matches(self, node: Node) -> bool:
        return all(p.matches(node) for p in self.parts)


class Or(Filter):
    def __init__(self, *parts: Filter) -> None:
        self.parts = parts

    def matches(self, node: Node) -> bool:
        return any(p.matches(node) for p in self.parts)


class Not(Filter):
    def __init__(self, part: Filter) -> None:
        self.part = part

    def matches(self, node: Node) -> bool:
        return not self.part.matches(node)

Because And, Or, and Not are themselves Filters that hold Filters, a query is a tree, not a list. A list nests zero levels deep; a tree nests as far as the query needs, which is what lets (a | b) & ~c exist at all.

flowchart TD
    and["And"] --> or["Or"]
    and --> not["Not"]
    or --> lt["LargerThan 10 MB"]
    or --> ext["Extension .log"]
    not --> up["UnderPath /tmp"]

Defining __and__, __or__, and __invert__ is the Python-specific move: those are the hooks Python calls for &, |, and ~, so the query reads (a | b) & ~c instead of And(Or(a, b), Not(c)). Two gotchas come with that. & binds tighter than |, so a | b & c means a | (b & c) and the parentheses in (a | b) & ~c are required. And these operators do not short-circuit the way the and/or keywords do; a | b always builds both Or operands (the predicates inside them still run later, in matches).

With combination handled by the type, each criterion shrinks to a constructor and one method. None mentions traversal, or any other criterion, or configuration.

class NameMatches(Filter):
    def __init__(self, pattern: str) -> None:
        self.pattern = pattern

    def matches(self, node: Node) -> bool:
        return fnmatch(node.name, self.pattern)


class Extension(Filter):
    def __init__(self, ext: str) -> None:
        self.ext = ext

    def matches(self, node: Node) -> bool:
        return node.name.endswith(self.ext)


class LargerThan(Filter):
    def __init__(self, nbytes: int) -> None:
        self.nbytes = nbytes

    def matches(self, node: Node) -> bool:
        return node.size() > self.nbytes


class UnderPath(Filter):
    def __init__(self, prefix: str) -> None:
        self.prefix = prefix

    def matches(self, node: Node) -> bool:
        p = self.prefix.rstrip("/") or "/"     # "/tmp/" and "/tmp" mean the same
        sep = "" if p == "/" else "/"          # or the root builds the prefix "//"
        return node.path == p or node.path.startswith(p + sep)


class IsFile(Filter):
    def matches(self, node: Node) -> bool:
        return isinstance(node, File)


def find(root: Node, where: Filter,
         descend: Filter | None = None) -> Iterator[Node]:
    """Traversal, predicate, and sink are three separate things."""
    return (n for n in walk(root, descend) if where.matches(n))

find is a single generator expression. It pulls nodes from walk and forwards the ones where accepts, and it names no criterion at all, which is exactly why adding one requires no edit to it. UnderPath is the longest criterion only because prefix normalisation is fiddly; the worked example below shows what those two lines prevent.

The directory tree: Composite

The tree half gives files and directories one interface, Node, so callers never ask which they are holding. Node promises two questions: path and size().

class Node(ABC):
    def __init__(self, name: str, mtime: float = 0.0) -> None:
        self.name, self.mtime = name, mtime
        self.parent: Directory | None = None

    @property
    def path(self) -> str:
        if self.parent is None:
            return self.name
        base = self.parent.path
        return f"{base}/{self.name}" if base != "/" else f"/{self.name}"

    @abstractmethod
    def size(self) -> int: ...


class File(Node):
    def __init__(self, name: str, size_bytes: int, mtime: float = 0.0) -> None:
        super().__init__(name, mtime)
        self.size_bytes = size_bytes

    def size(self) -> int:
        return self.size_bytes


class Directory(Node):
    def __init__(self, name: str, mtime: float = 0.0) -> None:
        super().__init__(name, mtime)
        self.children: list[Node] = []

    def add(self, *children: Node) -> "Directory":
        for c in children:
            if not isinstance(c, Node):
                raise TypeError(f"not a Node: {c!r}")
            if c.parent is not None:                   # a tree, not a DAG: a second
                raise ValueError(f"{c.name} already has a parent; "
                                 "a tree is not a DAG")   # parent rewrites `path`
            n = self
            while n is not None:                       # d.add(d) is a one-line cycle
                if n is c:
                    raise ValueError("that would make a cycle")
                n = n.parent
            c.parent = self
            self.children.append(c)
        return self

    def size(self) -> int:                    # the branch delegates to its leaves
        return sum(c.size() for c in self.children)

Three pieces of syntax do design work. @abstractmethod on size means any subclass that fails to define it cannot be instantiated, so the interface’s promise is checked by the language. @property lets path be written as a method but read as a field (node.path, no parentheses), hiding that it is computed by walking up the parent chain. mtime is the Unix modification time, seconds since 1970.

add is where the tree stays a tree

add looks like bookkeeping but is the only place that can enforce two structural invariants:

  • No re-parenting. Setting c.parent = self unconditionally would silently move a node when it is added to a second directory. It would then report a new path in the old tree too, so UnderPath answers differently for queries built before the move, and walk reports it twice. The if c.parent is not None guard refuses that.
  • No cycles. p.add(c); c.add(p) builds a cycle in one line, after which walk, size(), and path each recurse until Python gives up. The while n is not None loop walks up from the new parent looking for the child; finding it means the link would close a loop.

A tree is not a DAG and not a general graph, and the method that builds one is the only place that can say so.

What Directory.size() costs

Having Directory.size() sum the subtree is a decision, not an obvious truth, and it changes what a query means. Real find -size +10M tests the directory entry’s own block size (typically 4096 bytes on ext4), so a directory effectively never matches. This implementation makes /var match a 10 MB filter because its contents total 32 MB. Both are defensible; only one matches what a user expects.

That is the cost of Composite in general: a uniform interface invites you to ask leaf questions of branches, and some have no leaf-shaped answer. size() is one. mtime on a directory is another: it is when the directory’s own entry list changed, not the newest file inside it, a distinction that has burned every “sync only what changed” tool ever written.

The traversal

walk is where Composite pays off: a short pre-order recursion that never asks what kind of node it has except to decide whether to descend.

def walk(node: Node, descend: "Filter | None" = None) -> Iterator[Node]:
    yield node
    if isinstance(node, Directory):
        for child in node.children:
            if isinstance(child, Directory) and descend is not None \
                    and not descend.matches(child):
                continue                       # prune: skip the whole subtree
            yield from walk(child, descend)

yield makes this a generator: calling walk runs no code until asked, and each yield hands one node back and freezes the function until the caller wants the next. yield from relays everything a nested recursive call produces. Together they let the traversal be cheap and lazy, priced in the streaming section below. The three-line if is the pruning clause; deleting it leaves a working walker. descend is a second predicate, separate from where, and the reason it must be separate is the subject of the matching-vs-pruning section.

The class diagram

This is the whole design in one picture. Node and Filter are both abstract, each with a leaf-and-branch pair beneath it, and there is no Searcher class: the traversal is a plain function.

classDiagram
    class Node {
        <<abstract>>
        +str name
        +float mtime
        +path() str
        +size() int
    }
    class File {
        +int size_bytes
    }
    class Directory {
        +add(node) Directory
    }
    class Filter {
        <<abstract>>
        +matches(node) bool
    }
    class Clock {
        <<interface>>
        +now() float
    }
    class And
    class Or
    class Not
    class NameMatches
    class LargerThan
    class Extension
    class UnderPath
    class ModifiedWithin

    Node <|-- File : inheritance
    Node <|-- Directory : inheritance
    Directory "1" *-- "0..*" Node : composition
    Filter <|-- And
    Filter <|-- Or
    Filter <|-- Not
    Filter <|-- NameMatches
    Filter <|-- LargerThan
    Filter <|-- Extension
    Filter <|-- UnderPath
    Filter <|-- ModifiedWithin
    And "1" o-- "2..*" Filter : operands
    Or "1" o-- "2..*" Filter : operands
    Not "1" o-- "1" Filter : operand
    ModifiedWithin "1" --> "1" Clock : injected

Reading the notation: <|-- is inheritance (“is a”); *-- is composition (the part cannot outlive the whole); o-- is aggregation (the whole holds a reference to something with its own life); --> is a plain association. Multiplicities "1", "0..*", "2..*" mean exactly one, zero or more, two or more.

Two Composites appear, with the same shape but different arrows, and the difference is a claim about lifetime. Directory *-- Node is composition because removing a directory removes its contents. And o-- Filter is aggregation because a filter is intended to be shared between queries. Nothing enforces that sharing safely, though: Extension('.log').ext = '.txt' mutates a filter two live queries hold, and both silently change meaning. Freeze them with @dataclass(frozen=True) if you mean it; otherwise the honest arrow is composition.

The pipeline the diagram implies is three stages. The tree and the optional pruning filter feed the traversal, which streams nodes one at a time into the matching filter, which forwards accepted nodes to the sink (whatever the caller does with the iterator).

flowchart LR
    root[root Node] --> walk[walk: pre-order traversal]
    descend[descend Filter: prune subtrees] --> walk
    walk -->|one node at a time| where[where Filter: matches node]
    where -->|accepted| sink[sink: caller's for loop]

Worked example

Running the design against a small tree makes it concrete. Twelve nodes, seven directories and five files, with these sizes: python3 20 MB, old.log 30 MB, junk.log 50 MB, syslog.log 2 MB, doc.txt 1 KB.

MB = 1024 * 1024
root = Directory("/").add(
    Directory("usr").add(
        Directory("bin").add(File("python3", 20 * MB)),
        Directory("share").add(File("doc.txt", 1024)),
    ),
    Directory("var").add(
        Directory("log").add(File("syslog.log", 2 * MB), File("old.log", 30 * MB)),
    ),
    Directory("tmp").add(File("junk.log", 50 * MB)),
)

assert [n.path for n in walk(root)][:4] == ["/", "/usr", "/usr/bin", "/usr/bin/python3"]
assert root.size() == (20 + 30 + 2 + 50) * MB + 1024

The first assertion pins pre-order: a parent always before its children. The second confirms root.size() sums the whole subtree.

Now the requirement that broke the parameter list, as one expression:

# "over 10 MB OR ending in .log, and not under /tmp"
query = (LargerThan(10 * MB) | Extension(".log")) & ~UnderPath("/tmp")
hits = sorted(n.path for n in find(root, query))
assert hits == ["/", "/usr", "/usr/bin", "/usr/bin/python3",
                "/var", "/var/log", "/var/log/old.log", "/var/log/syslog.log"], hits

This result is correct, is exactly what the composition asked for, and is still not what the user meant. Five of the eight hits are directories (/, /usr, /usr/bin, /var, /var/log), present because Directory.size() means “everything underneath”, so /usr reports 20 MB and clears the filter. This is the honest state of the earlier size() decision, not a bug.

The fix is one object prepended to the query, and a brand-new criterion is one class with no registration step:

files_only = IsFile() & query
assert sorted(n.path for n in find(root, files_only)) == [
    "/usr/bin/python3", "/var/log/old.log", "/var/log/syslog.log"]


# Adding a criterion is a new class, not an edit. Nothing above changes.
class SmallerThan(Filter):
    def __init__(self, nbytes: int) -> None:
        self.nbytes = nbytes

    def matches(self, node: Node) -> bool:
        return node.size() < self.nbytes


tiny_docs = IsFile() & SmallerThan(4096) & NameMatches("*.txt")
assert [n.path for n in find(root, tiny_docs)] == ["/usr/share/doc.txt"]

The add guards can be exercised the same way. A string is not a Node; /usr already has a parent; and a two-node tree cannot be closed into a cycle.

for bad in ("/etc/passwd",            # not a Node at all
            root.children[0]):        # /usr already has a parent
    try:
        Directory("d").add(bad)
        raise AssertionError(f"add accepted {bad!r}")
    except (TypeError, ValueError):
        pass

p = Directory("p").add(Directory("c"))
try:
    p.children[0].add(p)              # p.add(c); c.add(p) -- one line, infinite recursion
    raise AssertionError
except ValueError:
    pass

UnderPath’s normalisation earns its two lines at the edges. "/" + "/" is "//", a prefix of nothing, and "/tmp/" would build the prefix "/tmp//":

assert [n.path for n in find(root, UnderPath("/tmp/"))] == ["/tmp", "/tmp/junk.log"]
assert len(list(find(root, UnderPath("/")))) == 12          # every node, not one
assert list(find(root, ~UnderPath("/"))) == []              # and nothing at all

Without it, UnderPath("/") would match only the root, ~UnderPath("/") would keep every node except the root, and UnderPath("/tmp/") would match nothing.

For contrast, the naive function on the same tree cannot express the requirement at all. Both attempts are wrong: one under-matches, one over-matches.

# Two criteria means AND, so this asks ">10 MB AND .log" -- misses the 2 MB syslog.log:
assert [n.path for n in find_naive(root, min_size=10 * MB, extension=".log")] == [
    "/var/log/old.log", "/tmp/junk.log"]

# Drop the size and it over-matches: junk.log is back, with no way to exclude /tmp:
assert [n.path for n in find_naive(root, extension=".log")] == [
    "/var/log/syslog.log", "/var/log/old.log", "/tmp/junk.log"]

A criterion that depends on more than the node

“Modified in the last 7 days” costs one class, with no edit to walk, find, Filter, or any existing criterion. It is also the first criterion whose answer depends on something other than the node: the current time.

class Clock(Protocol):
    def now(self) -> float: ...


class FakeClock:
    def __init__(self, t: float) -> None:
        self.t = t

    def now(self) -> float:
        return self.t


class ModifiedWithin(Filter):
    def __init__(self, seconds: float, clock: Clock) -> None:
        self.seconds, self.clock = seconds, clock

    def matches(self, node: Node) -> bool:
        return self.clock.now() - node.mtime <= self.seconds


DAY = 86400.0
clock = FakeClock(t=100 * DAY)
recent = Directory("/r").add(File("fresh.log", 10, mtime=97 * DAY),
                             File("stale.log", 10, mtime=80 * DAY))
week = ModifiedWithin(7 * DAY, clock)
assert [n.path for n in find(recent, IsFile() & week)] == ["/r/fresh.log"]
assert [n.path for n in find(recent, IsFile() & ~week)] == ["/r/stale.log"]

A Protocol describes a shape, not an ancestry: anything with a now() returning a float counts as a Clock, so FakeClock satisfies it without inheriting or being registered. Pinning “now” at day 100 makes the assertions pure arithmetic, and ~week (the same object, negated) picks out the complement.

ModifiedWithin takes its Clock at construction instead of reaching for a global. Handing an object its collaborators this way is dependency injection, and it is what lets the test freeze time. It also breaks a property the other filters had: matches is no longer a pure function of the node (same input, same answer), so the same query gives different answers at different times, and caching results is now unsound.

Matching versus pruning

There are two places a predicate can go, and the same object means different things in each:

  • As a matcher (where): ~UnderPath("/tmp") visits every node under /tmp, evaluates it, and discards it.
  • As a pruner (descend): ~UnderPath("/tmp") never enters the directory at all.

The answers can be identical while the work is not. Prune a 700,000-node subtree out of a million-node tree and you skip 70% of the walk for the same result. A Counting decorator (an object with the same interface that tallies every node it is asked about) makes the difference measurable instead of asserted:

counted: list[str] = []


class Counting(Filter):
    """Wraps a filter and records every node it is asked about."""

    def __init__(self, inner: Filter) -> None:
        self.inner = inner

    def matches(self, node: Node) -> bool:
        counted.append(node.path)
        return self.inner.matches(node)


not_tmp = ~UnderPath("/tmp")

counted.clear()
as_matcher = sorted(n.path for n in find(root, Counting(IsFile() & not_tmp)))
visited_matching = len(counted)

counted.clear()
as_pruner = sorted(n.path for n in find(root, Counting(IsFile()), descend=not_tmp))
visited_pruning = len(counted)

assert as_matcher == as_pruner                     # same answer
assert visited_matching == 12                      # every node, including /tmp/junk.log
assert visited_pruning == 10                       # /tmp and its child never existed

This walk prunes before yielding, so the pruned directory is not reported either, giving ten. Real find -prune reports the directory and skips its contents, a third defensible reading of “skip”. Because a seven-line traversal has three reasonable meanings of “skip”, the intent cannot be inferred from a single predicate: it has to be two arguments.

The sharpest case negates a predicate about node type. As a pruner, IsFile() collapses the walk:

# As a matcher, "not a file" means "do not report files".
assert len(list(find(root, ~IsFile()))) == 7       # the 7 directories

# As a pruner, IsFile() means "only descend into files"; no directory is a file,
# so none is ever entered and the walk yields only the root.
assert len(list(find(root, IsFile(), descend=IsFile()))) == 0
assert [n.path for n in walk(root, IsFile())] == ["/"]

The last assertion is the whole point: not “stops at depth 1” but “never reaches depth 1”. Negation is a property of a predicate in a position, not of the predicate itself. The find command exposes the same split as -not -path versus -prune, and -prune is famously confusing precisely because it hides that distinction.

The alternative, inferring pruning from the match expression, is a query optimiser. Doing it correctly requires knowing which criteria are monotone down the tree: if false for a directory, false for everything inside it. UnderPath is monotone; LargerThan is not, because a small directory can hold a huge file. Keeping the two predicates separate makes the distinction structural instead of folklore.

Streaming versus building a list

find returns a generator, so results come back one at a time. The Counting decorator shows what laziness saves:

from itertools import islice

counted.clear()
first = next(iter(find(root, Counting(Extension(".log")))))
assert first.path == "/var/log/syslog.log"
assert len(counted) == 9        # stopped 3 nodes early, mid-tree

counted.clear()
everything = list(find(root, Counting(Extension(".log"))))
assert len(counted) == 12       # the whole tree, every time
assert len(everything) == 3

assert [n.path for n in islice(find(root, Extension(".log")), 2)] == [
    "/var/log/syslog.log", "/var/log/old.log"]

next() abandons the walk the moment the first match appears; list() drains it and visits all twelve; islice takes the first n without materialising the rest. On a real root the difference dominates: a list of a million matches (at ~2 µs per node) stays silent for ~2 seconds before its first result and holds ~200 MB (about 200 bytes per match). A generator returns the first hit in microseconds and holds only a recursion stack proportional to the tree’s depth.

Streaming costs real properties, though:

CostConsequence
No len() or sorted() without buffering“How many matches?” reintroduces the 200 MB
Single-passA second iteration yields nothing, with no error
Deferred exceptionsA permission error surfaces inside the caller’s for loop, far from where a directory means anything
Deferred mutationThe tree can change between the first and last result, so output is not a snapshot of one instant
yield from depthEach item is relayed through one frame per level; at depth 20 that is 20 frames per result

Two have standard mitigations. For exceptions, pass a policy object (on_error(node, exc) -> bool) instead of a try inside walk, because walk cannot know whether a permission error should end the search or be skipped. For depth, use an explicit-stack traversal (the traversal keeps its own list of pending directories); it is faster and reads worse, so profile before paying for it.

What the design assumes

The load-bearing assumption is the signature matches(node) -> bool. Those four tokens commit to three things at once:

  1. A match is decidable from one node in isolation. No criterion may depend on another node, which quietly forbids “the ten largest files”, “duplicates”, and “newer than a sibling”.
  2. A match is a boolean, not a score. No ranking, so no fuzzy name matching and no “best” results.
  3. The criterion is opaque code, not inspectable data. That is what forbids pushing it into an index.

The structure also assumes the tree is finite and fully walkable (enforced by add), each node is visited once (the same guard), traversal is pre-order and not part of the query, and each node knows its parent so path can be derived instead of stored.

Relaxing any of the three commitments changes what you build:

  • If speed is the requirement, not expressiveness, the object model is the wrong artefact. Criteria become entries in an inverted index (a lookup from name or extension to matching paths), And/Or become set intersection and union, and the filter tree becomes a query plan to rewrite. Filters survive only if an optimiser can read them, which means filters must be data. That is locate, not find, and it can be stale.
  • If criteria are relational (“the ten largest”, “duplicates”), matches(node) -> bool is the wrong signature entirely. The design becomes a pipeline of stream operators where each stage sees the whole stream, with its own failure mode: stages that must buffer.
  • If results are ranked, matches returns a score, And/Or become something like multiply and maximum, and results must be sorted, which means buffering everything and losing the streaming property.
  • If the tree is remote and paginated (cloud object storage), Directory.children cannot be a list; children become an iterator that makes network calls, size() stops being cheap, and pruning becomes the difference between a query that finishes and one that does not.
  • If queries must be saved and reloaded, filters need a serialised form, adding a to_dict() or visitor to every criterion.

A few common follow-ups fall out of the same model:

  • Symlink loops need identity, not names: a set of (st_dev, st_ino) (device and inode numbers, which identify a file uniquely) checked before descending. This belongs in walk, which owns traversal history, not in a Filter, which cannot see it.
  • -maxdepth is a traversal parameter, not a filter: depth is a property of how you reached a node, not of the node.
  • “Why not pass lambdas?” For one criterion, do. Objects earn their place when criteria must be inspected: explain() on a zero-result query, optimisation, serialisation. A lambda can be called but not asked what it is.
  • Contradictions like LargerThan(10) & SmallerThan(5) return nothing, correctly, and no static check catches them; proving a filter tree unsatisfiable needs a domain-specific solver, out of scope here.

Conclusion

  • A parameter list is an implicit AND with no escape. The moment a query needs OR or NOT, each criterion has to become an object.
  • The Specification pattern makes combination a type: And, Or, and Not are Filters that hold Filters, so a query is a tree and a combination is usable wherever a single criterion is. Adding a criterion is a new class, not an edit.
  • The Composite pattern gives File and Directory one Node interface, so the traversal is one short recursion. Its cost is that leaf questions asked of branches (size(), mtime) have no honest answer.
  • Directory.add is the only place that can keep a tree a tree, refusing re-parenting and cycles.
  • Match and prune are two positions for the same predicate; negation means different things in each, so they must be separate arguments.
  • Streaming returns the first result in microseconds at constant memory, and gives up len(), second passes, in-place error handling, and snapshot consistency.
  • The whole design rests on matches(node) -> bool: one node, a boolean, opaque code. Relax any of the three and you are building an index, a stream pipeline, or a ranker instead.

One line to remember: a query earns its object the moment it needs OR or NOT, and once each criterion is an object, the next one is a new class with no callers to break.

Further reading

  • Gamma, Helm, Johnson, Vlissides, Design Patterns (1994): the Composite pattern (uniform leaf/branch) and the iterator that walk embodies.
  • Eric Evans and Martin Fowler, “Specifications”: the combinable-predicate pattern behind Filter.
  • The find(1) man page: how a real tool separates tests from actions, and why -prune behaves the way it does.
  • Python documentation on generators and yield and the itertools module, for the streaming mechanics.
  • Object-oriented programming fundamentals covers the open-closed principle, Composite, and Specification as topics in their own right. Optional.
Report a bug