Part IV · Attribution and Reporting · Chapter 14

Detection

Commands only. What each step does, why it is built this way, and the judgment behind it are in the book.

Infrastructure tier 8 labs ≈ 4.5–7.5 h Pure Python · no Docker Windows · macOS · Linux

Labs in this chapter

What you'll be able to do

Keep a timeline of monitoring snapshots and diff consecutive frames into a raw change feed, using content identity to separate a real change from a cosmetic re-render.
Build a watchlist directly from Chapter 13's high-confidence operators, so attribution decides what the monitor watches.
Classify each change into a dark-web taxonomy and recognise the one that matters most, a watched operator resurfacing under a new mask.
Score every event by severity and confidence, boost changes that touch a watched operator, and correlate cross-surface duplicates into a single alert.
Grade the monitor against ground truth on recall, precision, and the false-alert count, watch a naive monitor bury its criticals, and state plainly where detection stops; it watches, it never acts.

LAB 14.1

The monitoring shift and the watchlist

0.5–1 h the shift changefeed.py snapshots over time

From one capture to a timeline

Ubuntu

$ ls labs/artifacts/detect-monitor/corpus/
$ ./lab detect selftest     # the whole detection engine self-tests offline — no Docker, no Tor

Expected

selftest: three snapshots diff into a raw feed; victim/site/mirror/clone/persona
          appearances and churn page diffs all surface  -> PASS
selftest: the watchlist is built from Chapter 13's high-confidence operators
          (1 operator, 1 watched key)  -> PASS
selftest: every landmark change types correctly and the 10 banner flips are
          cosmetic_churn; the resurfacing persona maps to a watched operator  -> PASS
selftest: criticals score critical, a watched market going dark boosts to high,
          churn is suppress, and hard-identifier events are high confidence  -> PASS
selftest: 4 mirror-duplicated events collapse into their origin, the
          origin event is kept, and the new_mirror alert survives  -> PASS
selftest: full precision 1.00/false-alerts 0/criticals 2/2; naive precision 0.36/false-alerts 12  -> PASS
  detect self-tests passed (changefeed, watchlist, classify, score, correlate, pipeline+scorer)

What the monitor watches

Ubuntu

$ ./lab detect watchlist

Expected

  the watchlist — the operators this monitor watches (from Chapter 13's high-confidence clusters):
    [high] op-1: BlackVault, NightHawk, RedLattice, n1ghthawk
             keys ['F19B7A0C4E82D5613FA0']  wallets ['bc1qsynth0laba00…']

Verify in the Docker host

Ubuntu

$ ./lab check 14.1

The detect-lab corpus is present, three snapshots (t1, t2, t3) of a monitored world, and every file carries the synthetic watermark
./lab detect selftest passes offline across all six modules and the scorer
./lab detect watchlist shows the watchlist built from Chapter 13's high-confidence operators, carrying the watched key F19B7A0C…

What this lab looks like

Terminal showing ./lab check 14.1 and its output

Terminal: ./lab check 14.1

Firefox rendering an HTML view for lab 14.1

Firefox on this lab's HTML view

Mousepad showing an artifact dump from lab 14.1

Artifact dump in Mousepad

LAB 14.2

The change feed

0.5–1 h the feed changefeed.py change vs re-render

Diffing the timeline

Ubuntu

$ ./lab detect feed

Expected

  the raw change feed (every difference between consecutive snapshots, before judgement):
    raw events: 22
    t1->t2  victim   Northwind        @RedLattice     countdown->countdown
    t1->t2  victim   Meridian         @RedLattice     teased->published
    t1->t2  victim   Coastal          @RedLattice     -->teased
    t1->t2  victim   Northwind        @RedLattice-m1  -->countdown
    t1->t2  victim   Meridian         @RedLattice-m1  -->published
    t1->t2  victim   Coastal          @RedLattice-m1  -->teased
    t1->t2  site     NightHawkMkt     @NightHawkMkt   -->-
    t1->t2  mirror   RedLattice-m1    @RedLattice-m1  -->-
    t1->t2  page     churn-1          @churn-1        -->-
    t1->t2  page     churn-2          @churn-2        -->-
    t1->t2  page     churn-3          @churn-3        -->-
    t1->t2  page     churn-4          @churn-4        -->-
    t1->t2  page     churn-5          @churn-5        -->-
    t2->t3  victim   Northwind        @RedLattice     countdown->gone
    t2->t3  victim   Northwind        @RedLattice-m1  countdown->gone
    t2->t3  clone    NightHawkMkt-x   @NightHawkMkt-x -->-
    t2->t3  persona  n1ghthawk2       @n1ghthawk2     -->-
    t2->t3  page     churn-1          @churn-1        -->-
    t2->t3  page     churn-2          @churn-2        -->-
    t2->t3  page     churn-3          @churn-3        -->-
    t2->t3  page     churn-4          @churn-4        -->-
    t2->t3  page     churn-5          @churn-5        -->-

A real change is not a re-render

Verify in the Docker host

Ubuntu

$ ./lab check 14.2

./lab detect feed produces 22 raw events across the two transitions
The five churn pages appear as raw diffs in both transitions, banner changed, content identity unchanged
The byte-identical mirror re-lists RedLattice's victims, so those events appear on both the origin and the mirror surface

What this lab looks like

Terminal showing ./lab check 14.2 and its output

Terminal: ./lab check 14.2

Firefox rendering an HTML view for lab 14.2

Firefox on this lab's HTML view

Terminal output captured during lab 14.2

Terminal output for this lab

LAB 14.3

Classifying events

0.5–1 h the taxonomy classify.py naming the change

The taxonomy

Ubuntu

$ ./lab detect classify

Expected

  each raw change typed into the dark-web taxonomy:
    typed events: 22
    t1->t2  deadline_slip        Northwind        @RedLattice
    t1->t2  publication          Meridian         @RedLattice
    t1->t2  new_victim           Coastal          @RedLattice
    t1->t2  new_victim           Northwind        @RedLattice-m1
    t1->t2  new_victim           Meridian         @RedLattice-m1
    t1->t2  new_victim           Coastal          @RedLattice-m1
    t1->t2  market_down          NightHawkMkt     @NightHawkMkt
    t1->t2  new_mirror           RedLattice-m1    @RedLattice-m1
    t1->t2  cosmetic_churn       churn-1          @churn-1
    t1->t2  cosmetic_churn       churn-2          @churn-2
    t1->t2  cosmetic_churn       churn-3          @churn-3
    t1->t2  cosmetic_churn       churn-4          @churn-4
    t1->t2  cosmetic_churn       churn-5          @churn-5
    t2->t3  withdrawal           Northwind        @RedLattice
    t2->t3  withdrawal           Northwind        @RedLattice-m1
    t2->t3  new_clone            NightHawkMkt-x   @NightHawkMkt-x
    t2->t3  operator_resurface   n1ghthawk2       @n1ghthawk2
    t2->t3  cosmetic_churn       churn-1          @churn-1
    t2->t3  cosmetic_churn       churn-2          @churn-2
    t2->t3  cosmetic_churn       churn-3          @churn-3
    t2->t3  cosmetic_churn       churn-4          @churn-4
    t2->t3  cosmetic_churn       churn-5          @churn-5
    ---
     10  cosmetic_churn
      4  new_victim
      2  withdrawal
      1  deadline_slip
      1  publication
      1  market_down
      1  new_mirror
      1  new_clone
      1  operator_resurface

The type that ties detection to attribution

Verify in the Docker host

Ubuntu

$ ./lab check 14.3

./lab detect classify types all 22 raw events, with 10 labelled cosmetic_churn
The new_clone (a look-alike market with a swapped key) and the operator_resurface are both present at t3
The resurfacing persona is typed operator_resurface because its signed key is on the Chapter 13 watchlist

What this lab looks like

Terminal showing ./lab check 14.3 and its output

Terminal: ./lab check 14.3

Terminal output captured during lab 14.3

Terminal output for this lab

Firefox rendering an HTML view for lab 14.3

Firefox on this lab's HTML view

LAB 14.4

Scoring and prioritization

0.5–1 h severity score.py how much it matters

Severity by type

Ubuntu

$ ./lab detect score

Expected

  every change scored by severity (watched operators boosted; churn suppressed):
    critical new_clone            NightHawkMkt-x   @NightHawkMkt-x
    critical operator_resurface   n1ghthawk2       @n1ghthawk2 *watched*
    high     publication          Meridian         @RedLattice
    high     new_victim           Coastal          @RedLattice
    high     new_victim           Northwind        @RedLattice-m1
    high     new_victim           Meridian         @RedLattice-m1
    high     new_victim           Coastal          @RedLattice-m1
    high     market_down          NightHawkMkt     @NightHawkMkt *watched*
    medium   deadline_slip        Northwind        @RedLattice
    medium   withdrawal           Northwind        @RedLattice
    medium   withdrawal           Northwind        @RedLattice-m1
    low      new_mirror           RedLattice-m1    @RedLattice-m1
    suppress cosmetic_churn       churn-1          @churn-1
    suppress cosmetic_churn       churn-2          @churn-2
    suppress cosmetic_churn       churn-3          @churn-3
    suppress cosmetic_churn       churn-4          @churn-4
    suppress cosmetic_churn       churn-5          @churn-5
    suppress cosmetic_churn       churn-1          @churn-1
    suppress cosmetic_churn       churn-2          @churn-2
    suppress cosmetic_churn       churn-3          @churn-3
    suppress cosmetic_churn       churn-4          @churn-4
    suppress cosmetic_churn       churn-5          @churn-5

The watchlist boost

Verify in the Docker host

Ubuntu

$ ./lab check 14.4

The two criticals, new_clone and operator_resurface, score critical and sort to the top
The watched market_down is boosted from medium to high and tagged *watched*
Every cosmetic_churn event scores suppress; the resurface and clone are high confidence

What this lab looks like

Terminal showing ./lab check 14.4 and its output

Terminal: ./lab check 14.4

Mousepad showing an artifact dump from lab 14.4

Artifact dump in Mousepad

Firefox rendering an HTML view for lab 14.4

Firefox on this lab's HTML view

LAB 14.5

Correlation and dedup

0.5 h dedup correlate.py one event, one alert

The same event on two surfaces

Ubuntu

$ ./lab detect correlate

Expected

  collapsing cross-surface duplicates (a victim mirrored across two onions is one alert):
    collapsed 4 cross-surface duplicates; 18 events remain

Why the origin event wins

Verify in the Docker host

Ubuntu

$ ./lab check 14.5

./lab detect correlate collapses 4 cross-surface duplicates, leaving 18 of the 22 raw events
The surviving Northwind event at t2 is the origin deadline_slip, not the mirror's spurious new_victim
The new_mirror alert survives correlation, a mirrored victim collapses, but the mirror itself is still reported

What this lab looks like

Terminal showing ./lab check 14.5 and its output

Terminal: ./lab check 14.5

Firefox rendering an HTML view for lab 14.5

Firefox on this lab's HTML view

Mousepad showing an artifact dump from lab 14.5

Artifact dump in Mousepad

LAB 14.6

Noise, drift, and the flood

1 h the flood adversarial expected vs anomaly

Expected churn is not news

Ubuntu

$ ./lab detect monitor

Expected

  running the watch loop (full monitor) — the alert stream an analyst reads:
    full monitor: 8 alerts  (suppressed 10 churn, collapsed 4 duplicates)
    CRITICAL  new_clone            NightHawkMkt-x   @NightHawkMkt-x
    CRITICAL  operator_resurface   n1ghthawk2       @n1ghthawk2 *watched*
    HIGH      new_victim           Coastal          @RedLattice
    HIGH      publication          Meridian         @RedLattice
    HIGH      market_down          NightHawkMkt     @NightHawkMkt *watched*
    medium    deadline_slip        Northwind        @RedLattice
    medium    withdrawal           Northwind        @RedLattice
    low       new_mirror           RedLattice-m1    @RedLattice-m1

The alert-fatigue attack

Ubuntu

$ ./lab detect monitor --naive

Expected

  running the watch loop (naive monitor) — the alert stream an analyst reads:
    naive monitor: 22 alerts  (suppressed 0 churn, collapsed 0 duplicates)
    unranked  deadline_slip        Northwind        @RedLattice
    unranked  publication          Meridian         @RedLattice
    unranked  new_victim           Coastal          @RedLattice
    unranked  new_victim           Northwind        @RedLattice-m1
    unranked  new_victim           Meridian         @RedLattice-m1
    unranked  new_victim           Coastal          @RedLattice-m1
    unranked  market_down          NightHawkMkt     @NightHawkMkt
    unranked  new_mirror           RedLattice-m1    @RedLattice-m1
    unranked  cosmetic_churn       churn-1          @churn-1
    unranked  cosmetic_churn       churn-2          @churn-2
    unranked  cosmetic_churn       churn-3          @churn-3
    unranked  cosmetic_churn       churn-4          @churn-4
    unranked  cosmetic_churn       churn-5          @churn-5
    unranked  withdrawal           Northwind        @RedLattice
    unranked  withdrawal           Northwind        @RedLattice-m1
    unranked  new_clone            NightHawkMkt-x   @NightHawkMkt-x
    unranked  operator_resurface   n1ghthawk2       @n1ghthawk2
    unranked  cosmetic_churn       churn-1          @churn-1
    unranked  cosmetic_churn       churn-2          @churn-2
    unranked  cosmetic_churn       churn-3          @churn-3
    unranked  cosmetic_churn       churn-4          @churn-4
    unranked  cosmetic_churn       churn-5          @churn-5

Verify in the Docker host

Ubuntu

$ ./lab check 14.6

./lab detect monitor emits 8 ranked alerts, both criticals on top, reporting 10 churn suppressed and 4 duplicates collapsed
./lab detect monitor --naive emits 22 flat unranked alerts with the two criticals buried mid-stream
Both monitors catch both criticals, recall is not the differentiator; reading order is

What this lab looks like

Terminal showing ./lab check 14.6 and its output

Terminal: ./lab check 14.6

Firefox rendering an HTML view for lab 14.6

Firefox on this lab's HTML view

Terminal output captured during lab 14.6

Terminal output for this lab

LAB 14.7

Scoring the detector and the watch loop

0.5–1 h the scorecard detect-scorer crying wolf

Grading the alert stream

Ubuntu

$ ./lab detect grade

Expected

  ran the monitor (full monitor); grading the alert stream against ground truth:
scored the alert stream against detect-lab ground truth
  alert recall           8 / 8      1.00
  alert precision        8 / 8     1.00
  false alerts           0          (noise shown to the analyst — the crying-wolf count)
  criticals surfaced     2 / 2      (at critical severity, ranked to the top)
  churn suppressed       10
  duplicates collapsed   4

The naive monitor and the crying-wolf count

Ubuntu

$ ./lab detect grade --naive

Expected

  ran the monitor (naive monitor); grading the alert stream against ground truth:
scored the alert stream against detect-lab ground truth
  alert recall           8 / 8      1.00
  alert precision        8 / 22     0.36
  false alerts           12          (noise shown to the analyst — the crying-wolf count)
  criticals surfaced     0 / 2      (at critical severity, ranked to the top)
  churn suppressed       0
  duplicates collapsed   0

Reflection

Verify in the Docker host

Ubuntu

$ ./lab check 14.7

The full monitor scores 8/8 recall, 8/8 precision, 0 false alerts, 2/2 criticals surfaced, 10 churn suppressed, 4 duplicates collapsed
The naive monitor's precision collapses to 0.36 with 12 false alerts and 0/2 criticals surfaced on the same corpus
The false-alert count, the number of times the monitor cried wolf, is the metric the chapter foregrounds

What this lab looks like

Terminal showing ./lab check 14.7 and its output

Terminal: ./lab check 14.7

Firefox rendering an HTML view for lab 14.7

Firefox on this lab's HTML view

Mousepad showing an artifact dump from lab 14.7

Artifact dump in Mousepad

LAB 14.8

Detection over a real change feed

Runs the standing watch against a genuine public change feed, where new victims appear at their own pace rather than on a script.

Written up in the book, commands and all.