push to main

Showcase · The Circus · 03

Connie

An open-source Go engine that scans public job boards concurrently, and remembers every posting she has ever seen. Job postings are hiring signals. Connie is the memory that makes them legible.

This page is a walkthrough of that engine: what goes in, what runs, and what comes back. The scanner is the Go binary in the public repo.

Go SQLite Concurrency Open source
Read the code ↗ Read the case study ↗
3
ATS platforms in the case study, behind one adapter. The README also names Workable
1
static binary, one direct dependency, cross-compiles anywhere
0
mutexes in the worker pool. Channel-owned state, proven by the race detector

What it does

A job posting is a buying signal: it says what a company is about to invest in, the day they decide it. The public, unauthenticated APIs of the major applicant-tracking systems expose the boards. A scraper that only fetches them tells you what exists. Connie keeps the memory, so the next run can say what changed.

The case study names Ashby, Greenhouse, and Lever behind one adapter interface, with more platforms at about 100 lines each. The public README lists those three and Workable. She is named for Connie Sachs of Tinker Tailor Soldier Spy, the researcher whose head was the Registry.

How it works

Inputs

  • Board lists. A fresh clone ships hand-curated seeds per ATS. discover can add slugs harvested from the Wayback Machine.
  • A filter: a fuzzy title, a case-insensitive regex over descriptions, or an unfiltered run that takes every listed posting.
  • A contact string in the User-Agent. The README asks for one, and for concurrency left at the default of 8.

Steps

  • Scan. A bounded worker pool fetches the boards. One aggregator goroutine owns all mutable state. No mutexes, proven under the race detector. A dead board is counted. It does not abort the run.
  • Filter. A title match runs in both directions. A description regex keeps only the matched fragments. Full descriptions are not stored.
  • Remember. The Registry is SQLite, one transaction per scan. Each posting carries first_seen, last_seen, and closed_at.
  • Close, carefully. Only a full unfiltered scan may declare a posting gone, and only on the boards that run actually covered. A posting that reappears is reopened.
  • Discover. Board slugs come from the Wayback Machine CDX index, then a shape filter, then a probe against the live ATS. The union of seeds and discovered boards never shrinks.
  • Export. Matching postings go out as CSV and JSON. query reads the Registry offline, including what is open and new since a window such as 24h.

The only direct dependency named in the README is modernc.org/sqlite, so the binary builds with CGO off. Live tests sit behind a build tag. CI stays on recorded fixtures.

Outputs

  • A CSV and a JSON file of the postings that matched.
  • A Registry line: how many postings are new, updated, or closed, and how many boards came back dead or in error.
  • An offline query of the Registry. Run on a schedule, the diff is who started hiring for what, the day it opened.

Sample output

Scan

The example printed in the README, for connie scan --title "revenue operations".

Boards
35/35.
Registry
6 new, 0 updated, 0 closed, in connie.db.
Export
6 postings written to connie-jobs.csv and connie-jobs.json.
Health
0 dead boards. 0 errors.
One row
ashby, render, Revenue Operations Manager, with the job URL.

Second run

The receipt in the case study. A different run from the README example above.

Scope
67 seed boards, three ATSes, zero errors.
What she found
A remote "AI GTM Architect (Revenue Operations)" posting that an Ashby-only sweep could not see.

Source

Written from the Connie case study and the public repo github.com/joelrmartinez/connie. The README's design notes cover the worker pool, the Registry lifecycle, and why the binary stays static.