Career-Ops + Offbeat Job Radar — end-to-end job search automation

Five-stage pipeline

Collect → Keyword Filter (9 checks) → Smart Filter (Haiku) → Evaluate → Apply

• Collect. Three independent fetchers — scan.mjs hits 17 careers-page APIs (Databricks, MongoDB, Snowflake, Anthropic, Perplexity, PhonePe, Groww, …) with India-only location filtering at scrape time; the /fetch-gmail-alerts skill ingests LinkedIn's daily alert emails via the Gmail MCP; the offbeat radar pulls Reddit (forhire / hiring / MachineLearning / LocalLLaMA, plus HN /jobs and "who is hiring") through a free old.reddit.com + redlib fallback backend that I wrote after Apify started costing real money.
• Keyword filter (9 zero-token checks). Company type (no services / consulting / staffing), experience gate (≤3 yrs), Python primary stack, India location, headcount >10, BGV-risk employers, posting freshness, domain (AI/ML/GenAI/LLM, not QA/PM/sales), salary floor (≥₹18L if disclosed). 80% of the noise dies here for $0.
• Smart filter (Claude Haiku 4.5). Survivors get a structured score 1–5 on role fit, level match, and company quality. ~$0.0005 per call; ~$1–3/month at current cadence. The garbage-page detector skips JS-only LinkedIn renders gracefully so the budget never burns on a 404.
• Evaluate. When I paste a URL, the /career-ops Claude Code skill runs a full A–G evaluation (role match, comp band, legitimacy, risks, recommendation) and writes a numbered report under reports/.
• Apply. On APPLY / LEAN-APPLY verdicts, the system tailors my .tex resume against the JD and drops it in output/resume_{slug}.tex — I just open Overleaf and download the PDF.

Auxiliary surfaces

• Dashboard on :8001. FastAPI + vanilla JS. Five tabs: Overview (status flow + workflow guide), Pipeline (filterable table with status filters, Haiku score badges, color-coded row borders, action buttons), Applications (the canonical applications.md tracker), Scan History, Actions (one-click run-scan / smart-filter / fetch-gmail). Each row's status moves through New → Filtered → Ready → Analyzed → Applied, with manual Skip from any state.
• Chrome extension. A Manifest V3 extension I keep in Brave + Chrome on every machine. Right-click a JD anywhere → POST to jackal:8001/api/extension/jd → the worker triggers /career-ops evaluation and resume tailoring against the captured page. Tailscale-wide, systemd auto-start, token-authed, env in ~/.career-ops-extension.env.
• Daily digests. Four Gmail digests a day at IST 09:30 / 13:30 / 17:30 / 21:30 — empty digests are suppressed. Idempotent via digests.jsonl.
• Cross-source dedup. Both scanner and Gmail parser dedup against pipeline.md + scan-history.tsv + applications.md on URL, LinkedIn job ID, and a company::role key — so the same job from a Greenhouse link and a LinkedIn alert collapse to one row.

Backend rewrite — Apify → free direct

The first version paid Apify $3.40 per 1,000 scraped items. The seen-set didn't help me because Apify bills per returned item, not per new one. After May 6 the pipeline runs on a fetchers/reddit-direct.mjs backend that hits old.reddit.com/.json directly with a redlib mirror fallback and adaptive backoff via recentFailureRate(). The dispatcher reads a REDDIT_BACKEND env to swap implementations; Apify only runs on explicit opt-in. HN is a separate hn-jobs.mjs that reads the official Firebase API and HTML-strips the linked YC company pages for context. Apify line item: $0/mo. Haiku: $1–3/mo.

Architecture highlights

What I learned

• Cheap filters before expensive ones, always. The 9-check keyword prefilter kills 80% of postings for $0 in CPU time. Letting Haiku see the survivors is what makes the monthly budget sit in the noise.
• Idempotent timers beat event buses. systemd timers + a SQLite seen-set is the entire scheduler. No queue, no broker, no DLQ. The fetcher can crash, retry, double-fire — the seen-set deduplicates outcome state without me writing any retry logic.
• Owning the integrations is the moat. Apify was a build-vs-buy decision that flipped the second the bill arrived. Writing reddit-direct.mjs took an evening; it's been free and stable since.
• Auto-tailor on LEAN-APPLY, not on APPLY. A separate behavior I learned from running this against my own pipeline: the cost of writing a tailored .tex upfront for borderline roles is tiny, and the friction of doing it later is what kills the actual application. Cheap to optimize the wrong direction; expensive to optimize the right one.

This pipeline is for me, on my homelab, scoring my own job options. But the architecture — fan-in collectors, cheap-then-expensive filters, idempotent timers, structured-scoring with a small model, escalation to a bigger one only on survivors — is exactly the shape of a production AI-ops system. I just got to use my own job hunt as the dataset.

Stack

Node.js · Claude Haiku 4.5 · SQLite · FastAPI · Python · systemd timers · Gmail MCP · Gmail SMTP · Chrome MV3 (Manifest V3) · Tailscale · LaTeX (Overleaf)