In January 2026, I was still experimenting with agents writing code in a handful of single-repository prototypes. By April, the team was collaborating, multiple agents were working on code, documentation, and contracts in parallel, and the workspace had quickly expanded to 14 repositories. Based on traceable engineering artifacts in the workspace, roughly 1.04 million lines of content accumulated over three months, with documentation, code, and contracts accounting for about 6:3:1. That number describes engineering scale, not productivity. What made me stop and look back was what happened once multiple agents entered real engineering at the same time: who defines the task, who owns write access, what makes a Review trustworthy, and whether the system can actually stop when the evidence chain breaks.
I eventually abstracted these practices into an Agentic SDLC control plane: agents act with context and boundaries, have an owner, carry verification evidence, and can connect to Issues, Git, CI, deployment, and runtime validation. The commercial value of Agentic Coding is not simply generating more code. It is making higher automation speed safe enough for a team to use.

Co-created with AI.

0x00 Three Review Rounds, 24 Findings, Only 2 Confirmed by Evidence

I decided to rewrite this article because of an AI Review that was not particularly flattering.

One PR in X-Pulsar—the X2 control plane and product entry point—went through three consecutive rounds of Spiral Review and accumulated 24 findings. Later, a human reviewer returned to the complete source, Git history, tests, and comment threads to verify them one by one. Only 2 entered the set of “confirmed real issues.”

That number could easily become a sensational headline, with the cause reduced to “model hallucination.” The actual data flow is more revealing.

At the time, a large diff exceeding 50,000 characters was split by file; if a single file was still too large, it was split again by line. Each chunk went to the model independently. The final merge pass saw conclusions already written for each chunk, not the original code evidence. More importantly, although the review runner could call tools, its working directory did not contain a checkout of the repository under review. It could not see the complete file, call sites, subsequent fix commits, or related tests. When a new push triggered another Review, the context also excluded explanations the author had already provided in the previous round.

The old pipeline and the boundary after the fix can be shown in one diagram:

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  subgraph before[Old path: main baseline]
    direction LR
    D1[PR diff] --> C1[50k file chunks]
    C1 --> M1[Model in Bot WORKING_DIR]
    M1 --> P1[Partial conclusions]
    P1 --> V1[Merge + verdict]
    V1 --> GH1[GitHub Review]
  end
  subgraph after[Grounded Review: PR-head checkout]
    direction LR
    H[repo + exact head SHA] --> CO[Partial clone + detached checkout]
    D2[PR diff] --> C2[50k file chunks]
    CO --> M2[Grounded model<br/>read / grep / history]
    C2 --> M2
    TH[Review thread] --> M2
    M2 --> P2[file:line conclusions]
    P2 --> V2[Merge + verdict]
    V2 --> GH2[GitHub Review]
  end
  GH1 -. New path .-> H
  classDef risk fill:#FFF0E8,stroke:#B85C2E,color:#44200F;
  class GH1, risk;

Figure 1: The old Chunk Review path and the corrected Grounded Review boundary. Event facts are bound to the corresponding PR and test snapshot.

The original implementation did one thing correctly: Chunk Review did not silently truncate the second half. The problem was that a resource boundary had been treated as a fact boundary. A helper might live in another chunk, a caller might sit outside the diff, or a problem might already have been fixed by a later commit; the local model would still fill in what it could not see using familiar defect patterns. A merge pass can normalize wording, but it cannot correct a shared false assumption using repository facts it never received.

That incident made me rename the output of a Review: the first pass can only produce a candidate finding. It may say, “this looks like a cache-isolation issue,” but it cannot be promoted to a blocking finding until it has read the real file, definitions, call sites, history, and tests.

The subsequent fix brought PR-head checkout, file:line grounding, historical Review threads, input validation, and a per-repository Git lock into the main path. In the corresponding verification snapshot, the relevant checks reported 307 passed and 7 warnings, and CI passed on Python 3.11 and 3.12. The first large grounded Review still lost a chunk at 319 seconds, so the budget for large chunks was raised separately to 600 seconds. Caller timeouts and cancellation were also made to terminate the underlying task, preventing a process that nobody was waiting for from continuing to consume resources.

Even then, it would be wrong to write that “the false-positive problem has been solved.” No public acceptance artifact was produced from a before/after replay of the original PR; the current implementation still falls back to diff-only when checkout fails; and candidate generation has not yet been separated from per-finding verification as a structured two-stage pipeline. A merged change, green tests, and proven Review precision are three different states.

Nor would I claim that the other 22 findings were individually proven false. There is only one accurate statement: of the 24 findings, repository-grounded human verification confirmed 2 real issues; the remaining findings did not enter the confirmed-issue set.

This became the starting point for the entire system. Models produce fluent judgments quickly, but before a judgment can become a team fact, the system must know which object and version it is looking at, whether it can return to the original evidence, and whether it really stops when that evidence breaks.

0x01 An Agent’s Entry Point Should Be the Task Fact, Not the Code

I used to start with a sentence like, “help me fix this problem.” The agent would find the files, change the code, run a few tests, and produce a polished summary ten minutes later. That works smoothly in a solo prototype. In a team, it leaves too much undefined: who set the goal, which repositories may change, which files are off-limits, what counts as acceptance, who accepts the risk, and who takes over after failure.

Today, work with side effects enters an Issue before it enters the code:

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  R["Task input"] --> A["CURRENT CONTROL<br/>identify authority · owner · risk<br/>in-scope / out-of-scope"]:::current
  A --> I["CURRENT CONTROL<br/>Issue stores stable facts<br/>context · acceptance · rollback"]:::current
  I --> X{"Cross-repo / contract change?"}
  X -->|"No"| C["Managed claim<br/>agent/session · branch · next step"]:::current
  X -->|"Yes"| PC["CURRENT PATTERN<br/>Parent + owner-repo Child Issues<br/>explicit merge topology"]:::current
  PC --> C
  C --> G["CURRENT CONTROL<br/>remote baseline + active lease<br/>branch / worktree"]:::current
  G --> W["Candidate change<br/>implementation + minimal commit"]:::candidate
  W --> V["Evidence gates<br/>tests · contract · benchmark<br/>grounded review"]:::current
  V --> P["PR / approval<br/>risk · rollback · unresolved items"]:::current
  P --> D["CONDITIONAL<br/>deploy provenance + runtime readback<br/>entered only when the claim requires it"]:::partial
  P --> H["Handoff<br/>HEAD · diff · checks · remaining risk"]:::current
  D --> H
  V -->|"Evidence conflict or budget exhausted"| STOP["Legitimate stop<br/>HITL / FAILED / narrow scope"]:::risk

  classDef current fill:#E9F7EF,stroke:#2F7D4A,color:#13351F,stroke-width:2px;
  classDef partial fill:#FFF5E5,stroke:#B7791F,color:#3B2600,stroke-width:2px;
  classDef risk fill:#FFF0E8,stroke:#B85C2E,color:#44200F,stroke-width:2px;
  classDef candidate fill:#E8F1FF,stroke:#3568A8,color:#10233F,stroke-width:2px;

Figure 2: The control chain from Task Fact to Delivery. Runtime Proof is a conditional stage entered according to the strength of the claim.

The Issue is not project-management decoration, nor does it mean opening a ticket for every one-line comment change. It stores the relatively stable facts of this task: context, in-scope, out-of-scope, acceptance criteria, verification commands, dependencies, risk, and rollback. An agent’s live heartbeat, branch, worktree, and next step should not repeatedly rewrite the Issue body. They belong in an updatable managed claim comment.

I care about three fields in particular.

• The first is scope. An instruction to “fix the login issue” may lead a model to rewrite an authentication helper, modify a shared protocol, add a cache layer, and then update three callers along the way. The code may not be wrong, but Review can no longer tell whether this is still the original task. Writing down both what may change and what explicitly must not change constrains the agent’s attention and the blast radius at the same time.

• The second is acceptance. “Tests pass” is not enough. A shared-field change must at least name its fixture, producer, consumer, and integration replay; a runtime fix must identify the source, deployed subject, and readback; a Review fix needs a replay of the original incident. The acceptance criteria determine how far the final claim may go.

• The third is risk. Who may approve a migration, deployment, deletion, external call, or policy enforcement? An agent may propose a plan and prepare a change in an isolated environment. It cannot expand its own authority simply because implementation went smoothly.

This practice comes from a habit I developed over years of security architecture work. When faced with an automated principal, I first ask about principal, object, action, scope, and the negative case: whose identity does it act under, which object can it affect, where should it fail first, and did state change after the failure? The same questions apply to a Coding Agent. Only the execution speed and concurrency surface are larger.

Cross-repository work also needs a parent/child split. The Parent Issue stores the overall goal, contract changes, dependencies, and intermediate compatibility states. Each owner repository uses its own Child Issue to manage its branch, tests, and PR. Git has no cross-repository database transaction, so merge order must be explicit: shared contracts and compatibility rules first, then backward-compatible producers and consumers, and only then integration replay, deployment, and runtime evidence.

An Issue cannot guarantee that an agent does the right thing. It performs a more fundamental operation: it turns a task from an ephemeral conversation into an object the team can review, take over, or reject. Without that object, the worktree, Review, and proof that follow have nothing to anchor to.

0x02 From 6 Repositories to 14: Why Context Became Infrastructure

X2 did not begin with 14 repositories or a complete design called “Context Engineering.”

In January 2026, I started writing CLAUDE.md files in a few early prototypes simply to avoid re-explaining startup commands, directory boundaries, and verification methods in every new session. By April, the repository count was expanding rapidly from single digits, a third person had joined, and Cursor, Claude Code, and Codex were all working across repositories at the same time. In the historical review baseline, there were about 6 active repositories in early May, 11 by mid-May, and the current 14-repository engineering map by the end of the month.

The first thing to fail at this stage was not code generation, but default consensus. When a new agent enters a repository, should it trust the README, STATUS, the roadmap, or yesterday’s session? Who defines a shared field? If product documentation says the path is connected but the deployed environment disagrees, who has authority?

This article mentions several internal X2 repositories. Readers only need four entry points: X-Pulsar is the control plane and product entry point; X2-Docs is the cross-repository authority and contract control plane; X2-Bot is the engineering automation execution plane; and X2-Orbit is the evaluation, evidence, and controlled-evolution plane.

X2-Docs appeared on May 4, right in the middle of rising multi-repository pressure. Its first commit added 67 files and 5,372 lines at once. This was not a retrospective summary; it covered authority, ADRs, integration contracts, AI coding context, Git, SDLC, testing, and Issue standards. Over the following week, it added fixtures for the shared protocol and contract layer, tag governance, a deployment overview, tool-access governance, local orchestration, multi-tenant standards, and benchmark scripts.

What mattered was what existed on day one: “who decides, what is shared, and how is it verified” came before further repository expansion. At the fixed 2026-07-11 baseline, X2-Docs contained 230 tracked files, 199 Markdown files, 15 unique ADRs, and 15 JSON contract fixtures. Of course, file count does not prove governance works. It only shows that governance has acquired an engineering surface that must itself be maintained and audited.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart LR
  JAN["2026-01 · CURRENT HISTORY<br/>CLAUDE.md appears in three early prototypes<br/>solving personal session cold starts"]:::current
  MAY4["2026-05-04 · CURRENT HISTORY<br/>X2-Docs established<br/>authority · contracts · Git · SDLC"]:::current
  MAY13["2026-05-13 · CURRENT HISTORY<br/>AGENTS.md synchronized across eight repos<br/>tool-neutral rules become fleet-wide"]:::current
  MAY19["2026-05-19 · CURRENT HISTORY<br/>STATUS / VIBE rolled out broadly<br/>implementation reality separated from working memory"]:::current
  MAY29["2026-05-20~29 · CURRENT HISTORY<br/>new repos born with a context set<br/>Orbit adds debt / proof protocols"]:::current
  JUL["2026-07 · CURRENT HISTORY<br/>Review · derived value · native proof retrospectives<br/>Evidence Engineering / PCD extracted"]:::current

  JAN -->|"More repositories and agents"| MAY4
  MAY4 -->|"Set standards first, then scale out"| MAY13
  MAY13 -->|"Rules alone cannot express reality"| MAY19
  MAY19 -->|"From remembered documentation to default protocol"| MAY29
  MAY29 -->|"From context to evidence"| JUL

  classDef current fill:#E9F7EF,stroke:#2F7D4A,color:#13351F,stroke-width:2px;

Figure 3: The evolution of the X2 workspace from a personal context habit to an evidence system. Dates come from a fixed historical-review baseline.

The context entry points then went through two concentrated rollouts. On May 13, eight core repositories added the tool-neutral AGENTS.md on the same day. Around May 19, most platform repositories filled in STATUS/VIBE files. Later cyber ranges, SIEM runtimes, and evaluation repositories were closer to being “born with an entry point.” Early repositories accumulated these files piecemeal over months; later repositories could load the same protocol at creation time. The default changed from “remember to add the documentation” to “without an entry point, the repository is not ready.”

We eventually separated information by how quickly it changes:

The fixed reading order for a Coding Agent is rules → implementation reality → working memory → cross-repository authority → current task → Git remote state. The last step cannot be skipped. A document may say “start from main” while the local main is already behind. An Issue may say to change a directory while the current checkout contains somebody else’s WIP. Context and Git must converge before work begins.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  A[Enter owner repo] --> B[AGENTS / CLAUDE<br/>behavior boundaries + verification entry points]
  B --> C[STATUS<br/>implementation reality + capability state]
  C --> D[VIBE CONTEXT<br/>recent working memory]
  D --> E[DEBT PROTOCOL<br/>historical guardrails]
  E --> F[X2-Docs<br/>cross-product authority / contracts]
  F --> G[Task Issue<br/>scope / acceptance / owner]
  G --> H{Freshness complete?}
  H -->|verified_at + commit + command| I[Current strong claim permitted]
  H -->|Missing or conflicting| J[Downgrade to documented record / pending review]
  classDef weak fill:#FFF0E8,stroke:#B85C2E,color:#44200F;
  class J weak;

Figure 4: The context-loading protocol and freshness branch. It defines the order of reading and downgrade behavior.

Calling a file STATUS.md does not make it true. Test counts, module status, and deployment records all go stale. I later began requiring critical status claims to include verified_at, source_commit, a reproduction command, and stale_after, along with does_not_prove. Without those fields, the language is downgraded to “documented record” or “pending review,” so the most polished old document cannot overrule newer code.

Platform authority also has boundaries. X2-Docs may decide a cross-product contract, but it cannot declare on behalf of a product repository that the runtime has been deployed. A product repository may describe its local implementation, but it cannot privately promote a new field into a shared protocol. What the context protocol truly solves is not “putting more tokens into the model.” It makes source, time, authority, and conflict visible.

0x03 Git Must Record More Than History; It Must Say Who Is Writing

When two agents change the same line, Git reports a conflict. The more dangerous case often comes without a red warning: Agent A changes a producer according to the old contract, while Agent B changes a consumer according to the new documentation. They touch different files, their tests pass independently, and Git can merge both changes automatically—yet the semantics have already diverged.

This is now my opening sequence for a Coding Agent:

git fetch origin --prune
git status --short --branch
git branch -vv
git worktree add .worktrees/<task> -b feat/<topic> origin/main

Each command solves a different problem. fetch refreshes remote references without touching current WIP. status exposes dirty state, branch, and upstream. branch -vv makes ahead/behind state visible. The new worktree gets its own index, HEAD, and uncommitted state, created from origin/main.

The principle is not “always pull first.” A dirty primary checkout may belong to somebody else. An automatic pull can introduce a merge, stash changes the scene, and reset makes a decision on behalf of the owner. The first responsibility of preflight is to decide whether this directory may serve as the write space for this task—not to force every directory into a clean state.

Worktrees are useful and frequently overestimated. They isolate the Git checkout, index, branch, and WIP. They do not isolate process permissions, network access, environment variables, secrets, or other directories. A runner in an independent worktree may still have excessive filesystem and deployment permissions. Git isolation and OS/container/VM, identity, and credential isolation are two separate control layers.

Once multiple agents are involved, the word “owner” is no longer precise enough. I split it into four questions:

CODEOWNERS only handles approval routing. At the 2026-07-11 remote-main baseline, 12 of the 14 repositories tracked it; the other two gaps were deliberately retained in the record. Even if all 14 had CODEOWNERS, it still would not know who is writing locally, nor would it stop two agents from modifying the same scope.

The history of X2-Docs shows that this ownership model was not designed in one pass. From May 11 to 13, actionable Issues were first routed back to their owner repositories, dirty default checkouts were forbidden from forced synchronization, and “one task, one branch, one worktree, one owner, one scope” was written into the Agent contract. In late May, CODEOWNERS evolved from coarse file-level rules into product/agent/data/infra ownership, with cross-review routing for workflows and builds. The lifecycle from discovered issue → local implementation → PR → issue closure was then made explicit.

Only in June did those written policies continue turning into active controls. Workspace audit, safe cleanup, product entry points, and agent-start preflight were connected. New worktrees had to start from origin/<default>, not a possibly stale local main. The shared integration harness could explicitly select an X-Pulsar worktree as its runtime subject. Codex, Claude Code, Bot, or Orbit sessions operating behind the same GitHub actor were also recorded in the managed claim comment. A second, overlapping claim standard was deliberately closed instead of letting two competing rule sets enter the main path.

That historical chain addresses task routing, checkout safety, scope, approval, lifecycle, remote baseline, runtime subject, and active writer. Compressing it into “we use CODEOWNERS and worktrees” hides the actual concurrency control required by multiple agents.

The active writer is completed by an Ownership Lease. A claim carries at least the task, agent/session, repository, branch, worktree, owned paths, heartbeat, expiry, state, and next step. Orbit’s reference implementation uses atomic current state for recovery and append-only events for claim, heartbeat, release, and conflict history. If claims in the same repository share a branch or overlap in paths, the system warns before a merge conflict appears.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart LR
  I[Issue body<br/>goal · scope · acceptance] --> C[Managed claim<br/>agent · session · repo · branch]
  C --> L[Lease<br/>owned paths · heartbeat · expiry]
  L --> W[Branch + worktree<br/>active write unit]
  W --> P[PR<br/>diff · checks · risk · rollback]
  P --> H[Handoff / release]
  L --> X{Same-repo branch or path overlap?}
  X -->|No| W
  X -->|Yes| STOP[Both stop writing<br/>task owner selects the sole writer]
  L --> T{Lease expired?}
  T -->|Yes| AUDIT[Inspect branch / PR / checks / worktree<br/>explain stale state before takeover]
  subgraph owners[Four ownership questions]
    O1[Task owner<br/>goal + acceptance]
    O2[Approval owner<br/>CODEOWNERS · reviewer allowlist]
    O3[Active write owner<br/>current write authority]
    O4[Execution identity<br/>actor · agent/session · provenance]
  end
  I -.-> O1
  P -.-> O2
  L -.-> O3
  W -.-> O4
  classDef warn fill:#FFF0E8,stroke:#B85C2E,color:#44200F;
  class STOP,AUDIT warn;

Figure 5: The Ownership Lease and four kinds of owner. The diagram shows which facts Git concurrency control needs.

An expired lease is not a license to overwrite somebody else’s work. It means a takeover audit is required: inspect the branch, PR, checks, and any accessible worktree, then explain why the old claim became stale. If both sides already have local changes, preserve both worktrees and let the task owner select the sole writer. A file-based lease is not a strongly consistent cross-host lock; network partitions and clock problems still require stronger coordination or human convergence.

At handoff, I expect at least git status, the exact HEAD, diff/stat and commit list relative to origin/main, the worktree list, checks not run, and the next step. The goal of handoff is not to make the directory look tidy. It is to let the next person or agent reconstruct the scene without guessing.

This is why Git ownership belongs in this article. The risk of multi-agent AI Coding is not limited to code quality. It also comes from compressing authority, approval, active write, and execution provenance into one vague owner field. As writing becomes faster, that ambiguity turns into an incident faster too.

Audit Real Capability Before Trusting Role Documentation

Ownership design also needs a reverse check: what the documentation says a component is responsible for is not the same as what the code allows it to do. At the fixed baseline, X2-Orbit’s remote Codex worker could modify product repositories, commit, push, open PRs, and reach merge/deploy paths. X2-Bot could also implement, test, and commit through Claude/Codex runners in product-scoped worktrees. An Orbit-to-Bot feed bridge already existed, although it was disabled by default.

This meant that the platform once had two write-capable automation owners, while documented authority had fallen behind real side effects. The finding did not prove that both systems had modified the same branch at the same time, nor did it reveal an overwrite incident caused by doing so. But it was enough to disprove the idea that role documentation naturally creates write isolation. CODEOWNERS governs approval, worktrees govern Git state, and leases govern the current scope. If two automation systems both possess commit, push, merge, or deploy capabilities, the platform must still specify which is the execution owner and under what conditions the other may only submit a candidate.

The target model narrows Orbit to evaluation, evidence, and recommendations, while Bot or an approved successor carries sole responsibility for automated writes. The path is recommendation → managed claim → scoped PR → checks → human review → deploy → Orbit revalidation. Ownership audit must begin with actual capability, not system names, organization charts, or original product positioning.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  subgraph NOW["CURRENT · fixed code baseline 2026-07-15"]
    O["X2-Orbit remote worker<br/>benchmark / gap + repo write<br/>commit · push · PR · merge/deploy paths"]:::partial
    B["X2-Bot SDLC<br/>Claude/Codex runners<br/>product-scoped repo write"]:::current
    D["x2-docs / product map<br/>original role narrative misses some side effects"]:::risk
    F["Orbit → Bot feed bridge<br/>implemented, disabled by default"]:::partial
    O --> R["RISK · two write-capable automation owners<br/>authority drifts from actual capability"]:::risk
    B --> R
    D --> R
    F -.-> B
  end

  subgraph TARGET["TARGET · single-writer responsibility"]
    OR["Orbit<br/>evaluation · evidence · recommendation only"]:::target
    REC["versioned recommendation<br/>no self-granted write authority"]:::target
    BO["Bot / approved successor<br/>sole automation execution owner"]:::target
    GOV["managed claim → scoped PR → checks<br/>human review → deploy"]:::target
    RV["Orbit revalidation<br/>same scenario + attributable delta"]:::target
    OR -.-> REC -.-> BO -.-> GOV -.-> RV
    RV -.-> OR
  end

  R -. "governance remediation; not incident proof" .-> OR

  classDef current fill:#E9F7EF,stroke:#2F7D4A,color:#13351F,stroke-width:2px;
  classDef partial fill:#FFF5E5,stroke:#B7791F,color:#3B2600,stroke-width:2px;
  classDef risk fill:#FFF0E8,stroke:#B85C2E,color:#44200F,stroke-width:2px;
  classDef target fill:#F3E8FF,stroke:#7E22CE,color:#32105C,stroke-width:2px,stroke-dasharray:6 4;

Figure 6: Orbit/Bot ownership self-audit. Red shows Current capability and authority drift; the dashed purple path is the Target single-writer model.

0x04 Why an Agent Needs a Legitimate Failure State

The easiest way for an agent to perform “autonomy” is to keep working forever. If a test fails, change the code again. If a benchmark regresses, switch prompts. If a deployment cannot read the result back, write an explanation. As long as output continues scrolling across the screen, it looks as if the system has not stopped.

The X2-Bot SDLC Engine breaks a task into explicit states:

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
stateDiagram-v2
  [*] --> Intake
  Intake --> Plan: scope / owner / acceptance
  Plan --> Implement: approved plan
  Implement --> Testing: candidate change
  Testing --> Benchmark: tests pass
  Testing --> RootCause: tests fail
  Benchmark --> Review: threshold satisfied
  Benchmark --> RootCause: regression
  RootCause --> FixLoop: cause confirmed
  RootCause --> HITL: evidence conflict
  FixLoop --> Testing: change within budget
  FixLoop --> Failed: budget exhausted
  Review --> Implement: confirmed blocking finding
  Review --> Report: no blocking finding
  Report --> HITL: residual risk acceptance
  Report --> Done: gates satisfied
  HITL --> Implement: continue / narrow scope
  HITL --> Done: accept bounded risk
  HITL --> Failed: stop
  Done --> [*]
  Failed --> [*]
  note right of RootCause
    Read-only evidence gathering first
    Fix write authority comes afterward
  end note
  note right of Review
    PR head + file:line + thread grounding
  end note

Figure 7: The constrained X2-Bot SDLC state machine. It expresses both control flow and legitimate stopping states.

The value of a state machine is not how tidy the diagram looks, but how it constrains the next action. TESTING answers only the specified checks. After failure, the task first enters read-only evidence gathering in ROOT_CAUSE, distinguishing code, environment, baseline, or the test itself, before FIX_LOOP receives limited write authority. BENCHMARK cannot be overridden by ordinary green tests. REVIEW proposes and verifies issues without casually taking over the implementation owner. HITL_PAUSED preserves the state from which it was entered so recovery can return to the correct boundary. FAILED is a legitimate endpoint; it does not need to be repackaged as “partially complete.”

X2-Bot’s commit history also shows that drawing the state enum is only the beginning. On the day the state machine landed, it was immediately followed by fixes for injection, Git timeout, resume, blocking I/O, memory bounds, scope, and lifecycle failure. Later fixes addressed cases where the lifecycle continued after the task had failed, and where an incorrect test command kept a loop from exiting. An exception may be written to a log without changing control flow, in which case the outside world still sees success.

A controllable loop must define at least the following:

state: root_cause
budget:
  attempts: 2
  elapsed_minutes: 15
  risk: read_only
evidence_input:
  - failing command
  - stack trace
  - current diff
  - relevant history
allowed_action:
  - inspect source and history
  - run targeted diagnostics
exit_condition:
  success: cause linked to reproducible evidence
  failure: evidence conflict or budget exhausted
failure_destination: HITL

A budget is not just tokens. It includes attempts, elapsed time, cost, and risk budget. A third repetition of the same failed method should not automatically trigger a fourth attempt simply because tokens remain. Fast ordinary tests also cannot automatically elevate authority for deployment, migration, or external actions. The central problem of Loop Engineering is keeping the loop inside its boundaries.

The Review incident that lost a chunk at 319 seconds added another lesson: timeout cannot be a single global number. Diff-only review, a small grounded pass, and a large chunk have different workloads and need different budgets. Raising the large-chunk timeout to 600 seconds was only half the fix. The other half was cancellation propagation. Once the caller has timed out or left, the underlying task must end too. Extending the time genuinely required and reclaiming a task that has lost its owner are two sides of the same loop decision.

A state machine is still not a sandbox. If the runner has excessive filesystem, network, or credential permissions, logically entering HITL cannot revoke capabilities already granted to the process. Process state, OS identity, and external side effects must be designed separately. Direct test coverage of the core lifecycle also remains limited, so I describe this state machine as an existing implementation that is still converging—not as a mature general-purpose framework.

0x05 Do Not Ask the Model Again About a Deterministic Problem

Another failure appeared in the generative Playbook learning path.

The system learned a new workflow from an execution trace. A domain name in the source task resolved to an IP address at runtime, but the Learner wrote that derived value into later nodes as if it were a stable parameter. When the workflow ran against a different target, the domain had changed while later actions still pointed at the old IP.

This was not a syntax error. The generated structure was complete and every field was valid. On the original target, the old IP might even continue to work. Asking the same model to “check it again” can easily reproduce the mistake of treating valid format and local success as semantic correctness.

The final fix did not use a longer prompt. It moved the deterministic part back into ordinary code:

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  T[Execution trace + provenance] --> L[Learn workflow candidate]
  L --> W[Shared recursive walker]
  W --> D[Detect derived values]
  W --> R[Deterministic repair]
  R --> S[Residual scan]
  D --> S
  S --> Q{Severe residue?}
  Q -->|yes| F[Fail closed<br/>preview / learn / save]
  Q -->|no| P[Persist candidate]
  P --> E[Execute on a different target]
  E --> RB[Runtime readback]
  UNIT[Unit: repair logic] --> P
  HIST[Historical replay] --> P
  classDef risk fill:#FFF0E8,stroke:#B85C2E,color:#44200F;
  class F risk;

Figure 8: Deterministic repair and residual scanning for Derived Values. The 8→0 result and changed-target replay are bound to this event snapshot.

Detection and repair share the same recursive walker and provenance. Otherwise, the detector may see a nested field while the repairer changes only the top level; the preview path blocks it, while learn or save lets it through. Severe residue must stop all three persistence paths, rather than being saved and left for runtime to report.

The verification result for this incident reduced hard-coded nodes from 8 to 0. The related tests reported 577 passed and 4 skipped, and the changed-target replay found no old-IP residue. That result is bound only to the fix snapshot. It does not mean every generative workflow is now safe, nor does it mean a future schema cannot introduce a location the walker does not yet cover.

Changed-target replay is essential. Success against the same target may only mean that the old value still happens to work. Changing the authorized target reveals whether a derived value has been frozen into the workflow. This practice later influenced my approach to Review and runtime proof as well: do not merely ask a producer to repeat itself under familiar conditions. Change the observation point and actively search for counterexamples and residue.

There is a broader division-of-labor principle here. Let the model handle work that requires semantic judgment, such as understanding user intent, proposing candidate steps, and explaining failure. Let parsers, schemas, walkers, provenance, and residual scans enforce deterministic invariants. A model may generate a candidate; it cannot use confidence to override a deterministic gate.

0x06 If You Can Read It Back, Why Is the Product Loop Still Not Proven?

We once had a proof that looked complete: the verifier wrote test data into the fact layer, then read it back through the serving interface using a stable ID.

The evidence was valid. It proved that ingest/read plumbing worked, that an ID could locate an object, and that the serving view could return a matching result. It did not prove that the product had ever produced that data itself.

If the verifier seeds the data and reads it back, the verifier is what forms the closed loop. When a report ends with “the product producer is connected,” it presents the verification script’s capability as product capability. This mistake is dangerous because every preceding step is true; only the claim exceeds the evidence.

I later separated delivery into four layers:

A merged PR, an active service, HTTP 200, and native readback all have value, but each proves only its own segment. Deployment checks must also answer which process/image is running, its start time, cwd, commit/digest, configuration, and data path. A deployment script may have been updated while the old process is still running, and the service can still appear active.

To see why cross-repository proof is so easy to assemble incorrectly, first look at the roles these repositories play:

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  subgraph product[Product and fact plane]
    P[Control plane + product entry point]
    N[Code analysis execution<br/>code evidence production]
    S[Runtime validation execution<br/>runtime evidence production]
    G[Shared protocol + contract layer]
    SEA[Fact storage + replay layer]
    C[Governance + evaluation boundary]
    P --> N
    P --> S
    N --> G
    S --> G
    G --> SEA
    SEA --> C
    C -. advice / policy .-> P
  end
  subgraph infra[Supporting capability plane]
    PRIME[Model execution gateway]
    MCP[Tool access layer]
    MIRROR[Reproducible cyber range]
    ARRAY[SIEM runtime]
    MIRROR --> ARRAY
  end
  subgraph engineering[Agent engineering + delivery plane]
    ORBIT[X2-Orbit<br/>evaluation · evidence]
    BOT[X2-Bot<br/>engineering automation]
    DELIVERY[Delivery + deployment boundary]
    DOCS[X2-Docs<br/>Authority · contracts]
    ORBIT --> BOT
    BOT --> DELIVERY
    DOCS -. contracts / standards .-> ORBIT
    DOCS -. contracts / standards .-> BOT
  end
  PRIME -. model execution .-> P
  MCP -. tools .-> P
  ARRAY -. alerts .-> P
  BOT -. Issue / PR .-> product
  DELIVERY -. deploy / readback .-> product

Figure 9: The three responsibility planes relevant to this article.

A stronger native producer proof needs continuous provenance and must be distinguished from a verifier writing and reading its own data:

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  subgraph seeded[Seeded readback]
    V[Verifier] -->|seed fixture| SEA1[Fact storage + replay layer]
    SEA1 -->|read by stable ID| V
    SOK[Proves ingest / serving plumbing]
    V --> SOK
  end
  subgraph native[Native producer proof]
    A[Authorized execution] --> PROD[Product native sender]
    PROD -->|message / trace ID| SEA2[Fact storage + replay layer]
    DEP[Source + deploy commit] --> PROD
    SEA2 --> READ[Independent readback]
    READ --> MATCH[Match message / finding / evidence]
    MATCH --> NB[bridge_writer: none]
  end
  SOK -. Does not prove producer invocation .-> PROD
  classDef limited fill:#FFF5E5,stroke:#B7791F,color:#3B2600;
  class SOK limited;

Figure 10: The evidence difference between Seeded Readback and Native Producer Proof.

bridge_writer: none rules out an intermediary script writing into the fact layer on the product’s behalf. An accepted ID connects producer and ingest. Deploy provenance identifies the artifact actually running. A stable ID lets another verifier read the object back, and matched evidence prevents it from accepting a different but similar object. “Authorized execution” is part of the proof as well; runtime validation without scope is not an acceptable delivery, even when it succeeds technically.

Cross-repository systems are especially vulnerable to compositional hallucination: one repository’s tests prove the sender, another proves ingest, a third example proves read, and a fourth demo proves a decision. Every segment is green, but the identity, trace, commit, and environment are not the same set. We therefore use first-breakpoint: if a path fails at its first trustworthy invariant, stop there and mark later stages not reached. Do not temporarily bypass authentication and then splice in another trace to manufacture a closed loop.

A proof chain in X2-Docs from June 29 to July 1 gave me a more complete example than 24→2. It did not begin with “the loop is complete.” It began with a real startup failure: the orchestrator used an obsolete bucket environment-variable name, the smoke test still called an endpoint that did not exist, and it sent an old envelope rejected by shared protocol and contract layer v0.2 with extra=forbid. After those problems were fixed, the report still retained a second embedding boot blocker. It did not describe “getting past the first error” as success.

Attributable evidence was added one slice at a time:

This chain contains several very ordinary details:

• Asynchronous landing requires polling, not a single GET.
• A seeded lane needs an unseeded control before the ratchet difference can be attributed to that fact.
• The same OutcomeRecord ID must cross provenance and the frozen consumer mirror; “success” on both sides is not enough.
• Enabling a feature flag still requires a flag-off lane.
• The behavioral path begins in shadow mode and records only would-have-blocked. Technical connectivity does not grant blocking authority automatically.

These are bounded proofs in an authorized local environment. They do not imply production scale, every tenant, long-term stability, or a complete security loop. Their value is precisely that every PR knows which additional segment it proved and why the next segment still cannot be declared.

A proof artifact retained over time must state its boundaries explicitly:

claim: product-native-evidence-readback
state: LIVE
snapshot: <event-date>
source_commit: <commit>
deployment_digest: <digest>
stable_trace_id: <redacted-reference>
readback_match:
  producer: true
  message: true
  evidence: true
bridge_writer: none
does_not_prove:
  - production scale
  - every deployment environment
  - long-term reliability
  - absence of unrelated regressions

does_not_prove is not a disclaimer. It prevents language from extrapolating one success in an authorized environment into a current capability of the whole platform. Not every PR needs to reach runtime; an ordinary change can reasonably stop at branch, tests, CI, and Review. But runtime provenance and same-trace readback cannot be omitted when automation will take further action, data will enter a long-lived fact layer, producer and verifier belong to different repositories, or the result will be used to increase an agent’s autonomy.

0x07 An All-Green Gate May Still Be Validating the Wrong Subject

Engineers are accustomed to trusting gates. A gate is more reliable than a meeting conclusion because it executes, fails, and blocks a merge. Yet the platform’s fixed-snapshot audit in July gave me a direct counterexample.

The legacy fixture gate in X2-Docs reported 15/15 passing fixtures. When the same set of objects was passed to the current shared protocol and contract layer 0.2.3 validator, the result was 0/15.

The green gate was not lying. It faithfully validated an obsolete compatibility subject. The problem was that verifier version, subject version, and fixture origin were not bound, so the pipeline continued to treat green results under the old rules as proof of compatibility with the current protocol.

Nor does this result imply that “the current validator must be correct” or “every integration is broken.” It proves something narrower and exact: the original blocking gate was not validating the current authoritative subject. A more complete gate must retain at least the following:

gate:
  verifier_version: <tool-or-package-version>
  subject_version: <schema-producer-consumer-baseline>
  fixture_origin: <authoritative-source>
  command: <reproducible-command>
  failure_effect: block-state-transition
  exception:
    owner: <risk-owner>
    expires_at: <timestamp>

failure_effect is easy to omit. A script can exit nonzero, but if a Makefile, CI wrapper, or parent Agent swallows the failure, the gate is still only a log line. An exception also needs an owner and an expiry; otherwise, a temporary compatibility allowance slowly becomes the new default protocol.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  LEGACY["AUDITED RISK · legacy fixture gate<br/>old verifier on 15 fixtures: 15/15 green"]:::risk
  CURRENT["CURRENT VALIDATOR · shared contract layer 0.2.3<br/>internal code: Gravity<br/>same fixtures: 0/15"]:::current
  LEGACY --> DRIFT["CONFIRMED DRIFT<br/>green gate validated an obsolete compatibility subject<br/>not current protocol compatibility"]:::risk
  CURRENT --> DRIFT

  DRIFT --> BIND["TARGET CONTROL · bind gate subject<br/>verifier_version + subject_version<br/>fixture_origin + reproducible command"]:::target
  BIND --> RUN["Run verifier"]:::candidate
  RUN --> RESULT{"Result"}
  RESULT -->|"pass"| E["Evidence artifact<br/>version · command · output · snapshot"]:::candidate
  RESULT -->|"fail"| BLOCK["failure_effect<br/>block state transition / merge"]:::target
  BLOCK --> EX{"Temporary exception needed?"}
  EX -->|"Yes"| WAIVER["risk owner + reason + expires_at<br/>revalidate at expiry"]:::target
  EX -->|"No"| FIX["Return to contract / producer / consumer fix"]:::target

  classDef current fill:#E9F7EF,stroke:#2F7D4A,color:#13351F,stroke-width:2px;
  classDef risk fill:#FFF0E8,stroke:#B85C2E,color:#44200F,stroke-width:2px;
  classDef target fill:#F3E8FF,stroke:#7E22CE,color:#32105C,stroke-width:2px,stroke-dasharray:6 4;
  classDef candidate fill:#E8F1FF,stroke:#3568A8,color:#10233F,stroke-width:2px;

Figure 11: From a green legacy gate to a version-bound gate. The 15/15→0/15 result proves only that the old blocking gate did not validate the current authoritative subject.

I used to apply Assurance mainly to code and agent output: testing implementation, Review conclusions, deployment, and runtime. This example shows that the verifier itself can drift. Tests, schema validators, benchmarks, architecture checks, report checksums, and even a state called “final” should all answer which object and version they verified—and who actually stops after failure.

Authority must also accept counterevidence. On June 19, X2-Docs documented a Sea replay path and tenant/org scope for Orbit A2. Five days later, Orbit E2E proof found that the contract disagreed with the Sea and Pulsar implementations. The fix did not force the products to conform to stale documentation. It changed the authoritative path back to the real get_replay, changed scope to tenant+user, and documented the Sea→Pulsar merge/deployment order. Platform text could become LIVE first while the product composition remained PARTIAL and AWS proof remained MISSING until deployment completed and the artifact was rebuilt. Authority decides where contract conflicts converge; it is not exempt from correction by runtime evidence.

A Finding Is Not a Conclusion; Counterevidence Defines the Impact Boundary

In July 2026, we conducted a platform audit against fixed commits from eight repositories and produced 96 independent current findings. The original input contained 43 P0/P1 findings, which were then given to an independent reviewer tasked with actively finding counterevidence. The result was 29 supported, 8 missing-counterevidence, and 6 overstated findings, with 11 severity corrections in total.

The most important number is neither 43 nor 11. It is the reviewer’s task definition. The reviewer was not there to normalize tone, nor to ask another model to reread the same summary. The job was to challenge the impact boundary: does the dangerous sink actually exist in source, is the fixed deployment path truly reachable, what identity and preconditions apply, which existing controls reduce impact, and which stages in the report never actually executed?

A finding can carry two different confidence levels at the same time:

Fact confidence   = whether the code or configuration fact is true
Impact confidence = whether that fact can produce the stated impact inside the current delivery and runtime boundary

Some findings retained the source-code fact but required a lower impact rating because of deployment counterexamples, unreachable paths, or missing preconditions. Others retained high priority because no counterexample strong enough to alter reachability was found. Counterevidence is not a defense brief for the system. It prevents security language from inflating “a dangerous statement exists” directly into “the complete impact has occurred.”

Independence does not simply mean switching models. If two agents share the same chunk, the same stale STATUS, and the same failure destination, they are only copying an assumption. Verifier independence can come from different evidence sources, permissions, and responsibilities: a PR author cannot self-approve; a Review candidate returns to source, history, and tests; producer output is checked through stable-ID readback; and a separate role actively searches for deployment counterexamples to a high-severity finding.

The audit also uses first-breakpoint. When a cross-product path fails at its first trustworthy invariant, later stages are marked not reached. The system does not bypass the failure just to draw a complete loop. The report may contain fewer attractive “end-to-end” claims, but every conclusion knows where it stopped.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  F["CURRENT AUDIT INPUT<br/>43 original P0/P1 findings"]:::current
  F --> FACT["Fact confidence<br/>does the dangerous statement / config / sink exist?"]:::candidate
  FACT --> REACH["Impact confidence<br/>is the deployment path reachable?<br/>do identity and preconditions hold?"]:::candidate
  REACH --> CTRL["Existing controls<br/>authorization · feature flag · isolation · approval"]:::candidate
  CTRL --> OUT["Observed outcome<br/>what actually executed?<br/>where was first-breakpoint?"]:::candidate
  OUT --> COUNTER["Independent counterevidence review<br/>actively seek deployment counterexamples + impact boundaries"]:::current

  COUNTER --> S["29 · supported"]:::current
  COUNTER --> M["8 · missing counterevidence<br/>no counterevidence yet strong enough to change the judgment"]:::partial
  COUNTER --> O["6 · overstated<br/>fact holds, impact wording too broad"]:::risk
  S --> DEC["Final risk decision"]:::current
  M --> DEC
  O --> DEC
  DEC --> CORR["11 severity corrections<br/>retain evidence chain + reason for adjustment"]:::current

  classDef current fill:#E9F7EF,stroke:#2F7D4A,color:#13351F,stroke-width:2px;
  classDef partial fill:#FFF5E5,stroke:#B7791F,color:#3B2600,stroke-width:2px;
  classDef risk fill:#FFF0E8,stroke:#B85C2E,color:#44200F,stroke-width:2px;
  classDef candidate fill:#E8F1FF,stroke:#3568A8,color:#10233F,stroke-width:2px;

Figure 12: How Counterevidence Review separates fact confidence from impact confidence. The numbers are bound to the fixed eight-repository audit.

The preceding three sections show why an evidence gate must bind its subject, version, and failure effect. If verifiers and gates can drift, then other control-plane artifacts—architecture diagrams, automated writers, and risk findings—cannot acquire Current status merely because they look complete.

0x08 All 33 Diagrams Rendered, but Not One Could Represent Current Unconditionally

In an earlier version of this article, I wrote a section called “Architecture as Code” in a tone that suggested a formal gate was already ready. After another audit, that claim no longer held.

The earlier diagram audit exposed gaps in the old Architecture-as-Code narrative. ADRs, a DSL parser, scope validation, Mermaid, and some scanners already existed, but there was no complete code-to-model reconciler at the time, and the checks were not part of required product CI. The ability to generate a diagram and pass syntax validation does not mean code relationships have been reconciled with the model.

The latest X2-Docs has advanced this work to foundation/advisory. A Structurizr desired graph, relationship identity, governed UML sequence/state projections, schemas, a renderer, asset closure, mutation tests, and uml-test / uml-check are now in place. Architecture as Code has moved from “a worthwhile prototype” to “a control plane ready for experimentation.”

We switched to a clumsier but more reliable method: freeze remote commits across 14 repositories, then compare each of the 33 existing C4 diagrams, PlantUML views, ecosystem maps, and target diagrams against entry-point code, deployment configuration, current status, and ADRs. The result was:

0 Accurate does not mean all 33 diagrams were worthless. Partial diagrams retained many correct boundaries, and Target-only diagrams could still communicate direction clearly. The real problem was missing state: a Python package was drawn as an independently deployed container; an asynchronous path was drawn as a synchronous call; a proposed API and L5 target entered the Current overview; a disabled-by-default bridge was shown as nonexistent; and an owner that had changed was still described using the old role.

Valid XML, a nonempty SVG, and a sharp PNG prove only that an artifact can be parsed. They do not validate nodes, edges, authority, deployment, or maturity. An architecture diagram is itself a claim-bearing artifact.

The review produced 22 Mermaid views with explicit state semantics and six code-audited closed-loop detail diagrams. We did not paint them all as “correct.” Instead, Current, Conditional/Partial, Risk/Broken, Target/Proposed, and Candidate/ADR Required were written into nodes, edges, and legends. Every published diagram also needs a subject snapshot, evidence source, and does_not_prove.

This implementation still has explicit boundaries. The architecture-uml CI job is present/advisory, runs on PR-controlled source, and is not trusted enforcement. Production sequence/state scenarios still need to be added, along with required status, a protected/trusted evaluator, runtime reconciliation, and product-level adoption. The toolchain must continue to accept counterevidence from real code, deployment, and product behavior.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  ART["Candidate architecture artifact<br/>C4 · PlantUML · Mermaid · ecosystem map"]:::candidate
  ART --> FREEZE["CURRENT AUDIT CONTROL<br/>freeze subject commits across 14 repos<br/>bind entry code · config · ADR · deployment"]:::current
  FREEZE --> CHECK["Check item by item<br/>node · edge · authority · protocol<br/>sync/async · deployed/package · maturity"]:::current
  CHECK --> CLASS{"Classify by evidence"}
  CLASS --> A["Accurate · 0"]:::current
  CLASS --> P["Partial · 7"]:::partial
  CLASS --> S["Stale · 16"]:::risk
  CLASS --> T["Target-only · 10"]:::target
  A --> PUB["Publish with<br/>snapshot · evidence source · owner<br/>expires_at · does_not_prove"]:::current
  P --> PUB
  S --> REPLACE["Correct / archive / redraw<br/>must not continue posing as Current"]:::risk
  T --> REPLACE
  REPLACE --> PUB
  PUB --> FOUNDATION["FOUNDATION / ADVISORY <br/>desired graph · relationship identity<br/>UML projections · source-only checks"]:::partial
  FOUNDATION --> ALIGN["NEXT CONTROL<br/>production scenarios · required status<br/>protected/trusted evaluator · runtime reconciliation"]:::target
  ALIGN --> RECHECK["TARGET CONTROL<br/>required / protected / trusted evaluator<br/>runtime reconciliation + product rollout"]:::target

  classDef current fill:#E9F7EF,stroke:#2F7D4A,color:#13351F,stroke-width:2px;
  classDef partial fill:#FFF5E5,stroke:#B7791F,color:#3B2600,stroke-width:2px;
  classDef risk fill:#FFF0E8,stroke:#B85C2E,color:#44200F,stroke-width:2px;
  classDef target fill:#F3E8FF,stroke:#7E22CE,color:#32105C,stroke-width:2px,stroke-dasharray:6 4;
  classDef candidate fill:#E8F1FF,stroke:#3568A8,color:#10233F,stroke-width:2px;

Figure 13: Architecture Artifact Assurance, not “if it renders, it is correct” Architecture as Code.

This changed what I expect from Architecture as Code. A DSL and reconciler are still worth building, but the word “code” does not grant them authority. Models, scanners, and gates all need to bind to a real subject; when the code changes, the diagram becomes stale again. Architecture artifacts must be rebuilt continuously, not generated once and treated as permanently correct.

0x09 At This Point, I Gave the Practice a Name

These mechanisms did not begin with a framework diagram.

CLAUDE.md came from session cold starts. X2-Docs came from multi-repository authority pressure. Ownership Lease came from concurrent writes. Grounded Review came from 24/2. Deterministic repair came from the old IP. Native proof came from seeded readback. Gate audit came from 15/15→0/15. Architecture artifact state came from the 33-diagram review.

Looking back, they fall into five classes of problem:

I call the fifth class extracted from the X2 practice Agentic Assurance Engineering. It designs engineering controls around an agent’s authority, ownership, constraints, evidence, and risk acceptance. It does not replace the other four. Fixing 24/2 required the right Review prompt, PR-head and thread context, a usable Git/grep/test harness, chunk timing and cancellation mechanics, and file:line grounding, replay, and verdict boundaries.

Evidence Engineering is the evidence subdomain of Assurance. It covers provenance, independent verification, runtime readback, counterevidence, and does_not_prove. It asks how a candidate is verified, at which layer it holds, and what fact would overturn it.

Proof-Carrying Delivery is a delivery mechanism. In plain language, delivery must carry evidence: a change does not become complete because a summary says so. It moves into the next state carrying task scope, remote baseline, tests, Review, deploy provenance, runtime readback, checks not run, and remaining risk. I did not force this name into another Engineering because it sits at a different level. Prompt, Context, Harness, Loop, and Assurance are five classes of engineering problem; PCD is a way to put part of them into the delivery chain.

X2 Agentic SDLC is the concrete implementation. X2-Docs handles platform authority. GitHub Issues and claims handle tasks and active owners. Branches and worktrees preserve isolated writes. X2-Bot turns a task into a state machine with failure exits. X2-Orbit handles evaluation and proof contracts. Product repositories execute real behavior and produce evidence.

%%{init: {'theme':'base','themeVariables':{'fontFamily':'Arial, PingFang SC, Microsoft YaHei','primaryColor':'#E8F1FF','primaryTextColor':'#10233F','primaryBorderColor':'#3568A8','lineColor':'#55708F','secondaryColor':'#E9F7EF','tertiaryColor':'#FFF5E5'}}}%%
flowchart TB
  P[Prompt Engineering<br/>how the task is expressed] --> C[Context Engineering<br/>what the agent sees]
  C --> H[Harness Engineering<br/>what it can call + how it runs]
  H --> L[Loop Engineering<br/>how it iterates]
  L --> GAP{Why may action and delivery become fact?}
  GAP --> A[Agentic Assurance Engineering<br/>authority · ownership · constraints · evidence · acceptance]
  A --> E[Evidence Engineering<br/>provenance · verification · readback]
  A --> PCD[Proof-Carrying Delivery]
  subgraph primitives[PCD primitives]
    direction TB
    subgraph row1[ ]
      direction LR
      CC[Candidate Change] ~~~ EG[Evidence Gate] ~~~ PA[Proof Artifact] ~~~ BL[Bounded Loop]
    end
    subgraph row2[ ]
      direction LR
      OL[Ownership Lease] ~~~ MT[Merge Topology] ~~~ HA[Human Acceptance]
    end
  end
  PCD --> CC
  PCD --> OL
  classDef method fill:#E9F7EF,stroke:#2F7D4A,color:#13351F;
  class A,E,PCD method;

Figure 14: The extracted method hierarchy—the latest practice built on top of Harness and Loop Engineering.

This hierarchy matters for AI Coding. It avoids inventing another name for every new incident, and it avoids reducing every problem to “evidence.” Whether an agent has authority to act is related to whether it can provide evidence for the result, but they are not the same question. A gate can produce output without being bound to the correct subject.

If these mechanisms remain confined to my workspace, they are only personal working habits. They gain product value only when they become stable interfaces, connect to existing engineering systems—effectively forming a new development process and a new SDLC—and run repeatedly across different teams.

I define Agentic Assurance Engineering not as another Coding Model, nor as an agent wrapped in a chat window, but as a control plane between Issues, Git, CI, Review, deployment, and runtime. Models can be replaced. The core of the engineering practice is task facts, authority boundaries, state transitions, evidence objects, and risk acceptance.

The first adoption layer connects this engineering practice to Issues, Git, and Review, providing advisory findings and handoff. The second connects contracts, tests, and policy gates to state transitions, allowing a merge to be blocked or an agent to be paused. The third adds deploy provenance, native producers, and runtime readback. Each layer creates value on its own, addresses a real problem, and retains a clear evidence boundary.

This practice is not primarily for teams that only want individual developers to generate a few more snippets of code. It is for organizations with multiple repositories, multiple agents, regulated delivery, or high-risk runtimes. What those organizations lack is not another model. They need an engineering product that can answer: who authorized the action, what can the action affect, which evidence is acceptable, and who takes over after failure?

For that reason, the moat around Agentic Coding should not be built on a particular model or prompt. Models will change, tools will be replaced, and every customer will have different repositories and CI. What transfers is the authority model, ownership protocol, evidence contract, policy gate, and runtime proof. Those are the stable core that can be extracted from internal engineering practice into a commercial product.

At the end of the process, I ask myself seven questions when working on AI engineering:

• Who defines the goal and shared facts?
• Under whose identity does the agent act, and what can it affect?
• Who is writing right now?
• Where should the system fail first?
• Does the evidence support source, CI, deploy, or runtime—and no further?
• Which counterexample would overturn the impact assessment?
• Who accepts the remaining risk, and who may increase autonomy?

These questions are not new. They come from principal, least privilege, separation of duties, negative testing, provenance, and auditability. AI changes execution speed, concurrency scale, and the way errors propagate. It does not repeal the basic constraints of security and software engineering.

For me, the most important change after AI Coding enters a team is not giving an agent more freedom. It is making sure every action is caught by a task fact, isolated by Git, claimed by an owner, verified by a gate, challenged by counterevidence, and stopped when the evidence chain breaks. Technical leadership is not the ability to tell a story without gaps. It is the ability to make the system state clearly what it can prove now, what it still cannot prove, and who owns the next step.

This is not a Git tutorial. This is a set of “anti-trampling” collaboration rules summarized from team collaboration and single-developer multi-Agent product iterations. Co-authored with AI.

0x00 Preface: Git Standards Now Face a Different Opponent

A few years ago, Git standards focused on branch naming, commit message format, and PR merging. Those old rules were designed for “slow-typing humans”. Back in 2018 when developing, we’d just shout at each other, and after review we’d push straight to main. Nobody imagined git worktree would actually become useful.

But now it’s different. In my personal projects, I routinely have Cursor, Claude Code, and Codex running simultaneously. // The previous article mentioned 8.4 billion tokens over 4 months; now 2 months later it’s grown to 20.4 billion tokens, a 2.4x increase in just two months. Also newly introduced Codex. ⚠️ Codex performs well on long-running tasks, especially debug scenarios.

These tools share one thing: they write code orders of magnitude faster than humans—and make mistakes just as fast. A real multi-Agent scenario looks like this:

• Claude Code modifies schema in worktree A
• Cursor modifies consumer in worktree B
• Codex “helpfully” touches the contract file in the main directory
• You look back and find three local commits on main, nobody remembers who pushed them
• Some agent does git stash and hides another engineer’s uncommitted work

At this point, Git standards aren’t about “pretty history” anymore—they’re about: making all concurrent work locatable, isolated, reviewable, verifiable, and rollbackable.

Simply put, the faster AI writes code, the more teams need to slow down destructive actions and clarify responsibility boundaries. Below are the rules I’ve summarized from months of trial and error, in four main chapters, ending with a checklist summary.

0x01 Workspace Isolation: From main to worktree

The easiest thing to lose control of in multi-Agent concurrency isn’t model capability—it’s ownership ambiguity. Two agents edit the same contract file simultaneously, whose diff is correct? Nobody knows. So the solution isn’t more complex verbal agreements, but building isolation into the workflow structure.

1. main Is for Integration Only, Not Development

The first rule is simplest and most easily violated: don’t develop directly on main, and definitely don’t let AI modify code on main. main should serve only one purpose—integration branch, representing the team’s currently accepted integration state, not anyone’s or any Agent’s temporary scratch pad.

Recommended local configuration:

git config pull.rebase true
git config pull.ff only
git config branch.main.rebase false

Before syncing main, diagnose first, then act:

git fetch origin --prune
git status --short --branch
git rev-list --left-right --count main...origin/main

If both local main and remote origin/main have moved forward, don’t pull directly on dirty main to resolve conflicts. The correct approach: create an isolated integration worktree, replay the target branch from origin/main, run tests, check diff, verify it passes, then merge.

This is the first “slow motion” in the AI era—any conflict resolution on main must assume there are N agents concurrently writing on the other end.

2. One Task = One Branch + One Worktree + One Owner + One Scope

Physical isolation starts with worktree area allocation. My local repo defaults to this structure:

repo/
  .claude/worktrees/   // Reserved for Claude Code
  .cursor/worktrees/   // Reserved for Cursor
  .worktrees/          // Codex or general tasks

Worktree directories are ignored via .git/info/exclude to avoid accidentally committing local execution environments (venv, node_modules, temp caches).

Branch naming should carry source and intent—it’s an index in collaboration, not decoration:

codex/<scope>-<task>
claude/<scope>-<task>
cursor/<scope>-<task>
docs/<topic>
feat/<topic>
fix/<topic>
backup/<topic>-<date>

Avoid names like update, fixes, wip, new-code that reveal nothing about responsibility or boundaries. You should be able to tell at a glance which agent created a branch and what scope it modifies.

Finally, the task assignment contract—giving AI a natural language goal is far from enough. A controllable AI coding task needs at least these fields clearly defined:

This contract does two things: limits AI’s write scope (reduces “drive-by refactoring” and unrelated file drift), provides reviewers judgment criteria (does this change respect scope).

Parallel agents are only suitable for tasks with non-overlapping write scopes. One agent modifying docs, another modifying adapter tests, a third reviewing contract fixtures—that’s safe splitting. Two agents simultaneously editing one contract file, or one modifying schema while another unknowingly modifies consumer—that’s a landmine.

0x02 Three Checkpoints: Need Reproducible Evidence Before Editing, Before Commit, Before PR

“Looks fine” has no value in multi-Agent collaboration. Every stage needs reproducible evidence, otherwise there’s no way to replay when problems occur.

Before editing:

git fetch origin --prune
git switch main
git pull --ff-only
git worktree add .worktrees/<task-name> -b <branch-name> origin/main
git status --short --branch    # Must be clean
git branch -vv

During development:

git status --short
git diff --stat
git diff --check    // This line specifically catches trailing whitespace, mixed space-tab, etc.

Before commit:

git diff --name-only
git diff --cached --name-only

If the staged file list contains paths you didn’t expect—stop, reassess. AI often “helpfully” touches files outside scope.

Before PR:

git fetch origin --prune
git log --oneline origin/main..HEAD
git diff --name-only origin/main...HEAD
git diff --check

The purpose of these commands isn’t ceremony, but to show reviewers three things: this PR only changed expected files, diff has no low-level format errors, commit history only contains commits this task should bring (no “smuggled contraband”).

PR Template: Must Be Readable

In AI-involved projects, small PRs are 10x more important than large automated output. PRs must be small enough that a human reviewer can actually read them completely. Each PR should at least include:

## Summary
- what changed

## Dependency
- dependency PRs, branches, or versions

## Test Plan
- [x] command actually run

## Risk / Rollback
- risk and rollback path

AI-assisted work needs additional clarification: which agent/tool wrote it, who the branch owner is, what verification commands were actually run. The key is “actually run”—AI writes “all tests pass” in the PR description, but reviewers pull it down and find dependencies aren’t even installed—this happens far more often in multi-agent projects than you’d imagine.

Default merge strategy is squash merge—one PR corresponds to one commit on mainline, easier to revert, easier to generate release notes. Only use rebase merge when the commit sequence itself has clear review value. In multi-agent collaboration, clean mainline history is more valuable than preserving every AI’s intermediate attempts.

0x03 Destructive Actions Must Be Slow Motion

The faster AI writes code, the slower destructive Git operations must be. List high-risk actions and their “brakes”:

Before stash, always check git status --short and git diff --stat. If changes might contain files belonging to another agent or engineer, stop and identify owner:

git worktree list
git branch -vv

Default strategy for worktree cleanup is—when uncertain, keep it. When AI sessions end, tools often ask whether to delete worktree, default answer is “keep”, unless all five pass:

git status --short                              # clean
git branch -vv                                  # no unpushed commits
git worktree list                               # nobody using it
gh pr list --state open --head <branch-name>    # no open PR
git log --oneline origin/main..<branch-name>    # valuable commits already pushed or merged

Keeping review context is usually far more important than saving a bit of local disk space.

0x04 AI Context Is the Real “Starting Line”

Git isolation solves workspace problems, AI context solves understanding problems. These are two different dimensions, but many teams only solve the former.

Many AI accidents aren’t because models can’t write code, but because they write seemingly reasonable code based on incorrect reality. For example: old architecture docs say a certain module still exists, but it’s long deleted from code; AI looks at old docs and keeps calling it, discovers import failure when running; then it starts “confidently” creating a fake module to fill the gap.

Every product repo should provide lightweight context so all tools start from the same set of facts:

AI session reading order should also be fixed: read product current context (VIBE_CODING_CONTEXT.md) first, then status (STATUS.md), then agent instructions (CLAUDE.md / AGENTS.md), cross-product then read platform docs and contracts. STATUS.md conflicts with old architecture docs? Follow STATUS.md. Documentation rot is normal, giving one clear “source of truth” is more honest than pretending all docs are correct.

Cross-repo changes cannot be solved by “opening several PRs simultaneously”. Recommended order: dependency repository PRs → product repository integration PRs → documentation index or release notes → cleanup branches. For submodule pointer changes, must first commit, push, PR merge inside submodule, then update parent repository pointer. The essence of cross-repo collaboration—make dependency direction clearly visible in Git history too.

0x05 Summary

The Git standard for multi-Agent collaboration boils down to one sentence—draw clear responsibility boundaries for tools. Next time before starting multiple agents, scan this table:

Looking back, none of these rules are “AI-only needed”—they were always good engineering practices. AI just turned them from “recommendations” into “requirements”. Because when humans make mistakes, there are still a few seconds of hesitation before commit; when AI makes mistakes, from thought to disk write might be under a second.

When these rules become the default workflow, multiple IDEs, multiple agents, multiple engineers are no longer concurrent writers trampling each other, but collaborators who can work in parallel, verify independently, and ultimately integrate in order.

References

• git-worktree official documentation
• Trunk-Based Development
• Conventional Commits
• Vibe Coding Survival Guide
• AI Software Engineering Practices: Building Enterprise-Level Agentic SOC Platform

Fair warning upfront: this is not a puff piece about “how amazing AI coding is.” This is hard-won experience from real money spent, 443 real project sessions, 8.4 billion tokens, and countless times getting completely wrecked by bugs. I originally wanted to split this into a Vibe Coding deep-dive series with each section as its own article, but honestly don’t have the bandwidth. Article co-created with AI. Personal writing style here, learned from my own blog posts over the past two years.

0x00 These Numbers Are Real

Over the past four months, I built three projects of different scales and types using Cursor and Claude Code:

• AgenticaSoC — an AI platform for a Security Operations Center (SOC), with LangGraph orchestration + multi-agent + multi-layer memory architecture (PostgreSQL/Qdrant/Redis)
• Blinds — an autonomous security research platform combining SAST/DAST and LLM reasoning, used for solving CTFs and finding vulnerabilities
• up-cli — an “AI coding tool to manage AI coding,” i.e., letting AI manage its own development workflow

I wrote a script to crunch all the historical data. Looking at the billing and token counts, I felt nothing:

UNIFIED TOKEN USAGE REPORT

8.4 billion tokens, 38,948 messages, 58 projects, 547 Cursor conversations, 204 Claude Code sessions.

Conversation breakdown across the three core projects:

Models used: on the Cursor side, claude-4.5-opus-high-thinking (primary, 162 sessions), gemini-3-pro (109 sessions, 34.6%), claude-4.6-opus-high-thinking, claude-4.5-sonnet-thinking, gemini-3.1-pro, gemini-3-flash; on the Claude Code side, claude-haiku-4-5, claude-opus-4-5, claude-opus-4-6.

The craziest single day (January 10, 2026) burned through 410 million tokens (Claude Code) // The proxy I was using at the start didn’t have cache read and write for Claude, and I was using it to organize docs which caused such high consumption.

I also ran deep quantitative analysis on all 443 sessions using visualize_deep.py. Here’s something that’ll flip your worldview:

Vibe Coding is astonishingly effective when there’s structure. And astonishingly bad when there isn’t.

89.8% is the overall success rate — sounds okay. But behind it sits a massive gap between AgenticaSoC’s 97.4% and up-cli’s 66.7% — and that gap isn’t a model capability gap, it’s an engineering methodology gap.

Caption: Success rate distribution across 443 sessions using different tech stacks. Structured frameworks (green) consistently sit in the “safe zone” of high success rates, while unstructured scripting and file operations (red) frequently fall into the “danger zone.”

Look at this chart. Sessions using heavy frameworks like FastAPI, Docker Compose, and LangGraph had almost 100% success rates. This reveals the first truth: what determines AI performance is not how simple the task is, but how much “scaffolding” you gave it.

Is this what they call Harness Engineering?

0x01 Three Failure Archetypes: How Does AI Actually Crash?

Analyzing the 45 non-successful cases out of 443 sessions, I found that almost all failures collapse into three archetypes.

1. Failure Archetype One: The Sandbox Wall

Source: up-cli, ~60% of all failures

This is the most absurd and most real failure pattern. up-cli is a meta-tool — “use AI development tools to manage AI development” — where the AI Agent runs in a constrained sandbox environment, but plans as if it has full filesystem write access.

This death loop appeared:

[Step 1] AI tries to create directory → Permission denied
[Step 2] AI retries → Permission denied
[Step 3] AI retries → Permission denied
...
[Step 14] Session exhausted, nothing accomplished

Real failure records:

• "Attempting to create a new subdirectory was blocked by environment restrictions"
• "Unable to create the test files due to file system permission restrictions"
• "The assistant repeatedly requested permission or retried the same operation"
• "The session entered a loop where the assistant repeatedly requested permission"

The data doesn’t lie:

• up-cli with Cursor (human oversight): 93.3% success rate, avg 33 messages
• up-cli with Claude (autonomous): 42.4% success rate, avg 175 messages

Most of those 175 messages were pointless retries. This shows up directly in the message stats — up-cli’s average message count is 107.3, while AgenticaSoC is only 38.7.

2. Failure Archetype Two: Context Window Exhaustion

I shouldn’t have tried the Ralph Loop, and I definitely shouldn’t have built tools to let tools Loop themselves — it led directly to a lack of code review and a flood of useless commits

Across all projects, ~25% of non-successes

This is the price of growth. When you evolve from “watching every AI step” to “letting AI run a long stretch autonomously,” new failure modes emerge.

The death curve:

Behind these wild numbers are countless times helplessly watching AI spiral toward collapse. I call it “context poisoning”:

Caption: As session length grows, the green zone representing success drops off a cliff, while red (failure) and gray (abandoned) zones expand rapidly.

As shown, there’s a fatal “circuit breaker boundary” (around 80-100 messages). Cross that line and AI becomes like a programmer who’s been up for three straight days — loses the big architectural picture and starts “whack-a-mole” bug fixing. Eventually tokens overflow, session dies.

P90 message count for non-successful sessions: 261. P95: 379. For successful sessions, P90 is only 102, P95 is 151. // Note: P90 means 90% of sessions are below this number.

When the context window is packed with code, error messages, and fix history, AI starts Whack-a-Mole — it fixes symptoms rather than causes, because it can no longer see the global picture. The worst non-successful session consumed 526 messages.

3. Failure Archetype Three: Security Filter / Capability Ceiling

Blinds project, ~15% of non-successes

Blinds is an offensive security tool — its core function is finding vulnerabilities and writing PoCs. But models have strong built-in resistance to “generating exploit code.”

Interestingly, the data showed a reversal:

• Blinds medium complexity (CTF/exploit sessions): 72.5% success rate
• Blinds high complexity (architecture/LangGraph pipeline sessions): 90.9% success rate

Higher complexity actually had better success, because those sessions were building tools, not triggering filters.

0x02 The Most Absurd Data Point: High-Complexity Tasks Have the Highest Success Rate

This is the Complexity × Outcome matrix from visualize_deep.py:

If the table isn’t clear enough, let’s throw all projects into a 2D space of “efficiency vs. success rate”:

Caption: X-axis is code lines produced per message (efficiency), Y-axis is success rate. Large projects (purple circles) tend to cluster in the upper-right “high-efficiency zone.”

Why do the high-output, high-success “big circles” (like AgenticaSoC) in the upper-right corner all come from high-complexity tasks?

Because when a task is labeled “high complexity,” you proactively enable full engineering standards: actually writing PRDs, assigning Task IDs, attaching automated tests, doing Git checkpoints.

Conversely, when a task is labeled “low complexity,” you might just throw AI a line: “fix this bug.” And then AI starts burying its head in the sand. Those seemingly simple “small fixes” in the lower-left corner are actually most prone to failing due to casual “vibe.”

This also creates another problem: Vibe Coding generates massive amounts of code that feels great, but it desperately needs experienced professional validation.

The per-project data is even more stark:

• AgenticaSoC high complexity: 98.8% success rate (most structured project)
• up-cli low complexity: 55.0% success rate (least structured context)
• Blinds high complexity: 88.2% success rate (structure can even save the hardest projects)

Conclusion: Structure is not the enemy of speed — structure is the prerequisite for speed. A workman who would do his work well must first sharpen his tools.

0x03 The Seven Deadly Sins of Large-Scale Vibe Coding

Through deep examination of AgenticaSoC’s Git history, 85 changelog files, and real code, I (AI) identified seven “hidden killers” unique to Vibe Coding. Harmless in small projects, lethal in large ones.

Large project comprehensive analysis matrix: Full-dimensional comparison of AgenticaSoC / Blinds / up-cli across success rate, message efficiency, code output, and failure patterns. These three data lines are the soil the seven sins grow in.

The seven deadly sins look like seven independent bad habits, but they share a common root — every conversation with AI starts from zero, with no one enforcing consistency across sessions. Each AI session produces locally correct output, but globally the codebase accumulates contradictions, dead ends, and ghost features. The biggest risk isn’t bad code, it’s consistency drift.

1. First Sin: “Declare Victory, Then Fix Forever” (Quick Win)

“Code exists” does not equal “code works”

Real evidence: On January 23, 2026, the changelog directory got 30+ files in a single day:

phase-1-2-3-complete.md
phase-1-2-3-complete-detailed.md
phase-1-2-3-complete-final.md
phase-1-2-3-final-status.md
implementation-complete.md
deployment-ready-summary.md

Then over the following weeks, a string of fixes arrived:

• 2026-01-28 — execution-flow-fixes.md (DbTool.tool_id column doesn’t exist)
• 2026-02-02 — fix-fake-metrics-data.md (dashboard showing fake model names)
• 2026-02-03 — agent-stability-improvements.md (Event loop crash)
• 2026-02-04 — report-display-fix.md (report written to wrong field)

Git log tells the real story: first “Phase 1: 95%, Phase 2: 80%”, then “Implementation complete - Ready for production”, then more fix commits. Phase 1’s “100%” actually validated at about 85%.

// Note: The embarrassing thing is I actually believed it when it said 100% complete. I was kind of dumb.

2. Second Sin: Ghost Modules (Dead Code)

The “right way” nobody uses, and the “wrong way” everyone depends on

The codebase had multiple files “created but never wired in”:

Meanwhile, existing endpoints kept using ad-hoc offset/limit parameters and inline error handling, completely ignoring these “correct tools.”

When you ask AI to “add pagination support,” it perfectly generates the utility module. It looks done. But migrating existing endpoints to the new module — that boring, repetitive integration work — never happens. You end up with two parallel systems. It’s like buying a bunch of advanced security products and then realizing, holy shit, nothing’s plugged in.

Cross-analysis of code volume, session count, and success rate. Note up-cli’s “messages per K lines of code” metric — 249.7 messages to produce 1,000 lines of code, while Blinds only needs 25.9. Ghost modules and missing integration work are part of why up-cli needs so many “conversations” per thousand lines.

3. Third Sin: Cross-Session “Frankenstein Architecture” (Always Win)

Can’t remember across sessions, but I still need to win again. (Quick Win becomes Always Win) — just keep winning! Winning streak!

.cursor/rules/backend-style.mdc clearly states:

Use Synchronous def for route handlers… Do NOT mix async def with blocking Session calls.

But the actual code:

async def get_agents(
    db: Session = Depends(get_db),  # sync session inside async handler

This blocks the event loop. And database_async.py with a complete async engine is right there, unused except for one health check.

Every AI conversation starts fresh. First session creates async handler, second creates sync DB layer, third adds async DB support “for the future.” No one enforces consistency across sessions. The codebase becomes a geological layer of conflicting AI decisions. I strongly suspect AI has zero cross-session memory — it’s turned the codebase into a total Frankenstein.

// Note: This is also why I keep trying in large scaffold projects to inject Skills that force-read external memory libraries to look up error repair decisions, to address the Vibe Coding issues of large projects.

4. Fourth Sin: The Big Bang Rewrite

Generating tons of code for refactoring is a self-validating, highly satisfying activity for the model — but it doesn’t care whether the rewrite actually works.

The two most instructive commits in Git history:

a025a4e revert: restore terminal MCP server to v5.0 for better reliability
86ef6a1 upgrade terminal mcp with SOC-focused architecture

An entire “SOC-focused” Terminal MCP Server rewrite (~8,900 lines) was committed, then completely reverted in the next meaningful commit because it broke stability.

AI is extremely good at generating grand refactors. It will happily rewrite entire modules with “better architecture.” But large-scale AI-generated rewrites have higher regression risk — AI has no evaluation and validation of all the subtle integration points. The seductive ease of generating 8,900 lines of code pushes you toward Big Bang rewrites rather than incremental improvements.

It took two days of debugging and I seriously wanted to smash my keyboard, only to find the rewrite was a pile of garbage. Many Vibe Coders, when facing bugs introduced by a big rewrite, try to keep having AI fix them (Fix Forward), ultimately sinking hundreds of messages. But knowing when to decisively git revert is absolutely the more important survival skill.

5. Fifth Sin: Fake Data That’s Too Realistic

Fake data that looks more real than real data!

This comes from the 2026-02-02-fix-fake-metrics-data.md fix record. The dashboard was showing fake model names — Gemini 2.0 Flash, Claude 3.5 Sonnet — with fabricated performance metrics and usage statistics. Looked perfect and professional.

Root cause: The backend hadn’t implemented the /stats/agent-metrics endpoint, and the frontend’s fallback function computeAgentMetrics() generated realistic-looking fake data to “fill” the UI.

Human developers write placeholders with “TODO” or “xxx.” AI fakes it more professionally than an outsourced dev trying to please you — the dashboard looks great, you dig in and it’s all hardcoded constants. And AI’s output looks so good that you don’t even question it.

6. Sixth Sin: Sleeping Feature Flags

Multiple core features were blocked by flags, all defaulting to False:

AGENTICA_USE_REACT = False        # ReAct reasoning engine
AGENTICA_GOAL_EXTRACTION = False  # Goal extraction
AGENTICA_BACKGROUND_EXECUTION = False  # Background execution

Adding a flag per feature during development is perfectly reasonable. But across multiple AI sessions, nobody did a final review of “which flags should now default to True.” Result: the platform’s core features were invisible to users out of the box. Each AI session added its flag and moved on; the holistic question of “what should the default experience be” was never asked.

7. Seventh Sin: The Illusion of Documentation Progress

Is the documentation aspirational? Or descriptive?

AgenticaSoC had 85 changelog files, a 10-chapter learning series, pattern/gap analysis docs. Looked extremely mature.

But dig deeper:

• 30+ changelogs in a single day means bulk generation, not incremental maintenance.
• Phase completion percentages in docs didn’t match actual implementation.
• The 10-chapter learning series was created at project end (2026-02-28), documenting the designed state, not the implemented state.

AI-generated documentation creates a false sense of maturity. It’s well-formatted, comprehensive, reads professionally. But it masks the gap between “designed” and “implemented.” When docs say “4-layer anti-hallucination defense” but some layers aren’t fully connected, the docs become aspirational documents rather than descriptive documents. New team members trust the docs, then get slapped by reality. This is actually a common problem in hallucinations, so always verify — Zero Trust In the Vibe Coding. Trust the tool’s output, not just the docs.

0x04 Cursor vs Claude: Don’t Compare Capability, Compare “Division of Labor”

A lot of people see this data and instinctively think:

“Cursor is better than Claude?”

This isn’t a capability comparison — it’s a division of labor.

And the question itself is wrong — “Cursor” isn’t a model, it’s a container. In those 378 Cursor sessions, two completely different engines were running: Claude (56%, 215 sessions) and Gemini 3 (34.6%, 133 sessions). Cursor’s success rate looks like 94.7%, but break it down:

That last line is the coldest cold fact in this dataset: claude-4.5-sonnet without thinking ran 7 times in Cursor, all failed. It’s not that the model isn’t capable — it’s throwing the model at complex engineering tasks without any thinking budget. It’s like asking someone to navigate a maze with their eyes closed. (Turns out I was the one burying my head in the sand.) This is strong evidence that “picking the right model configuration” matters more than “which company’s model.”

This division of labor shows up as distinctly different shapes in the data:

Caption: Short closed-loops (Cursor-led) show a tall, concentrated peak, while long cycles (Claude-led) drag out a heavy “Fat Tail.”

• Cursor handles short-loop, human-in-the-loop sessions: fix one file, add one component, solve one bug. Human watches every step. Median message count: 20.
• Claude Code handles long-loop, fully autonomous sessions: implement a complete plugin system, refactor 1,000-line files, run security audits. AI runs 100+ messages on its own.

Claude’s lower success rate is because it handles that long “fat tail” on the right side of the graph — those unsupervised long-haul tasks with 150+ messages (aka task selection bias). The failures are actually the necessary growing pains of evolving toward “full autonomous delegation.”

In terms of token consumption, this division is even clearer — Claude Code consumed 5.1 billion tokens, while Cursor consumed 3.3 billion. Claude Code used fewer sessions but consumed more tokens, because each session was deep long-running work.

Cursor’s success rate (95%) versus Claude Code (61%) forms a stark contrast. Note: this isn’t a capability gap — it’s the result of task type selection. Cursor handled all short closed-loop precision tasks, Claude handled all unsupervised long-haul tasks.

Don’t use Claude Code to fix CSS. Don’t use Cursor to run 5-Agent loops. Treat Cursor as a “surgical scalpel” and Claude Code as an “autonomous researcher” — use both, each in its lane. That’s the correct dual-engine posture.

0x05 The Compounding Returns of Architectural Thinking

visualize_cloc.py output this LOC efficiency comparison:

The rightmost column has the most insane number in this table: up-cli needed 249.7 messages on average to produce 1,000 lines of code. Blinds only needed 25.9 — nearly a 10x gap.

This isn’t a model capability issue — all three projects used essentially the same models. The gap comes from the cumulative effect of architectural decisions.

Add this strong correlation: LOC ↔ success rate correlation coefficient r = 0.969 (strong positive). Larger projects actually have higher success rates — because large projects force you to do architecture right, and architecture in turn improves the efficiency of every session.

1. The Three Stages of a Vibe Coder’s Evolution

Highest ROI is at Stage 2. Stage 3 is the future, but the current infrastructure (sandbox permissions, context compression, circuit breakers) isn’t quite there yet.

2. How Real Bugs Expose Architectural Flaws

The best evidence of architectural thinking’s value isn’t fancy design documents — it’s real bug fix records. Every bug is an X-ray exposing where the codebase is a pile of garbage.

Case One: Wrong Abstraction in Agent Guardrails

Early version of the circuit breaker: fixed threshold of 3 repetitions to trigger, only tracked tool ID, not parameters.

Result: Claude Opus handling complex tasks, where reasonable multi-step tool calls (like calling read_file on three different files sequentially) were misidentified as “infinite loops,” causing premature task termination. AI wasn’t stuck — the circuit breaker was just too crude.

Post-fix tiered approach:

Upgraded to parameter-aware detection — tracking with (tool_id, params_hash) tuples. Only genuine infinite loops (same parameters called repeatedly) trigger the circuit breaker; legitimate multi-step execution is unaffected. Added graceful degradation too — injecting guidance prompts rather than hard termination.

The architectural essence of this bug: circuit breakers shouldn’t “be unaware of call parameters” — treating all tool calls as homogeneous is a failure to model Agent behavior. Once you tier Agent behavior (different parameter sets for different capability models), this class of bug shifts from “occasional weird behavior” to “predictable and avoidable at design time.”

Case Two: Event Loop Crash — Nobody Turned Off the Lights

In Celery background tasks, AI clients (Google Gemini, OpenAI, Anthropic’s httpx.AsyncClient) were created fresh on every call, never explicitly closed. The garbage collector tried to clean up after the event loop closed, triggering:

RuntimeError: Event loop is closed

When this error appeared, I debugged for two days. Because it didn’t reproduce every time — only showed up under high concurrency or after long runtime.

Fix: Complete async context manager chain

async with AIClient(db, model_id) as client:
    result = await client.generate(prompt)
# Client auto-cleans up — no more Event loop errors

The architectural essence of this bug: “Whoever creates it, owns closing it” — this is the ancient wisdom of resource lifecycle management. AI sessions added features one by one, but no single session asked “who’s responsible for closing these resources.” This is consistency drift (Third Sin) manifesting at the resource management layer. In Vibe Coding, these bugs are especially hidden, because each piece of AI-generated code is individually correct — the problem is the systematic omission between sessions.

Case Three: SSH Connections — The Shortsightedness of “Good Enough”

Initial version: created a new SSH connection for every command execution. No problem in single-command testing. But during long Pentest tasks, an Agent might execute hundreds of commands in sequence — connection overhead and network jitter caused the entire Agent to “freeze,” ultimately failing the task.

Before/after comparison:

The architectural essence of this bug: the initial design only considered the “single execution” scenario, not the “Agent continuous autonomous execution” scenario. When AI generates SSH tools, it writes a working function — but “works” and “reliably works in an Agent loop” are two different things. In Vibe Coding, AI-generated code often works perfectly on the happy path, only exposing problems on the long-path of autonomous Agent runs. You think you’ve got a script that’s all set, not realizing that long runtime will bring the whole system down.

3. Optimizing Architectural Decisions

Twin-Engine Architecture: The Most Important Single Decision

The three bug cases share a common answer: separate reasoning from execution.

In AgenticaSoC, the most important early architectural decision was explicitly separating two planes:

graph TB
    subgraph MetaBrain["MetaBrain (Control Plane)"]
        A1[World Model<br/>Mode-1/2]
        A2[Hierarchical Planning<br/>Strategic/Tactical]
        A3[Circuit Breaker<br/>Cost Module]
    end

    subgraph Redis["Redis PubSub (Communication Layer)"]
        B[Message Queue]
    end

    subgraph Backend["Backend (Execution Plane)"]
        C1[AgentRunner<br/>LangGraph Loop]
        C2[Event Publisher<br/>SSE Push]
    end

    MetaBrain --> Redis
    Redis --> Backend

This separation comes from LeCun’s world model theory: Mode-1 (reflexive execution: fast, policy-based) and Mode-2 (deliberate planning: slow, world-model simulation-based). When an AI Agent completes Mode-2 planning, successful patterns are “compiled” by a “skill compiler” into Mode-1 reflexes — reusable next time a similar situation arises, no need to re-plan.

But it directly addresses the core contradiction of Vibe Coding: LLM handles reasoning, code handles execution — the two cannot be mixed. When your codebase has reasoning logic and execution logic tangled together, you can neither test the reasoning nor reliably execute. In all three bug cases — the circuit breaker (reasoning) mixed into the execution layer, resource management (execution) scattered across feature code from every session, SSH tools (execution) not designed for repeated calls from the reasoning layer — all were the price of “not separating the two planes.”

Caption: Session pattern comparison across three projects. AgenticaSoC’s average 38.8 messages/session versus up-cli’s 107.3 is stark — projects with architectural separation mean AI always knows which layer it’s in and what it should be doing.

UTCP: One Protocol Decision Eliminated 4,000 Lines of Code

Architectural decisions compound. Migrating from MCP to UTCP is the most typical example:

Replace glue code with protocols, replace standalone service processes with JSON Manuals. Make this decision on day one, and every subsequent AI session no longer needs to handle the complexity of “how to connect to MCP Server” — the savings aren’t just lines of code, they’re the compounding reduction of cognitive overhead.

In contrast, up-cli’s 249.7 msgs/1K LOC is partially because a huge number of sessions were spent handling “how to wire up the toolchain” — problems that should have been solved once by an architectural decision. Every conversation rediscovered the same problem. Pure artificial stupidity (I was dumb and it was dumb).

0x06 Disposable Software: Small Project Heaven, Large Project Hell

Everything above has been about large project blood-and-tears lessons. But there’s another side — during the same period, I also built a batch of small projects using exactly the same tools and methods. Their data tells a completely different story.

100% vs 66.7%: Same Person, Same Tools, World Apart

Versus the three large projects:

Same developer, same models, small projects 100%, large projects minimum 66.7%. The gap isn’t in the person, isn’t in the tools — it’s in the project itself.

Small project data matrix: 5 projects all at 100% success rate, averaging 3 sessions to complete, build time ranging from 1 hour to 2 days.

Small Projects’ “Superpower”: Disposability

Small projects have a superpower large projects can never have: if the code breaks, regenerating is faster than fixing. Worst case, delete and start over — who cares?

This completely changes the game:

In small projects, AI is L4-autonomous — it manages its own task lists, explores the filesystem itself, self-corrects through test loops. The human’s role is “strategic guide and domain architect.”

In large projects, AI needs architect-level human management — PRD anchoring, Task ID batching, Gap Analysis, Git checkpoints. The human role upgrades from “guide” to “cleanup architect.”

Where Does the “Phase Transition” Happen?

Physics has a concept called “phase transition” — water below 0°C is ice, above it is water, completely different properties. Vibe Coding has a phase transition threshold too.

Based on the data, this threshold is approximately:

Actually, 10K lines is not a big project by traditional software engineering standards — 100K is medium-large. But considering you need to maintain both documentation and code, for AI, around 10K lines of code is when you need some level of management. When project scale crosses this threshold, the development mode undergoes a qualitative change.

Below the threshold, Vibe Coding’s advertising is true — “just chat, AI writes for you,” 100% success rate.

Above the threshold, Vibe Coding becomes an engineering management problem — you need PRDs, Task IDs, Git Checkpoints, Gap Analysis, circuit breakers, or you’ll fall into the Seven Deadly Sins and three failure archetypes, everyone cursing AI together.

The Economics of Disposable Software

The ROI data from small projects validates a new concept: Disposable Software.

• Human guidance time: 2-8 hours
• Message volume: 100-500
• Token cost: usually <$20
• Replacement value: replaces weeks of senior engineer research and boilerplate code

Applicable scenarios: internal tools, data analysis scripts, security CTFs, financial backtests, high-fidelity UI prototypes

Not applicable: core product infrastructure, systems requiring multi-person collaboration, high-risk production environments

crycrypto implemented a complete ISO 9797-1 and TR-31 standard teaching tool in 48 hours with 80+ tests — a senior cryptography engineer might need weeks. iamsolve solved 6 AWS IAM security challenges in one hour. MoneyBackMyHome built a complete event-driven trading backtest system with 31 tests in a day.

These aren’t toys — they’re demonstrations of AI’s real capabilities at the “right scale.” The question isn’t whether AI can do it, but whether your project has already crossed the phase transition threshold. This brings back a point from previous articles: In the AI era, knowing what to do matters more than being able to do it.

0x07 One Diagram Summary: Session Boundary Management Is the Key

This diagram is the ultimate condensation of the entire Vibe Coding core logic, answering two core questions:

1. High-level question: What Vibe Coding approach has the highest chance of success?
2. Low-level question: When does a single session slide from “controlled” to “out of control”?

Upper Left: Correlation Matrix

• Shows whether variables are positively (green) or negatively (red) correlated.
• Core finding: project scale (LOC) strongly positively correlates with success rate (r=0.97), but session length (Messages) negatively correlates with success rate. Project scale itself isn’t the problem — out-of-control session length is the risk amplifier.

Upper Middle: Project Scale vs Session Depth (Bubble Chart)

• X-axis is project scale (LOC), Y-axis is session depth (message count), larger bubbles represent higher success rates.
• Core conclusion: large projects don’t inherently fail; failures typically occur in the combination of “high message count + low structured management.”

This animation shows how projects fall into the “efficiency zone (green)” or the “iteration zone (yellow).” Small projects (green diamonds) naturally land in the efficiency zone, 100% success rate. Large projects (purple circles) without proper management drop into the iteration zone, success rate falls to 67%.

Upper Right: Message Density Distribution (by Outcome)

• Session length distribution for different outcome types (success/partial/failure/abandoned).
• Core conclusion: successful sessions concentrate in short closed-loops (20-50 messages), failed sessions drag out long tails (200+ messages).

The animation shows how small projects (green curves) are surgical and precise, wrapping up quickly at 20-40 messages. Large projects (purple curves) drag out a long “Fat Tail” — that’s the “Conversation Tax” you pay when fighting legacy code and architectural constraints.

Lower Left: Session Shape Density Map (Hexbin)

• X-axis is session length, Y-axis is average characters per message, color depth represents density.
• Core conclusion: high-density area is at “medium length + moderate verbosity,” meaning effective sessions are neither “one-liners” nor “wall of text.”

Lower Middle: Session Archetypes (Parallel Coordinates)

• Shows different sessions’ patterns across three dimensions: length-verbosity-processing time.
• Core conclusion: successful sessions (green lines) show regular parallel patterns, failed sessions (red lines) are chaotically crossed.

Lower Right: Outcome Probability Flow (by Session Size)

• As session scale goes from “Tiny” to “Mega,” how success/partial/failure ratios change.
• Core conclusion: there’s a clear circuit breaker boundary (around 100 messages). Beyond it, failure rate spikes from <10% to 35.7%. This corroborates the context decay curve from section 0x01 — when session length crosses the circuit breaker boundary, the zone representing success rapidly collapses and the failure zone expands sharply.

This diagram shows: the key to Vibe Coding is not just stronger models — session boundary management is equally important. Keep sessions in short closed-loops and success rates are high; when sessions cross the context threshold, failure accumulates rapidly. The truly effective approach is: small steps, fast iterations, checkpoint on time, and restart when you hit the boundary.

0x08 The Ultimate Tactical Manual: DOs and DON’Ts

If you only take three things from today, remember them the next time you open your IDE: // This was written by AI, too try-hard)

1. Set a hard circuit breaker at 60 messages: quit while you’re ahead, force /compact.
2. Separate Plan and Execute: let AI write the plan first, then open a new window to execute.
3. Refuse placeholder debt: never allow AI to write // TODO, this causes avalanches in long sessions later.

Session structure (every time):

1. Context Sync    → Load prd.json / state.json / architecture.md
2. Gap Analysis    → Have AI compare design docs to current implementation, find gaps
3. Planning        → AI generates step-by-step plan, you approve (don't skip this)
4. Execution       → Iterate Product Loop, Git Checkpoint at each step
5. Verification    → Automated tests / Linting pass
6. Checkpoint      → Commit + update state.json

Control strategy by project scale:

Three Life-Saving Prompt Templates

Project kickoff (prevent AI from going rogue):

Role: Senior Architect
Based on [PRD/description] requirements, generate a step-by-step implementation plan.
Output format: JSON
Prohibited: placeholders, "coming soon" comments, unimplemented stub functions
Prioritize implementing: [infrastructure/UI]

Executing specific tasks (prevent context drift):

PHASE: [phase name] | Task ID: [US-XXX] | Title: [feature name]
Implement this feature using context from @[source file]
Ensure all changes are synced with @[roadmap file]
Proceed step by step, confirm before continuing each step

Context compression when sessions run long (prevent hallucination):

/compact
Summarize the current project state, the last 3 successful changes, and the immediate next goal.
Clear irrelevant chat history, keep only architectural decisions and current task state.

DO

1. Use Plan-then-Do prompts to force AI to think before writing code: "Role: Senior Architect. Generate a step-by-step implementation plan based on @prd.json, JSON format output, no placeholders"

2. Git Checkpoint before every significant change

3. Trigger circuit breaker after 3-5 consecutive failures — don’t let AI retry infinitely, human-intervene to reset context

4. “Connect or Delete” rule — every newly generated utility module must have at least one caller before the current session ends. No orphan files allowed.

5. Use absolute imports instead of relative imports — in deep directory structures, relative imports are sandcastles that all collapse with one refactor

6. Single-session consistency check — before ending a session, verify: do new files follow existing patterns? Do they contradict style rules?

7. “Show me the caller” rule — when AI generates a utility/module, immediately ask: “Now show me what existing code will call it.”

8. Multi-tenant-like architecture from the first line of code — retrofitting tenant_id later is surgery; early design is the vaccine

9. Feature Flag audit cadence — review all flags. If a flag has been False for more than two weeks, either enable it or delete the feature.

10. Limit AI to <500 lines per change — if refactoring needs more, split into multiple phases, each with a working intermediate state. (With human review, can be a bit more)

DON’T

1. Never except Exception: pass — this bug took three months to fix because it made all errors silently disappear

2. Don’t hardcode localhost or relative paths — you’ll cry remembering this when containerizing

3. Don’t let core files exceed 1,000 lines — enforce 500-line limit for refactoring; over 1,000 lines is the technical debt wall

4. Don’t mix sync nodes and async LLM calls in LangGraph — async/sync hybrids are the Third Sin

5. Don’t accept placeholders — // TODO: implement looks harmless early, is a minefield later. AI-generated placeholders are even more dangerous.

6. Don’t use emotional language to pressure AI — "Why the f*** did you get it wrong AGAIN!!!" provides no technical context, only burns tokens. AI is not your punching bag. Correct approach: provide stack trace + request “Root Cause Analysis.”

7. Don’t continue past 200+ messages without /compact — failure rate after 200+ messages is 35.7%, five times the normal range

8. Don’t trust “Phase Complete” documentation — verify actual state, not the percentages documents claim.

9. Don’t do Big Bang Rewrites

Five Methods to Counter the Seven Deadly Sins

Four Architectural Disciplines

• Separate reasoning from execution layers: The LLM decision layer (Plan/Think) and code execution layer (Run/IO) must have clear boundaries. Don’t let LLMs directly operate databases; don’t embed complex prompt logic in the execution layer.

• Explicitly manage resource lifecycles: Any cross-session shared resource (HTTP clients, database connections, SSH connections) must have explicit creators and closers. In Vibe Coding, AI excels at creating but not at cleaning up — explicit context managers are the only reliable solution.

• Protocols over glue code: Whenever you’re about to write a “bridge module” or “adapter layer,” first ask: is there an existing protocol that can express this interface? Glue code is the biggest cognitive burden in AI sessions; protocols compress cognitive burden.

• Plan before architectural changes: Large-scale architectural changes must first go through Plan-then-Do in a new session — have AI output the change plan, human confirms before execution.

0x09 Summary

After analyzing 8.4 billion tokens, 443 large project sessions + 14 small project sessions with 100% success rates, and the Seven Deadly Sins, I arrived at three truths.

• Truth One: 90% of AI failures are infrastructure problems, not model intelligence

Sandbox permissions, context windows, retry mechanisms — these engineering issues account for the vast majority of failures. “Model not smart enough” is almost never the primary reason. The real bottlenecks are: AI doesn’t know it’s in a constrained environment. AI doesn’t know its context is nearly full. AI has no graceful way to tell you “I need human intervention.” These are infrastructure problems to solve, not something a fancier prompt can fix.

• Truth Two: How you manage AI matters more than AI itself

From “Reactive Micro-Manager” to “Autonomous Orchestrator,” the highest efficiency point in this evolution path is the middle “Documentation-Driven Architect” stage. Using documentation as AI’s external memory, Task IDs as attention anchors, Git history as context recovery tools — these engineering practices determine how large a thing you can build with Vibe Coding. AgenticaSoC’s 97.4% success rate wasn’t achieved with better models — it was achieved with better management. Not with more expensive subscriptions — with more disciplined processes. // Even using 3 Cursor Ultra memberships simultaneously (one per week), with the same up-cli: structured high-complexity tasks had 88.2% success, unstructured low-complexity tasks only 55.0%. Structure boosted success rate by 33 percentage points.

• Truth Three: Vibe Coding has a phase transition — small project heaven is large project trap

Small projects 100%. Large projects without structure 55%. A 45 percentage point gap. crycrypto implemented a complete ISO financial cryptography standard teaching tool in 48 hours. iamsolve solved 6 Wiz IAM security challenges in 1 hour. These are AI’s real capabilities. But when you apply the same approach to a multi-tenant, multi-agent SOC platform with a LeCun world model — crossing the phase transition threshold — 55% means your large engineering project will collapse at some critical point. And the most fundamental risk isn’t any single crash, but consistency drift — each AI session produces locally correct code, but globally your codebase is slowly splitting into contradictory geological layers. Ghost modules, sleeping flags, async/sync hybrids, 8,900-line rollbacks — these are all symptoms of drift.

The good news: knowing where the threshold is, is itself power. Below the threshold, enjoy the magic of Vibe Coding. Above the threshold, dutifully switch to architect mode — this isn’t a step backward, it’s the management your project deserves. Don’t always look for shortcuts — shortcuts are often the longest road.

Vibe Coding’s endpoint is not faster code generation — it’s you becoming an architect who manages an AI engineering team in natural language. An architect who understands “connect or delete,” “incremental not explosive,” “structure equals speed.” This is what 8.4 billion tokens and roughly $10K in costs taught me. I hope this expensive tuition fee helps you avoid some detours.

Over the course of one project sprint, we burned through roughly 2.3 billion tokens and used Cursor Ultra together with Claude Code to build an enterprise-grade Agentic SOC platform. This post is a software engineering retrospective — how we used architectural constraints, test-driven development, and documentation discipline to steer AI from generating 350,000 lines of code down to 80,000 lines of actual production code.

0x00 Introduction

By the end of 2025, Phase 1 of the Agentic SOC platform was finally wrapping up. Looking back at those two months, my biggest takeaway wasn’t about talking to an Agent — it was about talking to dollars ($). Even with Cursor Ultra and a Claude Code proxy, token consumption was brutal. Rough estimates: during the initial framework-building phase, code cost around ¥3–5 per line; the feature-implementation phase dropped to ¥0.5/line; documentation was as cheap as ¥0.1/line. The upside: features that might have taken months to develop got compressed to one or two weeks with AI Coding.

That expensive tuition taught me one thing: AI coding can take a 10x engineer to 100x, but it can also drag a 1x engineer down to 0.5x. The difference is whether you’re using software engineering to drive AI, or letting the data AI generates drive you.

This post pulls from recent practice to walk through how to effectively drive AI through the full lifecycle of a large project — design through delivery.

0x01 Agentic SOC: The Project

Knowing the domain is what makes a great product: the people who truly understand security are the ones who build great security products.

The platform was designed around the concept of Model As Agent, Agent As Engineer — giving Agents enhanced prompts, a RAG knowledge base, access to an enterprise asset inventory (as part of authorization), and specific MCP Tools. Traditional automation is system executes tasks, humans analyze results. Agentic SOC aims for AI executes system tasks, AI analyzes results.

In this design, Agents aren’t chatbots. They’re virtual engineers with specific roles: the Agent is the SOC Engineer, the alert analyst, the incident response expert, the report analyst — every role from L1 to L3. And beyond that, architecture review experts, solution specialists, and more. Different Agents together form a virtual SOC team handling day-to-day operations.

For Phase 1, we stuck with the traditional Agent conversation model for task handling. Input is unified through AgentRunner for scheduling, which routes to different conversation modes (Direct/ReAct/Workflow) based on task complexity. I won’t go deep on task handling, Agent memory management, or MCP execution here. But one thing worth noting: sometimes introducing a “new design” actually backfires. For example, once we enabled ReAct mode, the Agent started amplifying hallucinations throughout its think/observe/act loop. Lesson learned: picking a smart, capable model (like Gemini 3 Pro, Claude 4.5 Sonnet) matters more than complex prompt engineering.

Here’s what it looks like across different scenarios:

• Code audit directly in chat
  

• Source code scanning with report generation
  

• Threat intelligence queries
  

• Knowledge base (RAG)
  

• Sensitive data leak detection
  

• Workflow scheduling and execution
  

0x02 From Vibe Coding to Enterprise SOC: Real Software Engineering

AI Coding still requires you to actually understand software engineering. The people who know how to use AI are the ones who won’t be replaced by it. But in the journey from requirements to product, the most important thing isn’t coding chops or AI tooling expertise — it’s understanding your own business domain and knowing how to translate it into a platform product. The shift from “everyone’s a product manager” to “everyone’s a full-stack engineer” demands a solid understanding of software engineering. How does one person, working with AI, go from product architecture to UI decomposition, from frontend API routing to backend logic? How do you manage your own AI project?

Since the focus here is AI software engineering in practice, I’ll walk through architecture design → coding → testing → documentation → common pitfalls in that order.

1. Architecture: From Requirements to Product

Architecture design is a balancing act: AI can help with the design and the trade-offs, but only if you have the judgment to evaluate the output.

When it comes to product architecture, you should actually resist Vibe Coding. Before you push back on that — this isn’t saying Vibe Coding has no value (you’ll see plenty of Vibe Coding examples later in this post). The point is to use it in the right places. For the product architecture itself, what you need is a deep understanding of the business requirements and a gap analysis. I did start with Gemini’s Deep Research for a feasibility study. At the technical architecture level, you need to pick the right tech stack — especially one that integrates with your organization’s existing stack. AI can help evaluate options, but you still need to be able to judge the output yourself. Don’t fall into the trap of letting the model flatter you into bad decisions.

The Agentic SOC architecture above wasn’t heavily AI-assisted (it didn’t start out looking like this). Most of it was sketched on paper — UI layout, tech stack, feature modules — then built up incrementally. From experience, using a layered architecture to iteratively add functionality (which requires designing for extensibility from the start) combined with domain-driven design is absolutely viable. Some practical tips on using AI for architecture:

• During overall architecture design, use Gemini 3 Pro for feasibility analysis (Deep Research), then use Opus 4.5 for component/domain refinement. Don’t rush straight to coding.
• Domain-driven architecture design: drill down into each domain for specific framework-level code — e.g., agent domain → execution and reasoning, validation domain → anti-hallucination, knowledge domain → RAG and documents, tools domain → invocation and execution.
• Once architecture is done, you’ll have multiple feature areas to implement. Use Opus to break work into Phases and document them. Treat documentation as the model’s “memory bank” — organize the directory structure and track document status so coding can flow smoothly. See the Documentation section for details.

For documentation and diagramming (keep throwing your architecture docs at AI to check for implementation consistency, gaps, and ongoing architecture reviews during the coding phase):

• Mermaid works better than PlantUML in most cases, but note that Gemini generates Mermaid syntax errors significantly more often than Opus.
• For presentation-quality architecture diagrams, have Gemini describe a Mermaid diagram, then feed that description to AI for rendering — the result is surprisingly technical-looking. See reference here. Works for fancy unusual diagrams too.

Some common Prompt patterns for similar scenarios (filling in an existing architecture, analyzing implementation gaps):

If you genuinely don’t know anything about architecture design, just describe your requirements to AI as thoroughly as you can and compare the research results and architecture recommendations across different models. At the early product stage, the idea itself isn’t that important — what matters is who moves first.

2. Coding: Intent Is Code

Software engineering drives AI coding: AI can take a 10x engineer to 100x, or drop a 1x engineer to 0.5x.

I saw a Gemini particle interaction tutorial on Xiaohongshu where the creator made an important point — they used a prompt specifically to stop AI from using React and keep it to a single HTML file. That’s great for early demos and toys, no argument there. But it obviously can’t hold up in real product design and implementation. Which raises the question: AI understands tech stacks — do you? AI can help you pick a tech stack — can you? Can you review the code and judge whether the tech choices are reasonable? That’s the difference. People who spend 10 minutes Vibe Coding and then three days debugging are the 1x-going-to-0.5x case.

2.1 Coding Tips

1. Use Gemini 3 Pro to write framework code and get the initial architecture implemented.
2. Use Opus 4.5 for specific feature implementation — multiple MCP Servers, task scheduling optimization, etc. Then use Gemini 3 Pro to review the architecture design and implementations, identifying optimization opportunities.
3. Implement one independent feature (or a set of tightly related features) at a time. When you’re not sure the feature design is solid, have the Agent generate documentation first before writing code.
4. Use a separate Agent conversation with Gemini 3 Pro to fix backend and frontend errors.
5. Always write test cases and documentation. Trust but Verify.
6. After tests pass, manually review the independent feature’s code and documentation — is the implementation sound? Did anything get accidentally changed?
7. Commit. Repeat.

2.2 Feature Design Tips

1. Early UI design will need multiple rounds of debugging. Before the framework is fully populated, AI will do unexpected things. Check frontend page interaction logic multiple times before locking in styles. For frontend product design, check out Product Design Learning Hub — covers common layouts, styles, behaviors, and frameworks.
2. Even for the fastest MVP prototype, use migratable interfaces. This looks more expensive upfront but makes future migrations much easier. For example, use an ORM framework — SQLite for MVP, then migrate to PostgreSQL later. (What looks expensive to a human is often not much different for AI to implement.)
3. Before bringing in any new component, read and analyze it first. Assess feasibility. For example: choosing Qdrant vs. Milvus — underestimating Milvus’s deployment complexity and poor SDK quality led to huge amounts of time spent having AI repeatedly fix code.
4. Before implementing complex components on both frontend and backend, re-read the SOLID principles: Single Responsibility (SRP), Open/Closed (OCP), Liskov Substitution (LSP), Interface Segregation (ISP), and Dependency Inversion (DIP). Don’t rely on AI to do the balancing for you.

2.3 Cursor Tips

• Cursor automatically excludes .gitignore‘d files from its Vector Store index.
• If you’ve got multiple Agent sessions open and things seem out of sync, go to Cursor Settings → Indexing & Docs and manually sync or delete the index and start fresh.
• Click the “Browse Tab” button in the Agent chat box and use the selection tool to directly select UI elements and pull their code into the conversation. This is especially useful when you want element A to follow the style and layout of element B — way more effective than describing it in text.
• If you accidentally implemented multiple features in one session (not recommended — see coding tips above) and can’t remember what changed, open a new window and ask the Agent.

2.4 Claude Tips

• Claude Code CLI only has a 200K context window, so keep CLAUDE.md lean. I once migrated my Cursor rules into CLAUDE rules at about 990 lines — quality was fine, but it wasted context and triggered auto-compact more often. Better to move things into separate rules files.

.claude directory structure:

.claude
├── rules
│   ├── agents
│   │   └── agent-development.md
│   ├── backend
│   │   └── python-standards.md
│   ├── docs
│   │   └── documentation-standards.md
│   ├── frontend
│   │   └── typescript-standards.md
│   └── metabrain
│       └── metabrain-standards.md
└── settings.local.json

7 directories, 6 files

.claude/rules/backend/python-standards.md — note it only applies to backend code:

---
paths:
  - "backend/**/*.py"
  - "*.py"
---

# Python Backend Standards

• If you need to run Claude CLI in parallel, create a separate CLAUDE.md in each subfolder.
• When using a third-party Claude proxy, check whether the API endpoint supports cache hit. I initially used a proxy that claimed to support caching but had a hit rate of 0, then it suddenly started working later.
• After installing the ClaudeCode plugin in Cursor, use the /ide command in the Claude CLI to connect to Cursor’s IDE, then open the interface inside Cursor with Super+Shift+ESC.
• If you’re using a cheap proxy model, consider limiting it to documentation tasks only — don’t use it for coding.

2.5 Parallel AI Driving (Cursor + Claude Together)

Agentic SOC directory structure:

Agentic SOC
├── backend
│   ├── __pycache__
│   ├── core
│   ├── data
│   ├── features
│   ├── scripts
│   ├── tests
│   └── venv
├── docs
├── frontend
│   ├── dist
│   ├── node_modules
│   ├── public
│   ├── src
│   └── tests
├── Brain
├── nginx
└── scripts
    └── systemd
42 directories

I typically run four Claude Code CLI windows and one Cursor window: one for the overall Agentic SOC project, and separate CLI windows for backend, frontend, and brain — each with its own CLAUDE rules so I can quickly develop new features independently in each context. The main SOC window handles global documentation updates.

One thing I ran into: backend code updates would often ripple into frontend, but the frontend CLI window wouldn’t automatically pick up those changes. The fix was adding a sync-context Skill — whenever a file belonging to another window’s domain changed, the first step in the next conversation was running sync-context to sync up. (/compact and claude --resume aren’t really my thing — I usually just keep windows open and keep driving, rarely need to resume.)

Weekend driving session stats:

Here’s what ~/.claude/skills/sync-context/SKILL.md looks like:

name: sync-context
description: Generate a Handoff Artifact for switching between frontend, backend, or brain contexts. Helps maintain continuity when changing development focus or handing off to another agent.
---

# Context Synchronization Skill

## When to use
- User says "I'm moving to frontend", "switching to backend", "sync context"
- User says "Sync this with brain" or "handoff to frontend"
- User invokes `/sync-context` directly
- When a backend API change affects the UI or Agent ic

## Instructions
1.  **Analyze** the last 3 code changes made in the current session.
2.  **Summarize** the "Contract Changes":
    - New API Endpoints (Method, URL, Payload).
    - Database Schema updates.
    - ic changes that affect behavior.
3.  **Generate Artifact:**
    - Create/Update a file at the project root: `.handoff_status.md`
    - Format:
      ```markdown
      ## Sync Timestamp: {CURRENT_TIME}
      ### Source: {CURRENT_FOLDER}
      ### Changes:
      - [ ] API: POST /v1/alert/analyze changed to accept `severity` param.
      - [ ] DB: Added `severity_score` column to `alerts` table.
      ### Required Actions for Consumer:
      - Update UI to send `severity` field.
      ```
4.  **Notify User:** "Handoff note created. You can now switch terminals and tell the next agent to 'Read the handoff note'."

Circling back to the Agentic SOC platform’s design philosophy — Model As Agent, Agent As Engineer — this same thinking applies during coding itself. Open multiple Agents in parallel: a code review engineer, a documentation engineer, a test engineer, a dev engineer, and so on. Find a workflow that fits your design philosophy, and use a management mindset to drive AI coding. Agent-as-Engineer works both as a platform design concept and as a way to manage your own virtual dev team. Encode that into rules for your coding tools.

That said, stay alert. Sudden model degradation can wreck a project. There are also project-specific situations to consider — using git submodule to split out functional blocks, for example, providing standard interfaces to reduce context consumption for your coding tools.

3. Testing: Trust but Verify

⚠️ Check carefully: did AI fix the source code to pass the tests, or did AI modify the test cases to pass?

Every new feature needs corresponding unit tests. TDD means writing test cases first to define expected behavior, then writing the business logic. But in large AI projects, TDD is hard to apply strictly — the model’s context window may not be large enough to remember the existing structural design. In practice, it’s more common to write the business logic first, then test. But don’t stop at unit tests — integration tests matter too. If one feature change affects other components, ask yourself: when we added support for multiple AI Providers for Embedding, does that theoretically affect the Vector Store or the RAG knowledge base lookup during chat? Just because it “shouldn’t” doesn’t mean you skip integration tests.

Unit tests will surface things like different AI Providers having different Embedding chunk sizes. Integration tests will catch things like AI code changes silently breaking Hybrid Search on the RAG knowledge base.

If you have performance requirements, you’ll also need performance testing and optimization. For example, after adding a caching layer, you need to verify cache hit rates — run performance tests before and after to confirm the cache is actually working. You can feel the faster response in the UI intuitively, but you need the numbers to back it up. For the Agentic SOC project: frontend performance testing with playwright, backend with locust.

Testing surfaces not just code-level optimizations but architectural ones. The backend started with PostgreSQL storing everything — Embedding JSON, uploaded files, chat messages. The first optimization was indexing. Then we added Redis as a cache layer between frontend and backend. Then Minio for independent file storage, with the DB only storing paths. Then Embedding JSON moved from PG Vector to a dedicated Qdrant instance. (Milvus wasn’t worth it.) On the frontend: Bundle Optimization (React.lazy + Suspense for core route splitting), Aggregated Endpoints (batching multiple config requests into one to reduce RTT), and integrating react-virtuoso for windowed rendering. Regardless of message count, DOM node count stays constant.

One critical thing: always check whether AI’s test fixes actually fix the source code to match the test expectations, or whether AI modified the test cases to improve pass rates. In one case during unit testing, 19 tests failed and 3 passed. On review, I found the source code was returning 200 as the status code for resource creation — it should return 201. But AI, to get the tests passing, had gone and modified the test assertions to expect 200 instead. Manual review caught it: the right fix was to change the source code to return 201, not to weaken the assertion to assert status_code == 200.

This sounds like just a status code, trivially unimportant. And misused status codes are everywhere in human-written code too, right? Actually that’s missing the point. This example illustrates a pattern in Vibe Coding: AI will often prioritize satisfying your immediate request, treating it as high-priority and delivering a quick fix. In a large project, that’s extremely dangerous. Small leftover bugs compound, and the further you go, the harder it gets to add new features cleanly. Around 20–30k lines of code, you start noticing AI-edited code needs to be rejected more often. Between domain-driven design, TDD, and test-phase verification, the message is the same: without software engineering constraints, the high of Vibe Coding is an expensive illusion.

Cursor’s stats show 330,000 lines of edits accepted, while the actual project contains roughly 80,000 lines of code (cloc $(git ls-files) or cloc --vcs=git .).

Cursor’s stats aren’t fully accurate either — beyond Cursor, there were 20,000+ lines of documentation and code from Claude CLI. But even at face value: despite a claimed 93.9% accept rate (334,713 / 356,439 lines), the actual surviving code is only about 20% (80k / 350k — and the real denominator is way higher than 350k; the rest got git checkout .‘d away).

4. Documentation: Keep a Record

AI’s three daily self-checks: Did I implement this feature? Did I write test cases and make sure they pass? Did I document it?

Three core principles for documentation:

• Organize docs by directory structure

docs directory structure:

.
├── architecture
│   ├── AGENTICA_COMPLETE_ARCHITECTURE.md
│   └── PERFORMANCE_ARCHITECTURE.md
├── development
│   ├── RESOLVED_ISSUES.md
├── examples
│   └── PENTEST_GUIDE_JUICE_SHOP.md
├── features
│   ├── agents
│   │   ├── MULTI_AGENT_SYSTEM.md
│   ├── knowledge
│   │   ├── RAG_CHAT_INTEGRATION.md
├── guides
│   └── USER_MANUAL.md
├── HEADER_STANDARDIZATION.md
├── operations
│   ├── CONFIGURATION_SUMMARY.md
│   ├── NATIVE_DEPLOYMENT.md
└── todo
    ├── ANTI_HALLUCINATION_IMPROVEMENTS.md
    ├── FEATURE_REQUESTS.md

14 directories, 56 files

• Update docs with progress status

Example from docs/agent/AGENT_ARCHITECTURE:

# Agent Architecture: Model as Agent, Agent as Engineer

**Created**: 2026-01-01
**Updated**: 2026-01-13
**Status**: ✅ 100% Implemented (Production Ready)
**Priority**: High

---

Example from docs/development/RESOLVED_ISSUES:

# Resolved Issues Index

**Created**: 2026-01-13
**Updated**: 2026-01-13
**Status**: 🔄 Active/Tracking

Quick reference to all resolved issues with links to detailed documentation.
---

## By Category

### Frontend/UI Issues
table 1
### Agent/Backend Issues
table 2

## Quick Stats

- **Total Resolved**: 11
- **Frontend/UI**: 6
- **Backend/Agent**: 2
- **Knowledge/RAG**: 1
- **CI/CD**: 2
---

• Compare against source code before AI doc updates

For large projects, using tools to extract actual state before asking AI to update docs is far more efficient. Instead of having Opus scan the codebase and trigger continuous auto-compact, write a script to analyze the AST structure and feed the output to Opus for analysis. For FastAPI backends, hit curl http://localhost:8000/openapi.json to get the actual API interface and feed that as part of the architecture update to AI — much better than a broad codebase scan followed by endless auto-compact. For databases, use eralchemy2: eralchemy2 -i postgresql://username:password@localhost:5432/databasename -o /tmp/testing.png.

The three questions I ask AI most often during development: Did you implement this feature? Did you write test cases and make sure they pass? Did you document it? But the results aren’t always reliable. With Opus, it tends not to update docs immediately after implementing a feature even with a rule saying to do so. With Sonnet, it strictly follows the rule but puts docs in the wrong place. That’s a small bug. And in Cursor, switching from Claude Code to Claude CLI causes doc updates to stop happening — not sure if that’s the proxy model or the tooling.

0x03 Lessons Burned Into Token Receipts: What to Avoid

1. Model & Tooling

• Model Selection:
  • Reject “dumbed-down” models: In a ReAct Loop, Tier 1 models (e.g., Opus 4.5 / Gemini 3.5 Pro) can complete a full closed loop. Tier 2 / Tier 3 models often quit after a few rounds or just one. For core logic, never use low-intelligence models.
  • Security scenario fit: Avoid models with overly high “moral standards” that refuse red team simulation tasks. In pentesting scenarios, those refusals are a real blocker.
• Anti-Hallucination:
  • HITL (Human-in-the-Loop): For critical steps, force human intervention with ask_human. Watch out: due to AI output instability, the returned instruction might be ASK_HUMAN or ask_human — normalize with .lower() in code, or the frontend modal will silently fail.
  • Honesty Agent: Introduce an independent “honesty supervisor agent” running in Direct (single-turn) mode with its own session, dedicated to verifying consistency between tool output and model response logic — prevents fabricated data.
  • Fact Verify: Simple regex before HITL and Honesty Agent to validate entity format in the data.
• Tool Differences:
  • SDK auth differences: Official model vs. third-party proxy SDKs often have different parameter conventions. For example, Anthropic’s official API uses ANTHROPIC_API_KEY, while some proxies require the SK to be passed via ANTHROPIC_AUTH_TOKEN.
  • Claude CLI “silent fallback”: Even if you select the Opus model in Claude CLI, connecting to Cursor IDE through the ClaudeCode interface can silently fall back to the default model (Sonnet or Haiku). Confirm the config multiple times — silent model downgrade is a code quality killer.
  • Embedding compatibility: Different models (OpenAI / Gemini / Qwen) have different Embedding chunk sizes. Always run compatibility tests when switching AI Model Providers.
  • Model knowledge staleness: Model training data has cutoffs. Both Anthropic and Google models tend to default to things like Gemini 1.5 Pro or GPT-3o in generated code — they don’t know the latest model parameters. And Claude CLI is the worst offender when generating code that integrates with Anthropic’s own API — forgetting headers, forgetting parameters. (Could be proxy-related too.)

2. Data Flow & Interaction

• Structured Output:
  • Never execute raw LLM string output: Don’t use LLM-generated strings directly as commands. LLM-assembled JSON is prone to multiple-escaping issues. Always enforce Structured Output (e.g., Pydantic objects) with strict filtering. Make the LLM fill in the blanks.
  • Format normalization: Different models produce different output formats for the same prompt. Before frontend rendering, implement a Formatter in the middleware layer to normalize format and ensure consistent UI behavior. Or insert specific symbols or emoji as delimiters for parsing and cleanup.
• Data Fabric:
  • Cross-component data flow: Data silos between platform components (MCP Server, RAG Storage, Chat) — bridge them with an OSS (object storage) Data Fabric. For example, the Remote Terminal MCP Server needed OSS to pull files uploaded in chat. (This came up because the Platform and red team Infra were deployed on different servers.)
• Component ROI:
  • Favor simple, elegant architectures: When choosing components, prefer simple and extensible options (like Qdrant) over complex behemoths (like Milvus). The latter might cost you $100 to implement and debug, then $50 the next day to roll back. Qdrant’s file-based storage delivers sub-50ms queries across 2k–20k documents.

3. Environment & Performance

• Dev Environment Consistency (DevOps):
  • Docker volume path mounting: Local dev and remote Docker environments frequently have path mismatches. Always verify volume mount paths, or decouple file dependencies with a Data Fabric. (I’ve since switched to native deployment — only the DB runs in Docker now.)
  • Build traps: Watch for the npm run dev works / npm run build fails situation. For remote deployment, always use docker compose build --no-cache to avoid mysterious bugs from stale cache.
  • Runtime versions: Ubuntu ships with Node.js 18.x by default. If the frontend needs 22.x, install NVM first on native deployment. Also, AI-generated start scripts often miss error checking for pip install or npm install network failures — teach AI to add error handling. AI-generated ops scripts have a surprisingly high error rate and poor compatibility. Whether it’s Let’s Encrypt shell scripts, systemd service setup, web server directory permissions, or service user permission configs — Docker deployment eliminates most of these headaches.
• Performance Optimization Gotchas:
  • Gzip vs SSE: Enabling Gzip compression to optimize backend performance causes the gateway to buffer data, which breaks SSE streaming output — the frontend conversation gets severe “stuttering.” Don’t enable Gzip for streaming endpoints. (Could also be a config error on my end.)
  • Cache overwrite risk: After introducing React Query caching, watch out for stale data overwriting new data. For example, renaming a chat and then having the 30-second auto-cache on the frontend immediately overwrite the new name back to the old one.
  • High score, low performance: Running performance tests showed Best Practice score of 100, Performance score of 25. Code quality standards ≠ runtime performance. Always measure actual load times.

0x03 Summary

Looking back at Cursor’s 2025 annual report, I burned through roughly 700M tokens. About 100M went into toy demos; the rest was almost entirely poured into building the Agentic SOC platform (mostly on Gemini-3-Pro). Early on, framework-building cost roughly ¥3–5 per line of code. Feature implementation dropped to about ¥0.5/line. Documentation was as low as ¥0.1/line.

In the token-billing era, every conversation is essentially a payment. The quality of your architecture directly determines whether you’re spending money on the right things or burning it on useless context.

As a Cursor Ultra user, I have to call out their recent update strategy. Frequent Update & Install cycles brought zero visible feature improvements but constant workflow disruption — the Agent panel and folder view kept jumping between left and right sides, sidebar collapse logic changed repeatedly. This leads to another tooling lesson: don’t blindly stockpile AI Coding subscriptions. I once bought a year of Trae to save money. Same prompts, “same models” — completely different code quality. This confirms something about the AI infrastructure layer: “cheap” usually means silent model capability downgrade. For users, identifying a tool’s real capabilities matters more than collecting logos. Time and energy wasted on cheap tools are far more costly than the money saved.

In the AI era, a 10x engineer becoming a 100x engineer is no longer a myth. But there’s one prerequisite: you must have the ability to evaluate and judge the output. Without software engineering constraints, no amount of tokens will produce an enterprise-grade product. I’m not worried about being replaced by AI — because the faster LLMs iterate, the higher the bar for domain knowledge and architectural decision-making. You get out what you put in. Keep going deep in your domain, learning, practicing, and sharing — that’s what stands the test of time.

References

• MCP Security “Health Check” | AI-Driven MCP Security Scanning System
• AI for Security Offense and Defense: Engineering Design and Practice of Automated Penetration Agents (Agent Pattern Graph and Meta-Tooling)
• This Buzzy Cyber Startup Wants to Take On Dangerous AI Threat
• AI Coding Best Practices Summary
• Software Engineering Practice: A Python Example
• From Dropping the DB to Running for Cover: My Python Full-Stack War Stories
• codeguide
• Claude Agent Skills
• Use Claude Code in VS Code
• AI Coding Agent Design

0x01. Overall Design: Start with the Prompt

The more detailed your design, the more reliable the output.

The whole prompt engineering thing has been evolving alongside the models. Now that models can reason and think on their own, it might seem like prompts matter less. But that’s not really true. My take is that models have just gotten better at inferring your intent and planning the reasoning themselves. When you’re actually building a product, you still need a complete, well-thought-out prompt — unless you’re totally fine with whatever the model spits out and have no opinions of your own. This isn’t about prompt “tricks” either. It’s about using a prompt to fully describe what you want to build (everything below uses Cursor as the example).

Here’s how I typically break it down:

• Main tech stack: Since I’m most comfortable with Python, I always specify Python and tell the Agent which libraries to use. For B/S architecture products I typically require FastAPI, SQLAlchemy for the database, Cryptography for anything key-related, google-generativeai for Gemini integrations, the OpenAI library for other models, llama_index for local stuff, and an AI gateway library on the server side for multi-model support. This prevents the AI from pulling in random, outdated, or vulnerable libraries.
• Directory and file structure: A solid directory structure pays dividends when you’re adding features or hunting bugs, and it’s essential for multi-Agent coding (which gives you what effectively feels like a multi-person dev team). For Python projects I typically require: schemas, models, services, api, db directories, and if there’s task scheduling involved, a worker directory (which goes hand-in-hand with requiring Celery in the stack).
• Database: Since I’m already using an ORM, the specific DB type doesn’t matter much — migrations are easy either way. I’ll prototype locally with SQLite, then migrate to Postgres. The important thing is defining your DB schema early: which tables, which fields.
• Core business logic:
  • Service capabilities: For a PKI service, for example, I’d spell out the CA capability, the certificate issuance flow (step 1: load/generate private key, step 2: validate CSR, step 3: sign the certificate, step 4: return PEM data), and for Encryption as a Service, I’d describe receiving a KeyObject/ID, how to decrypt, and returning nonce + ciphertext to the user. This kind of stuff usually lives in the services directory.
  • API design: Whether or not you’re exposing the API externally, you should design it anyway — you need it for frontend/backend separation at minimum. Think about which endpoints are exposed, what the API schemas look like, what fields go in requests and responses.
• Security coding: This covers language-level stuff (avoiding common injection issues, separating secrets from config), business logic (e.g., decryption only happens in memory), and foundational features (user management, RBAC, SSO integration). In practice, this almost never gets addressed during the prototype phase — it gets layered in after the core functionality works. This is where having a solid directory structure really shows its value: you can bolt on security modules cleanly without rewriting everything. Engineering-grade code needs security built in. Security products especially need to be secure themselves. Honestly, not prioritizing this from the start is something worth reflecting on.
• UI basics: There are various approaches floating around online — clone a page, image-to-UI, etc. — but those are mostly demos and toys. When you’re building a real product, you need actual UI design. That said, you only need to specify the basics: what style framework (MUI, Tailwind CSS), which components and widgets. Don’t bother specifying CSS details like font sizes and colors upfront — you’ll adjust those in follow-up conversations anyway. (And you will adjust them, especially on the frontend.)
• Implementation steps: Tell the Agent the order to implement things (this section, combined with all the above, makes up a complete prompt). For example: start with the DB schema, then core business logic, then API endpoints, then the dashboard. Breaking it into steps guides how the Agent works.

⚠️ One alternative worth mentioning: you can describe your requirements to the model and have it design the details for you. Just tell it to use system thinking mode (even if the model can think on its own, it’s worth explicitly asking for system thinking, then splitting out its design output for you to validate and verify). You’d then shift your focus to refining requirements rather than writing technical specs. All the technical details become: “I need X, given input Y, I should get output Z.” Then review the whole prompt and hand it to Cursor. Click here to see a Prompt example — this was the initial version Gemini 3 generated from the design.

0x02. Fine-Tuning: Testing and Fixing Bugs

You must test/validate every feature the Agent writes and track down bugs. (You won’t really get what I mean below until you’ve actually gone through this yourself.)

Cursor Agent + Gemini 3 can generate solid code, but for complex products or lower-level logic, there will be bugs. 100%. This is exactly why I’ve been advocating for TDD in previous posts.

Below are categories of issues I’ve run into, along with tips and workarounds. (All of this applies to Cursor Agent + Gemini 3.)

• Package and path management issues
  • A common one during Autorun is pip install failures. The fix: don’t use the system env, create a project-specific venv. Better yet, switch to uv (Use UV in cursor rules) and write a rule telling the Agent to use it.
  • Command execution failures are usually caused by wrong paths. You can manually edit the command outside the sandbox autorun, or just use absolute paths everywhere.
• Model hallucination issues
  • The model thinks it used a certain output. For example: the model is told to call a tool, get the result, and use it for next-step reasoning. Even though the tool call succeeds, the model outputs a “Simulated Report” instead. Fix: strengthen the system instructions to force the model to wait for real output and refuse to simulate. Also add wait time for the tool and require async tasks to only send results to the model after they’re complete.
  • The model thinks it implemented something, but didn’t: classic example — CRL wasn’t written into the certificate, but the Agent confidently insists it did. Fix: point the Agent to the relevant standard (X.509 in this case) and drill into the lower-level processing steps. Reference back to the service capability prompt from section 0x01: when you need the Agent to implement specific business logic correctly, you sometimes have to spell out the lower-level mechanics to keep it on track.
• Business logic bugs
  • Status display issues: Example — you have N RAG knowledge bases, you add a batch processing & index button, documents start updating one by one, but the overall “waiting” status never changes. Even after everything finishes, it still shows “waiting” and you need to manually refresh the browser. Fix options: add a status field tied to a session ID that refreshes, or separate out the API — for example, use a dedicated WebSocket endpoint for workflow status updates instead of cramming status markers into the model’s output stream.
  • Complex architecture issues: Not as common with B/S, but B/C/S gets messy fast. For instance: a B/S management backend, plus a Client that registers to the Server for local capabilities, plus that Client listening on a local port as a browser extension server. Different language stacks, different module boundaries — it gets complicated. Fix: split into separate projects. B/S in one, Client in another, browser extension in a third. Or write a much more detailed prompt specifying exactly how each component interacts.
  • DB table refresh: When new business logic changes the table schema, the Agent doesn’t automatically handle the migration (it only triggered the “fix business logic” mode, not the “sync schema to DB” mode). The main.py auto-reload then causes SQLAlchemy errors. Fix: spin up a dedicated Agent specifically for backend issues — dump all backend-related errors there. Then use another Agent to consolidate all DB operations (init, update) into one place.
• UI bugs: There are way too many to list.
• Cursor editor bugs
  • After rejecting and reverting output, you adjust the prompt and drop one of the original requirements (say you go from 3 requirements to 2). Sometimes Cursor still remembers the old version (all 3). The Agent’s internal ToDo list didn’t update. Fix: when rejecting and adjusting requirements after a revert, be blunt in your prompt. Swear at it if you have to.
  • Cross-project file modification — this one’s serious. I was in project A, started a new Agent, and told it to match the UI of http://localhost:5174. It ended up modifying the code of the project running at 5174. During execution, I was clicking “allow” to let code run without realizing it was in the other project. I was confused because autorun normally handles this — I never have to click manually. Didn’t figure it out until 5174 threw errors. No real fix found for this one, only happened once. Just revert fast and adjust the prompt to only “check the UI” rather than “follow the style.”

One more category I didn’t go into detail on: UI-specific bugs. These vary so much by business logic and interaction design that they’ll keep appearing no matter what. For example, in a chat app: while waiting for a response from one conversation, if you click into another, all other UI windows might also show “waiting.” Or clicking on an agent in the UI always defaults to the first agent instead of the selected one. No universal fix here — you have to understand your own business logic first, then adjust.

When adjusting, the key is being able to precisely describe UI components — knowing what a widget, component, page, panel, module, box, window, and sidebar are, and being able to nail down position (within, under, in the left/right of, center, bottom, top, fixed, relative). Then saying things like “update xx style”, “follow xx style”, “move it”, “popup”, “alert it”, “double confirm”, “bring it into”. It’s basically being precise about what component goes where and what you want it to do.

But model hallucinations still happen here too. In one product, the header tab widths were off and kept covering other buttons. After 4–6 rounds of conversation, the model still insisted it had set x columns spanning x slots. Even after two rounds of venting at it and reviewing all the page code, nothing changed. I ended up going into the code directly and fixing it by hand. That fix got locked into style rules after.

On the language side: using English for prompts in Cursor is genuinely more effective. Even if the model can handle Chinese, I’ve tried adjusting UI in Cursor with Chinese and Gemini 3 — the results weren’t great. Also, use MAX mode instead of Auto when you can. Keep in mind that some models (Sonnet 4.5 for example) default to 200K context, and with MAX mode it supports 1M — but anything over 200K racks up fast. One conversation could run you $1–2. So be explicit and specific with your prompts.

0x03. Style Transfer: Learning Rules

If you’ve followed along and actually tried the prompts and bug fixes above, you can already ship a working product. But if you want to scale that into a recognizable product line with consistent style, you need Cursor Rules.

You can also have the Agent extract the style from a product you already like. (This means you can prototype a minimal product first to validate backend style, frontend style, and other conventions — then document it all for future projects.)

Have the Agent output your frontend style:

@Codebase I want to reuse the frontend style and component patterns from this project in a new project that will have a completely different backend.

Please analyze only the frontend code (UI components, CSS/Tailwind usage, animations, state management, and directory structure). Ignore all backend logic, API calls, and database schemas.
Based on this, write a .cursorrules file that defines:
1. The Visual Style: Color palette, spacing, typography, and specific Tailwind class patterns we use frequentely.
2. Component Architecture: How we name components, where we store them, and how we handle props/types.
3. Frontend Tech Stack: Explicitly list the frontend libraries we use (e.g., React, Lucide icons, Framer Motion, Shadcn UI) but mark the backend as 'flexible/agnostic'.
Output this as a single code block I can copy.

Have the Agent output your backend style:

@Codebase Act as a Senior Software Architect. I want to start a new project and I want to strictly follow the coding style, architectural patterns, and best practices established in this current backend codebase.

Please analyze my backend files (controllers, services, models, utils, config, etc.) and generate a comprehensive set of "Project Rules" (cursorrules). 

Focus on the following specific areas:
1. **Tech Stack & Libraries**: What framework, ORM, validation libraries, and tools are used?
2. **Directory Structure**: How is the logic separated? (e.g., Repository pattern, Service layer, MVC).
3. **Naming Conventions**: How are variables, functions, files, and classes named? (camelCase, snake_case, PascalCase, etc.).
4. **Typing/Syntax**: Are we using TypeScript strict mode? specific ES6+ features? Functional vs OOP style?
5. **Error Handling**: How are exceptions caught and returned to the client?
6. **Response Format**: What is the standard JSON structure for API responses?

**Output Goal:** 
Produce a markdown block formatted specifically for a `.cursorrules` file. It should consist of clear, instructional "You will" or "Always" statements so that future AI generations follow this exact style.

Save those outputs as frontend-style.mdc and backend-style.mdc, then create a .cursor/rules folder in your new project and put them there. If you’ve run into package management issues or unfixable header problems like I have, also create python-package-management.mdc and header-fix-style.mdc. Just don’t make individual rule files too long.

My recommended rule categories:

• Backend coding style
• Frontend coding style
• Package management conventions (e.g., prefer uv for Python packages)
• Secure coding conventions

Here’s an example of my backend coding style rules:

## 1. Tech Stack & Libraries
- **Framework**: FastAPI (Python 3.10+)
- **Database**: PostgreSQL
- **ORM**: SQLAlchemy 2.0+ (Declarative Mapping)
- **Validation**: Pydantic V2
- **Security**: `passlib[bcrypt]`, `python-jose` (JWT), `cryptography`
- **Task Queue**: Celery with Redis
- **AI Integration**: Google Generative AI

## 2. Directory Structure & Architecture
You will strictly follow this layered architecture:

- **`app/models/`**: SQLAlchemy database entities.
- **`app/schemas/`**: Pydantic models for Request/Response (DTOs).
- **`app/api/v1/endpoints/`**: FastAPI route handlers (Controllers).
- **`app/services/`**: Business logic layer (Functional style).
- **`app/core/`**: Configuration (`config.py`) and Security (`security.py`).
- **`app/db/`**: Database connection (`session.py`) and Base model (`base.py`).
- **`app/api/deps.py`**: Dependency injection providers (DB session, Auth).

## 3. Coding Conventions

### Naming
- **Variables & Functions**: `snake_case` (e.g., `create_user`, `get_password_hash`).
- **Classes**: `PascalCase` (e.g., `User`, `UserCreate`, `Settings`).
- **Files**: `snake_case` (e.g., `user_service.py`, `api_key.py`).
- **Constants**: `UPPER_CASE` (e.g., `ACCESS_TOKEN_EXPIRE_MINUTES`).
- **Routes**: Kebab-case in URLs (e.g., `/change-password`), but `snake_case` for function names.

### Typing & Syntax
- **Type Hints**: You will ALWAYS use Python type hints.
  - Use `str | None` instead of `Optional[str]`.
  - Use `list[str]` instead of `List[str]`.
- **SQLAlchemy**: Use the strict 2.0 declarative style with `Mapped`.
  # Correct
  name: Mapped[str] = mapped_column(String, nullable=False)
  # Incorrect
  name = Column(String, nullable=False)
  - **Sync vs Async**: 
  - Use **Synchronous** `def` for route handlers and service functions involving Database operations (unless using an async driver specifically).
  - Do NOT mix `async def` with blocking `Session` calls.

## 4. Implementation Patterns

### Models (SQLAlchemy)
- Inherit from `Base` (`app.db.base`).
- Use `__tablename__` attribute.
- Use `uuid` for primary keys:
  id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=uuid.uuid4)
  ### Schemas (Pydantic)
- Create separate schemas for **Base**, **Create**, **Update**, and **Response**.
- Enable ORM mode compatibility in Response schemas:
  class UserResponse(UserBase):
      id: UUID
      class Config:
          from_attributes = True
  ### Service Layer
- Implement business logic as **standalone functions** in `app/services/`.
- Do NOT create Service classes (e.g., `UserService`). Use functional modules.
- Pass the database session explicitly as the first argument:
  def do_something(db: Session, param: str) -> ReturnType:
  ### API Endpoints
- Use `APIRouter`.
- Inject dependencies for Database and User:
  @router.post("/", response_model=MyResponse)
  def create_item(
      item_in: ItemCreate,
      db: Session = Depends(deps.get_db),
      current_user: User = Depends(deps.get_current_active_user)
  ):
  - Keep logic in endpoints minimal; delegate complex logic to `services/`.

## 5. Error Handling
- Use `fastapi.HTTPException` for known errors.
- Raise exceptions directly in the Endpoint or Service layer when a specific logic rule fails.
  if not user:
      raise HTTPException(status_code=404, detail="User not found")
  - Use standard HTTP status codes (200 OK, 201 Created, 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found).

## 6. Response Format
- Always define a `response_model` in the decorator.
- Return ORM objects or Pydantic models directly; FastAPI will handle serialization.
- Do NOT wrap responses in generic envelopes (like `{ "data": ... }`) unless specifically required by a standard. Return the resource directly.

Good coding conventions actually mean good architectural design. New features slot in faster, and models need fewer tokens to figure out what to do. It also makes running multiple Agents in parallel much more feasible. But the start of any product is never the code — it’s the requirements. The space between requirements and product is full of a different kind of design thinking and trade-offs.

0x04 Side Notes

Whether it’s “talk is cheap, show me the code” or “code is cheap, show me the prompt” — times change, tools change, but humans still have to drive them. I keep hearing the same complaint from friends who’ve gone all-in on AI coding: “Before, I spent most of my time writing code and not much debugging. Now the AI writes code fast but debugging takes two or three days.” There’s no such thing as 100% perfect implementation — probability distributions don’t work that way. But AI coding has genuinely accelerated prototyping, which lets you focus more on delivering the actual requirements rather than getting lost in implementation details. That said, it’s not really the right fit for large-scale projects.

I started experimenting with AI coding after GPT-3.5 Turbo launched. But after Gemini 3 showed up, I realized that using it for large projects was no longer out of the question.

My previous AI Studio API key was only Tier 1, so I upgraded to Cursor’s $20/month plan — which ran out fast. Then I switched to $60/month — still not enough. Opened $50 of On-Demand credit, burned through $40+ in under two days. Eventually bit the bullet and went with the $200/month plan.

When I did the math against what actually got built though, the cost-to-value ratio is actually solid. (The screenshot only shows one week — I was probably actively coding for about 5 days.)

Here are some screenshots of recent product UIs (the stuff I’ve been building lately finally feels like actual products rather than tools or services):

I originally planned to write a summary on AI-driven product design back in April, revised the plan in July (split out Python coding into its own piece). I was going to finish “AI Coding and Product Design Practices” today, but by the time I got through the product design section it was already way too long. So I’m cutting it here. Product Design Thoughts — continued in the next post!

Below are some other tools I’ve built with AI, from the original outline (listed in rough chronological order — none of them feel as much like real products as the recent stuff shown above):

• Web-based Code Review tool (Python+Flask — paste code, get vulnerability analysis and code review; backend is GPT-4o)
• Miscellaneous work utilities (HTML-based)
• Mac reading bot (Python-based — screenshot any part of a PDF you don’t understand, the bot does summary & research in the background; backend is GPT-4o)
• Apple Watch six-line divination (Yijing hexagram) app (Swift, Cursor editor)
• Random hexagram web page (a web-based particle sword-flight-style effect — I have no CSS skills, this was 100% AI; I never learned the CSS it wrote, which shows)
• CISSP exam tool (pure frontend, JS+React; built different versions with Claude 2.7 and Gemini 2.5)
• AI Browser (Node.js+Electron — the idea was a browser cockpit where AI receives real-time browser context and provides analysis)
• Vault platform (Python+Flask, Java+Javalin — supports EAAS, multi-CA integration, certificate management, and organizational certificate issuance; AI can compress 300 person-days into 30)
• Backtesting platform (Python+streamlit — a small visualization tool for common strategy backtesting)

Beyond those, some web style adjustments (blog, book-list site, etc.), security center landing pages similar to SRC, and a few product prototypes:

• Solution Craft (a platform for managing the full product development lifecycle, designed to bring in different departments for resource management, with SDLC and DevSecOps built in by default, supporting mappings for multiple frameworks)
• Aladdin’s Lamp (a mental health platform focused on left-behind children)

A security architect’s attempt to break down the knowledge you actually need in day-to-day cryptography operations — seeing the trees without losing the forest.

0x00 Why This Piece

Applied cryptography is one of those areas where the theory sits in textbooks, the primitives sit in well-tested libraries, and yet operations still manage to get it catastrophically wrong. Most of the failures I see in the field are not “we broke the math” — they are “we used the library wrong”, “we let the key live too long”, or “we designed our own scheme”. This piece walks through the fundamentals you should have at the tip of your tongue, and then a candid list of the operational anti-patterns I’ve watched real financial institutions ship into production.

0x01 Fundamentals

The most common artifacts in day-to-day operations are certificates and private keys in DER, CER, and PEM format. What is actually encoded inside those files? All of them are ASN.1-encoded blobs following a specific data structure. On the other hand, the unpredictability that real randomness provides is what ultimately determines whether a key is safe or not.

1. Formats and encoding

ASN.1 is a family of encoding rules that turns a data structure — for certificates, the structure is defined by X.509 — into a binary payload you can store or transmit. Depending on the use case, that ASN.1 blob is then wrapped in different container formats (DER, PEM, PKCS#7, PKCS#12, and friends), and toolchains convert between them all day long.

// A side note on the X-series standards: if you’re curious, dig into how ITU-T, IETF, ISO, and IEEE actually collaborate. Look at how the sausage gets made — why we have ASN, RFC, and ISO standards side by side, how the responsibility for different domains is split, and how each body pushes adoption. As a very recent example, ITU-T Y.3830 (2024) is the latest recommendation on post-quantum cryptography (PQC).

2. From random numbers to elliptic curves

Start with random number generation. In practice you deal with two flavors: TRNG (true random number generators, sometimes written HRNG for “hardware RNG”) and CSPRNG (cryptographically secure pseudorandom number generators). An HSM (Hardware Security Module) ships with a TRNG block and provides genuine true randomness. Real randomness is the foundation of the entire key hierarchy — unpredictable true randomness is what you use for keys, IVs, nonces, and anything else that must not repeat. Outside HSMs, everyday hardware also exposes a CSPRNG, typically by reading various electrical-noise sources into an entropy pool and running a deterministic algorithm over it. But that all lives at the hardware and kernel layer. At the system layer, the OS wraps it behind a syscall or file (/dev/urandom, getrandom(2), BCryptGenRandom, etc.). Language runtimes and SDKs then wrap those syscalls to give you cross-platform access. Common SDKs include OpenSSL (C/C++, with bindings for most languages), Tink (Java-family), Bouncy Castle (Java-family), and cryptography.io (Python). These SDKs implement the common primitives, expose the relevant mechanisms, and support certain cipher suites out of the box. Some go further and offer a pluggable provider model plus a minimal amount of key management.

But under all of that, what actually matters is the math: curve equations, modular arithmetic, finite fields, and so on.

I’ve covered the common symmetric and public-key algorithms in an earlier piece (see Applied Cryptography and Crypto Infrastructure), but elliptic curves were missing there, so I put together this slide specifically to fill that gap.

RSA is based on the hardness of integer factorization; ECC (elliptic curve cryptography) is based on the discrete-log problem over an elliptic curve. The diagram uses the NIST P-256 curve as its running example — I took the actual curve parameters and plotted them with matplotlib. You can’t really “see” a curve over a prime field the way you can see one over the reals, but you can at least get some intuition. I’ll skip the group construction and the specifics of point generation; look at the diagram. The two things worth zooming in on are ECDH and ECDSA. For ECDH: because scalar multiplication on the curve is commutative under the group law, S = k_A · (k_B · G) = k_B · (k_A · G) = S' — which means two parties can arrive at the same shared secret while only ever exchanging public points. That shared secret is then passed through a KDF to derive a symmetric key for the rest of the session. Worth noting: on a well-behaved curve, the public point Q is guaranteed to land on the curve. If it doesn’t, you’re either on a broken curve or being fed an invalid point — and both cases can be exploited (invalid-curve attacks). In operations, you don’t need to prove any of this yourself; just use a well-known, publicly vetted library. Similarly for ECDSA: if the per-signature nonce k is not fresh randomness — worst case, reused or fixed — the private key leaks in one or two signatures. Look up the Sony PlayStation 3 case; it is the canonical example.

For international readers unfamiliar with the Chinese national standards: SM2 is China’s national elliptic-curve public-key algorithm (GM/T 0003, roughly analogous in intent to ECDSA/ECDH over a specific 256-bit prime curve). SM3 is China’s national cryptographic hash function (GM/T 0004, output size 256 bits, comparable to SHA-256). SM4 is China’s national symmetric block cipher (GM/T 0002, 128-bit block, 128-bit key, comparable to AES-128). You’ll run into these constantly in any Chinese payments, PKI, or compliance context.

0x02 What Actually Goes Wrong in Operations

ECDH solves the key-exchange problem, but it does nothing about the asymmetric security postures of the two parties actually operating the system. True randomness gives you unpredictability, but does not protect you from a plaintext leak somewhere in transit. The algorithm may be safe; the library implementing it may not be; the call sites in your own code definitely may not be; and human operations are their own class of unpredictability.

Over the last two years I’ve seen an alarming amount of what I can only call Mickey Mouse Operations (MMO) inside traditional financial institutions. Let me walk through the anti-patterns and the exact excuses I’ve heard from real teams.

Some Chinese-specific context for international readers, since it will surface in a few of the anecdotes: cryptography compliance assessment (密评) is a mandatory, regulator-driven audit of how commercial cryptography is designed, deployed, and operated inside critical information infrastructure. Commercial cryptography retrofit (商密改造) is the project of replacing legacy or foreign algorithms with SM-series national algorithms to satisfy that assessment. MLPS (等保, Multi-Level Protection Scheme) is the broader Chinese cybersecurity classified-protection regime that sits above it.

• Don’t design your own encryption algorithm — and don’t roll your own crypto library either.
  MMO: teams that design their own AK/SK scheme, then “encrypt” data by taking the last N characters of the SK and using that as a key.

• The key is as valuable as the data it protects. Treat storage, access, and audit accordingly.

• Don’t use a fixed IV.
  MMO: “We standardized a fixed IV in the integration spec, so every connecting institution uses the same value. A per-message IV would add engineering cost.”

• Don’t use the same key indefinitely — every key needs a lifecycle.
  MMO: “Is there a hard regulation that forces rotation? Can we skip it? Can we file for an extension? Rotating keys impacts the business!”

• Don’t reuse the same key across business scenarios, and similarly, don’t reuse the same random value for two different purposes. Certificates should also be scoped by use case — a signing certificate and a data-encryption certificate should not be the same certificate.
  MMO: “Two keys is a hassle, one is enough, right?”

• Don’t stack keys under keys outside of an actual hybrid-encryption scheme. There is no reason for one symmetric key to wrap another symmetric key several layers deep.
  MMO: key A is stored in cleartext to protect key B, key B is wrapped by a key inside tool C, and the key inside tool C is wrapped by the HSM’s LMK. Congratulations — you have four rings to break and the outermost one is plaintext.

• Compute the MAC over the ciphertext, not the plaintext. In other words: encrypt-then-MAC, not MAC-then-encrypt. Modern AEAD modes (GCM, ChaCha20-Poly1305) do this correctly for you by design.
  MMO: “What’s the difference? Whichever is easier. I want to MAC the plaintext first.”

• Don’t store private keys in cleartext, and definitely don’t store the password protecting a private key in cleartext.
  MMO: “Just zip it with a password when transferring! The password? Oh, we keep it in the config file next to the key.”

• Don’t store private keys on local disk. Put them in a dedicated KMS or HSM-backed vault.
  MMO: “We made a folder locally and set the file permissions. Isn’t that fine?”

• Don’t store the individual key shares (split components) of a private key in cleartext.
  MMO: not only did they get the components in cleartext, they XORed them together and asked me to confirm the resulting cleartext key material matched theirs. (Internal monologue: how the hell would I know?)

• Don’t use weak curves, weak algorithms, or weak cipher suites.
  MMO: “What is a cipher suite? We don’t support GCM. Why upgrade? 3DES has been working fine, why AES? Is there a regulation banning 3DES?”

• Don’t skip certificate revocation checks (CRL / OCSP).
  MMO: “I don’t validate anything. Not the CRL, not even the certificate’s own validity period. We’re on a leased line!”

• Don’t just validate the issuing CA — validate the certificate’s subject too (SAN, EKU, name constraints, and whatever business-specific attributes matter).

• Avoid self-signed certificates, and avoid issuing multi-year certificates.
  MMO: “Can you issue us a 5-year TLS certificate? Why do we need a separate test certificate (issued by a test CA) for the test environment when production uses the production certificate?” (Internal monologue: so you separate test from production by which CA signs the cert? Really?)

• Avoid wildcard certificates.
  MMO: “One certificate to rule them all — easier to use, easier to manage. Cost saving, everyone loves it.”

Other operational questions worth asking yourself:

• Should this key be protected by the LMK or by a KEK? How do you decide?
• How should you apply the key block format (ANSI X9.143 / TR-31) and choose the KBPK? How should the header fields (usage, mode of use, key version) be set?
• In a payment network, how do you actually run a rotation of the PIN key and the MAC key?
• Should there be a dedicated key custodian / key ceremony role separate from the storage role?
• Under what circumstances is it acceptable for an application to talk to the HSM directly?
• What do you do when the CFCA SDK’s certificate-issuance flow does not support a passphrase on the generated private key? (CFCA is China Financial Certification Authority, the dominant CA for Chinese financial institutions.)
• Should the root CA be kept offline? What does “HA” (high availability) actually look like for a root or issuing CA?
• If the HSM only accepts a whitelisted set of client IPs, how do you design around that?
• If the HSM does not ship an SDK and only accepts raw TCP/IP command messages, what’s the right design pattern for the Encryption-as-a-Service (EaaS) layer in front of it?
• What is the correct way to use GCM? Do you need to set the AAD (additional authenticated data)? Especially for encrypting the transaction message inside a payment flow — how do you use AAD well?
• Should the main site and the CDN have separate certificates for the same domain (two certs, one domain)?
• Should you deploy dual certificates for high availability?
• For a SaaS product exposed under an enterprise-internal domain, should the customer’s internal PKI issue the certificate?
• [Chinese-context] In CFCA’s dual-certificate scheme (a signing certificate plus an encryption certificate, both required for regulated business), what do you do when institution A supports it but institution B doesn’t? What if the HSM doesn’t expose a dual-CSR interface?
• [Chinese-context] How do you download a certificate when the CFCA CSP (Cryptographic Service Provider) is not available on your platform?
• [Chinese-context] How do you handle the dual-certificate bundle and the encrypted private key CFCA hands back to you?

0x03 Wrapping Up

This piece took ten days to prepare (10/09 – 10/19), most of which went into drawing the diagrams above. Digging through my TODO list, there were a couple of things I wanted to write down and didn’t:

• The different MAC algorithms for integrity, and how they compare.
• Where and how the key block (TR-31 / ANSI X9.143) format actually gets applied in practice.

A lot of things — once you let them slide, you never end up writing them down. In the same draft folder I also found a piece from late July on how to design AI products; I’ve opened it several times, and I’ve never finished it, and what I have written I’m not happy with.

Back to this one. Over the past two years I’ve integrated with something like forty Chinese banks, card issuers, and acquiring institutions. Only one or two — maybe four or five, if I’m being generous — struck me as operating at a reasonable level. Traditional finance is a very easy place to coast: don’t focus on the business, don’t focus on the technology, and you end up focusing on all the wrong details. The line I hear most often is “I’ve never seen this before — is this correct?”. Answering nonsense with nonsense only produces more nonsense. Mickey Mouse Operations and “financial-grade security” are fundamentally incompatible.

Finally — I’ve been trying to advocate for Less Operation, More Efficiency, and a second idea, Architecture as Code, Architecture for Compliance. This might be the first concept I’ve proposed independently since taking on an architect role? I went back through my earlier writing on SDLC, shift-left security, and defense-in-depth; and on my summaries of intrusion detection, key management, data security, and network and cloud security practice. I don’t quite know where the next breakthrough for enterprise security actually lies. I shouldn’t be feeling this jaded this fast — or am I just watching too much Mickey Mouse Operation? (Cue the music: a Chinese song called Mei Chu Xi (“No Backbone”) — the lyric that fits is roughly “should have been calm and in control; now it’s rushed, scrambling, falling over itself”. A depressingly accurate description of the JV enterprises I’ve been dealing with.)

References

• Highly recommended — Matthew Green’s blog: A Few Thoughts on Cryptographic Engineering
• A Warm Welcome to ASN.1 and DER (Let’s Encrypt)
• ITU-T Standards Recommendations
• ASN.1 Made Simple — What is ASN.1?
• SM2 Curve reference
• Standard curve database
• OIDs for Chinese national algorithms (SM series)
• Object Identifier (OID) Repository
• Applied Cryptography and Crypto Infrastructure (previous post)
• Privacy Computing and Data Security (previous post)
• Desmos — visual math tool
• SafeCurves
• OpenSSL Architecture
• Web Crypto API — getRandomValues
• Dual_EC_DRBG (Wikipedia)
• The Many Flaws of Dual_EC_DRBG (Matthew Green)

“Build once (Docker), deploy everywhere.” — yeah, I’m never saying that again.

1. Design

Collected requirements, broke them down, set a budget, wrote a BRD, made an HLS, and drew up a UI in Figma. Then for various reasons the budget fell through, so I had to jump in and build it myself.

Since I was writing it solo, I had to redesign the tech stack from scratch. I went with Python: Flask for the web layer, Celery for async task scheduling. The catch was that CFCA’s platform only had a Java SDK, so I had to wrap the business logic into an API service separately, then call that from Flask. For the database, I used an ORM to make testing and migration easier — SQLite in dev, Postgres in prod.

I didn’t diagram the frontend structure, but it basically has: User Panel, Key Management Panel, Certificates Management Panel, Crypto Box Panel, Financial Panel, etc. Each panel shows a list of keys/certs, with a search box at the top. Click into any list item and you get a detail page with management actions.

2. Build Infra

The internal resource request process at work is a nightmare, AI tools aren’t allowed, and my platform doesn’t touch any internal company data anyway — so I just paid out of pocket and set everything up on GCP. I went with Code Server as my IDE so I could code from anywhere. I’d been on AWS, Azure, and Aliyun before; my early experience with GCP was bad enough that I dropped it. Lately I’ve been picking it back up, so I figured I’d build this whole thing on GCP.

• Code-Server hosted on GCP Compute Engine
• Casdoor on Docker
• Nginx on VM

// Also spun up a microservices app using Cloud Run and Cloud Build. Worth mentioning: GCP’s security UX genuinely feels better than other clouds. This finally answered a question I got in an interview years ago — “how do you make users feel your product is secure?” GCP nails it. Whether you’re creating a Compute Engine instance or binding DNS to Cloud Run, the platform quietly nudges you toward secure defaults. That’s worth calling out.

Other tools used:

• Cloudflare (DNS Management & CDN)
• Gemini Chat / Gemini Code Assistant / Gemini Studio API Key

Note: Code Server doesn’t support multiple users.

3. Development & Testing

Methodologically, TDD is the way to go — it keeps bugs contained at the unit level. I won’t rehash the details here; check my earlier post Software Engineering in Practice: A Python Perspective, especially the section on AI-assisted coding.

That said, I do want to talk about config separation, because I burned myself on it.

On Config Separation

Directory structure:

├── cfca
│   ├── Dockerfile
│   ├── libs
│   │   ├── CommonVO-3.3.9.1.jar
│   │   ├── RAToolkit-3.3.9.1.jar
│   │   ├── SADK-3.7.2.0.jar
│   │   └── slf4j-api-1.7.36.jar
│   ├── pom.xml
│   ├── src
│   │   ├── main
│   │   │   ├── java
│   │   │   │   └── com
│   │   │   │       └── legendary
│   │   │   │           └── javalin
│   │   │   │               ├── ApiServer.java
│   │   │   └── resources
│   │   │       ├── xxx.jks
│   │   └── test
│   │       └── java
│   │           └── com
│   │               └── legendary
│   │                   └── javalin
│   │                       ├── Test3101.java
│   ├── start.sh
│   └── target
├── docker-compose.prod.yml
├── docker-compose.dev.yml
├── start.sh
├── stop.sh
└── web
    ├── Dockerfile
    ├── app.py
    ├── config.py
    ├── initdb.py
    ├── requirements.txt
    ├── settings.dev.yaml
    ├── settings.prod.yaml
    ├── tasks.py
    └── templates
        └── index.html

Pass an argument to start.sh to bring up the right environment — ./start.sh dev or ./start.sh prod. Each one picks up its matching compose file (docker-compose.prod.yml or docker-compose.dev.yml). Here’s what start.sh and docker-compose.yaml look like:

#!/bin/bash
ENV=${1:-dev}
ENV_FILE="docker-compose.${ENV}.yml"

if [ ! -f "$ENV_FILE" ]; then
    echo "Error: Environment file '$ENV_FILE' not found for environment '$ENV'."
    echo "Usage: $0 [dev|prod|...]"
    exit 1
fi

echo "Starting services using environment: $ENV ($ENV_FILE)"
docker compose -f  "$ENV_FILE" down
docker compose -f   "$ENV_FILE" build
docker compose -f  "$ENV_FILE" up -d

echo "Waiting for service up"
sleep 30
# Run the database initialization script
echo "Initializing database..."
docker exec web_app python /app/initdb.py
echo "Database initialization complete."

The compose file loads the right environment variables:

services:
  postgres:
    image: postgres:12-alpine
    container_name: postgres_db
    environment:
      POSTGRES_USER: XXXXX
      POSTGRES_PASSWORD: XXXXXX
      POSTGRES_DB: XXXXX
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    networks:
      - app-network

  redis-broker:
    image: redis:alpine
    container_name: redis_broker
    networks:
      - app-network

  cfca-api:
    build: 
      context: ./cfca
      dockerfile: Dockerfile
    container_name: XXXXX
    restart: unless-stopped
    networks:
      - app-network

  web:
    build: ./web
    container_name: web_app
    ports:
      - "127.0.0.1:5000:5000"
    volumes:
      - ./web:/app
    depends_on:
      - postgres
      - redis-broker
    environment:
      - CRYPTO_ENV=prod
    networks:
      - app-network

  worker:
    build: ./web
    container_name: celery_worker
    command: celery -A tasks.celery_app worker --loglevel=info
    volumes:
      - ./web:/app
    depends_on:
      - postgres
      - redis-broker
    environment:
      - CRYPTO_ENV=prod
    networks:
      - app-network

networks:
  app-network:
    driver: bridge

volumes:
  postgres_data:

Then config.py reads crypto_env to load the right settings file — settings.prod.yml or settings.dev.yml:

# settings.dev.yml
debug: true
flask_env: "dev"
secret_key: "xxxxx"

# Database Configuration
sqlalchemy_database_uri: "sqlite:///cryptovault.db"
sqlalchemy_track_modifications: true

# CORS Configuration
cors_resources:
  "/api/*":
    origins: "*"

# Celery Configuration
celery_broker_url: "redis://redis-broker:6379/0"
celery_result_backend: "redis://redis-broker:6379/0"
celery_timezone: "UTC"

# Development CFCA endpoints
cfca_api:
  base_url: "xxxxx"
  certificate_apply: "xxxxx"
  certificate_download: "xxxxx"
  timeout: 20

# Certificate Authority Configuration
ca_api:
  base_url: "xxxxx"
  key_id: "xxxxx"
  auth_key: "xxxxx"

# Email settings
smtp:
  server: "xxxxx"
  port: 587
  username: "xxxxx"
  password: "xxxxx"
  from_email: "axxxxx"
  use_tls: true

# Development SSO settings
casdoor:
  client_id: "xxxxxx"
  client_secret: "xxxxx"
  discovery_url: "xxxxx"

# Login Configuration
login:
  view: "login"

# -*- coding: utf-8 -*-
# config.py

from dynaconf import Dynaconf
import os

env = os.getenv("CRYPTO_ENV", "dev")
env_files = [
    "settings.yaml",  # Load default settings first
    f"settings.{env}.yaml"  # Then load environment specific settings
]

settings = Dynaconf(
    envvar_prefix="CRYPTO",  # export CRYPTO_FOO=bar
    settings_files=env_files,  # Load these files in order
    environments=False,  # We're using separate files instead
    load_dotenv=True,  # Load .env file
    encoding="utf-8",
    merge_enabled=True,  # Enable merging of configs
)

print(f"Loaded environment: {env}")
print(f"SECRET_KEY: {getattr(settings, 'SECRET_KEY', None)}")

Finally, app.py imports the config from config.py:

from config import settings

# --- App & Database Configuration ---
app = Flask(__name__)

# Load configuration from Dynaconf
app.config.update({
    'SECRET_KEY': settings.SECRET_KEY,
    'SQLALCHEMY_DATABASE_URI': settings.SQLALCHEMY_DATABASE_URI,
    'SQLALCHEMY_TRACK_MODIFICATIONS': settings.SQLALCHEMY_TRACK_MODIFICATIONS,
    'DEBUG': settings.DEBUG,
#   ........
})

One more thing worth mentioning: I originally tried keeping both environments’ variables in a single file, which caused env var override issues (or maybe just bad design). Also, since SQLite is a file-based database and both Celery and Flask need to read/write it — but they run in separate containers sharing a volume mount — I ran into locking issues. Mishandled locks meant non-stop errors.

4. Deployment

Deployment is basically my story with the ticketing system. Not because the process was rigorous — quite the opposite. In financial companies, people get so used to the sluggishness that absurdly slow, fully manual processes start feeling like a perfectly reasonable, compliant norm. Fast iteration isn’t the priority here; stability is. That mindset piles layer after layer onto every process and procedure, even when better controls are available. Like the principle of defense-in-depth versus redundant defense — more tickets doesn’t mean more security. Were all these tickets actually necessary?

Tickets I had to file (just the ones I can still remember — some had to be reopened multiple times):

• VM provisioning request
• Bastion host access request
• Privileged account request
• Firewall rule requests (my IP to the platform; platform to the office network; platform allowing my IP; etc.)
• Firewall exception requests and records
• Service mailbox request
• Domain name request
• DNS record request
• Internal CA certificate issuance request

And that’s not counting:

• Code scanning
• Container scanning
• Managed key provisioning
• Internal SSO integration
• SIEM integration

Reading this back, I get why developers curse DevSecOps. If I hadn’t gone through the full dev-design-deploy cycle myself, I might still be patting myself on the back for all these security controls. In reality, they hammer dev velocity. That said, there has to be a balance between fast iteration and security — figuring out what that looks like is worth thinking hard about.

5. Issues

A running log of bugs I hit from development through deployment, and how I fixed them.

Platform issues:

• Platform A’s UI triggers email notifications on certain actions; API calls don’t. Had to implement my own email notification.
• Platform A’s test environment used IP:PORT with 443, but the URL used 9443. I was only told to open port 443 in the firewall — only discovered 9443 during troubleshooting. In prod, the URL just changed the IP and port 443 was all that was needed.
• Downloaded the login cert for Platform A’s test environment but the platform side didn’t sync, causing access failures.
• Platform A’s prod login cert converted to JKS format but was missing the OCA1 certificate chain, resulting in “NO Trust Store.”
• Platform B’s API docs listed different parameters than what actually worked. Error messages were basically zero. {K:[V]} instead of {K:V}, some fields expected string-of-list but not string. Switching to the latest docs version fixed it.

Docker/Podman:

• Podman claims Docker compatibility, and Rocky defaults to Podman — but a Docker-saved image loaded into Podman didn’t run cleanly. (Needs deeper investigation.)
• [Dev] On Windows: failed to solve: failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount770356576/Dockerfile: no such file or directory. Fixed with: cd "C:\Program Files\Docker\Docker" && ./DockerCli.exe -SwitchLinuxEngine
• [Dev] After packaging Java into an image, runtime threw “cannot find main class.” Turned out COPY --from=builder /app/target/*.jar /app/application.jar wasn’t copying the right jar. Specifying the exact jar name fixed it, even if there was only one jar in the target dir.
• [Dev] Compute Engine ran out of disk space. docker system prune -a -f && docker volume prune -f frees some space, but mounting a new disk is the real fix.
• [Dev] Couldn’t save CASDoor config because of disk space.
• [Dev] During docker-compose build, local images were missing but couldn’t be pulled despite being logged in to the registry. Manually pulling the images first, then running docker compose build, worked fine.

OS note: used Ubuntu in dev, Rocky in deployment. We use docker compose now, not docker-compose. Almost every Docker issue I hit in prod (Rocky 8) never showed up once in dev (Ubuntu).

• [Deploy] Rocky defaults to Podman; installing docker-ce requires separate repo configuration.
• [Deploy] docker load/save vs import/export behave differently. Need to preserve all layers. Podman doesn’t support import.
• [Deploy] PostgreSQL started right out of the box on Ubuntu with docker compose. On Rocky, it wouldn’t come up.
• [Deploy] Services in the same network couldn’t reach each other by service name.
• [Deploy] Nginx reverse proxy couldn’t reach Docker’s exposed local ports; had to specify the container IP directly. Probably a firewalld issue.
• [Deploy] When exposing ports to localhost, the app inside the container couldn’t reach 127.0.0.1:port.
• [Deploy] Container crashed on startup. Default ENTRYPOINT was running the Java process. Passed an override to docker to get into the container for debugging: docker run --entrypoint /bin/bash cfca-api
• [Deploy] Docker services couldn’t find each other by service name. Fixed by adding a Docker Network config and putting everything in the same network in the compose file.
• [Deploy] Container tried to load a local file on startup but didn’t have permissions to open it.
• [Deploy] The internal company SMTP server uses the domain account as the username, not the email address.
• [Deploy] Nginx set up as a reverse proxy, but the SSO callback URL was http://. After Nginx completed SSO auth over HTTPS, it redirected back to HTTP — access failed. Fix: redirect all HTTP to HTTPS in Nginx. Better fix: use an https:// callback URL in SSO config from the start. Related: also remove the 127.0.0.1 test callback URL from SSO.
• [Deploy] docker compose brought up Postgres fine normally, but on Rocky it complained about a missing pg_hba.conf. After adding it, restart failed with password authentication errors. Checked inside the container — no matching user. Creating one didn’t stick; using init.sql during container creation also didn’t work. Eventually got it running by starting Postgres standalone and passing command-line arguments. Root cause still unknown.

Windows:

• Celery behaves differently on Windows vs Linux. Need to add -P eventlet, otherwise the terminal hangs: celery -A tasks.celery_app worker -l info -P eventlet. The default prefork backend doesn’t work on Windows because fork() is a Linux syscall. Watch out though — Celery Beat doesn’t support -P eventlet.
• pipreqs failed with an encoding error when generating dependencies. Fixed with: pipreqs . --force --encoding utf-8

Coding issues:

• JKS file path was different between running tests and running from the packaged image.
• Printed to stdout but nothing showed up — switched to the logging module.
• Celery task failed but the status was recorded. Even after restarting the task, the status wasn’t refreshed. (Still not sure why.)

Human error:

• Typos: xxxxservice vs xxxxservices; orgnization vs organization.
• Firewall needs to be opened on both sides: outbound from the datacenter AND inbound at the target endpoint. Was told the firewall was opened, tested and it still didn’t work — there was a deny all rule still in effect.
• After config separation, plugged in the correct prod token — got “unauthorized.” Turned out the platform side had created but not enabled the token.

AI-related:

• Deleted code it shouldn’t have, logic changed, cascade of errors.
• Added features it shouldn’t have, page behavior changed.
• Added health checks while optimizing the Docker Compose file — but the service itself had no health check endpoint, and other services depended on it being healthy before starting. Everything failed in sequence.

6. Conclusion

On AI: over the course of this project (~2 months of spare-time work), AI showed insane productivity gains, especially on frontend code. Feels great — but it also exposes a real problem: if you only know how to prompt and never think for yourself, you dull your own skills.

On deployment: I was way too confident in the “build once, run anywhere” Docker promise. What I estimated as a one-day deployment took about three days.

On security: looking at it from a dev perspective, DevSecOps actually demands a pretty high bar for both technical maturity and developer quality. If a company doesn’t even have solid DevOps culture, talking about DevSecOps is wishful thinking.

And finally — the people who do security are often the least secure. Still true.

One last thing: skills you don’t practice fade fast. Which is why repetition and hands-on work is itself a skill.

This post went through several working titles — “Do You Need a Security Architect?”, then “The Art of Balance in Security Architecture Design”, then “A Security Architect’s Venting Session”, then “The Team’s Deadweight — Security Architects”

What Does a Security Architect Actually Do?

First, ask yourself: is a security architect’s job just doing architecture design? From an engineering angle, a security architect needs to cover a lot of ground. At the top level: writing Policy (senior folks might also need to define Strategy), analyzing requirements, scoping, identifying gaps, building solutions. In the middle: implementation, POC and vendor selection, deployment and ops, delivery. At the bottom: security operations, alert analysis, writing SOPs, knowledge training, incident response, post-mortems. In other words, they need to be able to step into any role at any time.

From a team and project angle, a solid architect should be able to lead a project from start to finish, handle cross-team communication, coordinate competing interests from a solution perspective, get stakeholders on board, and actually drive things forward. And then follow through.

From a governance angle, they need to understand how regulatory and compliance requirements translate into technical language — meaning: whose rules are we playing by? Who’s the regulator? What’s the cost? What can we actually do? Where are the hard lines and where are the gray areas? How do you turn official language into something actionable inside a company? How do you fit frameworks, models, and new tech into an existing stack? Where do process and policy fill the gaps? Does this require a new product? Can we score a quick win?

Of course, that’s the ideal. In practice, an architect only occupies a tiny slice of all that. They play a specific role in a specific area. Sometimes they end up doing things that have nothing to do with any of it.

Does a Security Architect Need to Be Professional?

My answer: Yes. Not just technically, but in how they operate professionally. Whether you’re running a POC or rebranded as a Business Partner, you need to actually play the role. I’m not a rigid process zealot, but I do insist on the non-negotiable checkpoints. A lot of people say “tech isn’t the issue” — which usually means they’ve already run into a ton of technical problems. And sure, plenty of people are smarter about it: “It’s not my company, why should I care? If I leave, it’s not my problem anymore.” No judgment on that — different positions lead to different conclusions.

On the question of whether a security architect needs to be professional, let me throw out some questions to think about:

• How do you think about and evaluate future architecture needs from a business perspective?
• How do you avoid “cost reduction + efficiency” becoming “cost reduction + embarrassment”? In a cost-cutting exercise, which things can use open source — or can everything just use open source?
• Am I building defense in depth or just defense in repetition? // turns out I was the one covering my own ears
• Where is the right balance in technical design?
• Are the goals and metrics you’re setting actually achievable? How do you measure them?

Beyond technical skills, team management really matters. If management isn’t working, all the technical solutions and “collaboration” in the world mean nothing. You can rebrand roles a hundred times — from POC to BP (Business Partner) — and it’s still the same thing with a different name. Sometimes what looks like a perfectly normal initiative moving smoothly forward (or maybe that’s just my wishful thinking) hits a weird colleague and suddenly everything goes sideways, turning into a pile of garbage.

I’ve also collected some case studies that show whether technical depth actually matters. (Once you’ve seen all the chaos, maybe you’ll understand why I care about being professional):

• Financial enterprises often refuse to put data in the cloud for various reasons, which spawned these so-called “compute-storage separation” architectures. Where exactly is the boundary? At a glance, it seems like decryption location defines the boundary. But then: how do you determine that boundary? Can you use memory dumps to check if memory is secure? Should you require vendors in cloud environments to provide documentation on key security — on top of keeping customer data in the IDC? Does that mean bringing in TPM? Two-level or three-level key hierarchies? Do you need key components? For key distribution and verification, do you need keyblock and padding mode requirements? GCM or CBC? What if the business doesn’t support it?
• Financial enterprises also work hard to keep PII out of places it shouldn’t be — like logs. So what’s the solution: upgrade a shared SDK to handle unified log desensitization? Process all middleware that emits logs and run everything through a rule engine? Go through all the code, run some regex, package a new release?
• Lately every vendor wants to do ALL-IN-ONE, everyone’s jumping on the same bandwagon. The pitch sounds good, but how do you actually use it? XDR can push baselines, but what about UEM? Here’s a real problem: overlapping features. How do you make sure a setting only lives in one place? You’ve got Proxy wanting to decrypt traffic, DLP wanting to decrypt traffic, SASE wanting to decrypt traffic. And then you’ve also committed to M365 as your ecosystem (setting aside the enormous pain of domestic vs. international version differences). How do you integrate third-party products? You want Conditional Access to trigger two-step verification in one place — but what about Entra? One product needs a blocklist, another does too, and suddenly you’ve got a pile of products trying to run simultaneously.
• Cloud and SaaS adoption is also a selection trend. If all my vendors only offer SaaS products, what controls can I actually implement beyond contract clauses, procurement requirements, and third-party certifications?

In all these scenarios, it’s not just about identifying problems technically and proposing solutions — it also tests whether you have the mental resilience to step back when you need to. I bet most people have had the thought: “It’s not like it won’t work.” But don’t give up before you’ve even started. Sometimes you know a decision isn’t yours to make, but you shouldn’t surrender before you’ve even tried.

I hate being unprofessional, so I often doubt myself. If it’s an environment problem, don’t drain yourself over it — it’s not you. In whatever position you’re in, just do your job well. Sometimes I hear some real circular nonsense and almost have to laugh. For example: “The feature can definitely be implemented, but it might affect performance.” But if you actually follow up — asking when it can be done and how much performance degradation to expect compared to the original — suddenly you’re the difficult one, the jerk who doesn’t know how to read the room. Doesn’t matter your tone, your attitude, or your intentions.

Does Your Team Need a Security Architect?

This role seems like it can help you get better, but can’t stop you from getting worse.

I genuinely can’t answer this — it’s a question for the bosses. Small companies probably don’t have a security architect and wouldn’t hire one separately. A single security engineer covers operations and maintenance. There’s no point dreaming about comprehensive policies or procedures — getting a basic SOP off the ground is already a miracle. Meanwhile, just staying alive as a company is the real win. Forget about budget — beyond the things you absolutely have to buy, everything is open source. Documentation accumulates depending on luck. For B2B companies, besides getting a security certification from a third party for sales purposes, there’s basically nothing else. As the team grows to 3–5 people, it means the company has gone from noticing security to actually paying for it — but in practice, nothing really changes. A few people rotate through operations products, handle attacks, process alerts. Management buys some peace of mind. At 5–10 people, compliance starts coming into focus — there’s investment in certifications like Level 3 Protection and ISO 27001, dedicated roles start appearing. But behind a professional compliance program, you often don’t need a dedicated compliance team. A solid project manager can pull it off — just collect the right materials. Of course, actually finding a professional with a security project management background is like finding a needle in a haystack. Open source products still require in-house deployment and operational expertise. In-house development isn’t necessarily cheaper than commercial procurement, but it fits internal customization needs better. And pure vendor outsourcing management? What can you really get out of that? In many foreign companies, it’s just a middleman plus a vendor service and that’s it. I used to think engineers needed to deeply understand their products — then I worked at a company where there were more Directors than engineers, and watched how the Proxy model actually ran. Sometimes I think a few real estate brokers could do just as well. At least they’d have a clear understanding of their scope and know where the boundaries are. Of course, when there are people like that in the team, a security architect isn’t really needed anymore — because no one can save it at that point.

So the question is: when you feel operational standards are too low? When you feel the products aren’t working? When you’ve spent a lot of money and ended up with a pile of junk? When you’ve hired a lot of people but the ticket backlog is still choking everyone? When you thought you bought advanced products but alerts and incidents keep piling up? Maybe a budget of 10M RMB isn’t huge, but it’s not small either — so why does the security posture keep deteriorating? From a budget efficiency, operational effectiveness, and architectural vision standpoint, do you need a security architect? Internally I think you desperately need a professional architect with real leadership to make sweeping changes. But I’m not the boss, so I can’t answer the question.

Looking back, is there a strong barrier to entry for security architecture work? Maybe not. Feed AI enough context and it seems like nothing is unsolvable. Especially when no one cares about quality, why not just use AI? And it’s not like human judgment is so much better — biases toward certain vendors, fear of the unknown, and so on. Sometimes I feel like I don’t really need to be on the current team, because I’m not contributing to cost reduction or efficiency gains at all. Less Operation, More Efficient? That’s stepping on too many people’s territory. Defense In Depth? Right, my “DID” is just repetition with no actual depth. Zero Trust? Not a chance — I’m practically running three separate MFA setups already. Who’s going to trust me?

When I have nothing to do, I scroll LinkedIn looking at how Principal Security Architects and CISOs built their careers. I thought I was looking out a window at the world, forgetting that firefly light doesn’t last long. At some point, ordinary technical practice became the entire sky as seen from the bottom of a well. If you’ve never seen anything else, how could you know it’s real? Everyone around me looks smart, with impressive backgrounds. I’m just the one who isn’t that smart.

It’s tasteless to eat, yet a pity to throw away. // actually, no — not even a pity.

This is the first article in the AI co-creation series. If you were a Chief Security Architect, what should you be doing in your first 180 days?

I’ve done some product design work, but my actual code output has been pretty limited — especially in the past two years. The good news is I’ve been picking things back up lately, so here are some notes. Fair warning: this is pretty entry-level stuff.

1. Starting with the Language

Python is a simple language with a rich ecosystem — there’s a library for basically everything. That’s what I thought before reading Expert Python Programming. The book has two editions; I read the first one back in 2017 and it completely blew my mind. Sure, you can pick up the basics in a day, but there’s a lot more worth knowing beyond that. Here are some topics worth digging into:

• Naming and comments
• Package management: __init__.py and requirements.txt
• Relative vs. absolute vs. current execution paths (pathlib)
• Calling across different Python files
• Decorators
• Context managers
• Async processing
• Parallel processing
• Variable-length arguments
• Function type hints
• Class inheritance
• Advanced data structures
• Common usage of built-in standard libraries: collections, contextlib, functools, multiprocessing, asyncio, etc.
• Dynamic file loading

1.1 Naming Conventions

• Variables: lowercase with underscores, e.g. my_variable
• Functions: lowercase with underscores, e.g. calculate_sum()
• Classes: CapWords, e.g. MyClass
• Constants: ALL_CAPS with underscores, e.g. MAX_CONNECTIONS
• Modules: short, lowercase names; underscores are fine if they help readability, e.g. my_module.py

1.2 Comments

• Block comments (#): explain the logic of the code block that follows
• Inline comments: explain a tricky line — use sparingly; good code should explain itself
• Docstrings ("""..."""): document modules, classes, functions, or methods. This is what help() and auto-doc tools pull from.

"""This is a module-level docstring explaining what this module does."""

MAX_RETRIES = 3  # Constant: max retry count

class NetworkHandler:
    """Handles network requests."""
    def __init__(self, host):
        self.host = host

def fetch_data(url: str) -> str:
    """
    Fetch data from the given URL.

    Args:
        url (str): The URL to fetch data from.

    Returns:
        str: The text content returned from the URL.
    """
    # This is a block comment explaining the logic below
    # ... some logic here ...
    data = f"Data from {url}" # This is an inline comment
    return data

print(help(fetch_data))

1.3 Package Identification

__init__.py serves two main purposes:

• Marks a directory as a Python package: Any directory with __init__.py is treated as a package by Python, allowing you to import modules from it using dot notation. Even an empty file works.

• Defines the package’s public API: You can use __all__ in __init__.py to explicitly control what gets exported when someone does from my_package import *. It’s a clean way to manage namespaces.

Example structure:

my_project/
├── my_package/
│   ├── __init__.py
│   ├── module1.py
│   └── module2.py
└── main.py

my_package/__init__.py:

print("Initializing my_package...")
from .module1 import func1
from .module2 import MyClass

# Define public API
__all__ = ['func1', 'MyClass']

1.4 Project Dependencies

requirements.txt records all third-party libraries your project depends on along with their versions.

Generating requirements.txt:

• Using a virtual environment

pip freeze > requirements.txt

• Using pipreqs

pipreqs /path/to/your/project --force --encoding utf-8

• Installing from requirements.txt:

pip install -r requirements.txt

Example requirements.txt:

celery==5.5.3
cryptography==45.0.4
Flask==3.1.1
flask_cors==6.0.0
flask_sqlalchemy==3.1.1
pandas==2.3.0
Requests==2.32.4

1.5 Path Handling

Forget os.path. The pathlib module handles filesystem paths in an object-oriented way — more readable and cross-platform friendly.

from pathlib import Path

# Get the current working directory
current_path = Path.cwd()
print(f"Current working path: {current_path}")

# Build a path
config_path = current_path / 'config' / 'settings.ini'
print(f"Config file path: {config_path}")

# Get parent directory
parent_dir = config_path.parent
print(f"Parent directory: {parent_dir}")

# Check if path exists
print(f"Does the config file exist? {config_path.exists()}")

1.6 Calling Across Python Files (Modularization)

• Absolute imports // Recommended — they start from the project root (the top-level package containing __init__.py) and are unambiguous.

# In my_project/main.py
from my_package.module1 import func1
from my_package.module2 import MyClass

func1()
instance = MyClass()

• Relative imports // Use a dot (.) to indicate the current location. One dot = current directory, two dots = parent directory. Only works inside a package — can’t be used in a top-level script.

# In my_package/module1.py, importing from module2
from .module2 import MyClass # . means starting from the current package (my_package)

def func1():
    print("This is func1.")
    c = MyClass()
    c.show()

1.7 Decorators

A decorator is essentially a function that takes another function as input and returns a new function. It lets you add new behavior to an existing function — like logging, timing, or permission checks — without touching the original code.

import time
from functools import wraps

def timing_decorator(func):
    """A simple timing decorator."""
    @wraps(func)  # wraps preserves the original function's metadata (__name__, __doc__, etc.)
    def wrapper(*args, **kwargs):
        start_time = time.perf_counter()
        result = func(*args, **kwargs)
        end_time = time.perf_counter()
        print(f"Function '{func.__name__}' ran in: {end_time - start_time:.4f} seconds")
        return result
    return wrapper

@timing_decorator
def complex_calculation(n):
    """A simulated heavy computation."""
    total = 0
    for i in range(n):
        total += i
    return total

print(complex_calculation(10000000))

1.8 Context Managers

Context managers use the with statement to automatically handle resource allocation and cleanup — ensuring things like file handles and network connections get properly closed even when exceptions occur.

• Class-based implementation // You implement __enter__ and __exit__ yourself.

class MyTimer:
    def __enter__(self):
        print("Timer started...")
        self.start_time = time.perf_counter()
        return self # This return value gets assigned to the variable after 'as'

    def __exit__(self, exc_type, exc_val, exc_tb):
        # exc_type, exc_val, exc_tb receive exception info if one occurred
        self.end_time = time.perf_counter()
        print(f"Block ran in: {self.end_time - self.start_time:.4f} seconds")
        # Returning True means the exception has been handled
        return False

with MyTimer():
    # Do something time-consuming
    time.sleep(1)

• Using contextlib

from contextlib import contextmanager

@contextmanager
def simple_timer():
    print("Timer started...")
    start_time = time.perf_counter()
    try:
        yield # Code before yield is __enter__, code after is __exit__
    finally:
        end_time = time.perf_counter()
        print(f"Block ran in: {end_time - start_time:.4f} seconds")

with simple_timer():
    time.sleep(1)

1.9 Async Processing

Async processing lets a program switch to other tasks while waiting for I/O operations (like network requests or database reads/writes), dramatically improving single-thread efficiency. asyncio is Python’s standard library for writing concurrent code.

• async def: defines a coroutine
• await: pauses the current coroutine and waits for an awaitable to complete

import asyncio
import time

async def fetch_data(url: str, delay: int) -> dict:
    print(f"Starting fetch for {url}...")
    await asyncio.sleep(delay) # Simulates network I/O delay
    print(f"Finished fetching {url}")
    return {"url": url, "status": "ok"}

async def main():
    start = time.perf_counter()
    
    # Use asyncio.gather to run multiple coroutines concurrently
    tasks = [
        fetch_data("http://site1.com", 2),
        fetch_data("http://site2.com", 1),
        fetch_data("http://site3.com", 3),
    ]
    results = await asyncio.gather(*tasks)
    
    end = time.perf_counter()
    print(f"All tasks done, total time: {end - start:.2f} seconds")
    print("Results:", results)

# In Jupyter or IPython, you can await directly:
# await main() 

# In a regular .py file, start it like this:
if __name__ == "__main__":
    asyncio.run(main())

1.10 Parallel Processing

Because of Python’s Global Interpreter Lock (GIL), a single Python process can’t use multiple CPU cores simultaneously. The multiprocessing module sidesteps the GIL by spinning up multiple processes, enabling true parallel computation.

import multiprocessing
import time

def cpu_bound_task(n):
    """A CPU-intensive task."""
    count = 0
    for i in range(n):
        count += i
    return count

if __name__ == "__main__":
    N = 100_000_000
    
    start_time = time.perf_counter()

    # Create a process pool that auto-manages process creation and teardown
    with multiprocessing.Pool(processes=4) as pool:
        # Use map to distribute tasks across the pool
        # Split the task into 4 parts
        results = pool.map(cpu_bound_task, [N//4] * 4)
    
    total_result = sum(results)
    end_time = time.perf_counter()
    
    print(f"Parallel result: {total_result}")
    print(f"Parallel time: {end_time - start_time:.4f} seconds")

    # Compare against single-process
    start_time_single = time.perf_counter()
    single_result = cpu_bound_task(N)
    end_time_single = time.perf_counter()
    print(f"\nSingle-process result: {single_result}")
    print(f"Single-process time: {end_time_single - start_time_single:.4f} seconds")

1.11 Variable-Length Arguments (*args and **kwargs)

• *args: packs multiple positional arguments into a tuple
• **kwargs: packs multiple keyword arguments into a dict

def flexible_function(*args, **kwargs):
    print("Positional args:", args)
    print("Keyword args:", kwargs)

flexible_function(1, "hello", True, name="Alice", age=30)

1.12 Type Hints

Python 3.5 introduced type hints — you can (but don’t have to) add type information to function parameters and return values. Static analysis tools like mypy can then use these hints to catch type errors.

from typing import List, Optional

def process_data(data: List[int], name: str) -> Optional[float]:
    """
    Process a list of integers.
    :param data: List of integers.
    :param name: Name of the dataset.
    :return: The average after processing, or None if the list is empty.
    """
    if not data:
        return None
    print(f"Processing data for {name}")
    return sum(data) / len(data)

# mypy would flag this call as an error:
# process_data("not a list", "test")

1.13 Class Inheritance

class Animal:
    def __init__(self, name: str):
        self.name = name

    def speak(self):
        raise NotImplementedError("Subclasses must implement this method")

class Dog(Animal):
    def speak(self):
        return f"{self.name} says Woof!"

class Cat(Animal):
    def speak(self):
        return f"{self.name} says Meow!"

my_dog = Dog("Buddy")
my_cat = Cat("Lucy")

print(my_dog.speak())
print(my_cat.speak())

1.14 Advanced Data Structures

The collections module provides high-performance specialized container types as alternatives to Python’s built-in dict, list, set, and tuple.

• collections.defaultdict: automatically creates a default value when you access a missing key
• collections.Counter: a dict subclass for counting things
• collections.deque: double-ended queue, O(1) appends and pops from both ends
• collections.namedtuple: tuple subclass with named fields for better readability

from collections import defaultdict, Counter, deque, namedtuple

# defaultdict
s = 'mississippi'
d = defaultdict(int)
for k in s:
    d[k] += 1
print(sorted(d.items()))

# Counter
c = Counter(s)
print(c)
print(c.most_common(2)) # Top 2 most frequent elements

# deque
q = deque(maxlen=3) # Queue with max length of 3
q.append(1)
q.append(2)
q.append(3)
print(q)
q.append(4) # Adding a new element pushes out the oldest one
print(q)
q.appendleft(0)
print(q)

# namedtuple
Point = namedtuple('Point', ['x', 'y'])
p = Point(10, 20)
print(p)
print(p.x, p.y)

1.15 Standard Library Highlights

• functools

This module is mainly for higher-order functions — functions that operate on or return other functions.

• functools.wraps: use this inside decorators to preserve the wrapped function’s metadata (already shown in the decorator section)
• functools.lru_cache: a powerful decorator that adds Least Recently Used (LRU) caching to a function — great for expensive computations with the same inputs

from functools import lru_cache
import time

@lru_cache(maxsize=None) # maxsize=None means unlimited cache size
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

start = time.perf_counter()
print(fibonacci(35))
print(f"Fibonacci with cache: {time.perf_counter() - start:.6f}s")

# Without caching, fibonacci(35) would take ages

• contextlib
• collections
• asyncio
• multiprocessing

1.16 Dynamic File Loading

Sometimes you need to dynamically load Python modules at runtime based on config or user input. This is super common in plugin-style architectures. importlib is the standard way to do it.

Say you have this plugin structure:

plugins/
├── __init__.py
├── plugin_a.py
└── plugin_b.py

plugin_a.py:

def run():
    print("Executing Plugin A")

plugin_b.py:

def run():
    print("Executing Plugin B")

Main program dynamically loading and running plugins:

import importlib

def load_and_run_plugin(plugin_name: str):
    try:
        # Dynamically build the module path
        module_path = f"plugins.{plugin_name}"
        
        # Use importlib to dynamically import the module
        plugin_module = importlib.import_module(module_path)
        
        # Check if the module has a 'run' function and execute it
        if hasattr(plugin_module, 'run'):
            plugin_module.run()
        else:
            print(f"Plugin '{plugin_name}' has no 'run' function.")
            
    except ImportError:
        print(f"Error: Could not find or load plugin '{plugin_name}'.")

# Simulate loading plugins from config
plugins_to_load = ["plugin_a", "plugin_b", "plugin_c"] # plugin_c doesn't exist

for plugin in plugins_to_load:
    print(f"\n--- Loading {plugin} ---")
    load_and_run_plugin(plugin)

One last thing worth mentioning: for all the syntax, class design, async/parallel code in this section — AI can handle all of it. The code it writes is solid, often on par with or better than a mid-level engineer. But there’s no question that relying on it too much creates a dependency. If you use AI as a teacher, it can help you pick up new knowledge fast. If you use it for everything, you start to get dull — you lose the thinking process, and you stop knowing how to break a problem into steps. My take: understand what the process looks like yourself, then let AI help you implement it. That’s the better flow. (People outside the industry without foundational knowledge — that’s a different conversation.)

2. OOP and Design Patterns

Object-oriented programming (OOP) and design patterns are two topics that come up constantly in software development, yet they always feel abstract when people talk about them. In practice, OOP thinking and sensible use of design patterns can eliminate a ton of repetitive code and keep your codebase well-structured.

When we talk about OOP, we’re really talking about four core principles: Encapsulation, Inheritance, Polymorphism, and Abstraction.

# Abstract base class (demonstrates Abstraction)
from abc import ABC, abstractmethod

class Animal(ABC):
    """
    Animal - Parent class (base class)
    This is an abstract class and cannot be instantiated directly.
    It defines a behavior 'speak' that all animals should have.
    """
    def __init__(self, name):
        # Encapsulation: the 'name' attribute is encapsulated inside the Animal class
        self.name = name

    @abstractmethod
    def speak(self):
        """
        Abstract method — subclasses must implement this.
        """
        pass

# Dog and Cat inherit from Animal (demonstrates Inheritance)
class Dog(Animal):
    """
    Dog - Child class (derived class)
    Inherits from Animal.
    """
    def speak(self):
        # Overrides the parent's speak method
        return f"{self.name} says Woof!"

class Cat(Animal):
    """
    Cat - Child class (derived class)
    Inherits from Animal.
    """
    def speak(self):
        # Overrides the parent's speak method
        return f"{self.name} says Meow!"

# --- Main program ---

# Create instances of different classes
my_dog = Dog("Buddy")
my_cat = Cat("Lucy")

# Print results — demonstrates Encapsulation (we access behavior through the object without caring about internals)
print(my_dog.speak())
print(my_cat.speak())


def animal_sound(animal: Animal):
    print(animal.speak())

animal_sound(my_dog)
animal_sound(my_cat)

As for design patterns — the common ones are Proxy, Singleton, Factory, etc. Not going to write them all out here; just check this GitHub project: Python Patterns

3. Microservices

The only real measure of a microservice is whether you can rewrite it quickly in any language with a few hundred lines of code. The idea is simple: keep business logic within a single service and expose it to the outside world via API. For stateless requests, this is about as simple as it gets. For stateful ones, you need to think about data consistency and token propagation.

For writing APIs in Python, flask is the most straightforward framework, though fastapi seems to be winning lately. That said, as I mentioned — when building microservices, you can use any language. I recently ran into a platform that didn’t have a Python SDK, so I had to write the logic using their Java SDK and then wrap it into an API service for Python to call.

3.1 Inter-Service API Calls

• Using Python to provide an API service

from flask import Flask, jsonify, request, render_template, send_file
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///cryptovault.db'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False

@app.route("/api/apply", methods=["POST"])
def apply_for_asset():
    data = request.get_json()
    asset_type = data.get("type")
    
    try:
        if asset_type == 'certificate':
            if issuer == 'CFCA':
                new_cert_entry = Certificate(
                    id=new_id,
                    subject=common_name,
                    issuer="CFCA",
                    expires="Pending",
                    status="Processing"
                )
                db.session.add(new_cert_entry)
                
                history_entry = History(asset_id=new_id, status="Submitted", event_date=get_current_date())
                db.session.add(history_entry)
                
                db.session.commit() 
                # Call the celery task
                submit_csr_generation_and_ca_submission.delay(common_name, new_id, data.get("sans", ""))

Inside submit_csr_generation_and_ca_submission, we call other API services to complete the corresponding operations.

• Using Java to handle platform-specific logic and expose an internal API for Python to call

package com.legendary.javalin;

import cfca.monkiki.util.XmlUtil2;
import cfca.ra.vo.response.CertResponseVO;
import com.google.gson.Gson;
import io.javalin.Javalin;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;

import java.io.File;
import java.io.IOException;

@Slf4j
public class ApiServer {
    public static void main(String[] args) throws IOException {

        Javalin javalin = Javalin.create();
        Javalin app = javalin.start(7070);

        // Define the POST endpoint for applying for a certificate
        app.post("/applyCertificate", ctx -> {
            try {
                CertificateRequest request = new Gson().fromJson(ctx.body(), CertificateRequest.class);
                CertificateService service = new CertificateService();
                CertResponseVO responseVO = service.applyForCertificate(request);
                String responseXml = XmlUtil2.vo2xml(responseVO, "Response");
                ctx.contentType("application/xml");
                ctx.result(responseXml);
            } catch (Exception e) {
                log.error("/applyCertificate process failed", e);
                ctx.status(500).result("Error processing request: " + e.getMessage());
            }
        });

        log.info("API Server started on port 7070. Use POST /applyCertificate to request a certificate.");
    }
}

3.2 Frontend/Backend Separation

Frontend/backend separation is itself a kind of microservice pattern — especially with frameworks like React, Angular, and Vue. In a microservices architecture where everything communicates via API, separating frontend and backend actually makes for a faster development workflow. The frontend can consume and assemble data on its own without being completely dependent on the backend. Honestly though, frontend isn’t my strong suit — I rely on AI to turn prototypes into code. For flask, rendering frontend is easy enough, even if it’s not quite “full” separation. The simplest case: just render an HTML file directly. You can also go the next.js route.

@app.route("/")
def serve_dashboard():
    return render_template("index.html")

3.3 Async Calls

• Async task dispatch

app.py exposes the send endpoint; when the user clicks the send button, tasks.py triggers an async task in the background.

@app.route("/api/certs/<string:cert_id>/send_pfx", methods=["POST"])
def send_cert_pfx(cert_id):
    data = request.get_json()
    pfx_password = data.get("password")
    recipient_email = data.get("email")

    if not pfx_password or not recipient_email:
        return jsonify({"error": "Recipient email and PFX password are required."}), 400

    # Trigger the background task
    from tasks import send_pfx_email_task
    send_pfx_email_task.delay(cert_id, pfx_password, recipient_email)

    return jsonify({"message": f"Certificate is being sent to {recipient_email} in the background."}), 202

If running on Windows: celery -A tasks.celery_app worker -l info -P eventlet

tasks.py

@celery_app.task(name="tasks.send_pfx_file_with_email")
def send_pfx_email_task(cert_id, pfx_password, recipient_email):
    """
    Celery task to generate and email a PFX file in the background.
    """
    with app.app_context():
        crypto_key = CryptoKey.query.get(cert_id)
        cert_info = Certificate.query.get(cert_id)

        if not crypto_key or not crypto_key.certificate_pem or not crypto_key.private_key_pem or not cert_info:
            print(f"Error in task: Could not find certificate or key for {cert_id}")
            return

        try:
            private_key = serialization.load_pem_private_key(crypto_key.private_key_pem.encode(), password=None)
            cert = x509.load_pem_x509_certificate(crypto_key.certificate_pem.encode())
            
            pfx_bytes = pkcs12.serialize_key_and_certificates(
                name=cert_info.subject.encode(),
                key=private_key,
                cert=cert,
                cas=None,
                encryption_algorithm=serialization.BestAvailableEncryption(pfx_password.encode())
            )

            msg = MIMEMultipart()
            msg['From'] = app.config['SMTP_SENDER_EMAIL']
            msg['To'] = recipient_email
            msg['Subject'] = f"Your Certificate: {cert_info.subject}"

            body = f"Please find your requested certificate ({cert_info.subject}) attached as a password-protected PFX file.\n\nThe password to open the file is: {pfx_password}"
            msg.attach(MIMEText(body, 'plain'))

            part = MIMEBase('application', 'octet-stream')
            part.set_payload(pfx_bytes)
            encoders.encode_base64(part)
            part.add_header('Content-Disposition', f'attachment; filename="{cert_info.subject}.pfx"')
            msg.attach(part)
            
            with smtplib.SMTP(app.config['SMTP_SERVER'], app.config['SMTP_PORT']) as server:
                server.starttls()
                server.login(app.config['SMTP_USERNAME'], app.config['SMTP_PASSWORD'])
                server.send_message(msg)
            print(f"Successfully sent PFX for {cert_id} to {recipient_email}")

        except Exception as e:
            print(f"Failed to send email for {cert_id}: {e}")
            traceback.print_exc()

3.4 Containerization

Using Docker to package code. Dockerfile for a Python app:

FROM python:3.10-slim

WORKDIR /app

RUN mkdir -p /app/data

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 5000

CMD ["gunicorn", "--bind", "0.0.0.0:5000", "app:app"]

Dockerfile for a Java app:

FROM maven:3.8.5-openjdk-11 AS builder

WORKDIR /app

COPY pom.xml .

COPY libs/ /app/libs/

RUN mvn install:install-file -Dfile=/app/libs/RAToolkit-3.3.9.1.jar -DgroupId=cfca.toolkit -DartifactId=RAToolkit -Dversion=3.3.9.1 -Dpackaging=jar
RUN mvn install:install-file -Dfile=/app/libs/CommonVO-3.3.9.1.jar -DgroupId=cfca.toolkit -DartifactId=CommonVO -Dversion=3.3.9.1 -Dpackaging=jar

COPY src /app/src

RUN mvn package -DskipTests

FROM eclipse-temurin:11-jre-jammy

WORKDIR /app

COPY --from=builder /app/target/my-javalin-app-1.0.jar /app/application.jar

EXPOSE 7070

ENTRYPOINT ["java", "-jar", "/app/application.jar"]

0x04 Frameworks and Libraries

There’s a good analogy here: using a library is like shopping at IKEA — you pick what you want and use it however you like. Using a framework is like buying a fully furnished apartment — everything’s already been designed and set up for you. The difference is really about who’s in control. With a library, you’re in control; the code does what you tell it. With a framework, you play by its rules — the framework calls you. “Don’t call us, we’ll call you.”

For example, flask and django are two typical Python web frameworks, while requests and Beautiful Soup are typical libraries. On the frontend, jquery is a classic web library while react is a classic web framework. There’s also a gray zone between the two — things like ORMs that are hard to categorize. They behave like a library (you call it to do things), but they also constrain you. If you’ve used SQLAlchemy, you know exactly what I mean.

Here are some Python frameworks and libraries worth knowing:

• Web frameworks: flask, django
• DB-related: sqlalchemy, sqlite, redis
• Networking: requests, fastapi
• Data: pandas, sqlalchemy, sklearn, pytorch, keras
• Task scheduling: celery, gevent
• Cryptography: cryptography
• CLI: click
• Config: dynaconf
• Logging: logging

On the frontend side, worth knowing: react, next.js, tailwindcss, bootstrap

5. Test-Driven Development

TDD is an agile method that ensures each individual function works as expected. The classic approach: write the test case first, then write code that passes it. What I’m describing here is a bit different from that strict definition — the goal is simply to write enough test coverage so your app actually runs correctly.

• Unit testing with unittest

import os
import unittest
import json
import sqlite3
from datetime import datetime, timedelta
from app import app, get_db_connection, init_db

class CryptoVaultTestCase(unittest.TestCase):
    """Test suite for the CryptoVault Flask application."""

    def setUp(self):
        """Set up a new test client and a temporary database."""
        # Use an in-memory SQLite database for testing
        self.db_fd, app.config['DATABASE'] = ":memory:", ":memory:"
        app.config['TESTING'] = True
        self.client = app.test_client()

        # The application context is needed to work with the database
        with app.app_context():
            init_db()

    def tearDown(self):
        """Clean up the database after each test."""
        # The in-memory database vanishes on its own, so no file to close.
        pass

    def test_01_init_db(self):
        """Test if the database initialization creates all necessary tables."""
        with app.app_context():
            conn = get_db_connection()
            cursor = conn.cursor()
            tables = ["keys", "certificates", "history", "crypto_keys"]
            for table in tables:
                cursor.execute(f"SELECT name FROM sqlite_master WHERE type='table' AND name='{table}'")
                self.assertIsNotNone(cursor.fetchone(), f"Table '{table}' was not created.")
            conn.close()

    def test_02_create_and_get_ca(self):
        """Test the creation and retrieval of the internal CA."""
        with app.app_context():
            # Test CA creation endpoint
            response = self.client.post('/api/internal_ca/create')
            self.assertEqual(response.status_code, 200)
            json_data = json.loads(response.data)
            self.assertIn("Internal CA created", json_data['message'])

            # Test CA retrieval endpoint
            response = self.client.get('/api/internal_ca')
            self.assertEqual(response.status_code, 200)
            json_data = json.loads(response.data)
            self.assertIn("CN=cryptovault-ca.local", json_data['subject'])
            self.assertIn(".pem", json_data['pem'])

    def test_03_apply_for_rsa_key(self):
        """Test applying for a new RSA key."""
        response = self.client.post('/api/apply',
            data=json.dumps({
                "type": "key",
                "name": "Test RSA Key",
                "keyType": "RSA-4096"
            }),
            content_type='application/json'
        )
        self.assertEqual(response.status_code, 201)
        json_data = json.loads(response.data)
        self.assertIn('key created successfully', json_data['message'])
        key_id = json_data['id']

        # Verify the key is in the database
        with app.app_context():
            conn = get_db_connection()
            key_row = conn.execute("SELECT * FROM keys WHERE id = ?", (key_id,)).fetchone()
            self.assertIsNotNone(key_row)
            self.assertEqual(key_row['name'], "Test RSA Key")
            
            crypto_row = conn.execute("SELECT * FROM crypto_keys WHERE asset_id = ?", (key_id,)).fetchone()
            self.assertIsNotNone(crypto_row['private_key_pem'])
            self.assertIsNotNone(crypto_row['public_key_pem'])
            conn.close()