In my last post, I quit playing whack-a-mole with my coding agents. This time it’s the how: where the linters hold the mallet while my agents build, and what an agent sees when it misses a rule.
Two layers
The first layer is stock: the checks that ship with RuboCop and its plugins, plus Herb, a linter for ERB view templates. I start from plain RuboCop defaults, with RuboCop’s newer pending cops turned on, not the rubocop-rails-omakase config. On top of that I add plugins, including rubocop-rails, rubocop-minitest, rubocop-erb (so cops can read the Ruby inside ERB templates), rubocop-thread_safety and rubocop-i18n.
The second layer is custom. These are checks written for Boswell’s own rules. There are 51 custom RuboCop cops now (34 in October) and 2 custom Herb rules.
boswell-app/
├── .rubocop.yml # stock defaults + plugins, requires the custom cops
├── lib/rubocop/cop/boswell/ # 51 custom cops
├── config/rubocop/ # one settings file per cop group
├── test/lib/rubocop/cop/boswell/ # one test per cop
├── .herb.yml
├── .herb/rules/ # 2 custom Herb rules
├── .githooks/pre-commit # lint on the staged files
└── .claude/hooks/ # Claude Code hooks
Where the checks fire
Each stage is tagged by who settles it. [script] means a script determines the result. [model] means an agent judges it. [person] means I do.
ALWAYS ON (Claude Code hooks, every agent session)
session start ..... [script] set core.hooksPath, so agent commits go
through the git hook
shell command ..... [script] refuse commands that would read the
Rails credentials
new file / edit ... [script] refuse a hand-made file that a generator
should create
[script] keep a coding agent inside its own checkout
after a tool call . nothing. No hook lints on edit.
THE BUILD
1. Before intent -> spec -> plan [model] agents write them
[person] I mark the spec ready
2. Write coder agent writes code [model]
on every git commit [script] pre-commit: RuboCop, then
Herb, i18n, Prettier and the
spec lints, on staged files
on every hand-back [script] writer gate: run the tests
the diff reaches.
1st red: sent back.
2nd red: to an advisor agent.
3. Check on every new commit to the branch, in parallel
lint + tests, changed [script] then mutation testing
files only
verify acceptance [model]
criteria
code review, then triage [model] first round only. fix now /
file an issue / discard
QA on changed screens [model] final round only
4. Fix coder fixes open items, [script] limits: 3 fix rounds before
then check again advice, 6 rounds total
5. Final full bin/ci, whole repo [script] once, at the end
6. Finish merge from posted statuses [script] a merge never deploys
change to the agent rules [person] waits for me
files (CLAUDE.md, ...)
Scripts determine the order, the limits, and pass or fail. Agents write, verify, review, triage and advise.
Claude Code’s SessionStart hook sets core.hooksPath to .githooks in the project checkout. That setting lives in each clone and is never committed, so the hook sets it at the start of every agent session. A commit can still skip the hook with --no-verify. bin/ci, which runs every check on the whole repo, must pass before the build merges a PR.
The pre-commit hook runs RuboCop with --fail-level=convention, so anything at convention severity or higher stops the commit.
One theme up close: guarding customer data
Each Boswell customer has its own database. Two kinds of checks guard that data: some keep each customer’s data in its own database, and some limit what each user can reach inside it. I’ll use them as the example.
Keeping customers apart:
- NoShardSwitch: no switching databases by hand. Code switches only through
tenant.with_shard. - ShardAwareJob: a job that reads customer data has to run in that customer’s database, or it “silently reads no tenant’s rows.”
What each user may reach:
- LoadThroughPolicyScope: load or query records through the policy scope or a parent record, never straight from the model class. A policy scope is Pundit’s way of limiting a query to the records the current user may see, so an admin and a staff member can get different rows.
- ControllerAuthorizes: every action authorizes, or says why it skips.
Violating code in app/controllers/households_controller.rb:
class HouseholdsController < ApplicationController
def show
@household = Household.find(params[:id])
end
end
The cops’ response:
app/controllers/households_controller.rb:2:3: E: Boswell/ControllerAuthorizes: Authorize the show action: call authorize, or skip_authorization with a reason comment on the line before (docs/standards/authorization.md, "When to skip authorization").
app/controllers/households_controller.rb:3:18: E: Boswell/LoadThroughPolicyScope: Start the query on Household at policy_scope(Household) or a parent relation, not Household.find: the policy scope decides which rows the user may reach (docs/standards/controllers.md).
The second cop, LoadThroughPolicyScope:
# lib/rubocop/cop/boswell/load_through_policy_scope.rb
# Trimmed. Cut: the module nesting, the doc comment, and the private helpers (marked below).
# Comments starting with >> are mine, added for this post.
class LoadThroughPolicyScope < Base
# >> OUTPUT: the message the agent sees when this check fails.
MSG = "Start the query on `%<model>s` at `policy_scope(%<model>s)` or a parent relation, not `%<source>s`: " \
"the policy scope decides which rows the user may reach (docs/standards/controllers.md).".freeze
# >> TRIGGER: every method call in a controller. There is no list of query methods,
# >> so any call on a tenant model counts as a query unless the config says otherwise.
def on_send(node)
# >> Pass: the call isn't made on a class, like policy_scope(Household).find or @household.people.
return unless node.receiver&.const_type?
# >> Pass: the class isn't tenant data, like Current or a platform model.
model = node.receiver.const_name
return unless tenant_model?(model)
# >> Pass: a class method that reads no rows, like new or model_name, listed in the config.
return if non_query_methods.include?(node.method_name.to_s)
# >> Fail: the first call on the class, so one offense per chain.
source = "#{model}.#{node.method_name}"
add_offense(node, message: format(MSG, model:, source:))
end
# [cut: tenant_model?, which ShardAwareJob shares: a model whose superclass chain is tenant data.
# And non_query_methods, read from the cop's settings, one reason comment per entry.]
end
Any query that starts at a tenant model class is an offense: Household.find(id), Household.where(active: true), Household.kept and custom class methods all fail. A short list in the cop’s settings names the class methods that read no rows, like new and model_name. Anything not on the list counts as a query. policy_scope(Household).find(params[:id]) and @household.people.find(params[:id]) pass, because they start from a scope or a record the user may reach.
The same action after the fixes passes both cops:
class HouseholdsController < ApplicationController
def show
@household = policy_scope(Household).find(params[:id])
authorize @household
end
end
Each message names the file that explains the rule. Some, like ShardAwareJob, also cite the numbered tenant-isolation rules in AGENTS.md.
Violating code in app/jobs/overdue_activity_alert_job.rb:
# Tells the staff of one organization which activities turned overdue in the last hour.
class OverdueActivityAlertJob < ApplicationJob
queue_as :default
def perform(tenant_id:)
tenant = Tenant.find(tenant_id)
overdue = Activity.in_progress.includes(:activity_kind).select(&:overdue?)
return if overdue.empty?
OverdueActivitiesMailer.with(tenant:, activity_ids: overdue.map(&:id)).alert.deliver_now
end
end
ApplicationJob is what bin/rails g job writes, and nothing errors. But a plain job runs on the main database, not the customer’s, so the query finds nothing in production. The tests still pass, because the test customer’s data lives in the main database.
The cop’s response:
app/jobs/overdue_activity_alert_job.rb:7:15: E: Boswell/ShardAwareJob: Activity is tenant data, and this job inherits ApplicationJob, so the query reads the primary database. Inherit ShardAwareJob, or read Activity inside tenant.with_shard { } for each tenant. When Activity is a platform model, add it to lib/rubocop/platform_models.yml (AGENTS.md "Tenant isolation" invariant §1 and §2, docs/standards/jobs.md).
overdue = Activity.in_progress.includes(:activity_kind).select(&:overdue?)
^^^^^^^^
After the fix, it passes:
# Tells the staff of one organization which activities turned overdue in the last hour.
class OverdueActivityAlertJob < ShardAwareJob
queue_as :default
def perform(tenant_id:)
overdue = Activity.in_progress.includes(:activity_kind).select(&:overdue?)
return if overdue.empty?
OverdueActivitiesMailer.with(tenant_id:, activity_ids: overdue.map(&:id)).alert.deliver_now
end
end
ShardAwareJob runs the job in the customer’s database and time zone.
Mutation testing
Mutation testing changes your code on purpose and checks that a test fails. If no test fails, the change survived, and your tests have a gap.
Boswell uses the Mutineer gem by David Teren. It runs in the scripted check on every new commit to the branch, after the tests that cover the change pass. It only mutates changed lines in changed Ruby source files (app and lib, not tests), and a score of 90% or higher passes. The build runs it, not the agent.
A survivor goes back to the coder as one line that ends with “add a test that fails under this change.” The coder adds that test, or, when the mutant behaves the same as the original code, adds it to an ignore list with a reason.
A cold Bootsnap cache once made every mutant count as killed. A commit with 4 real survivors reported 0 until we set DISABLE_BOOTSNAP_COMPILE_CACHE=1. If your score looks perfect, check for something like that.
Start from a generator, not a blank file
Lint catches a miss after the code is written. This check works before the file exists.
Boswell has generators for certain file types. You can make a generator for anything: models, controllers, service objects, whatever. We have ones for agent tools, agents and tool policies. A Claude Code PreToolUse hook reads .claude/create-blocks.yml, which maps paths to generators. Each entry has a name, the paths, the generator to run, and why. If an agent tries to hand-write a new file at app/ai/tools/*.rb, the hook refuses it. Editing an existing file is still allowed.
The entry for agent tools:
- name: agent tool
paths:
- app/ai/tools/*.rb
generator: bin/rails generate boswell:ai_tool <name>
why: >-
Every tool inherits Ai::ApplicationTool, the one base whose requires_approval hook asks the tool's
Pundit policy whether a call waits for an operator (docs/standards/ai.md A2, A3). ...
The refusal template in the hook:
create-block: a new #{name} starts from its generator, not a hand-written file (#{relative}).
Run: #{generator}
Why: #{why}
Editing an existing file stays allowed. The blocked paths are in .claude/create-blocks.yml.
Every tool, read or write, inherits one base class. RubyLLM gives tools a requires_approval hook, and the base answers it by asking the tool’s Pundit policy:
# app/ai/application_tool.rb
# Trimmed. Cut: the doc comment, tool_name, the initializer, and the hold check (marked below).
# Comments starting with >> are mine, added for this post.
class ApplicationTool < RubyLLM::Tool
requires_approval
# >> The operator's recorded decision comes first, then the tool's policy.
def approval_resolver
self.class.approval_resolver ||
->(tool_call) { ToolApproval.decide(tool_call, operator_required: operator_required?, hold: held?(tool_call)) }
end
private
# >> No policy, or no answer: the call waits for an operator.
def operator_required?
Pundit::PolicyFinder.new(self.class).policy!.new(nil, self.class).operator_required?
rescue Pundit::NotDefinedError
true
end
# [cut: held?, the hook a reply uses to wait on a handed-off conversation.]
end
So one Pundit policy answers two questions for a group of tools: who may use them, and whether a call waits for a person.
# app/policies/setup_tool_policy.rb (doc comment cut)
class SetupToolPolicy < ApplicationPolicy
READS = %w[list_programs list_person_fields].freeze
def invoke?
admin?
end
def operator_required?
!READS.include?(record.tool_name)
end
class Scope < ApplicationPolicy::Scope
def resolve
scope.select { |tool| SetupToolPolicy.new(user, tool).invoke? }
end
end
end
The generator is a plain Rails generator. Each method writes one piece:
# lib/generators/boswell/ai_tool/ai_tool_generator.rb
# Trimmed. Cut: the module nesting, the doc comment, the locale constants, and one method body (marked below).
# The comment starting with >> is mine, added for this post.
class AiToolGenerator < Rails::Generators::NamedBase
source_root File.expand_path("templates", __dir__)
# >> Rails runs each public method in order. Each one writes one piece.
def create_tool
template "tool.rb.tt", "app/ai/tools/#{file_name}.rb"
end
def create_partial
template "partial.html.erb.tt", "app/views/agentify/tools/_#{file_name}.html.erb"
end
def add_locale_keys
# [cut: 4 lines. Renders the tool's locale keys and inserts them into every language file.]
end
def create_test
template "tool_test.rb.tt", "test/ai/tools/#{file_name}_test.rb"
end
end
Here is what it writes for a new write tool, bin/rails generate boswell:ai_tool create_program. The generator gets the shape right and leaves a marker at each choice only a person should make. Each marker names the rule that settles it:
module Ai
module Tools
# TODO(boswell-gen): say what the tool reads, writes or sends, for which agent, and what the model supplies (A9).
#
# Every tool inherits the one tool base (A2). Whether a call waits for an operator is the policy's
# answer, never this class's (A3). The tenant comes from the agent's `tools` block, never from a
# parameter (A7).
class CreateProgram < ApplicationTool
# TODO(boswell-gen): write the description the model reads, and never mention approval (A6).
description "Create program."
# TODO(boswell-gen): declare a `parameter` for each keyword of execute, and none that carries an id (A7, A11).
# TODO(boswell-gen): name the agent's tool policy; its invoke? and operator_required? answer for this tool (A3, A20).
def self.policy_class = ApplicationPolicy
def initialize(chat:, tenant:)
super(chat:)
@tenant = tenant
end
# TODO(boswell-gen): call the command or model method the admin controller uses, in the shard, and return primitives (A8, A9, A14).
def execute
tenant.with_shard do
{}
end
end
private
attr_reader :tenant
end
end
end
Until someone names the real policy, the placeholder ApplicationPolicy fails closed: every call waits for an operator. The shard block is already there, so even a new tool starts on the right side of the tenant rules. A small script fails bin/ci while any marker is left:
app/ai/tools/create_program.rb:3: a generator TODO(boswell-gen) marker is left: decide what it asks, then delete the marker
app/ai/tools/create_program.rb:9: a generator TODO(boswell-gen) marker is left: decide what it asks, then delete the marker
app/ai/tools/create_program.rb:12: a generator TODO(boswell-gen) marker is left: decide what it asks, then delete the marker
app/ai/tools/create_program.rb:14: a generator TODO(boswell-gen) marker is left: decide what it asks, then delete the marker
app/ai/tools/create_program.rb:22: a generator TODO(boswell-gen) marker is left: decide what it asks, then delete the marker
You can do the same for controllers, models, policies, jobs and the rest. It takes one more entry in the config file per kind of file.
What the agent reads when a check fails
Every custom check’s message says what to change and which file explains it, and most say why. When a check fails, the message is what the agent sees, so it has to be enough to fix the problem.
From Boswell’s maintainer standards:
A tool before prose. When a cop, a Herb rule, i18n-tasks or a
bin/ciscript can tell whether code follows a rule, the rule lives in that tool. The written standard drops the rule and keeps only its reason, and the tool’s message names the file that holds the reason.
If you copy this
- Turn a check on over old code by putting the files that already break it in an Exclude list. Only offenses in files outside the list fail, and cleanup work removes entries later.
- Lint the staged files, not the working tree. Boswell’s hook lints a temp copy of what’s staged, so an unstaged edit can’t pass or fail a commit.
- Set core.hooksPath at session start so agent commits go through the git hook.
- Use
Severity: infofor advice that prints without failing the commit. - With mutation testing, watch for anything that makes every mutant count as killed.