[Pelis Agent Factory Advisor] Agentic Workflow Advisor Report — March 2026

[github-actions[bot]]

📊 Executive Summary

The gh-aw-firewall repository has a strong and mature agentic workflow foundation — 28 compiled workflows covering security scanning, smoke testing, CI investigation, documentation, and issue management. Compared to Pelis Agent Factory best practices, the biggest untapped opportunities are: code quality improvement agents (continuous refactoring/simplification), observability/metrics for the agents themselves, and a handful of high-value missing pieces like issue triage, breaking change detection, and schema consistency checking.

---

🎓 Patterns Learned from Pelis Agent Factory

The Pelis Agent Factory runs 100+ specialized workflows organized into these key categories:

Category	Key Patterns	In This Repo?
Issue Management	Triage + labeling, issue arborist, sub-issue closer	Partial (issue-monster dispatches but doesn't triage)
Code Quality	Continuous simplicity, refactoring, style agents	❌ Missing
Fault Investigation	CI Doctor, schema checker, breaking change checker	Partial (ci-doctor present)
Metrics & Analytics	Metrics collector, portfolio analyst, audit workflows	❌ Missing
Security	Daily malicious code scan, secrets analysis, static analysis	Strong (6 security workflows)
Documentation	Daily updater, noob tester, glossary maintainer	Partial (doc-maintainer only)
Operations/Release	Changeset generator, workflow updater	Partial (update-release-notes only)
ChatOps	Q optimizer, grumpy reviewer, workflow generator	Partial (plan slash command only)
Testing & Validation	Daily test improver, CI coach, workflow health manager	Partial (test-coverage-improver)
Meta-agents	Workflow health manager, audit-workflows	Partial (this advisor only)

Key Pelis lessons applicable here:

• Specialization beats monoliths — the 3 separate secret-digger engines (Claude/Codex/Copilot) is excellent Pelis pattern
• Meta-agents are valuable — monitoring how well the agents themselves perform is missing
• Context is king for ChatOps — the plan slash command is a good start; more ChatOps would help
• Trust but verify — continuous validation workflows prevent regressions (need more here)

---

📋 Current Agentic Workflow Inventory

Workflow	Purpose	Trigger	Assessment
build-test-{lang} ×8	Build/test PRs in sandboxed env per language	PR	✅ Excellent coverage, domain-specific value
smoke-claude/codex/copilot/chroot ×4	End-to-end firewall smoke tests per engine	PR + schedule	✅ Core value prop validated continuously
secret-digger-* ×3	Red-team hourly secret hunting per engine	Hourly cron	✅ Exceptional — 3 independent engines
security-guard	Claude-based PR security review	PR	✅ Domain-appropriate, high quality
security-review	Daily comprehensive threat modeling	Daily	✅ Evidence-based, creates discussions
dependency-security-monitor	CVE detection + dep update PRs	Daily	✅ Actionable outputs
ci-doctor	Investigates failed workflow runs	workflow_run	✅ High merge rate historically
ci-cd-gaps-assessment	Daily CI/CD gap analysis	Daily	✅ Produces discussion reports
cli-flag-consistency-checker	CLI flag vs docs alignment	Weekly	✅ Domain-specific, prevents drift
doc-maintainer	Docs sync with code changes	Daily	✅ skip-if-match avoids PR floods
test-coverage-improver	Generates test PRs for security paths	Weekly	✅ Security-focused coverage
issue-duplication-detector	Deduplication with cache-memory	On issue open	✅ Uses cache-memory well
issue-monster	Dispatches issues to Copilot agent	On issue open + hourly	✅ Task orchestrator
plan	Slash command for project planning	/plan comment	✅ ChatOps pattern
update-release-notes	Updates notes on release publish	On release	✅ Basic release automation
pelis-agent-factory-advisor	This workflow	Daily	✅ Meta-advisory

---

🚀 Actionable Recommendations

P0 — Implement Immediately

🏷️ Issue Triage Agent

What: Add a triage agent that labels newly opened issues with appropriate categories (bug, feature, security, documentation, chore, etc.) and leaves a welcoming comment.

Why: Currently issue-monster dispatches issues but doesn't label them. The issue tracker shows many CI-doctor failure issues ([aw] Build Test X failed, [Security], [Deps] prefixes) but no systematic labeling of user-reported issues. Triage is the "hello world" of agentic automation with a fast payoff.

How: Add issue-triage.md triggered on issues: [opened, reopened] with lockdown: false for public issues. Labels should include bug, feature, security, documentation, question, firewall-config, container, performance.

Effort: Low

---
on:
  issues:
    types: [opened, reopened]
  lockdown: false
permissions:
  issues: read
tools:
  github:
    toolsets: [issues, labels]
safe-outputs:
  add-labels:
    allowed: [bug, feature, security, documentation, question, help-wanted, firewall-config, container]
  add-comment:
    max: 1
timeout-minutes: 5
---
# Issue Triage Agent
Label newly opened issues in $\{\{ github.repository }}. Analyze the title and body in the context of a network firewall tool (Docker + Squid + iptables). Apply one label and leave a brief welcoming comment explaining the label and pointing to relevant docs.

---

🔄 Breaking Change Checker

What: A PR-triggered agent that watches for changes that could break backward compatibility — particularly CLI flag removals/renames, API/config schema changes, or container image changes that would break existing user setups.

Why: AWF is a security tool used in production CI pipelines. Breaking changes to CLI flags or container behavior can silently break users' workflows. This repo already has cli-flag-consistency-checker for docs alignment, but nothing that alerts on breaking changes in PRs before merge.

How: Trigger on pull_request, read the diff, check for: removed CLI flags, changed flag defaults, renamed environment variables, changed Docker network topology, modified squid config schema.

Effort: Low

---
on:
  pull_request:
    types: [opened, synchronize, reopened]
  roles: all
permissions:
  contents: read
  pull-requests: read
tools:
  github:
    toolsets: [default]
safe-outputs:
  add-comment:
    max: 1
timeout-minutes: 10
---
# Breaking Change Checker
Review this PR diff for breaking changes to: CLI flag names/defaults, `--allow-domains` behavior, `WrapperConfig` interface fields, Docker network topology (172.30.0.0/24), environment variable names passed to containers, and container image tag conventions. If breaking changes are found, comment with a clear warning and migration guidance.

---

P1 — Plan for Near-Term

📊 Workflow Metrics Collector / Audit Agent

What: A daily meta-agent that reviews the performance of all other agentic workflows — cost, error rates, PR proposal/merge rates, discussion creation — and produces a dashboard discussion.

Why: With 28 compiled workflows, there's no observability into how well the agents are actually performing. The Pelis Factory's audit-workflows and metrics-collector patterns were game-changing — they identified overactive agents, stale outputs, and cost inefficiencies. At this scale, a meta-agent is increasingly valuable.

How: Use agenticworkflows-logs tool + GitHub Actions API to aggregate run data. Output a weekly discussion with tables showing: runs per workflow, success rates, token cost estimates, PR outcomes.

Effort: Medium

---

🧹 Continuous Simplicity Agent

What: A weekly agent that searches for code complexity — overly long functions, duplicated logic, unnecessary abstractions — and proposes focused refactoring PRs.

Why: src/docker-manager.ts is already 1500+ lines and growing. The Pelis Factory's Continuous Simplicity agent had a 78%+ merge rate. This is the single biggest missing code quality automation.

How: Add continuous-simplicity.md on weekly schedule with skip-if-match to avoid PR flooding. Focus on: function length > 100 lines, duplicated error handling patterns, repeated string constants that should be shared.

Effort: Low

---

🔍 Schema Consistency Checker

What: A weekly agent that verifies alignment between: src/types.ts interfaces, docker-manager.ts generated configs, squid-config.ts, and documentation in docs/.

Why: As the codebase evolves, it's easy for WrapperConfig fields to diverge from what docker-manager.ts actually uses, or for docs/environment.md to list environment variables that were renamed. The Pelis Factory's schema checker caught exactly this kind of drift. For a security tool, config drift can have serious consequences.

How: Weekly schedule, reads all relevant source files and docs, creates a discussion report on any mismatches found.

Effort: Medium

---

📝 Documentation Noob Tester

What: An agent that periodically reads key documentation files (README.md, docs/quickstart.md, docs/chroot-mode.md) through the eyes of a first-time user — attempting to follow the instructions literally and identifying confusing or missing steps.

Why: AWF has extensive documentation (15+ docs files) targeting a sophisticated user base but the quick-start experience matters for adoption. The Pelis Factory's noob tester achieved 43% merge rate through causal chain — valuable signal generation even with lower direct merge rates.

Effort: Low

---

📦 Changeset Generator Enhancement

What: Enhance the existing update-release-notes workflow to also propose a version bump in package.json and update CHANGELOG.md by analyzing commits since last release.

Why: The current workflow updates release notes but doesn't manage semantic versioning. This adds a significant release automation gap. The Pelis Factory's Changeset agent had 78% merge rate.

Effort: Low

---

P2 — Consider for Roadmap

🛡️ Daily Malicious Code Scan

What: A daily scan of recent commits for suspicious patterns — obfuscated code, unexpected network calls added to container scripts, hardcoded IPs, unusual dependency additions.

Why: AWF runs agent code inside containers with controlled network access — it's a high-value target for supply chain attacks. The Pelis Factory runs this daily. The repo already has secret diggers but no general malicious pattern scan.

Effort: Low

---

🤖 Grumpy Reviewer (ChatOps)

What: A reaction-triggered code reviewer that performs opinionated, personality-driven code reviews — catching security anti-patterns, Docker best practices, iptables rule ordering issues.

How: Trigger on 👀 reaction on a PR, use Claude with a grumpy-but-helpful persona. Especially valuable for: iptables rule ordering, Squid ACL precedence, capability management.

Effort: Low

---

🗂️ Glossary Maintainer

What: An agent that maintains a glossary of AWF-specific terms in the documentation — TCP_TUNNEL, TCP_DENIED, NET_ADMIN, DNAT, ACL, etc. — keeping it synchronized with how terms are actually used in code and docs.

Why: The firewall domain has significant specialized vocabulary. New contributors struggle with terms like "CONNECT method", "Squid ACL", "capsh", "NAT table". A maintained glossary reduces onboarding friction.

Effort: Low

---

🧪 CI Optimization Coach

What: A weekly agent that analyzes CI pipeline configuration and suggests optimizations — caching strategies, parallelization, unnecessary test steps.

Why: With 8 language-specific build-test workflows + 4 smoke tests, CI is expensive. The Pelis Factory's CI Coach had 100% merge rate on all 9 proposed PRs.

Effort: Medium

---

P3 — Future Ideas

🌐 Firewall Domain Allowlist Advisor

What: A workflow that analyzes which domains are commonly needed by each supported language ecosystem (npm, cargo, go mod, pip, etc.) and maintains recommended domain allowlists in documentation.

Why: One of the most common user pain points is figuring out which domains to allow. An agent could automatically test fresh builds of each ecosystem's hello-world and document the needed domains.

Effort: High

---

📈 Performance Regression Detector

What: Track container startup time and command execution overhead across releases, alerting if the firewall overhead increases significantly.

Effort: High

---

🔗 Issue Arborist

What: Links related issues as sub-issues — grouping security issues, feature requests by component, etc. Especially useful given the many CI-failure issues from dependabot PRs.

Effort: Medium

---

📈 Maturity Assessment

Dimension	Current Score	Notes
Security automation	5/5	Exceptional — 3 independent secret diggers, daily threat modeling, CVE monitoring
Smoke testing	5/5	Multi-engine, multi-mode, reaction-triggered
CI fault investigation	4/5	CI Doctor is well-configured; missing breaking change detection
Documentation	3/5	Daily maintainer but missing noob tester, glossary, unbloat
Code quality	2/5	Test coverage improver and CLI checker, but no continuous refactoring
Issue management	2/5	Issue monster + dedup detector, but no triage/labeling
Meta-observability	1/5	Only this advisor — no metrics collection or audit agent
Release automation	2/5	Release notes update only

Current Level: 3/5 — Well-established agentic automation with strong security focus and testing coverage. The foundation is solid but several high-value pattern categories are underutilized.

Target Level: 4/5 — Achievable within 2-3 months by adding issue triage, breaking change checker, continuous simplicity, and a metrics collector.

---

🔄 Comparison with Pelis Factory Best Practices

What this repo does exceptionally well:

• 🥇 Multi-engine approach — Running the same security test (secret digger) on Claude, Codex, and Copilot in parallel is a Pelis-level pattern rarely seen elsewhere
• 🥇 Domain-specific security tooling — security-guard, daily security review, and dependency monitoring are well-tailored to the firewall domain
• 🥇 Smoke test rigor — Testing actual AWF container startup for each engine on every PR is production-grade validation
• ✅ skip-if-match patterns — doc-maintainer and test-coverage-improver correctly use throttling to prevent PR floods
• ✅ Modular shared imports — shared/mcp-pagination.md, shared/secret-audit.md follow DRY principles

What it could improve vs. best practices:

• 🔴 No meta-agent observability — The factory's biggest lesson: you need agents watching agents. No metrics on whether these 28 workflows are actually delivering value.
• 🟡 Code quality gap — Zero continuous refactoring/simplification agents. As docker-manager.ts grows, this becomes increasingly important.
• 🟡 Issue triage missing — The simplest, most impactful workflow not yet implemented.
• 🟡 Release automation incomplete — Changeset/version management is manual work that agents excel at.

Unique opportunities given the domain (firewall/security):

• A firewall-specific validation agent that verifies iptables rules, Squid ACL ordering, and container capability drops are still correct after each PR — using the extensive integration test suite as reference
• A domain allowlist health checker that periodically tests whether the documented domain lists are still accurate for each supported tool ecosystem

---

Report generated by the Pelis Agent Factory Advisor workflow on 2026-03-10. Notes saved to cache-memory for trend tracking across future runs.

AI generated by Pelis Agent Factory Advisor

• expires on Mar 17, 2026, 3:22 AM UTC

[github-actions[bot]]

🔮 The ancient spirits stir; the smoke-test wanderer passed through and left clear signs in the ether. By starlight, the wards hold.

🔮 The oracle has spoken through Smoke Codex