[Pelis Agent Factory Advisor] Pelis Agent Factory Advisor: Agentic Workflow Recommendations (2026-03-05)

[github-actions[bot]]

📊 Executive Summary

The gh-aw-firewall repository has a strong and growing agentic workflow infrastructure with 26 compiled agentic workflows already in place — well above typical repository maturity. The security domain context creates unique opportunities for high-value automations not seen in typical repositories. Key gaps include: a workflow health meta-agent, issue triage labeling, a breaking change checker, and domain-specific security automation unique to this firewall tool.

---

🎓 Patterns Learned from Pelis Agent Factory

From the Documentation Site

The Pelis Agent Factory (github/gh-aw) operated over 100 agentic workflows — the key philosophical insight is: specialization reveals possibilities. Focused agents find more useful applications than monolithic ones. Key patterns:

Pattern	Description	Applicability
Read-only analysts	Agents that only read and produce discussion reports	✅ Already used here (security-review, ci-cd-gaps-assessment)
PR-proposing agents	Agents that proactively create PRs	✅ Already used (test-coverage-improver, doc-maintainer)
Meta-agents	Agents that monitor other agents	❌ Missing: Workflow Health Manager
Orchestrator workflows	Agents that dispatch tasks to other agents	✅ Partially via issue-monster
Cache memory for continuity	Using persistent cache across runs	✅ Used in issue-duplication-detector
Multi-engine smoke tests	Testing with Claude, Codex, Copilot	✅ Best-in-class here (3 engines!)
skip-if-match throttling	Prevent agent overload	✅ Used in multiple workflows

From the Agentics Repository

The githubnext/agentics repo contains ~40 reusable workflow templates. Notably absent from this repo: daily-malicious-code-scan, link-checker, issue-triage, ci-coach, repository-quality-improver, duplicate-code-detector, daily-file-diet, and grumpy-reviewer.

Comparison to Current Implementation

This repo leads the factory in:

• Security-focused workflows (secret-digger across 3 engines, security-guard, security-review)
• Multi-engine smoke testing (unique to this codebase's domain)
• Language-specific build-test PRs (8 language adapters)

This repo lags behind in:

• Meta-observability (no workflow health manager or metrics collector)
• Issue lifecycle management (no triage labels, no issue arborist)
• Code quality improvement agents (no simplicity/refactoring/style agents)

---

📋 Current Agentic Workflow Inventory

Workflow	Purpose	Trigger	Assessment
build-test-{8 langs}	PR validation via Copilot coding agent	PR opened/sync	✅ Excellent coverage
ci-doctor	CI failure root cause analysis	workflow_run failure	✅ 69% merge rate in Pelis
ci-cd-gaps-assessment	Daily CI/CD gap identification	Daily	✅ Good, meta-monitoring
cli-flag-consistency-checker	Docs/code CLI flag sync	Weekly	✅ Active, valuable
dependency-security-monitor	CVE detection + dep update PRs	Daily	✅ High-value security
doc-maintainer	Docs sync with code changes	Daily	✅ Working well
issue-duplication-detector	Detect duplicate issues	Issue opened	✅ Cache-memory pattern
issue-monster	Assign issues to Copilot agent	Hourly + issue opened	✅ Task dispatcher
pelis-agent-factory-advisor	Agentic workflow advisor (this!)	Daily	✅ Meta-learning
plan	/plan slash command	Slash command	✅ ChatOps pattern
secret-digger-{claude,codex,copilot}	Hourly secret scanning	Hourly (3 engines)	✅ Exceptional coverage
security-guard	PR security review	PR opened/sync	✅ Critical for this domain
security-review	Daily comprehensive security review	Daily	✅ Thorough
smoke-{chroot,claude,codex,copilot}	End-to-end smoke tests	PR + schedule	✅ Multi-engine excellence
test-coverage-improver	Weekly test coverage PRs	Weekly	✅ Focused on security paths
update-release-notes	Enhance release notes	Release published	✅ Release automation

---

🚀 Actionable Recommendations

P0 — Implement Immediately

🔥 P0.1: Issue Triage Agent (Labeling + Classification)

What: Add an issue triage agent that automatically labels incoming issues (bug, feature, enhancement, security, documentation, question) and comments with a brief triage summary.

Why: issue-monster assigns issues to Copilot but doesn't label or classify them. Without labels, maintainers must manually scan every new issue to understand priority and type. In Pelis, even a minimal triage agent dramatically reduces maintainer review burden.

How: Add an issue-triage.md triggered on issues.types: [opened, reopened] with lockdown: false so external contributors' issues are processed. Use safe-outputs.add-labels and safe-outputs.add-comment.

Effort: Low

Example:

---
on:
  issues:
    types: [opened, reopened]
  lockdown: false
permissions:
  issues: read
tools:
  github:
    toolsets: [issues, labels]
safe-outputs:
  add-labels:
    allowed: [bug, feature, security, documentation, question, enhancement, good-first-issue]
  add-comment: {}
timeout-minutes: 5
---
# Issue Triage Agent
Analyze new issues in this AWF firewall repository. Label each issue appropriately
and add a triage comment explaining the label and possible next steps.
Security issues (iptables, domain bypass, credential exposure) are highest priority.

---

🔥 P0.2: Breaking Change Checker (PR Trigger)

What: A PR-triggered workflow that analyzes code changes for backward-incompatible behavior in CLI flags, API interfaces, Docker container configurations, and domain filtering rules.

Why: Users depend on AWF in their CI pipelines. A breaking change to --allow-domains syntax or container startup behavior could silently break dozens of downstream pipelines. Pelis's Breaking Change Checker has proven this pattern valuable. This is especially important for a tool that other agents use as infrastructure.

How: Trigger on pull_request, analyze diffs in src/cli.ts, src/types.ts, containers/agent/, check against documented API surface in docs. Create alert issues for breaking changes.

Effort: Low-Medium

---

🔥 P0.3: Daily Malicious Code Scan

What: Daily scan of recent commits and PRs for suspicious patterns in iptables rules, Squid config generation, Docker configurations, and domain whitelist handling.

Why: AWF is itself a security tool used by AI agents. A supply chain compromise here could compromise every agent using it. Pelis ran this as a standard security workflow. For AWF specifically, the attack surface is high-value (firewall bypass, credential exfiltration, domain bypass). This is the highest-ROI security workflow for this specific codebase.

How: Daily trigger, scan last 7 days of commits via GitHub tools, focus on: regex patterns that could match too broadly, domain whitelist bypass patterns, iptables rules that could allow unrestricted traffic, credential handling.

Effort: Low (can use githubnext/agentics/daily-malicious-code-scan as base)

Add-wizard command:

gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-malicious-code-scan.md
```

---

### P1 — Plan for Near-Term

#### 🚀 P1.1: Workflow Health Manager (Meta-Agent)

**What**: A weekly meta-agent that monitors the health of all other agentic workflows in this repository — tracking which workflows are running, which are producing results, which are failing, and which may need attention.

**Why**: With 26 agentic workflows, it's hard for maintainers to know which are working well vs. silently broken. In Pelis, the Workflow Health Manager **created 40 issues, leading to 34 PRs** — it's one of the highest-leverage workflows because it protects all other automation investment. A workflow that never runs is wasted infrastructure.

**How**: Weekly schedule, use `agentic-workflows` tool to get status, use GitHub Actions API to check recent run outcomes, create issues for workflows that haven't run in >2x their expected interval or have >50% failure rate.

**Effort**: Medium

---

#### 🚀 P1.2: Changeset / Version Bump Automation

**What**: Automated PR generation for version bumps and CHANGELOG updates after releases. Analyzes commits since last release tag to determine appropriate semantic version bump (major/minor/patch) and generates changelog entries.

**Why**: The existing `update-release-notes` workflow enhances notes *after* a release is published, but there's no automation for *preparing* releases. Maintainers must manually decide version bumps and update CHANGELOG. In Pelis, Changeset achieved **78% merge rate** across 28 proposed PRs.

**How**: Triggered on push to main after merging a batch of PRs (or weekly), analyze merged PRs for conventional commit types (`feat:` → minor, `fix:` → patch, `feat!:` → major), generate PR updating package.json version and CHANGELOG.

**Effort**: Medium

---

#### 🚀 P1.3: Link Checker for Documentation

**What**: Weekly check of all documentation files for broken links, both internal (cross-references between docs) and external (GitHub links, URLs in READMEs).

**Why**: The repo has extensive documentation across ~20 markdown files in `docs/`, `README.md`, `AGENTS.md`, `CLAUDE.md`, and the `docs-site/`. As the codebase evolves, internal references break. Pelis's link-checker found broken links reliably. For a security tool, broken docs reduce trust.

**Effort**: Low

```
gh aw add-wizard githubnext/agentics/link-checker
```

---

#### 🚀 P1.4: Container Image Vulnerability Scanner Workflow

**What**: A domain-specific security workflow that monitors the Squid proxy and agent base images for newly-published CVEs using Trivy or container scanning tools, distinct from the dependency-security-monitor.

**Why**: `dependency-security-monitor` covers npm packages but **not Docker base images** (`ubuntu/squid:latest`, `ubuntu:22.04`). A CVE in the base Ubuntu image or Squid binary could compromise all AWF users. The existing `container-scan.yml` runs on PRs but there's no proactive daily/weekly monitoring and alerting.

**How**: Weekly trigger, run `trivy image` against the currently-in-use image tags, compare against last week's results from cache-memory, create issues for new HIGH/CRITICAL CVEs.

**Effort**: Medium (requires bash + docker in workflow)

---

#### 🚀 P1.5: PR Review Quality Agent (General Code Quality)

**What**: A lightweight PR reviewer that complements `security-guard` by checking for general code quality issues: TypeScript type safety, error handling completeness, test coverage for new functions, and conventional commit compliance.

**Why**: `security-guard` reviews security posture on every PR — but there's no agent reviewing general code quality. The `build-test-*` workflows verify *other* codebases work with AWF, not AWF's own code quality. A "grumpy reviewer" pattern from Pelis can catch issues before human review.

**How**: Trigger on PR, use `safe-outputs.add-comment` with max 1, check for: untested new functions, missing error handling, TypeScript `any` usage, conventional commit format in PR title.

**Effort**: Low-Medium

```
gh aw add-wizard githubnext/agentics/grumpy-reviewer

---

P2 — Consider for Roadmap

📋 P2.1: Issue Arborist (Sub-Issue Linking)

What: Periodic scan for related issues that should be linked as sub-issues, building a dependency tree of related work items.

Why: As AWF grows in scope (chroot mode, api-proxy, tmpfs, rate limiting), related issues accumulate without visible grouping. Pelis's Issue Arborist created 77 discussion reports and 18 parent issues. For a tool with multiple subsystems (proxy, container, CLI, MCP), this organization helps prioritize work.

Effort: Medium

---

📋 P2.2: Squid Config Regression Auditor

What: A domain-specific workflow that verifies the generated Squid configuration never weakens the security model — checking that http_access deny all is present as final rule, blocked-by-default is preserved, and allowed-domains-only filtering is intact.

Why: src/squid-config.ts generates Squid configs dynamically. A regression here could silently allow all traffic through. This is unique to this domain and not available as a Pelis template — it would be a custom security regression detector specific to AWF's trust model.

How: Weekly, use bash to run the config generator with various inputs, parse output for critical security invariants (deny-by-default, localhost rules, Squid proxy exemption).

Effort: Medium

---

📋 P2.3: Metrics Collector / Workflow Audit

What: Daily collection of agentic workflow performance metrics — runs, success rates, token usage, cost estimates, PRs created/merged — into a discussion report.

Why: With 26 workflows running daily/hourly, cost visibility matters. Pelis's Metrics Collector generated 41 daily metrics discussions and helped identify overly-chatty agents. With 3 secret-digger instances running hourly, a cost spike could be missed without visibility.

Effort: Medium (requires agentic-workflows tool + GitHub Actions API)

---

📋 P2.4: Integration Test Drift Detector

What: Weekly analysis of the integration test suite vs. the implementation to identify new features without tests and existing tests that no longer test the right thing.

Why: The repo has ~25 integration tests but the codebase is evolving rapidly (tmpfs, docker cp, api-proxy, rate limiting, token unsetting). The test-coverage-improver generates new test PRs but doesn't detect drift where existing tests test outdated behavior. Custom to this domain.

Effort: Medium

---

P3 — Future Ideas

💡 P3.1: Daily Repository Status Report

A daily summary discussion of repository health: open PRs, stale issues, recent merges, upcoming milestones. Analogous to Pelis's daily-repo-status.md workflow. Low-cost observability for maintainers.

💡 P3.2: ChatOps Help Command (/help, /explain)

Add /help slash command to explain AWF behavior to contributors, and /explain to analyze a Squid log snippet or iptables rule. Extends the existing /plan ChatOps pattern.

💡 P3.3: Continuous Simplicity Agent

Weekly PR-proposing agent that identifies overly complex functions in the AWF codebase and proposes simplifications. Pelis's Continuous Simplicity had high merge rates. Given AWF's security-critical code, simpler code is safer code.

💡 P3.4: Copilot Setup Steps Validator

An agent that tests the copilot-setup-steps.yml configuration by running Copilot CLI against it periodically and verifying it installs correctly across configurations.

---

📈 Maturity Assessment

Dimension	Score	Notes
Security automation	5/5	Exceptional: secret-digger ×3, security-guard, security-review, dependency-security-monitor
CI/CD quality	4/5	CI Doctor, smoke tests ×4, build-tests ×8; missing: breaking change checker
Code quality agents	2/5	test-coverage-improver exists; no refactoring, simplicity, or general PR review
Documentation automation	4/5	doc-maintainer, CLI flag checker, release notes; missing: link checker
Issue management	3/5	issue-monster, dedup detector; missing: triage labels, arborist
Meta-observability	2/5	ci-cd-gaps-assessment exists; no workflow health manager or metrics
Release automation	3/5	update-release-notes; no changeset automation

Current Level: 3.3/5 — "Established Practitioner" — Well above average, with strong security automation and multi-engine testing. Gaps are in meta-observability and code quality agents.

Target Level: 4/5 — "Advanced Factory" — Add workflow health monitoring, issue triage, and the P0 security workflows to reach this level.

Gap Analysis: 3 P0 workflows + Workflow Health Manager + link checker = Level 4. Est. 1-2 days of implementation effort.

---

🔄 Comparison with Pelis Best Practices

What this repo does well ✅

• Multi-engine testing: Smoke tests across Claude, Codex, and Copilot is unique and valuable
• Security depth: More security-focused workflows than Pelis itself (domain-appropriate)
• Cache-memory usage: Correctly used for issue-duplication-detector persistence
• skip-if-match throttling: Prevents agent overload in doc-maintainer, issue-monster
• Convention enforcement: Commitlint + husky + PR title check maintains quality

What it could improve 🔺

• Meta-observability: Pelis invested heavily in metrics/audit workflows; this repo has none
• Issue lifecycle: Issues enter via issue-monster but aren't triaged/labeled first
• Code quality agents: Security guard exists but no general quality review agent
• Changeset management: Release notes are enhanced post-publish but pre-release automation is missing

Unique opportunities given the domain 🎯

1. Squid config security invariant tests (P2.2) — unique to this firewall tool
2. Container base image CVE scanner (P1.4) — more important here than typical repos
3. AWF self-hosting validation — AWF could be used to test AWF workflows, creating a meta-loop
4. Domain whitelist regression testing — automated verification that the firewall actually blocks what it should

---

📝 Notes for Future Runs

Analysis timestamp: 2026-03-05. Cache saved to /tmp/gh-aw/cache-memory/notes.txt.

Key trends to monitor in future runs:

• Whether the open Dependabot/[Deps] PR backlog gets cleared
• Whether the PR review workflow (security-guard) is generating useful comments
• Adoption rate of any newly-added workflows from these recommendations
• Whether the secret-digger hourly runs are finding or creating any issues

AI generated by Pelis Agent Factory Advisor

• expires on Mar 12, 2026, 3:25 AM UTC

[github-actions[bot]]

The oracle has watched the smoke and judged the omens. A quiet sign marks this place: the smoke test agent was here, and the path forward is clear.

🔮 The oracle has spoken through Smoke Codex