Automated Content Quality Check with GitHub Actions

The Markup AI Content Guardian runs the Style Agent against the content files in your repository — Markdown, plain text, HTML, DITA, and XML — and posts the results straight into your GitHub workflow. It returns a content-risk assessment and issues grouped by severity, so you can catch style and brand problems before they reach your users.

The action adapts to the GitHub event that triggered it: it reviews changed files on pull requests and pushes, audits the whole repository on manual and scheduled runs, and surfaces results where your team already works — as a pull request comment, inline review comments, a commit status, or a job summary.

Install it from the Markup AI Content Guardian listing on the GitHub Marketplace, and this guide will walk you through configuring it for your repository.

What Markup AI does for you in GitHub

• One step, agentic workflow: A single action submits each content file to the Markup AI Style Agent and waits for the result. Long documents won’t stall the job — the action submits each run, polls until it reaches a terminal state, and collects the full result.
• Style Agent: Check your content against the Style Guide that captures your team’s standards. Pick one per workflow, or leave it unset to use your organization’s default.
• Risk-based feedback, where you work: Every run leads with a content-risk assessment — a risk label and issue counts grouped by severity (High, Medium, Low) — posted as a pull request comment, inline review comments, a commit status, or a job summary depending on the event.
• Event-aware analysis: The action analyzes only the files that matter for each event — files changed in a pull request, files touched in a push, or every supported file in the repository for manual and scheduled audits.

Which agents are supported

• Style Agent

GitHub Action Use Cases

The Markup AI Content Guardian works seamlessly with different GitHub workflows and event types:

🔄 Pull Request Analysis

• When to Use: Review content changes before merging
• What it Does: Analyzes only files modified in the pull request
• Benefits: Catch content issues early, ensure quality before merge
• Perfect For: Teams that want to maintain content quality gates

🚀 Push Event Monitoring

• When to Use: Monitor direct commits to branches
• What it Does: Analyzes files changed in the push
• Benefits: Quick feedback on direct commits, maintain quality standards
• Perfect For: Small teams or solo developers

🔧 Manual Workflow Analysis

• When to Use: Comprehensive repository-wide content audit
• What it Does: Analyzes all supported files in the repository
• Benefits: Complete content health assessment, identify areas for improvement
• Perfect For: Regular content audits, compliance checks

📊 Continuous Content Monitoring

• When to Use: Ongoing quality assurance for content repositories
• What it Does: Runs on a schedule or manual trigger
• Benefits: Proactive content quality management, trend analysis
• Perfect For: Documentation teams, content-heavy projects

Prerequisites

To set up the Markup AI Content Guardian, you’ll need:

• A GitHub repository with content files you want to analyze, and GitHub Actions enabled.
• A Markup AI account with the Style Agent enabled for your organization. The Style Agent is not on self-serve signup yet — talk to us to get it enabled.
• A Markup AI API token generated from the Markup AI console.
• Content files in a supported format (Markdown, text, HTML, DITA, or XML).
• A GitHub repository secret to store your API token securely.

Supported File Types

The action automatically detects and analyzes the following file types:

• Markdown: .md, .markdown, .mdown, .mkd
• Text: .txt, .text
• HTML: .html, .htm
• DITA: .dita, .xml

Files with any other extension are skipped automatically, so only relevant content is analyzed.

Quick Start Guide

1. Add the Action to Your Repository

Create a .github/workflows/markupai-analysis.yml file in your repository:

name: Markup AI Content Analysis
on:
  pull_request:
  push:
    branches: [main]
  workflow_dispatch:
permissions:
  contents: read
  pull-requests: write
  statuses: write
jobs:
  analyze-content:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Run Markup AI Analysis
        uses: markupai/content-guardian-action@v2
        with:
          markup_ai_api_key: ${{ secrets.MARKUP_AI_API_KEY }}
          github_token: ${{ secrets.GITHUB_TOKEN }}
          # style_guide is optional — omit it to use your organization's
          # default Style Guide, or set it to pin a specific one (see below).

Install the latest release from the Markup AI Content Guardian listing on the GitHub Marketplace.

2. Configure Repository Secrets

1. Go to your repository’s Settings → Secrets and variables → Actions.
2. Add a new repository secret:
   • Name: MARKUP_AI_API_KEY
   • Value: Your Markup AI API token from the Markup AI console.

GITHUB_TOKEN is provided automatically by GitHub Actions — you don’t need to create it.

3. Choose Your Style Guide

The Style Guide captures your team’s standards — dialect, tone, and the rules the Style Agent checks against are all configured on the guide itself in the Markup AI console, so your workflow stays clean.

The style_guide input is optional:

• Omit it to use your organization’s default Style Guide. Most teams set the default once in the console and never pass it in the workflow.
• Set it to a Style Guide ID or display name (case-insensitive) to pin a specific guide:

- name: Run Markup AI Analysis
  uses: markupai/content-guardian-action@v2
  with:
    markup_ai_api_key: ${{ secrets.MARKUP_AI_API_KEY }}
    github_token: ${{ secrets.GITHUB_TOKEN }}
    style_guide: "Microsoft Manual of Style" # Style Guide ID or display name
    add_commit_status: "true"

Configuration Options

Advanced Configurations

Pull Request Analysis

name: PR Content Review
on:
  pull_request:
    paths:
      - "docs/**"
      - "README.md"
      - "CHANGELOG.md"
permissions:
  contents: read
  pull-requests: write
# Recommended: cancel the previous in-flight run when a new commit lands on the
# same PR. The action reconciles its inline review comments against the latest
# analysis, and two overlapping runs can race on that reconciliation.
concurrency:
  group: markup-ai-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true
jobs:
  content-review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Analyze PR Content
        uses: markupai/content-guardian-action@v2
        with:
          markup_ai_api_key: ${{ secrets.MARKUP_AI_API_KEY }}
          github_token: ${{ secrets.GITHUB_TOKEN }}

Push Event Monitoring

name: Content Quality Check
on:
  push:
    branches: [main]
permissions:
  contents: read
  statuses: write
jobs:
  quality-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Check Content Quality
        uses: markupai/content-guardian-action@v2
        with:
          markup_ai_api_key: ${{ secrets.MARKUP_AI_API_KEY }}
          github_token: ${{ secrets.GITHUB_TOKEN }}

Manual and Scheduled Repository Audit

name: Full Content Audit
on:
  workflow_dispatch:
  schedule:
    - cron: "0 0 * * 1" # Every Monday at midnight
permissions:
  contents: read
jobs:
  content-audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Full Repository Analysis
        uses: markupai/content-guardian-action@v2
        with:
          markup_ai_api_key: ${{ secrets.MARKUP_AI_API_KEY }}
          github_token: ${{ secrets.GITHUB_TOKEN }}
          # Optional: narrow a repo-wide audit to specific paths
          paths: |
            docs/
            README.md

How Results Show Up

The action leads with a content-risk assessment on every run, and adapts where it posts based on the event:

• Pull requests — a single summary comment plus inline review comments on the flagged lines.
• Pushes — a commit status on the pushed commit.
• Manual and scheduled runs — a job summary written to the workflow run.

Risk-based assessment

Each file gets a risk label derived from its worst issue — 🔴 High, 🟡 Medium, 🟢 Low, or ✅ No issues — alongside a breakdown of issue counts by severity (High, Medium, Low). The overall risk is the worst level across all analyzed files.

Example PR comment

🔍 Markup AI Analysis Results

This summary was automatically generated by the Markup AI Content Guardian for the pull_request event.

📊 Summary

Overall Risk: 🔴 High

Files Analyzed: 4

Total Issues: 31 (High: 9, Medium: 14, Low: 8)

Each inline review comment points at the flagged phrase and includes the Style Agent’s suggestion and explanation.

Commit status

On push events, the action posts a commit status summarizing the run, for example:

Risk High | Files 2 | Issues 107 (High: 71, Medium: 36, Low: 0)

The status state — success, failure, or error — is derived from the overall risk so your branch protection checks behave consistently.

Action Outputs

The action provides outputs you can use in subsequent workflow steps:

Available Outputs

• event-type: The GitHub event that triggered the action (push, pull_request, workflow_dispatch, schedule).
• files-analyzed: Number of files that were analyzed.
• results: JSON string containing the full per-file analysis results, including every issue and its severity. When numeric scoring is enabled for your organization, the scores are included too.

Using Outputs in Workflows

- name: Run Markup AI Analysis
  id: markupai
  uses: markupai/content-guardian-action@v2
  with:
    markup_ai_api_key: ${{ secrets.MARKUP_AI_API_KEY }}
    github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Process Results
  run: |
    echo "Event type: ${{ steps.markupai.outputs.event-type }}"
    echo "Files analyzed: ${{ steps.markupai.outputs.files-analyzed }}"
    echo "Results: ${{ steps.markupai.outputs.results }}"

Troubleshooting

Action Not Triggering

• Check file paths: Ensure your workflow’s event and paths filters cover the files you expect.
• Verify event triggers: Confirm the workflow runs on the events you want (pull_request, push, workflow_dispatch, schedule).
• Check branch filters: Make sure the workflow runs on your target branches.

Style Agent or Token Issues

• Style Agent not enabled: The action fails fast if the Style Agent isn’t enabled for your organization — reach out to support to get it turned on.
• Invalid token: Verify your Markup AI API token is correct and active, and that the MARKUP_AI_API_KEY secret is configured in repository settings.
• Style Guide not found: If you set style_guide, the value must match an enabled Style Guide’s ID or display name. The action logs the available Style Guides when resolution fails — copy the exact value from there.

No Files Analyzed

• File type support: Verify files use a supported extension (.md, .markdown, .mdown, .mkd, .txt, .text, .html, .htm, .dita, .xml).
• File content: Ensure files contain actual text content and aren’t empty.
• Path filters: If you set the paths input, confirm it intersects with the files the event surfaced.

Results Not Posted

• Missing permissions: Posting a PR comment needs pull-requests: write; posting a commit status needs statuses: write. Add the permissions block shown in the examples.
• Dry-run enabled: With dry_run: true, analysis runs but nothing is written back to GitHub — the rendered output is logged instead.

Analysis Failures

• Workflow ID: Per-file failures are logged with a workflow_id. Include it when you contact support about a specific run.
• API limits: Check whether you’ve exceeded your rate limits.
• Network issues: Verify GitHub Actions can reach the Markup AI API.

Best Practices

Workflow Design

1. Target specific paths: Use event paths filters or the paths input to run analysis only on content directories.
2. Event-specific workflows: Create different workflows for pull requests, pushes, and scheduled audits.
3. Quality gates: Combine strict_mode with the risk-based commit status to gate merges on content quality.
4. Scheduled audits: Run a regular repository-wide analysis to monitor content quality over time.

Configuration Management

1. Secrets for secrets: Store your API token as a repository secret, never in the workflow file.
2. Set the org default: Configure your default Style Guide once in the Markup AI console so most workflows don’t need the style_guide input.
3. Consistent settings: Use the same Style Guide across workflows that cover the same content type.
4. Test with dry-run: Use dry_run: true to validate a workflow on a real PR without posting anything.

Team Integration

1. Agree on the risk gate: Define what good enough looks like (for example, no High severity issues) before enforcing it.
2. Feedback loop: Use the PR comment and inline reviews to give contributors actionable, in-context feedback.
3. Continuous improvement: Revisit your Style Guides and risk gates as you review real results — aim for high signal, not high volume.

Need Help?

Reach out at the Markup AI Community. Include the workflow_id from the run’s output (and a short description of what you saw) — that’s everything support needs to investigate a specific run.

Working Example

To see the Markup AI Content Guardian in action, check out the content-guardian-action repository. It demonstrates:

• Real-world implementation of the action across event types
• Sample workflows for pull requests, pushes, and scheduled audits
• Example content files with various quality issues
• PR comments and commit status examples

Visit github.com/markupai/content-guardian-action to explore the working example, and install the action from the GitHub Marketplace.