How To Set Up AI Code Review On GitHub Pull Requests

Setting up AI code review on GitHub takes an afternoon, but setting it up well takes a few decisions most teams skip: which repositories the reviewer can read, which settings live in version control, whether its result can block a merge, and how you will know if it is helping. This guide walks through each step for an AI code review GitHub rollout, using CodeRabbit, GitHub Copilot code review and Greptile as working examples.

Setting names and file paths below come from each vendor’s documentation as of November 2026. Interfaces change, so check the linked docs if a menu label looks different.

Step 1: Choose Your AI Code Review GitHub Tool

Start with three filters: where your code is hosted, how you want to pay, and whether code can leave your infrastructure. For a GitHub-only team the main options are:

• GitHub Copilot code review: built in for teams with paid Copilot plans, configured with rulesets and instruction files.
• CodeRabbit: a dedicated reviewer installed as a GitHub App, configured with a .coderabbit.yaml file, also available on GitLab, Bitbucket and Azure DevOps.
• Greptile: indexes a graph of your repository for context, configured with a .greptile folder, priced per review credit.
• Others worth a look include Graphite for stacked pull requests, Cursor Bugbot for Cursor users, and the open source PR-Agent if you must self-host.

Pick one tool for the pilot. Running two bots on the same pull requests from day one makes it impossible to tell which comments are useful.

Step 2: Install The GitHub App With Least Privilege

Third-party reviewers install as GitHub Apps. The single most important choice in the install screen is repository access.

Pick Only Select Repositories

GitHub offers All repositories, which covers every current and future repository in the organization including public ones, or Only select repositories. Choose Only select repositories and start with one or two active, non-sensitive services. You can widen access later from the organization’s GitHub App settings.

Read The GitHub App Permissions

Check each permission against what the tool needs. CodeRabbit’s docs, for example, list read access to actions, discussions, members, metadata and merge queues, and read and write access to checks, code, commit statuses, issues, pull requests and workflows. The docs explain that workflows access exists so agents can edit files under .github/workflows and that review-only installs do not use it. If a permission is not explained in the vendor’s docs, ask before approving.

Install As An Organization Owner

Install at the organization level rather than on a personal account, so access survives staff changes and shows up in your organization’s audit log. Cursor’s Bugbot docs, for instance, require both a Cursor admin and a GitHub organization admin for setup.

Copilot Code Review Needs No App

Copilot code review is part of GitHub, so there is nothing to install. Organization owners decide who can use it through Copilot policies, and GitHub’s docs say members without a Copilot license can use it on GitHub.com only when an admin enables paid AI credit usage and the sub-policy for unlicensed members on Business or Enterprise plans.

Step 3: Put The Configuration In Version Control

Store reviewer settings in the repository so changes go through review like any other code.

CodeRabbit: .coderabbit.yaml

CodeRabbit reads .coderabbit.yaml from the repository root, using the version on the branch under review. A minimal starting point:

language: "en-US"
reviews:
  profile: "chill"
  high_level_summary: true
  auto_review:
    enabled: true
    drafts: false
    base_branches:
      - "^develop$"
  path_filters:
    - "!**/*.lock"
    - "!src/generated/**"
  path_instructions:
    - path: "src/api/**"
      instructions: "Flag missing input validation and auth checks."
knowledge_base:
  opt_out: false

The profile accepts quiet, chill or assertive. Base branch entries are unanchored regular expressions, so use ^develop$ for an exact match. Comment @coderabbitai generate configuration on a pull request and CodeRabbit opens a pull request that adds the file for you, and @coderabbitai configuration shows the resolved settings.

Copilot Code Review: Instruction Files

Copilot reads repository-wide guidance from .github/copilot-instructions.md and path-specific guidance from NAME.instructions.md files under .github/instructions. Path files use frontmatter with applyTo globs, and excludeAgent can stop one feature from using them:

---
applyTo: "src/api/**/*.ts"
---
Check every new endpoint for input validation and an auth guard.
Prefer early returns over nested conditionals.

Add excludeAgent: “cloud-agent” to the frontmatter if a file is meant only for code review, or “code-review” to hide it from the reviewer. Copilot reads these files from the head branch of the pull request.

Greptile: .greptile/config.json

Greptile recommends a .greptile folder at the repository root, or in any subdirectory for scoped settings, holding config.json, an optional rules.md and an optional files.json for context documents. The older greptile.json at the root still works, but the folder takes precedence.

{
  "strictness": 2,
  "commentTypes": ["logic", "syntax"],
  "rules": [
    {
      "id": "no-raw-sql",
      "rule": "Use parameterized queries.",
      "scope": ["src/db/**"],
      "severity": "high"
    }
  ]
}

Step 4: Turn On Automatic Reviews With Rulesets

Decide when the reviewer runs. Third-party apps usually review every non-draft pull request by default, which you control in their config. For Copilot code review, GitHub uses branch rulesets.

1. Open the repository’s Settings, then Rulesets, and choose New ruleset, then New branch ruleset.
2. Set Enforcement Status to Active and add your target branches, usually the default branch.
3. Under Branch rules, select Automatically request Copilot code review.
4. Optionally tick Review new pushes and Review draft pull requests, then click Create.

The same rule is available at the organization level, targeting repositories by include or exclude pattern, and at the enterprise level. Start with Review new pushes off to keep costs and noise down during the pilot.

Step 5: Wire Up Branch Protection And Required Checks

Your branch protection rules decide whether AI review is advisory or a gate. Keep humans in charge of approvals.

Keep Human Approvals Required

In the ruleset for your default branch, enable Require a pull request before merging with at least one required approval, and turn on Dismiss stale pull request approvals when new commits are pushed. Requiring conversation resolution is useful too, because it forces someone to read and resolve each AI thread.

Decide Whether AI Approval Counts

By default Copilot’s reviews do not count toward required approvals. GitHub has a public preview that lets Copilot approve pull requests and count toward merge requirements, which admins enable at repository, organization or enterprise level, optionally limited to up to 15 file globs. A new push dismisses Copilot’s approval. For most teams, leave this off until you trust the reviewer on low-risk paths such as documentation.

Add The Reviewer As A Required Status Check

Under Require status checks to pass before merging, you can add the reviewer’s check. CodeRabbit publishes a CodeRabbit check run and a separate CodeRabbit Security check, and it also publishes a passing check on merge queue commits so merge queues do not stall. Adding it to your required status checks ensures a review ran, not that the code is correct, so pair it with human approval.

Step 6: Tune The Noise

Developers mute bots that comment too much. Spend the first two weeks tightening scope.

• Exclude generated code, lock files, vendored dependencies and snapshots with path filters or, in Graphite, linguist-generated=true in .gitattributes.
• Lower the tone: CodeRabbit’s quiet or chill profile, or a lower Greptile strictness level.
• Write a few specific rules for the problems you actually care about, one concern per rule, with a bad and good example.
• React to comments. Greptile and CodeRabbit both learn from replies and reactions, and Greptile says it stops repeating ignored feedback after two to three weeks.
• Skip reviews you do not need, such as dependency bump pull requests, using title keyword filters like CodeRabbit’s ignore_title_keywords.

Use comment commands to control a single pull request. With CodeRabbit, @coderabbitai pause stops automatic reviews on that pull request, @coderabbitai resume restarts them, and @coderabbitai full review re-reviews every file.

Step 7: Answer The Security And Data Retention Questions

Before you widen the rollout, get written answers to these questions from the vendor or from its trust center:

• Does the vendor train models on our code? CodeRabbit states it never uses customer code for training.
• Which model providers see our code, and under what terms?
• What is cached or stored, for how long, and can we opt out? CodeRabbit caches encrypted archives and vector representations, and knowledge_base.opt_out disables features that need retention.
• Is self-hosting available? CodeRabbit, Greptile, Sourcery, Qodo and Bito offer it on higher plans, and PR-Agent is self-hosted by design.
• Which compliance reports (such as SOC 2 Type II) can we review?

AI review also plays a part in your wider AI security posture. If more of your code is written by agents, our guide to vibe coding security risks lists the issues a reviewer should be catching, and our roundup of observability tools for AI coding agents covers the runtime side.

Step 8: Measure Impact

Decide on metrics before the pilot starts, and compare the pilot repositories against a similar repository without AI review.

• Acceptance rate: the share of AI comments that lead to a code change or a resolved thread. Graphite’s dashboard reports acceptance and upvote rates per rule, and CodeRabbit includes usage analytics on every plan.
• Time to first review and time to merge, from GitHub’s pull request timestamps.
• Review cycles per pull request, which should fall if the bot catches problems before humans look.
• Escaped defects: bugs found after merge in code the reviewer saw.
• Cost per merged pull request, including AI credits or seats.

If acceptance stays below a level your team agrees on after a month of tuning, try a different tool on the same repositories rather than adding more rules.

Frequently Asked Questions

Does Copilot Code Review Cost Extra?

Copilot code review is included in paid Copilot plans, and usage is measured in AI credits. GitHub’s docs estimated a Lite review at $0.05 to $1 and a Balanced review at $0.25 to $5 in AI credits in November 2026, rising with pull request size.

Where Does The .coderabbit.yaml File Go?

In the root of the repository. CodeRabbit reads it from the feature branch being reviewed, so you can test configuration changes in the same pull request that makes them.

Can AI Review Block A Merge?

Only if you make its check required or let its approval count toward merge requirements. A failing or missing required status check blocks merging, while comment-only reviews do not.

Should AI Review Run On Draft Pull Requests?

Usually not during a pilot. Reviewing drafts adds cost and noise on code that is still changing. Both CodeRabbit and Copilot code review let you turn draft reviews on later.

How Many Repositories Should A Pilot Cover?

Two or three active repositories with different owners is enough to see patterns in a month, while keeping the GitHub App’s access narrow.

Conclusion

A good AI code review GitHub setup is mostly about restraint: narrow repository access, configuration in version control, human approvals that stay required, and a few weeks of tuning before you scale. Get those right and the reviewer becomes a dependable first pass instead of another notification to ignore.

Start with one tool, one or two repositories and clear metrics, then expand once your team is accepting its comments.