Active repository AI-assisted development MIT licensed

PatchTrack

PatchTrack is a tool and replication package for analyzing how ChatGPT influences pull request outcomes. It classifies whether AI-generated patches are applied, partially integrated, or ignored, giving the lab a grounded way to study real AI-assisted development behavior.

All projects Selected projects GitHub repository

Project image for PatchTrack — Illustrative view of pull request outcomes, patch classification, and integration trajectories for ChatGPT-generated code suggestions.

Overview

{"PatchTrack was built to answer a practical question about generative AI in software development" => "when developers mention ChatGPT in pull requests, how often do those suggestions actually make it into the final code, and in what form? The tool analyzes pull requests containing self-admitted ChatGPT usage and classifies suggested patches as applied, not applied, or not suggested."}

The replication package reported 338 pull requests from 255 GitHub repositories, 645 AI-generated snippets, and 3,486 developer-authored patches. Its findings show that full adoption of ChatGPT output is rare, with a median integration rate of 25%, and that developers often treat AI output as a starting point for iterative refinement rather than as final code.

What the project focuses on

Classifying AI-suggested patches within pull request workflows.
Measuring how often ChatGPT code is integrated, transformed, or discarded.
Supporting reproducible notebook- and CLI-based analysis of adoption behavior.
Connecting AI-generated suggestions to developer decision-making in context.

Key takeaways

Reveals that full ChatGPT patch adoption is uncommon in real pull requests.
Supports both notebook-based reproduction and CLI-based exploration.
Provides a concrete bridge between AI coding assistance and empirical software engineering evidence.

Why it matters

Claims about AI coding tools often stop at benchmark quality. PatchTrack instead shows how those suggestions are negotiated inside collaborative development, which is the level where practical engineering impact is actually decided.