An LLM can't go for a walk in the woods

MP 169: But we still can, and we should keep doing so.

Regular readers of Mostly Python are probably aware that I'm currently working on gh-profiler. It helps open source maintainers make better-informed decisions about how much time to invest with users who open new PRs and issues on their projects.

I was considering a few different ways to implement a new feature, and had settled on one approach. It was a bit clunky, but I was pretty sure it would work. The LLM I was using at the time didn't offer any better suggestions, and I was ready to implement the new feature. But implementation would have to wait until later in the day, because it was the last dry morning for a bit, and I wanted to go for a walk in the woods to look for birds. In the middle of the quiet woods, I came up with a much better approach to implementing the new feature.

Maintaining open source projects in 2026

These days, when they get a new PR or issue from a GitHub user they don't recognize, many maintainers are clicking over to that user's profile page before deciding how to proceed. They're looking for signs of whether the new contributor is a human doing well-intentioned open source work, a human driving an AI bot to make a high volume of contributions, or a semi-autonomous bot doing whatever it will in our communities.

Instead of looking for that kind of information manually, you can run gh-profiler against any GitHub username, PR number, or issue number in the repository you're working in.

I've been actively using gh-profiler in the gh-profiler repo itself:

gh_profiler$ uvx gh-profiler 129 --concise
GitHub user: <redacted>
🟢 No concerns found with user's profile.
🟢 No concerns found with recent PR activity.
🟢 No concerns found with recent issue activity.

For a more detailed report, run `gh-profiler <redacted>`.

This has been much easier than clicking around on people's profiles. I know exactly the kind of information I'm going to see, I can easily get a more complete report, and I can click over to their profile any time I want more context before deciding how to respond. Other maintainers are reporting similar experiences.

Previewing usage in a repo

Using gh-profiler to get a snapshot of an individual user's recent activity is pretty straightforward. The core usage looks like this:

$ uvx gh-profiler <target>

Here the target can be a username, a PR number, or an issue number. Whichever one you pass, the tool makes sense of and runs the appropriate profile. This is a nice simple user experience.

But I found myself running gh-profiler over and over again while looking at PR and issue tabs in a variety of open source projects. I've heard from multiple people who've written small scripts to run gh-profiler against a number of PRs and issues at once, to look at overall contribution patterns rather than an individual user's recent activity.

I wanted to build that feature right into gh-profiler itself, to make it easier for maintainers to see how well its output would align with the decisions they've already been making.

I thought this new feature would be implemented as a subcommand named preview:

$ uvx gh-profiler preview <repo-url> -n 10

This should work reasonably well. When gh-profiler preview is run, it should look at the last n closed PRs, profile each contributor, and show the profiler output alongside information about whether the PR was merged or closed.

The only problem was that gh-profiler doesn't have any other subcommands yet. Once you want to use a subcommand, a CLI works more smoothly if every action is implemented as a subcommand. This wasn't appealing, because I didn't want to add a command for profiling a user. Calling gh-profiler with a single target is a nice simple usage that I didn't want to give up.

I finally decided to parse the target that the user provided, and check if it was preview. If it was, I'd call the previewing code. It shouldn't cause any conflicts, because there aren't (and shouldn't be) any GitHub users named "preview". It's clunky, though, because it doesn't play well with the auto-generated help output you get from Click. It also leads to a somewhat kludgy implementation; when you use a framework like Click against its best practices, there's almost certainly going to be some ugliness in the implementation. The LLM was not offering any better suggestions.

A walk in the woods, and a moment of clarity

Since moving to western NC a couple years ago, I like taking monthly walks somewhere along the Blue Ridge Parkway to look for birds. It's a fantastic place to go birding, because such a wide variety of birds spend some time on the ridge at various times of the year.

I got up early for a sunrise bird walk, and once into the woods it was perfectly still except for a few interesting bird calls. While picking up my binoculars to look at a bird, I suddenly figured out a better implementation for this new feature. Instead of looking for a target named "preview", I could just let users pass a repo URL as the target. There's no need for a new subcommand. A repo is just a natural third kind of target, which the tool will know what to do with, just as it now knows how to handle usernames and PR/issue numbers.

I took out my little pocket notebook, and jotted this quick note:

No subcommand! If target is a URL, do URL things.

That was a really nice passing thought, and I don't think it ever would have come up if I'd stayed home working instead of going to the woods. If I'd stayed home, I would have just gotten to work implementing the original approach. I can't imagine the LLM would have come up with that cleaner solution either. It would have just supported the direction I had already chosen, because it can't stop to get a fresh perspective on a project like we can.

I enjoyed the rest of the walk even more, knowing I'd return home to implement an approach I was much more confident in. I wasn't really looking forward to building the kludgier solution, because I had a strong feeling there was a better approach.

Current usage

Now you can run gh-profiler against any GitHub repo, in a number of different ways:

$ uvx gh-profiler <repo-url>
$ uvx gh-profiler <repo-url> -n 20
$ uvx gh-profiler <repo-url> --back
$ uvx gh-profiler <repo-url> --back -n 20
$ uvx gh-profiler <repo-url> --table-only
$ uvx gh-profiler <repo-url> --back -n 20 --table-only

With no other arguments, the profiler will report on the 10 most recently submitted open PRs. You can include as many PRs as you want, up to 100. If you pass the --back option, it will look back at the most recently closed PRs. The --table-only option shows a table summarizing what gh-profiler found.

Here's what the output looks like for the 25 most recent closed/merged PRs in the Requests repository:

This usage makes it easy to see how well gh-profiler output is aligning with maintainer decisions. Here most of the submissions that raised red flags under gh-profiler were ultimately closed without merging by the maintainers of Requests.

Running this same command on a variety of projects, with n maxed out to 100, lets you see the literal waves of AI slop submissions that maintainers are dealing with. I know gh-profiler won't save open source alone, but I'm hoping it helps maintainers see and understand the patterns in the submissions they're getting on their own projects. I'm hoping it continues to be a way to share and make visible the meaningful signals we're seeing in our open source work, both for identifying problematic "contributors", and for identifying the kind of actual people we're all still wanting to work with.

If you've been thinking of getting out for a walk lately and haven't because an LLM or agent is never getting tired, remember that they can't choose to go for a walk. That's your decision to make, whenever it's right for you.