Shunsuke SuzukiThis post explains how to gather GitHub Issues and Pull Requests across GitHub repositories into GitHub Projects automatically.
This post is based on my post written in Japanese. 自分が管理する全 OSS の Issue や Pull Request を 1 つの GitHub Project に集約
Managing GitHub Issues and Pull Requests across multiple repositories can be challenging.
By adding all of them to GitHub Projects, you can manage them in one place.
To do this, you need to add them to GitHub Projects somehow.
There are several ways to add them to GitHub Projects automatically.
- Using the built-in automations
- Running GitHub Actions by issues and pull requests' opened events
- (Recommendation) Running GitHub Actions by schedule event to search issues and pull requests and add them to projects
I recommend the third approach and will describe it in detail.
1. Using the built-in automations
GitHub provides the built-in automations, but this feature has several limitations.
- You have to configure workflows for each repository, which is cumbersome
- You can create only five auto-add workflows. To create more workflows, you have to upgrade your plan
2. Running GitHub Actions by issues and pull requests' opened events
You can add issues and pull requests to projects using GitHub Actions.
GitHub provides an official action for this.
https://github.com/marketplace/actions/add-to-github-projects
However, I don't recommend this approach because you have to add workflows for each repository, which makes it hard to maintain.
3. Running GitHub Actions by schedule event to search issues and pull requests and add them to projects
You can run GitHub Actions by schedule event to search issues and pull requests and add them to projects.
The benefit of this approach is that you only have to maintain a single workflow.
I've developed a command-line tool for this purpose.
https://github.com/suzuki-shunsuke/ghproj
This tool performs the following tasks.
- Reads a configuration file
- Searches issues and pull requests using GitHub GraphQL API
- Excludes some issues and pull requests based on an expression
- Adds issues and pull requests to projects
ghproj init # Scaffold a configuration file ghproj.yaml
ghproj add [-config (-c) <configuration file path>] # Add issues and pull requests to GitHub Projects
Here is an example of a configuration file ghproj.yaml.
entries:
# query: The query of GitHub GraphQL API to search issues and pull requests which are added to GitHub Projects
- query: |
is:open
-project:suzuki-shunsuke/5
archived:false
owner:suzuki-shunsuke
owner:aquaproj
owner:lintnet
is:public
# expr: An expression filtering the search result.
# The expression is evaluated using github.com/expr-lang/expr.
# The expression is evaluated per item in the search result.
# The evaluation result must be a boolean.
# If the result is false, the item is excluded.
# expr is optional. If expr isn't set, all search results are used.
expr: |
(! Item.Repo.IsFork) &&
(Item.Title != "Dependency Dashboard")
# project_id: GitHub Project id where issues and pull requests are added.
# You can get the id by GitHub CLI's gh project list command.
project_id: PVT_kwHOAMtMJ84AQCf4
For more information on the query syntax, please see the GitHub documentation:
You can test queries using the search box on GitHub: https://github.com
GitHub Access Token
A GitHub access token is needed to add issues and pull requests to projects.
There are several types of GitHub access tokens.
- GitHub Actions token: This token doesn't support adding items to GitHub Projects
- GitHub App: This token doesn't support managing GitHub Users' Projects
- OAuth App (GitHub CLI)
- Personal Access Token (PAT)
- classic PAT: This token is insecure because you can't restrict permissions and scopes flexibly
- fine-grained PAT: This token doesn't support GitHub Projects
Fine-grained PAT doesn't support GitHub Projects.
https://github.com/orgs/community/discussions/36441
There are also some APIs that do not yet support the fine-grained permission model, that we'll be adding support for in time:
- Packages
- Projects
- Notifications
So there are two options.
- Pull together issues and pull requests to GitHub Users' Projects using a classic PAT
- (Recommendation) Pull together issues and pull requests to GitHub Organizations' Projects using a GitHub App
GitHub App is more secure than classic PAT, so I recommend the second option.
Even if you want to manage your personal issues and pull requests, I recommend creating a GitHub Organization and adopting the second option.
1. Using a classic PAT
You should configure the expiration date.
The required scopes are read:org and project.
2. Using a GitHub App
- Create a GitHub App
The required permissions are
-
Repository permissions:metadata: read-only -
Organization permissions:Projects: Read and write
- Install the GitHub App into a repository where GitHub Actions is run
- Create an installation access token using an action such as tibdex/github-app-token or actions/create-github-app-token
To create an installation token, you have to install the GitHub App to repositories.
To do this, you have to grant any Repository permissions to the GitHub App.
If you want to manage issues and pull requests of private repositories, you have to grant Pull Requests: read-only and Issues: read-only permissions to the GitHub App and install it to private repositories.
Archiving GitHub Projects Items Based on Conditions
You can add only 1,200 items in each project.
To add more items to a project, you have to archive existing items.
GitHub Project supports workflows to archive items automatically.
Archiving items will help you stay below the limit of 1,200 items in each project.
This feature is useful, but sometimes you may want to archive items based on specific conditions.
For example, you may want to archive issues and pull requests if their repositories are archived.
The above tool ghproj supports archiving them.
e.g. ghproj.yaml
entries:
- expr: |
(Item.Open && Item.Repo.IsArchived) ||
(Item.Title == "Dependency Dashboard")
action: archive
project_id: PVT_kwHOAMtMJ84AQCf4
GitHub GraphQL API can't query GitHub Project items, so ghproj retrieves all items and filter them using expr-lang/expr.
Automation with GitHub Actions
You can gather issues and pull requests into GitHub Projects automatically by running ghproj periodically via GitHub Actions schedule events.
This is the workflow I actually use.
https://github.com/szksh-lab/.github/blob/main/.github/workflows/update-project.yaml
This workflow pulls together all issues and pull requests to the GitHub Project:
https://github.com/orgs/szksh-lab/projects/1
Sorting issues and pull requests to multiple projects
The tool ghproj supports multiple projects, so you can sort issues and pull requests to different projects.
e.g. ghproj.yaml
entries:
# Add issues and pull requests with the label team/sre to the project of the SRE team
- query: |
is:open archived:false -project:szksh-lab/2
owner:szksh-lab label:team/sre
project_id: PVT_SRE0000000000000
# Add issues and pull requests whose repositories are managed by SER team to the project of the SRE team
- query: |
is:open archived:false -project:szksh-lab/2
repo:szksh-lab/k8s-clusters
repo:szksh-lab/aws-org
project_id: PVT_SRE0000000000000
# Add issues and pull requests which are assigned to the security team to the project of the Security team
- query: |
is:open archived:false -project:szksh-lab/2
owner:szksh-lab is:pr
team-review-requested:szksh-lab/security
project_id: PVT_SECURITY00000000
Conclusion
In this post, I explained how to automatically gather GitHub Issues and Pull Requests across multiple GitHub repositories into GitHub Projects.
I'm maintaining a lot of OSS projects, and managing their issues and pull requests has been challenging.
Using this approach, I could manage them all in one GitHub Project, which was incredibly helpful.
https://github.com/orgs/szksh-lab/projects/1/views/1
I hope this post helps you improve the management of your issues and pull requests.
