Amazon Kendra is no longer open to new customers. For capabilities similar to Amazon Kendra, explore Amazon Bedrock Knowledge Bases. Learn more.
GitHub
GitHub is a web-based hosting service for software development providing code storage and management services with version control. You can use Amazon Kendra to index your GitHub Enterprise Cloud (SaaS) and GitHub Enterprise Server (On Prem) repository files, issue and pull requests, issue and pull request comments, and issue and pull request comment attachments. You can also choose to include or exclude certain files.
Note
Amazon Kendra now supports an upgraded GitHub connector.
The console has been automatically upgraded for you. Any new connectors you create in
the console will use the upgraded architecture. If you use the API, you must now use the
TemplateConfiguration object instead of the
GitHubConfiguration object to configure your
connector.
Connectors configured using the older console and API architecture will continue to function as configured. However, you won’t be able to edit or update them. If you want to edit or update your connector configuration, you must create a new connector.
We recommended migrating your connector workflow to the upgraded version. Support for connectors configured using the older architecture is scheduled to end by June 2024.
You can connect Amazon Kendra to your GitHub data source using the Amazon Kendra console
For troubleshooting your Amazon Kendra GitHub data source connector, see Troubleshooting data sources.
Supported features
Amazon Kendra GitHub data source connector supports the following features:
-
Field mappings
-
User access control
-
Inclusion/exclusion filters
-
Full and incremental content syncs
-
Virtual private cloud (VPC)
Prerequisites
Before you can use Amazon Kendra to index your GitHub data source, make these changes in your GitHub and AWS accounts.
In GitHub, make sure you have:
-
Created a GitHub user with administrative permissions to the GitHub organization.
-
Configured a personal access token in Git Hub to use as your authentication credentials. See GitHub documentation on creating a personal access token
. Note
We recommend that you regularly refresh or rotate your credentials and secret. Provide only the necessary access level for your own security. We do not recommend that you re-use credentials and secrets across data sources, and connector versions 1.0 and 2.0 (where applicable).
-
Recommended:Configured an OAuth token for authentication credentials. Use OAuth token for better API throttle limits and connector performance. See GitHub documentation on OAuth authorization
. -
Noted the GitHub host URL for the type of GitHub service that you use. For example, the host URL for GitHub cloud could be
https://api.github.comand the host URL for GitHub server could behttps://on-prem-host-url/api/v3/. -
Noted the name of your organization for GitHub the GitHub Enterprise Cloud (SaaS) account or GitHub Enterprise Server (on-premises) account you want to connect to. You can find your organization name by logging into GitHub desktop and selecting Your organizations under your profile picture dropdown.
-
Optional (server only): Generated a SSL certificate and copied the path to the certificate stored in an Amazon S3 bucket. You use this to connect to GitHub if you require a secure SSL connection. You can simply generate a self-signed X509 certificate on any computer using OpenSSL. For an example of using OpenSSL to create an X509 certificate, see Create and sign an X509 certificate.
-
Added the following permissions:
For GitHub Enterprise Cloud (SaaS)
-
repo:status– Grants read/write access to commit statuses in public and private repositories. This scope is only necessary to grant other users or services access to private repository commit statuses without granting access to the code. -
repo_deployment– Grants access to deployment statuses for public and private repositories. This scope is only necessary to grant other users or services access to deployment statuses, without granting access to the code. -
public_repo– Limits access to public repositories. That includes read/write access to code, commit statuses, repository projects, collaborators, and deployment statuses for public repositories and organizations. Also required for starring public repositories. -
repo:invite– Grants accept/decline abilities for invitations to collaborate on a repository. This scope is only necessary to grant other users or services access to invites without granting access to the code. -
security_events– Grants: read and write access to security events in the code scanning API. This scope is only necessary to grant other users or services access to security events without granting access to the code. -
read:org– Read-only access to organization membership, organization projects, and team membership. -
user:email– Grants read access to a user's email addresses. Required by Amazon Kendra to crawl ACLs. -
user:follow– Grants access to follow or unfollow other users. Required by Amazon Kendra to crawl ACLs. -
read:user– Grants access to read a user's profile data. Required by Amazon Kendra to crawl ACLs. -
workflow– Grants the ability to add and update GitHub Actions workflow files. Workflow files can be committed without this scope if the same file (with both the same path and contents) exists on another branch in the same repository.
For more information, see Scopes for OAuth apps
in GitHub Docs. For GitHub Enterprise Server (On Prem)
-
repo:status– Grants read/write access to commit statuses in public and private repositories. This scope is only necessary to grant other users or services access to private repository commit statuses without granting access to the code. -
repo_deployment– Grants access to deployment statuses for public and private repositories. This scope is only necessary to grant other users or services access to deployment statuses, without granting access to the code. -
public_repo– Limits access to public repositories. That includes read/write access to code, commit statuses, repository projects, collaborators, and deployment statuses for public repositories and organizations. Also required for starring public repositories. -
repo:invite– Grants accept/decline abilities for invitations to collaborate on a repository. This scope is only necessary to grant other users or services access to invites without granting access to the code. -
security_events– Grants: read and write access to security events in the code scanning API. This scope is only necessary to grant other users or services access to security events without granting access to the code. -
read:user– Grants access to read a user's profile data. Required by Amazon Q Business to crawl ACLs. -
user:email– Grants read access to a user's email addresses. Required by Amazon Q Business to crawl ACLs. -
user:follow– Grants access to follow or unfollow other users. Required by Amazon Q Business to crawl ACLs. -
site_admin– Grants site administrators access to GitHub Enterprise Server Administration API endpoints. -
workflow– Grants the ability to add and update GitHub Actions workflow files. Workflow files can be committed without this scope if the same file (with both the same path and contents) exists on another branch in the same repository.
For more information, see Scopes for OAuth apps
in GitHub Docs and Understanding scopes for OAuth Apps in GitHub Developer. -
-
Checked each document is unique in GitHub and across other data sources you plan to use for the same index. Each data source that you want to use for an index must not contain the same document across the data sources. Document IDs are global to an index and must be unique per index.
In your AWS account, make sure you have:
-
Created an Amazon Kendra index and, if using the API, noted the index ID.
-
Created an IAM role for your data source and, if using the API, noted the ARN of the IAM role.
Note
If you change your authentication type and credentials, you must update your IAM role to access the correct AWS Secrets Manager secret ID.
-
Stored your GitHub authentication credentials in an AWS Secrets Manager secret and, if using the API, noted the ARN of the secret.
Note
We recommend that you regularly refresh or rotate your credentials and secret. Provide only the necessary access level for your own security. We do not recommend that you re-use credentials and secrets across data sources, and connector versions 1.0 and 2.0 (where applicable).
If you don’t have an existing IAM role or secret, you can use the console to create a new IAM role and Secrets Manager secret when you connect your GitHub data source to Amazon Kendra. If you are using the API, you must provide the ARN of an existing IAM role and Secrets Manager secret, and an index ID.