View a markdown version of this page

Migrate polling pipelines to use event-based change detection - AWS CodePipeline

Migrate polling pipelines to use event-based change detection

AWS CodePipeline supports full, end-to-end continuous delivery, which includes starting your pipeline whenever there is a code change. There are two supported ways to start your pipeline upon a code change: event-based change detection and polling. We recommend using event-based change detection for pipelines.

Use the procedures included here to migrate (update) your polling pipelines to the event-based change detection method for your pipeline.

The recommended event-based change detection method for pipelines is determined by the pipeline source, such as CodeCommit. In that case, for example, the polling pipeline would need to migrate to event-based change detection with EventBridge.

How to migrate polling pipelines

To migrate polling pipelines, determine your polling pipelines and then determine the recommended event-based change detection method:

  • Use the steps in Viewing polling pipelines in your account to determine your polling pipelines.

  • In the table, find your pipeline source type and then choose the procedure with the implementation you want to use to migrate your polling pipeline. Each section contains multiple methods for migration, such as using the CLI or CloudFormation.

How to migrate pipelines to the recommended change detection method
Pipeline source Recommended event-based detection method Migration procedures
AWS CodeCommit EventBridge (recommended). See Migrate polling pipelines with a CodeCommit source.
Amazon S3 EventBridge and bucket enabled for event notifications (recommended). See Migrate polling pipelines with an S3 source enabled for events.
Amazon S3 EventBridge and an AWS CloudTrail trail. See Migrate polling pipelines with an S3 source and CloudTrail trail.
GitHub (via GitHub App) Connections (recommended) See Migrate polling pipelines for a GitHub (via OAuth app) source action to connections.
GitHub (via OAuth app) Webhooks See Migrate polling pipelines for a GitHub (via OAuth app) source action to webhooks.
Important

For applicable pipeline action configuration updates, such as pipelines with a GitHub (via OAuth app) action, you must explicitly set the PollForSourceChanges parameter to false within your Source action's configuration to stop a pipeline from polling. As a result, it is possible to erroneously configure a pipeline with both event-based change detection and polling by, for example, configuring an EventBridge rule and also omitting the PollForSourceChanges parameter. This results in duplicate pipeline executions, and the pipeline is counted toward the limit on total number of polling pipelines, which by default is much lower than event-based pipelines. For more information, see Quotas in AWS CodePipeline.

Viewing polling pipelines in your account

As a first step, use one of the following scripts to determine which pipelines in your account are configured for polling. These are the pipelines to migrate to event-based change detection.

Viewing polling pipelines in your account (script)

Follow these steps to use a script to determine pipelines in your account that are using polling.

  1. Open a terminal window, and then do one of the following:

    • Run the following command to create a new script named PollingPipelinesExtractor.sh.

      vi PollingPipelinesExtractor.sh
    • To use a python script, run the following command to create a new python script named PollingPipelinesExtractor.py.

      vi PollingPipelinesExtractor.py
  2. Copy and paste the following code into the PollingPipelinesExtractor script. Do one of the following:

    • Copy and paste the following code into the PollingPipelinesExtractor.sh script.

      #!/bin/bash set +x POLLING_PIPELINES=() LAST_EXECUTED_DATES=() NEXT_TOKEN=null HAS_NEXT_TOKEN=true if [[ $# -eq 0 ]] ; then echo 'Please provide region name' exit 0 fi REGION=$1 while [ "$HAS_NEXT_TOKEN" != "false" ]; do if [ "$NEXT_TOKEN" != "null" ]; then LIST_PIPELINES_RESPONSE=$(aws codepipeline list-pipelines --region $REGION --next-token $NEXT_TOKEN) else LIST_PIPELINES_RESPONSE=$(aws codepipeline list-pipelines --region $REGION) fi LIST_PIPELINES=$(jq -r '.pipelines[].name' <<< "$LIST_PIPELINES_RESPONSE") NEXT_TOKEN=$(jq -r '.nextToken' <<< "$LIST_PIPELINES_RESPONSE") if [ "$NEXT_TOKEN" == "null" ]; then HAS_NEXT_TOKEN=false fi for pipline_name in $LIST_PIPELINES do PIPELINE=$(aws codepipeline get-pipeline --name $pipline_name --region $REGION) HAS_POLLABLE_ACTIONS=$(jq '.pipeline.stages[].actions[] | select(.actionTypeId.category == "Source") | select(.actionTypeId.owner == ("ThirdParty","AWS")) | select(.actionTypeId.provider == ("GitHub","S3","CodeCommit")) | select(.configuration.PollForSourceChanges == ("true",null))' <<< "$PIPELINE") if [ ! -z "$HAS_POLLABLE_ACTIONS" ]; then POLLING_PIPELINES+=("$pipline_name") PIPELINE_EXECUTIONS=$(aws codepipeline list-pipeline-executions --pipeline-name $pipline_name --region $REGION) LAST_EXECUTION=$(jq -r '.pipelineExecutionSummaries[0]' <<< "$PIPELINE_EXECUTIONS") if [ "$LAST_EXECUTION" != "null" ]; then LAST_EXECUTED_TIMESTAMP=$(jq -r '.startTime' <<< "$LAST_EXECUTION") LAST_EXECUTED_DATE="$(date -r ${LAST_EXECUTED_TIMESTAMP%.*})" else LAST_EXECUTED_DATE="Not executed in last year" fi LAST_EXECUTED_DATES+=("$LAST_EXECUTED_DATE") fi done done fileName=$REGION-$(date +%s) printf "| %-30s | %-30s |\n" "Polling Pipeline Name" "Last Executed Time" printf "| %-30s | %-30s |\n" "_____________________" "__________________" for i in "${!POLLING_PIPELINES[@]}"; do printf "| %-30s | %-30s |\n" "${POLLING_PIPELINES[i]}" "${LAST_EXECUTED_DATES[i]}" printf "${POLLING_PIPELINES[i]}," >> $fileName.csv done printf "\nSaving Polling Pipeline Names to file $fileName.csv."
    • Copy and paste the following code into the PollingPipelinesExtractor.py script.

      import boto3 import sys import time import math hasNextToken = True nextToken = "" pollablePipelines = [] lastExecutedTimes = [] if len(sys.argv) == 1: raise Exception("Please provide region name.") session = boto3.Session(profile_name='default', region_name=sys.argv[1]) codepipeline = session.client('codepipeline') def is_pollable_action(action): actionTypeId = action['actionTypeId'] configuration = action['configuration'] return actionTypeId['owner'] in {"AWS", "ThirdParty"} and actionTypeId['provider'] in {"GitHub", "CodeCommit", "S3"} and ('PollForSourceChanges' not in configuration or configuration['PollForSourceChanges'] == 'true') def has_pollable_actions(pipeline): hasPollableAction = False pipelineDefinition = codepipeline.get_pipeline(name=pipeline['name'])['pipeline'] for action in pipelineDefinition['stages'][0]['actions']: hasPollableAction = is_pollable_action(action) if hasPollableAction: break return hasPollableAction def get_last_executed_time(pipelineName): pipelineExecutions=codepipeline.list_pipeline_executions(pipelineName=pipelineName)['pipelineExecutionSummaries'] if pipelineExecutions: return pipelineExecutions[0]['startTime'].strftime("%A %m/%d/%Y, %H:%M:%S") else: return "Not executed in last year" while hasNextToken: if nextToken=="": list_pipelines_response = codepipeline.list_pipelines() else: list_pipelines_response = codepipeline.list_pipelines(nextToken=nextToken) if 'nextToken' in list_pipelines_response: nextToken = list_pipelines_response['nextToken'] else: hasNextToken= False for pipeline in list_pipelines_response['pipelines']: if has_pollable_actions(pipeline): pollablePipelines.append(pipeline['name']) lastExecutedTimes.append(get_last_executed_time(pipeline['name'])) fileName="{region}-{timeNow}.csv".format(region=sys.argv[1],timeNow=math.trunc(time.time())) file = open(fileName, 'w') print ("{:<30} {:<30} {:<30}".format('Polling Pipeline Name', '|','Last Executed Time')) print ("{:<30} {:<30} {:<30}".format('_____________________', '|','__________________')) for i in range(len(pollablePipelines)): print("{:<30} {:<30} {:<30}".format(pollablePipelines[i], '|', lastExecutedTimes[i])) file.write("{pipeline},".format(pipeline=pollablePipelines[i])) file.close() print("\nSaving Polling Pipeline Names to file {fileName}".format(fileName=fileName))
  3. For each Region where you have pipelines, you must run the script for that Region. To run the script, do one of the following:

    • Run the following command to run the script named PollingPipelinesExtractor.sh. In this example, the Region is us-west-2.

      ./PollingPipelinesExtractor.sh us-west-2
    • For the python script, run the following command to run the python script namedPollingPipelinesExtractor.py. In this example, the Region is us-west-2.

      python3 PollingPipelinesExtractor.py us-west-2

    In the following sample output from the script, the Region us-west-2 returned a list of polling pipelines and shows the last execution time for each pipeline.

    % ./pollingPipelineExtractor.sh us-west-2 | Polling Pipeline Name | Last Executed Time | | _____________________ | __________________ | | myCodeBuildPipeline | Wed Mar 8 09:35:49 PST 2023 | | myCodeCommitPipeline | Mon Apr 24 22:32:32 PDT 2023 | | TestPipeline | Not executed in last year | Saving list of polling pipeline names to us-west-2-1682496174.csv...%

    Analyze the script output and, for each pipeline in the list, update the polling source to the recommended event-based change detection method.

    Note

    Your polling pipelines are determined by the pipeline's action configuration for the PollForSourceChanges parameter. If the pipeline source configuration has the PollForSourceChanges parameter omitted, then CodePipeline defaults to polling your repository for source changes. This behavior is the same as if PollForSourceChanges is included and set to true. For more information, see the configuration parameters for your pipeline's source action, such as the Amazon S3 source action configuration parameters in Amazon S3 source action reference.

    Note that this script also generates a .csv file containing the list of polling pipelines in your account and saves the .csv file to the current working folder.

Migrate polling pipelines with a CodeCommit source

You can migrate your polling pipeline to use EventBridge to detect changes in your CodeCommit source repository or your Amazon S3 source bucket.

CodeCommit -- For a pipeline with a CodeCommit source, modify the pipeline so that change detection is automated through EventBridge. Choose from the following methods to implement the migration:

Migrate polling pipelines (CodeCommit or Amazon S3 source) (console)

You can use the CodePipeline console to update your pipeline to use EventBridge to detect changes in your CodeCommit source repository or your Amazon S3 source bucket.

Note

When you use the console to edit a pipeline that has a CodeCommit source repository or an Amazon S3 source bucket, the rule and IAM role are created for you. If you use the AWS CLI to edit the pipeline, you must create the EventBridge rule and IAM role yourself. For more information, see CodeCommit source actions and EventBridge.

Use these steps to edit a pipeline that is using periodic checks. If you want to create a pipeline, see Create a pipeline, stages, and actions.

To edit the pipeline source stage
  1. Sign in to the AWS Management Console and open the CodePipeline console at http://console.aws.amazon.com/codesuite/codepipeline/home.

    The names of all pipelines associated with your AWS account are displayed.

  2. In Name, choose the name of the pipeline you want to edit. This opens a detailed view of the pipeline, including the state of each of the actions in each stage of the pipeline.

  3. On the pipeline details page, choose Edit.

  4. In Edit stage, choose the edit icon on the source action.

  5. Expand Change Detection Options and choose Use CloudWatch Events to automatically start my pipeline when a change occurs (recommended).

    A message appears showing the EventBridge rule to be created for this pipeline. Choose Update.

    If you are updating a pipeline that has an Amazon S3 source, you see the following message. Choose Update.

  6. When you have finished editing your pipeline, choose Save pipeline changes to return to the summary page.

    A message displays the name of the EventBridge rule to be created for your pipeline. Choose Save and continue.

  7. To test your action, release a change by using the AWS CLI to commit a change to the source specified in the source stage of the pipeline.

Migrate polling pipelines (CodeCommit source) (CLI)

Follow these steps to edit a pipeline that is using polling (periodic checks) to use an EventBridge rule to start the pipeline. If you want to create a pipeline, see Create a pipeline, stages, and actions.

To build an event-driven pipeline with CodeCommit, you edit the PollForSourceChanges parameter of your pipeline and then create the following resources:

  • EventBridge event

  • IAM role to allow this event to start your pipeline

To edit your pipeline's PollForSourceChanges parameter
Important

When you create a pipeline with this method, the PollForSourceChanges parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see Valid settings for the PollForSourceChanges parameter.

  1. Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named MyFirstPipeline, run the following command:

    aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json

    This command returns nothing, but the file you created should appear in the directory where you ran the command.

  2. Open the JSON file in any plain-text editor and edit the source stage by changing the PollForSourceChanges parameter to false, as shown in this example.

    Why am I making this change? Changing this parameter to false turns off periodic checks so you can use event-based change detection only.

    "configuration": { "PollForSourceChanges": "false", "BranchName": "main", "RepositoryName": "MyTestRepo" },
  3. If you are working with the pipeline structure retrieved using the get-pipeline command, remove the metadata lines from the JSON file. Otherwise, the update-pipeline command cannot use it. Remove the "metadata": { } lines and the "created", "pipelineARN", and "updated" fields.

    For example, remove the following lines from the structure:

    "metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" },

    Save the file.

  4. To apply your changes, run the update-pipeline command, specifying the pipeline JSON file:

    Important

    Be sure to include file:// before the file name. It is required in this command.

    aws codepipeline update-pipeline --cli-input-json file://pipeline.json

    This command returns the entire structure of the edited pipeline.

    Note

    The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the start-pipeline-execution command to manually start your pipeline.

To create an EventBridge rule with CodeCommit as the event source and CodePipeline as the target
  1. Add permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see Using resource-based policies for Amazon EventBridge.

    1. Use the following sample to create the trust policy that allows EventBridge to assume the service role. Name the trust policy trustpolicyforEB.json.

      JSON
      { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
    2. Use the following command to create the Role-for-MyRule role and attach the trust policy.

      aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json
    3. Create the permissions policy JSON, as shown in this sample, for the pipeline named MyFirstPipeline. Name the permissions policy permissionspolicyforEB.json.

      JSON
      { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codepipeline:StartPipelineExecution" ], "Resource": [ "arn:aws:codepipeline:us-west-2:111122223333:MyFirstPipeline" ] } ] }
    4. Use the following command to attach the CodePipeline-Permissions-Policy-for-EB permissions policy to the Role-for-MyRule role.

      Why am I making this change? Adding this policy to the role creates permissions for EventBridge.

      aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
  2. Call the put-rule command and include the --name, --event-pattern , and--role-arn parameters.

    Why am I making this change? This command enables CloudFormation to create the event.

    The following sample command creates a rule called MyCodeCommitRepoRule.

    aws events put-rule --name "MyCodeCommitRepoRule" --event-pattern "{\"source\":[\"aws.codecommit\"],\"detail-type\":[\"CodeCommit Repository State Change\"],\"resources\":[\"repository-ARN\"],\"detail\":{\"referenceType\":[\"branch\"],\"referenceName\":[\"main\"]}}" --role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule"
  3. To add CodePipeline as a target, call the put-targets command and include the following parameters:

    • The --rule parameter is used with the rule_name you created by using put-rule.

    • The --targets parameter is used with the list Id of the target in the list of targets and the ARN of the target pipeline.

    The following sample command specifies that for the rule called MyCodeCommitRepoRule, the target Id is composed of the number one, indicating that in a list of targets for the rule, this is target 1. The sample command also specifies an example ARN for the pipeline. The pipeline starts when something changes in the repository.

    aws events put-targets --rule MyCodeCommitRepoRule --targets Id=1,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline
  4. (Optional) To configure an input transformer with source overrides for a specific image ID, use the following JSON in your CLI command. The following example configures an override where:

    • The actionName, Source in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.

    • The revisionType, COMMIT_ID in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.

    • The revisionValue, <revisionValue> in this example, is derived from the source event variable.

    { "Rule": "my-rule", "Targets": [ { "Id": "MyTargetId", "Arn": "pipeline-ARN", "InputTransformer": { "sourceRevisions": { "actionName": "Source", "revisionType": "COMMIT_ID", "revisionValue": "<revisionValue>" }, "variables": [ { "name": "Branch_Name", "value": "value" } ] } } ] }

Migrate polling pipelines (CodeCommit source) (CloudFormation template)

To build an event-driven pipeline with AWS CodeCommit, you edit the PollForSourceChanges parameter of your pipeline and then add the following resources to your template:

  • An EventBridge rule

  • An IAM role for your EventBridge rule

If you use CloudFormation to create and manage your pipelines, your template includes content like the following.

Note

The Configuration property in the source stage called PollForSourceChanges. If that property isn't included in your template, then PollForSourceChanges is set to true by default.

YAML
Resources: AppPipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: codecommit-polling-pipeline RoleArn: !GetAtt CodePipelineServiceRole.Arn Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: CodeCommit OutputArtifacts: - Name: SourceOutput Configuration: BranchName: !Ref BranchName RepositoryName: !Ref RepositoryName PollForSourceChanges: true RunOrder: 1
JSON
"Stages": [ { "Name": "Source", "Actions": [{ "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "CodeCommit" }, "OutputArtifacts": [{ "Name": "SourceOutput" }], "Configuration": { "BranchName": { "Ref": "BranchName" }, "RepositoryName": { "Ref": "RepositoryName" }, "PollForSourceChanges": true }, "RunOrder": 1 }] },
To update your pipeline CloudFormation template and create EventBridge rule
  1. In the template, under Resources, use the AWS::IAM::Role CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:

    • The first policy allows the role to be assumed.

    • The second policy provides permissions to start the pipeline.

    Why am I making this change? Adding the AWS::IAM::Role resource enables CloudFormation to create permissions for EventBridge. This resource is added to your CloudFormation stack.

    YAML
    EventRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - events.amazonaws.com Action: sts:AssumeRole Path: / Policies: - PolicyName: eb-pipeline-execution PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: codepipeline:StartPipelineExecution Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
    JSON
    "EventRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "events.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "Policies": [ { "PolicyName": "eb-pipeline-execution", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "codepipeline:StartPipelineExecution", "Resource": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ...
  2. In the template, under Resources, use the AWS::Events::Rule CloudFormation resource to add an EventBridge rule. This event pattern creates an event that monitors push changes to your repository. When EventBridge detects a repository state change, the rule invokes StartPipelineExecution on your target pipeline.

    Why am I making this change? Adding the AWS::Events::Rule resource enables CloudFormation to create the event. This resource is added to your CloudFormation stack.

    YAML
    EventRule: Type: AWS::Events::Rule Properties: EventPattern: source: - aws.codecommit detail-type: - 'CodeCommit Repository State Change' resources: - !Join [ '', [ 'arn:aws:codecommit:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref RepositoryName ] ] detail: event: - referenceCreated - referenceUpdated referenceType: - branch referenceName: - main Targets: - Arn: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] RoleArn: !GetAtt EventRole.Arn Id: codepipeline-AppPipeline
    JSON
    "EventRule": { "Type": "AWS::Events::Rule", "Properties": { "EventPattern": { "source": [ "aws.codecommit" ], "detail-type": [ "CodeCommit Repository State Change" ], "resources": [ { "Fn::Join": [ "", [ "arn:aws:codecommit:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "RepositoryName" } ] ] } ], "detail": { "event": [ "referenceCreated", "referenceUpdated" ], "referenceType": [ "branch" ], "referenceName": [ "main" ] } }, "Targets": [ { "Arn": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] }, "RoleArn": { "Fn::GetAtt": [ "EventRole", "Arn" ] }, "Id": "codepipeline-AppPipeline" } ] } },
  3. (Optional) To configure an input transformer with source overrides for a specific image ID, use the following YAML snippet. The following example configures an override where:

    • The actionName, Source in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.

    • The revisionType, COMMIT_ID in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.

    • The revisionValue, <revisionValue> in this example, is derived from the source event variable.

    • The output variables for BranchName and Value are specified.

    Rule: my-rule Targets: - Id: MyTargetId Arn: pipeline-ARN InputTransformer: sourceRevisions: actionName: Source revisionType: COMMIT_ID revisionValue: <revisionValue> variables: - name: BranchName value: value
  4. Save the updated template to your local computer, and then open the CloudFormation console.

  5. Choose your stack, and then choose Create Change Set for Current Stack.

  6. Upload the template, and then view the changes listed in CloudFormation. These are the changes to be made to the stack. You should see your new resources in the list.

  7. Choose Execute.

To edit your pipeline's PollForSourceChanges parameter
Important

In many cases, the PollForSourceChanges parameter defaults to true when you create a pipeline. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see Valid settings for the PollForSourceChanges parameter.

  • In the template, change PollForSourceChanges to false. If you did not include PollForSourceChanges in your pipeline definition, add it and set it to false.

    Why am I making this change? Changing this parameter to false turns off periodic checks so you can use event-based change detection only.

    YAML
    Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: CodeCommit OutputArtifacts: - Name: SourceOutput Configuration: BranchName: !Ref BranchName RepositoryName: !Ref RepositoryName PollForSourceChanges: false RunOrder: 1
    JSON
    { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "CodeCommit" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "BranchName": { "Ref": "BranchName" }, "RepositoryName": { "Ref": "RepositoryName" }, "PollForSourceChanges": false }, "RunOrder": 1 } ] },
Example

When you create these resources with CloudFormation, your pipeline is triggered when files in your repository are created or updated. Here is the final template snippet:

YAML
Resources: EventRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - events.amazonaws.com Action: sts:AssumeRole Path: / Policies: - PolicyName: eb-pipeline-execution PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: codepipeline:StartPipelineExecution Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] EventRule: Type: AWS::Events::Rule Properties: EventPattern: source: - aws.codecommit detail-type: - 'CodeCommit Repository State Change' resources: - !Join [ '', [ 'arn:aws:codecommit:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref RepositoryName ] ] detail: event: - referenceCreated - referenceUpdated referenceType: - branch referenceName: - main Targets: - Arn: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] RoleArn: !GetAtt EventRole.Arn Id: codepipeline-AppPipeline AppPipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: codecommit-events-pipeline RoleArn: !GetAtt CodePipelineServiceRole.Arn Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: CodeCommit OutputArtifacts: - Name: SourceOutput Configuration: BranchName: !Ref BranchName RepositoryName: !Ref RepositoryName PollForSourceChanges: false RunOrder: 1 ...
JSON
"Resources": { ... "EventRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "events.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "Policies": [ { "PolicyName": "eb-pipeline-execution", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "codepipeline:StartPipelineExecution", "Resource": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] } } ] } } ] } }, "EventRule": { "Type": "AWS::Events::Rule", "Properties": { "EventPattern": { "source": [ "aws.codecommit" ], "detail-type": [ "CodeCommit Repository State Change" ], "resources": [ { "Fn::Join": [ "", [ "arn:aws:codecommit:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "RepositoryName" } ] ] } ], "detail": { "event": [ "referenceCreated", "referenceUpdated" ], "referenceType": [ "branch" ], "referenceName": [ "main" ] } }, "Targets": [ { "Arn": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] }, "RoleArn": { "Fn::GetAtt": [ "EventRole", "Arn" ] }, "Id": "codepipeline-AppPipeline" } ] } }, "AppPipeline": { "Type": "AWS::CodePipeline::Pipeline", "Properties": { "Name": "codecommit-events-pipeline", "RoleArn": { "Fn::GetAtt": [ "CodePipelineServiceRole", "Arn" ] }, "Stages": [ { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "CodeCommit" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "BranchName": { "Ref": "BranchName" }, "RepositoryName": { "Ref": "RepositoryName" }, "PollForSourceChanges": false }, "RunOrder": 1 } ] }, ...

Migrate polling pipelines with an S3 source enabled for events

For a pipeline with an Amazon S3 source, modify the pipeline so that change detection is automated through EventBridge and with a source bucket that is enabled for event notifications. This is the recommended method if you are using the CLI or CloudFormation to migrate your pipeline.

Note

This includes using a bucket that is enabled for event notifications, where you do not need to create a separate CloudTrail trail. If you are using the console, then an event rule and CloudTrail trail are set up for you. For those steps, see Migrate polling pipelines with an S3 source and CloudTrail trail.

Migrate polling pipelines with an S3 source enabled for events (CLI)

Follow these steps to edit a pipeline that is using polling (periodic checks) to use an event in EventBridge instead. If you want to create a pipeline, see Create a pipeline, stages, and actions.

To build an event-driven pipeline with Amazon S3, you edit the PollForSourceChanges parameter of your pipeline and then create the following resources:

  • EventBridge event rule

  • IAM role to allow the EventBridge event to start your pipeline

To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy
  1. Grant permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see Using resource-based policies for Amazon EventBridge.

    1. Use the following sample to create the trust policy to allow EventBridge to assume the service role. Name it trustpolicyforEB.json.

      JSON
      { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
    2. Use the following command to create the Role-for-MyRule role and attach the trust policy.

      Why am I making this change? Adding this trust policy to the role creates permissions for EventBridge.

      aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json
    3. Create the permissions policy JSON, as shown here for the pipeline named MyFirstPipeline. Name the permissions policy permissionspolicyforEB.json.

      JSON
      { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codepipeline:StartPipelineExecution" ], "Resource": [ "arn:aws:codepipeline:us-west-2:111122223333:MyFirstPipeline" ] } ] }
    4. Use the following command to attach the new CodePipeline-Permissions-Policy-for-EB permissions policy to the Role-for-MyRule role you created.

      aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
  2. Call the put-rule command and include the --name, --event-pattern, and --role-arn parameters.

    The following sample command creates a rule named EnabledS3SourceRule.

    aws events put-rule --name "EnabledS3SourceRule" --event-pattern "{\"source\":[\"aws.s3\"],\"detail-type\":[\"Object Created\"],\"detail\":{\"bucket\":{\"name\":[\"amzn-s3-demo-source-bucket\"]}}}" --role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule"
  3. To add CodePipeline as a target, call the put-targets command and include the --rule and --targets parameters.

    The following command specifies that for the rule named EnabledS3SourceRule, the target Id is composed of the number one, indicating that in a list of targets for the rule, this is target 1. The command also specifies an example ARN for the pipeline. The pipeline starts when something changes in the repository.

    aws events put-targets --rule EnabledS3SourceRule --targets Id=codepipeline-AppPipeline,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline
To edit your pipeline's PollForSourceChanges parameter
Important

When you create a pipeline with this method, the PollForSourceChanges parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see Valid settings for the PollForSourceChanges parameter.

  1. Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named MyFirstPipeline, run the following command:

    aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json

    This command returns nothing, but the file you created should appear in the directory where you ran the command.

  2. Open the JSON file in any plain-text editor and edit the source stage by changing the PollForSourceChanges parameter for a bucket named amzn-s3-demo-source-bucket to false, as shown in this example.

    Why am I making this change? Setting this parameter to false turns off periodic checks so you can use event-based change detection only.

    "configuration": { "S3Bucket": "amzn-s3-demo-source-bucket", "PollForSourceChanges": "false", "S3ObjectKey": "index.zip" },
  3. If you are working with the pipeline structure retrieved using the get-pipeline command, you must remove the metadata lines from the JSON file. Otherwise, the update-pipeline command cannot use it. Remove the "metadata": { } lines and the "created", "pipelineARN", and "updated" fields.

    For example, remove the following lines from the structure:

    "metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" },

    Save the file.

  4. To apply your changes, run the update-pipeline command, specifying the pipeline JSON file:

    Important

    Be sure to include file:// before the file name. It is required in this command.

    aws codepipeline update-pipeline --cli-input-json file://pipeline.json

    This command returns the entire structure of the edited pipeline.

    Note

    The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the