wp-s3-action

Woodpecker CI plugin to perform S3 actions.

steps:
  - name: sync
    image: quay.io/thegeeklab/wp-s3-action
    settings:
      action:
        - upload
      access_key: randomstring
      secret_key: random-secret
      region: us-east-1
      bucket: my-bucket.s3-website-us-east-1.amazonaws.com
      source: folder/to/archive
      target: /target/location

access_key required string

S3 access key.

Default: none

action required list

S3 action to execute. Actions are low-level operations that can be combined to compose higher-level workflows such as a full sync. Supported actions: upload, download, delete, redirect, invalidate-cloudfront.

• upload: Uploads files from the local source directory to the S3 bucket target path. When combined with the upload_delete setting, remote files that no longer exist locally are removed.
• download: Downloads files from the S3 bucket target path to the local source directory. Requires an explicit non-empty target so the entire bucket cannot be pulled by accident.
• delete: Deletes all files from the S3 bucket target path. Requires an explicit non-empty target so the entire bucket cannot be wiped by accident.
• redirect: Creates redirect objects from the redirects setting under the target path. Requires a target server that supports the S3 website redirect feature.
• invalidate-cloudfront: Invalidates the configured CloudFront distribution path. This action is AWS-specific and does not work with S3-compatible endpoints such as Garage.

Default: none

archive optional bool

Enable archive mode for the upload and download actions. When enabled, the source directory is serialized into a single tar (optionally compressed) object at the target key instead of one object per file. Intended for CI caches with a large number of small files where per-object transfers are slow and costly.

In archive mode target is interpreted as the full object key rather than a key prefix, and it must be non-empty for both upload and download.

Default: none

archive_compression optional string

Compression algorithm used in archive mode. Supported values are gzip and none. gzip is the default.

Default: gzip

bucket required string

Name of the bucket.

Default: none

checksum_calculation optional string

Checksum calculation mode. Supported values are required and supported. For third-party S3 implementations, required must most likely be used.

Default: required

cloudfront_distribution optional string

ID of cloudfront distribution to invalidate.

CloudFront is an AWS service and is not supported by S3-compatible endpoints such as Garage.

Default: none

dry_run optional bool

Dry run disables API calls. When enabled, the plugin logs the actions it would take without actually uploading, downloading, deleting, or redirecting.

Default: none

endpoint optional string

Endpoint for the s3 connection.

Default: none

insecure_skip_verify optional bool

Skip SSL verification.

Default: none

log_level optional string

Plugin log level.

Default: info

max_concurrency optional integer

Customize number concurrent files to process.

Default: 100

path_style optional bool

Enable path style for bucket paths.

Default: none

redirects optional dict

Map of source paths to redirect destinations. Each key is created as an S3 object under the configured target path with the x-amz-website-redirect-location header set to the corresponding value.

The target server must support the S3 website redirect feature, so check your provider’s documentation for static website hosting or website redirect support before using the redirect action.

Default: none

region optional string

S3 region.

Default: us-east-1

secret_key required string

S3 secret key.

Default: none

source optional string

Local working directory. Files are read from here during upload and written here during download.

Default: .

target optional string

S3 path prefix for the bucket operations. Used by all actions to scope the S3 key namespace (a leading / is stripped). Empty means the bucket root. The delete and download actions require an explicit non-empty value.

In archive mode target is the full object key and is rendered as a Go template against the Woodpecker pipeline metadata before the upload or download runs. Templating is not applied to the other actions.

Default: none

upload_acl optional dict

Access control list.

Default: none

upload_allow_empty_source optional bool

Allow empty source directory. By default this setting will prevent deleting all files from the target if upload_delete: true is set and the source directory is empty.

Default: none

upload_cache_control optional dict

Cache-control settings for uploads.

Default: none

upload_content_encoding optional dict

Content-encoding settings for uploads.

Default: none

upload_content_type optional dict

Content-type settings for uploads.

Default: none

upload_delete optional bool

Delete remote files that are not present in the local source directory during upload.

Default: none

upload_metadata optional dict

Additional metadata for uploads.

Default: none

Actions are executed in the order they are listed. To mirror the local source directory with the S3 bucket target path, combine upload with upload_delete to remove remote files that no longer exist locally. Add redirect to create redirect objects and invalidate-cloudfront to invalidate a CloudFront distribution after the upload. redirect requires the server to support website redirects, and invalidate-cloudfront only works with AWS CloudFront. Check your provider’s documentation before using either action.

steps:
  - name: sync
    image: quay.io/thegeeklab/wp-s3-action
    settings:
      action:
        - upload
        - redirect
        - invalidate-cloudfront
      upload_delete: true
      access_key: randomstring
      secret_key: random-secret
      region: us-east-1
      bucket: my-bucket.s3-website-us-east-1.amazonaws.com
      source: folder/to/archive
      target: /target/location
      redirects:
        old/path: https://example.com/new/path
      cloudfront_distribution: E1ABCDEFGHIJKL

Customize the headers and user metadata attached to uploaded objects with upload_acl, upload_content_type, upload_content_encoding, upload_cache_control, and upload_metadata:

steps:
  - name: sync
    image: quay.io/thegeeklab/wp-s3-action
    settings:
      action:
        - upload
      access_key: randomstring
      secret_key: random-secret
      region: us-east-1
      bucket: my-bucket.s3-website-us-east-1.amazonaws.com
      source: folder/to/archive
      target: /target/location
      upload_acl:
        "public/*": public-read
        "private/*": private
      upload_content_type:
        ".svg": image/svg+xml
      upload_content_encoding:
        ".js": gzip
        ".css": gzip
      upload_cache_control: "public, max-age: 31536000"
      upload_metadata:
        "*.html":
          Author: "release-team"

Each map parameter accepts either a map keyed by a pattern that selects a subset of files, or a plain string that applies the same value to every uploaded file:

upload_content_encoding: gzip

The map and string forms are mutually exclusive within a single setting.

• For the upload_acl parameter the key must be a glob. Files without a matching rule default to private.
• For the upload_content_type parameter, the key must be a file extension (including the leading dot). To apply a configuration to files without an extension, set the key to an empty string "". Files without a matching rule get their content type detected automatically.
• For the upload_content_encoding parameter, the key must be a file extension (including the leading dot). To apply a configuration to files without an extension, set the key to an empty string "". Files without a matching rule are uploaded without a Content-Encoding header.
• For the upload_cache_control parameter, the key must be a glob. Files without a matching rule are uploaded without a Cache-Control header.
• For the upload_metadata parameter, the key must be a glob and the value is a map of user metadata header names to values. S3 stores user metadata under the x-amz-meta- prefix, so it is not interchangeable with the standard Cache-Control, Content-Type, or Content-Encoding headers — use upload_cache_control, upload_content_type, or upload_content_encoding for those instead. Only the first matching glob is applied per file.

Archive mode packs the source directory into a single compressed tar object on S3 instead of transferring each file separately. It is the recommended approach for any directory that contains many small files, e.g. Go build cache (GOCACHE), Go module cache (GOMODCACHE), node_modules, or a Python virtual environment. Archive mode only applies to the upload and download actions.

steps:
  - name: restore-cache
    image: quay.io/thegeeklab/wp-s3-action
    settings:
      action:
        - download
      archive: true
      access_key: randomstring
      secret_key: random-secret
      region: us-east-1
      bucket: my-cache-bucket
      source: .cache/go-build
      target: cache/go-build.tar.gz

  - name: build
    image: docker.io/library/golang:1.27
    commands:
      - export GOCACHE="$CI_WORKSPACE/.cache/go-build"
      - go build ./...

  - name: save-cache
    image: quay.io/thegeeklab/wp-s3-action
    settings:
      action:
        - upload
      archive: true
      access_key: randomstring
      secret_key: random-secret
      region: us-east-1
      bucket: my-cache-bucket
      source: .cache/go-build
      target: cache/go-build.tar.gz

To cache GOMODCACHE alongside GOCACHE, run a second restore/save pair that points source at the module cache directory and uses a different target key (for example cache/gomod.tar.gz) so the two archives do not overwrite each other.

In archive mode target is the full object key rather than a key prefix, and it must be set for both upload and download. Restore and save are each a single GET or PUT of the compressed archive. If the object does not yet exist, the download step logs the miss and exits successfully, so the first build against a fresh cache key simply runs with a cold cache. Restored files keep their original permissions and modification times, which matters for caches such as GOCACHE that rely on them to detect reuse.

Keep caches for different inputs apart with distinct target keys, for example one per Go version, branch, or pipeline. The key is rendered as a Go template against the wp-plugin-go Metadata, so you can derive it automatically:

target: cache/{{ .Repository.Branch }}-{{ .Pipeline.Number }}.tar.gz

Common fields include .Repository.Name, .Repository.Owner, .Repository.Branch, .Pipeline.Number, .Pipeline.Event, and .Curr.Sha. The template also exposes the sprig function map plus sha256file, which returns the SHA-256 digest of one or more files. Key the cache on a lockfile instead of a pipeline identifier to invalidate it automatically when dependencies change:

target: cache/go-build-{{ sha256file "go.sum" }}.tar.gz

Pass several paths to hash them in order, for example sha256file "go.mod" "go.sum". sha256file reads each path relative to the workspace and fails the step if a path does not exist, so only reference files that are checked into the repository or generated by an earlier step. Templating applies only in archive mode.

The default compression is gzip. Set archive_compression: none to store an uncompressed tar instead. The archive is streamed directly between the local filesystem and S3.

To sync to a self-hosted S3-compatible server, point endpoint at the server and enable path_style. Path-style addressing works out of the box, while virtual-hosted-style addressing requires additional DNS configuration.

steps:
  - name: sync
    image: quay.io/thegeeklab/wp-s3-action
    settings:
      action:
        - upload
      endpoint: https://s3.example.com
      access_key: randomstring
      secret_key: random-secret
      bucket: my-bucket
      source: folder/to/archive
      target: /target/location
      path_style: true

The plugin logs each file operation with a status marker (+ for additions, - for deletions, ~ for modifications, > for redirects) and prints a summary line at the end of each action showing the total counts (for example, upload: 3 to add, 1 to destroy). When all counters are zero, the summary reports no changes.

ANSI color codes are enabled by default. Set the NO_COLOR environment variable to any non-empty value to disable colored output.

Build the binary with the following command:

make build

Build the container image with the following command:

docker build --file Containerfile.multiarch --tag thegeeklab/wp-s3-action .

docker run --rm \
  -e PLUGIN_ACTION=upload \
  -e PLUGIN_BUCKET=my_bucket \
  -e AWS_ACCESS_KEY_ID=randomstring \
  -e AWS_SECRET_ACCESS_KEY=random-secret \
  -v $(pwd):/build:z \
  -w /build \
  thegeeklab/wp-s3-action