S3

The S3 plugin allows AWS S3 to be used as a remote cache for qik.

Installation

The S3 plugin requires additional dependencies. pip install "qik[s3]" to install them.

Note

One can manually install boto3 too.

After this, configure the plugin in qik.toml:

[plugins]
s3 = "qik.s3"

Configuration

After installation, define a custom cache in qik.toml by leveraging the type = "s3" cache provided by the plugin:

[caches.my-remote-cache]
type = "s3"
bucket = "my-cache-bucket"

Ensure that you either have AWS environment variables or an AWS config file.

You can also supply authentication information via context:

ctx = ["aws_access_key_id", "aws_secret_access_key"]

[caches.my-remote-cache]
type = "s3"
bucket = "my-cache-bucket"
aws-access-key-id = "{ctx.aws_access_key_id}"
aws-secret-access-key = "{ctx.aws_secret_access_key}"

You can also configure region-name and aws-session-token.

Usage

After configuring an S3 cache, it can be used like any other cache:

[commands.my-command]
exec = "echo hi"
cache = "my-remote-cache"

When using an S3 cache, all command results and artifacts are stored. We recommend configuring a lifecycle policy on your bucket to delete old entries.

Whenever cached entries are found, they're downloaded to the local cache at ._qik/cache first. This local cache serves as a hot cache if the commands are executed again.