What is AWS CLI pagination?

What is AWS CLI pagination?

January 7, 2021 / Nirav Shah

AWS CLI pagination, you can actually control the number of items included in the output when you run an AWS CLI command.

By default, it’s going to return 1000 items per API call. The AWS CLI uses a page size of 1,000.
But, what if you want to run a command which is going to return more than 1,000 objects?

For example:

If you run AWS S3 API list objects on a bucket, which contains over 2,000 objects. But if it contained 2,500 objects, then the CLI is actually going to make three separate API calls to S3.

But then, it’s going to display the entire output in one go. So, it’s actually making those three
API calls in the background. However, for some situations, you might find that this default page size of 1,000 is actually too high.

So if you had an S3 bucket with well over 1,000 objects, and you were trying to list the individual objects within that S3 bucket, you might see something like a time out or timed out error because the API calls have exceeded the maximum allowed time to fetch the required results.

It’s just taking too long to return the number of results that you have.

So, how do you fix this?

When you run your command. You basically just add this minus (–) is page size option to tell the CLI to request a smaller number of items from each API call. So, the CLI will still retrieve the full list of objects, but it’s just going to perform a larger number of API calls in the background. So, it makes multiple API calls to get your results and then retrieves a smaller number of items with each call, and then it will just return to you the full list of items.

Let us show you a couple of commands for better understanding,

Aws s3api list-object –bucketmy -bucket –page-size -10

This one is this AWS S3 API list objects, and then you specify the bucket name,

In this example, we’re setting the page size to 100 instead of 1,000.

If you had an S3 bucket which contained over 1,000 items, and you wanted to limit the page size to 100, it would just end up making over 10 API calls to get the results and then it will just return all of the results to you in one go.

Aws s3api list-object –bucketmy -bucket –Max-size -100

There’s this other option that you can use with the list objects command, and it’s called max items. This allows you to return fewer items in the CLI output. In the example command, we’ve got AWS S3 API list objects, and then we’re setting the bucket to our bucket, and then we’re setting the max items to 100. In this case, instead of returning all of the items in the bucket, it’s just going to return the first 100.

For more information, you can visit the AWS CLI usage pagination page.