Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(js): list api subpathStrategy parameter #7794

Merged
merged 13 commits into from
Jul 9, 2024
Merged
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 88 additions & 27 deletions src/pages/[platform]/build-a-backend/storage/list-files/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,23 +29,17 @@ export function getStaticProps(context) {
};
}

You can list files without having to download all the files. You can do this by using the listFile API from the Amplify Library for Storage. You can also get properties individually for a file using the getProperties API.
You can list files without having to download all the files. You can do this by using the `list` API from the Amplify Library for Storage. You can also get properties individually for a file using the getProperties API.

## List Files

<InlineFilter filters={["react", "angular", "javascript", "vue", "nextjs", "react-native"]}>
```javascript
import { list } from 'aws-amplify/storage';

try {
const result = await list({
path: 'photos/',
// Alternatively, path: ({identityId}) => `album/{identityId}/photos/`
});
} catch (error) {
console.log(error);
}
```
try { const result = await list({ path: 'photos/', // Alternatively, path: ({identityId}) => `album/{identityId}/photos/` }); } catch (error) { console.log(error); }

````
haverchuck marked this conversation as resolved.
Show resolved Hide resolved

Note the trailing slash `/` - if you had requested `list({ path : 'photos' })` it would also match against files like `photos123.jpg` alongside `photos/123.jpg`.

Expand All @@ -63,9 +57,18 @@ The format of the response will look similar to the below example:
// ...
],
}
```
{/* in other files we're referring to paths instead of folders, can we be consistent on terminology? */}
Manually created folders will show up as files with a `size` of 0, but you can also match keys against a regex like `file.key.match(/\.[0-9a-z]+$/i)` to distinguish files from folders. Since "folders" are a virtual concept in Amazon S3, any file may declare any depth of folder just by having a `/` in its name. If you need to list all the folders, you'll have to parse them accordingly to get an authoritative list of files and folders:
````

{/* in other files we're referring to paths instead of folders, can we be consistent on terminology? */} Manually created folders will show up as files with a `size` of 0, but you can also match keys against a regex like `file.key.match(/\.[0-9a-z]+$/i)` to distinguish files from folders. Since "folders" are a virtual concept in Amazon S3, any file may declare any depth of folder just by having a `/` in its name.

To access the contents and subpaths of a "folder", you have two options:

1. Request the entire path and parse the contents.
2. Use the subpathStrategy option to retrieve only the files within the specified path (i.e. exclude files under subpaths).

### Get all nested files within a path

This retrieves all files and folders under a given path. You may need to parse the result to get only the files within the specified path.

```js
function processStorageList(response) {
Expand Down Expand Up @@ -109,10 +112,66 @@ function processStorageList(response) {

This places each item's data inside a special `__data` key.

### Excluding subpaths

In addition to using the `list` API to get all the contents of a path, you can also use it to get only the files within a path while excluding files under subpaths.

For example, given the following keys in your `path` you may want to return only the jpg object, and not the "vacation" subpath and its contents:

```
photos/photo1.jpg
photos/vacation/
```

This can be accomplished with the `subpathStrategy` option:

```ts title="src/main.ts"
import { list } from "aws-amplify/storage";
const result = await list({
path: "photos/",
options:{
subpathStrategy: { strategy:'exclude' }
}
});
```
ashika112 marked this conversation as resolved.
Show resolved Hide resolved

The response will include only the objects within the `photos/` path and will also communicate any excluded subpaths:

```js
{
haverchuck marked this conversation as resolved.
Show resolved Hide resolved
excludedSubpaths: [
'photos/vacation/'
],
items: [
{
path: "photos/photo1.jpg",
eTag: "30074401292215403a42b0739f3b5262",
lastModified: "Thu Oct 08 2020 23:59:31 GMT+0800 (Singapore Standard Time)",
size: 138256
},
]
}
```

The default delimiter character is '/', but this can be changed by supplying a custom delimiter:

```ts title="src/main.ts"
const result = await list({
// Path uses '-' character to organize files rather than '/'
path: 'photos-',
options: {
subpathStrategy: {
strategy: 'exclude',
delimiter: '-'
}
}
});
```

haverchuck marked this conversation as resolved.
Show resolved Hide resolved
### More `list` options

Option | Type | Description |
| -- | -- | ----------- |
| Option | Type | Description |
| --- | --- | --- |
| listAll | boolean | Set to true to list all files within the specified `path` |
| pageSize | number | Sets the maximum number of files to be return. The range is 0 - 1000 |
| nextToken | string | Indicates whether the list is being continued on this bucket with a token |
Expand Down Expand Up @@ -242,7 +301,7 @@ You can list all of the objects uploaded under a given path by setting the `page
```swift
let options = StorageListRequest.Options(pageSize: 1000)
let listResult = try await Amplify.Storage.list(
path: .fromString("public/example/path"),
path: .fromString("public/example/path"),
options: options
)
listResult.items.forEach { item in
Expand All @@ -258,7 +317,7 @@ listResult.items.forEach { item in
let sink = Amplify.Publisher.create {
let options = StorageListRequest.Options(pageSize: 1000)
try await Amplify.Storage.list(
path: .fromString("public/example/path"),
path: .fromString("public/example/path"),
options: options
)
}.sink {
Expand All @@ -280,18 +339,19 @@ receiveValue: { listResult in

### More `list` options

Option | Type | Description |
| -- | -- | ----------- |
| Option | Type | Description |
| --- | --- | --- |
| pageSize | UInt | Number between 1 and 1,000 that indicates the limit of how many entries to fetch when retrieving file lists from the server |
| nextToken | String | String indicating the page offset at which to resume a listing. |

</InlineFilter>

<InlineFilter filters={["flutter"]}>

This will list all files located under path `album` that:
* have `private` access level
* are in the root of `album/` (the result doesn't include files under any sub path)

- have `private` access level
- are in the root of `album/` (the result doesn't include files under any sub path)

```dart
Future<void> listAlbum() async {
Expand Down Expand Up @@ -347,8 +407,8 @@ Future<void> listAllUnderPublicPath() async {

### More `list` options

Option | Type | Description |
| -- | -- | ----------- |
| Option | Type | Description |
| --- | --- | --- |
| excludeSubPaths | boolean | Whether to exclude objects under the sub paths of the path to list. Defaults to false. |
| delimiter | String | The delimiter to use when evaluating sub paths. If excludeSubPaths is false, this value has no impact on behavior. |

Expand All @@ -365,14 +425,15 @@ import { getProperties } from 'aws-amplify/storage';

try {
const result = await getProperties({
path: "album/2024/1.jpg",
path: 'album/2024/1.jpg'
// Alternatively, path: ({ identityId }) => `album/{identityId}/1.jpg`
});
console.log('File Properties ', result);
} catch (error) {
console.log('Error ', error);
}
```

The properties and metadata will look similar to the below example

```js
Expand All @@ -388,8 +449,8 @@ The properties and metadata will look similar to the below example

### More `getProperties` options

Option | Type | Description |
| -- | -- | ----------- |
| Option | Type | Description |
| --------------------- | ------- | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| useAccelerateEndpoint | boolean | Whether to use accelerate endpoint. | [Transfer Acceleration](/[platform]/build-a-backend/storage/extend-s3-resources/#example---enable-transfer-acceleration) |

<Callout>
Expand Down