S3 Directory Loader
S3 Directory Node v V2
The S3 Directory Loader enables you to load multiple files from an entire Amazon S3 bucket directory. It pulls all valid files located within a specified bucket prefix.
Inputs
- Bucket: The name of your S3 bucket.
- Region: AWS region where the bucket is located.
- Connect Credential: AWS credentials to access the bucket.
- Prefix (optional): The directory prefix to load files from.
- Text Splitter (optional): A text splitter to process the extracted content.
- Additional Metadata (optional): JSON object with extra metadata.
- Omit Metadata Keys (optional): Keys to omit from metadata.
Outputs
- Document: Array of document objects containing metadata and pageContent
- Text: Concatenated string from pageContent of documents
Features
- Scalable directory processing
- Built-in unstructured file processing capabilities
- Supports fetching numerous documents simultaneously
- Automatic pagination for large directories
- Region selection