"Verify the multipart upload implementation works correctly by running integration tests against MinIO and validating file integrity, parallel uploads, and error handling."
Core APIs:
- Add InitiateMultipartUpload for starting multipart uploads
- Add UploadPart for uploading individual parts (5MB-5GB)
- Add CompleteMultipartUpload to finalize uploads
- Add AbortMultipartUpload to cancel and cleanup uploads
- Add ListParts to query uploaded parts
High-Level Features:
- Add FileUploadMultipart API with automatic chunking
- Add uploadPartWithRetry with exponential backoff (3 retries default)
- Add parallel upload worker pool with configurable concurrency
- Add progress callbacks for real-time upload tracking
- Add GeneratePresignedUploadPartURL for browser-based uploads
Advanced Features:
- Automatic file chunking with configurable part sizes (5MB-5GB)
- Parallel upload worker pool with configurable concurrency
- Retry logic with exponential backoff (configurable retries)
- Real-time progress tracking with upload speed calculation
- Automatic cleanup on errors via AbortMultipartUpload
- Support for resumable uploads via ListParts
- Comprehensive input validation for all parameters
- File integrity verification across all upload modes
Testing:
- Add comprehensive integration tests (500+ lines)
- Test sequential uploads (15MB, 3 parts, ~45 MB/s)
- Test parallel uploads (20MB, 4 parts, concurrency=3)
- Test small files (3MB, single part)
- Validate abort and ListParts functionality
- Test input validation and error cases
- Verify file integrity with hash comparison
- All tests passing against MinIO
Documentation:
- Update README with multipart upload examples
- Add high-level and low-level API usage
- Document browser-based multipart uploads
- Mark multipart upload implementation as complete in ROADMAP
- Update gitignore to exclude *.bin and test-output.log
Verified against MinIO with complete test coverage.
- Add List method with ListInput and ListResponse structs following simples3 patterns
- Implement pagination, prefix filtering, and delimiter grouping
- Add comprehensive test suite for List functionality
- Update tests to use environment variables instead of hardcoded values
- Fix deprecated io/ioutil usage throughout codebase
- Add comprehensive README with API documentation and examples
- Add Nix development environment with flake.nix
- Add Justfile for modern command runner
- Add Docker Compose setup for local MinIO testing
- Update GitHub Actions to use Nix and Justfile for CI consistency
- Fix test naming and setup flow issues
New API:
s3.List(ListInput{
Bucket: "my-bucket",
Prefix: "documents/",
MaxKeys: 100,
})
All tests pass with new simplified API design.
Rohan Verma
Liut