diff --git a/README.md b/README.md index 51318b5..1f790f7 100644 --- a/README.md +++ b/README.md @@ -7,16 +7,16 @@ in S3 buckets using REST API calls or Presigned URLs signed using AWS Signature Version 4. **Features:** -- 🚀 **Simple, intuitive API** following Go idioms -- 🪣 **Complete S3 operations** - Upload, Download, Delete, List, Details -- 🔐 **AWS Signature Version 4** signing -- 🌐 **Custom endpoint support** (MinIO, DigitalOcean Spaces, etc.) -- 📋 **Simple List API** with pagination, prefix filtering, and delimiter grouping -- 🔄 **Iterator-based ListAll** for memory-efficient large bucket iteration (Go 1.23+) -- 🔗 **Presigned URL generation** for secure browser uploads/downloads -- 🪪 **IAM credential support** for EC2 instances -- ✅ **Comprehensive test coverage** -- 🎯 **Zero dependencies** - uses only Go standard library +- **Simple, intuitive API** following Go idioms +- **Complete S3 operations** - Upload, Download, Delete, List, Details +- **AWS Signature Version 4** signing +- **Custom endpoint support** (MinIO, DigitalOcean Spaces, etc.) +- **Simple List API** with pagination, prefix filtering, and delimiter grouping +- **Iterator-based ListAll** for memory-efficient large bucket iteration (Go 1.23+) +- **Presigned URL generation** for secure browser uploads/downloads +- **IAM credential support** for EC2 instances +- **Comprehensive test coverage** +- **Zero dependencies** - uses only Go standard library ## Install @@ -39,11 +39,11 @@ func main() { // Initialize S3 client s3 := simples3.New("us-east-1", "your-access-key", "your-secret-key") - // Use MinIO or other S3-compatible services - s3.SetEndpoint("https://s3.amazonaws.com") - // Upload a file - file, _ := os.Open("my-file.txt") + file, err := os.Open("my-file.txt") + if err != nil { + log.Fatal(err) + } defer file.Close() resp, err := s3.FileUpload(simples3.UploadInput{ @@ -99,6 +99,9 @@ defer file.Close() // Read the content data, err := io.ReadAll(file) +if err != nil { + log.Fatal(err) +} ``` #### Delete Files @@ -159,12 +162,12 @@ if err != nil { // Process objects for _, obj := range result.Objects { - fmt.Printf("📄 %s (%d bytes)\n", obj.Key, obj.Size) + fmt.Printf("%s (%d bytes)\n", obj.Key, obj.Size) } // Process "directories" (common prefixes) for _, prefix := range result.CommonPrefixes { - fmt.Printf("📁 %s/\n", prefix) + fmt.Printf("%s/\n", prefix) } ``` @@ -180,7 +183,7 @@ seq, finish := s3.ListAll(simples3.ListInput{ }) for obj := range seq { - fmt.Printf("📄 %s (%d bytes)\n", obj.Key, obj.Size) + fmt.Printf("%s (%d bytes)\n", obj.Key, obj.Size) } // Check for any errors that occurred during iteration @@ -231,9 +234,9 @@ func main() { log.Fatal(err) } - fmt.Printf("Total objects: %d\\n", len(allObjects)) + fmt.Printf("Total objects: %d\n", len(allObjects)) for _, obj := range allObjects { - fmt.Printf("- %s (%d bytes)\\n", obj.Key, obj.Size) + fmt.Printf("- %s (%d bytes)\n", obj.Key, obj.Size) } } ``` @@ -344,13 +347,7 @@ export AWS_S3_BUCKET="testbucket" ## Contributing -We welcome contributions! Please: - -1. Fork the repository -2. Create a feature branch -3. Add tests for new functionality -4. Ensure all tests pass: `just test-local` -5. Submit a pull request +Contributions welcome! Check [ROADMAP.md](ROADMAP.md) for planned features. Please add tests and ensure `just test-local` passes before submitting PRs. ## Author diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..8e2b136 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,195 @@ +# Project Roadmap + +## Current: v0.x (Pre-Stable) + +### Completed Features +- File Upload (PUT via FilePut, POST multipart via FileUpload) +- File Download (GetObject) +- File Delete (DeleteObject) +- File Details (HeadObject) +- List Objects (ListObjectsV2) with pagination +- ListAll iterator (Go 1.23+) for memory-efficient iteration +- Presigned URLs (GET/PUT with expiry) +- Upload policies for browser POST uploads +- IAM Role Support (EC2 instances with IMDSv2) +- IAM token auto-renewal +- Custom Metadata +- Custom Endpoints (MinIO, DigitalOcean Spaces, etc.) +- Content-Disposition and ACL support +- Zero dependencies (stdlib only) + +## v0.10.2 - Code Organization (Refactor) +**Scope**: Internal restructuring, no API changes +**Size**: Medium (refactor ~1064 LOC) + +Split simples3.go into logical modules: +- [ ] `simples3.go` - Core (S3 struct, New(), config methods: SetEndpoint/SetToken/SetClient) +- [ ] `iam.go` - IAM (fetchIMDSToken, fetchIAMData, NewUsingIAM, renewIAMToken, SetIAMData) +- [ ] `object.go` - Object ops (FileUpload, FilePut, FileDownload, FileDelete, FileDetails) +- [ ] `list.go` - List ops (List, ListAll) +- [ ] `helpers.go` - Utils (encodePath, detectFileSize, getFirstString, getURL) +- [ ] Ensure all tests pass after refactor + +**Why**: 1064-line file hard to navigate. Clean foundation before adding features. + +--- + +## v0.11.0 - Bucket Operations +**Scope**: Essential bucket management for CLI +**Size**: Small (3 APIs, ~200 LOC) + +- [ ] ListBuckets +- [ ] CreateBucket (with region support) +- [ ] DeleteBucket + +**Why**: These 3 operations unblock CLI development. Minimal but complete. + +--- + +## v0.12.0 - Object Operations (Copy & Batch Delete) +**Scope**: Object manipulation for CLI mv/sync +**Size**: Small (2 APIs, ~150 LOC) + +- [ ] CopyObject (within/across buckets) +- [ ] DeleteObjects (batch delete up to 1000 objects) + +**Why**: CopyObject enables `mv` command. Batch delete improves performance. + +--- + +## v0.13.0 - Multipart Upload +**Scope**: Large file support +**Size**: Large (5 APIs, ~400 LOC) + +- [ ] InitiateMultipartUpload +- [ ] UploadPart (with retry logic) +- [ ] CompleteMultipartUpload +- [ ] AbortMultipartUpload +- [ ] ListParts + +**Why**: Critical for large files. Enables resumable uploads. + +--- + +## v0.14.0 - Server-Side Encryption +**Scope**: Security basics +**Size**: Small (~150 LOC) + +- [ ] SSE-S3 support (AES256) +- [ ] SSE-KMS support (key ARN) +- [ ] Encryption headers in Put/Upload/Copy + +**Why**: Security is table stakes. Simple to add to existing operations. + +--- + +## v0.15.0 - Object Tagging +**Scope**: Metadata management +**Size**: Small (4 APIs, ~200 LOC) + +- [ ] PutObjectTagging +- [ ] GetObjectTagging +- [ ] DeleteObjectTagging +- [ ] Tagging support in Put/Upload/Copy + +**Why**: Common requirement for organization/billing. Simple XML operations. + +--- + +## v0.16.0 - Object Versioning +**Scope**: Version control +**Size**: Medium (5 APIs, ~300 LOC) + +- [ ] PutBucketVersioning +- [ ] GetBucketVersioning +- [ ] ListObjectVersions +- [ ] GetObjectVersion +- [ ] DeleteObjectVersion (with version ID) + +**Why**: Important for data safety. More complex due to version handling. + +--- + +## v0.17.0 - ACLs & Lifecycle +**Scope**: Advanced management +**Size**: Medium (9 APIs, ~500 LOC) + +ACLs: +- [ ] PutObjectAcl +- [ ] GetObjectAcl +- [ ] PutBucketAcl +- [ ] GetBucketAcl + +Lifecycle: +- [ ] PutBucketLifecycle +- [ ] GetBucketLifecycle +- [ ] DeleteBucketLifecycle + +**Why**: Grouped advanced features. Completes library API surface. + +--- + +## v0.18.0 - CLI Complete +**Scope**: Full-featured CLI with all library features +**Size**: Large (~2000 LOC) + +CLI (`cmd/simples3/`): +- [ ] Project structure (cobra or custom parser) +- [ ] S3 URI parsing (`s3://bucket/key`) +- [ ] Config (env vars: AWS_*, flags: --region/--profile) +- [ ] AWS credentials file support (~/.aws/credentials) +- [ ] Basic commands: + - [ ] ls (list buckets/objects) + - [ ] cp (upload/download, with multipart for large files) + - [ ] rm (delete single/batch) + - [ ] mb (make bucket) + - [ ] rb (remove bucket) + - [ ] mv (move/rename using copy+delete) + - [ ] presign (generate URL) + - [ ] sync (local ↔ S3 with diff algorithm) +- [ ] Flags: + - [ ] --recursive (cp/rm) + - [ ] --sse/--sse-kms-key-id (encryption) + - [ ] --tags (tagging) + - [ ] --version-id (versioning) + - [ ] --acl (canned ACLs) + - [ ] --delete (sync) + - [ ] --exclude/--include (patterns) + - [ ] --json (output mode) + - [ ] --dry-run +- [ ] Features: + - [ ] Progress indicators + - [ ] Parallel transfers + - [ ] Better error messages +- [ ] Subcommands: + - [ ] acl get/set + - [ ] lifecycle get/set/delete + - [ ] tags get/set/delete + +**Why**: Library complete. Build full-featured CLI supporting all operations. + +--- + +## v1.0.0 - Stable Release +**Focus**: Production-ready stability +**Size**: Documentation, tests, polish + +Library: +- [ ] All tests passing with 80%+ coverage +- [ ] API documentation complete +- [ ] Performance benchmarks +- [ ] Security audit of signing/crypto code + +CLI: +- [ ] CLI tests with real/mocked S3 +- [ ] CLI documentation (man pages, --help) +- [ ] Installation guides (brew, apt, etc.) +- [ ] Shell completions (bash, zsh, fish) + +Release: +- [ ] CHANGELOG.md complete +- [ ] Migration guide from v0.x +- [ ] Examples for all features +- [ ] API stability guarantee + +**Why**: v1.0 = stability commitment. Library + CLI production-ready. \ No newline at end of file