eiac/simples3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b43fd913edcdf04feac26ed49a082bc39ef7f07a
simples3/ROADMAP.md
T
Rohan Verma b43fd913ed feat: add ACL and Lifecycle support 
Implements ACL and Lifecycle management capabilities (v0.16.0) including:
- Bucket APIs: PutBucketAcl, GetBucketAcl, PutBucketLifecycle, GetBucketLifecycle, DeleteBucketLifecycle
- Object APIs: PutObjectAcl, GetObjectAcl
- Integration: Support for Canned ACLs (private, public-read, etc.) and Full Access Control Policies
- Lifecycle Configuration: Rules, Filters, Expiration, Transitions

Verified integration with MinIO:
- Note: MinIO returns 501 Not Implemented for Canned ACLs via headers, so tests for those are skipped while Custom ACLs (XML) are fully verified.

Includes comprehensive integration tests covering all operations.
Updated ROADMAP.md to mark v0.16.0 as completed.
2025-11-27 17:04:27 +05:30

7.3 KiB
Raw Blame History

Project Roadmap

Current: v0.x (Pre-Stable)

Completed Features

  • File Upload (PUT via FilePut, POST multipart via FileUpload)
  • File Download (GetObject)
  • File Delete (DeleteObject)
  • File Details (HeadObject)
  • List Objects (ListObjectsV2) with pagination
  • ListAll iterator (Go 1.23+) for memory-efficient iteration
  • Presigned URLs (GET/PUT with expiry)
  • Upload policies for browser POST uploads
  • IAM Role Support (EC2 instances with IMDSv2)
  • IAM token auto-renewal
  • Custom Metadata
  • Custom Endpoints (MinIO, DigitalOcean Spaces, etc.)
  • Content-Disposition and ACL support
  • Zero dependencies (stdlib only)

v0.10.2 - Code Organization (Refactor) ✅ COMPLETED

Scope: Internal restructuring, no API changes Size: Medium (refactor ~1064 LOC) Released: Nov 26, 2025 (v0.10.2)

Split simples3.go into logical modules:

  • simples3.go - Core (S3 struct, New(), config methods: SetEndpoint/SetToken/SetClient)
  • iam.go - IAM (fetchIMDSToken, fetchIAMData, NewUsingIAM, renewIAMToken, SetIAMData)
  • object.go - Object ops (FileUpload, FilePut, FileDownload, FileDelete, FileDetails)
  • list.go - List ops (List, ListAll)
  • helpers.go - Utils (encodePath, detectFileSize, getFirstString, getURL)
  • Ensure all tests pass after refactor

Why: 1064-line file hard to navigate. Clean foundation before adding features.

v0.11.0 - Bucket & Object Operations ✅ COMPLETED

Scope: Essential bucket management + object manipulation for CLI Size: Medium (5 APIs, ~400 LOC) Released: Nov 26, 2025

Bucket Operations:

  • ListBuckets
  • CreateBucket (with region support)
  • DeleteBucket

Object Operations:

  • CopyObject (within/across buckets)
  • DeleteObjects (batch delete up to 1000 objects)

Why: These operations unblock CLI development (mv, sync, batch operations). Combined release for efficiency.

Note: DeleteObjects has a known compatibility issue with MinIO that requires further investigation. The implementation follows AWS S3 specification correctly.

v0.12.0 - Multipart Upload ✅ COMPLETED

Scope: Large file support with parallel uploads, progress tracking, and presigned URLs Size: Large (10 APIs/features, ~800 LOC) Released: Nov 27, 2025

Core APIs:

  • InitiateMultipartUpload
  • UploadPart
  • CompleteMultipartUpload
  • AbortMultipartUpload
  • ListParts

High-Level Features:

  • FileUploadMultipart (automatic chunking)
  • uploadPartWithRetry (exponential backoff retry logic)
  • uploadPartsParallel (concurrent part uploads with worker pool)
  • Progress callbacks (real-time upload progress)
  • GeneratePresignedUploadPartURL (browser-based multipart uploads)

Advanced Features Implemented:

  • Automatic file chunking with configurable part sizes (5MB-5GB)
  • Parallel upload worker pool (configurable concurrency)
  • Retry logic with exponential backoff (3 retries default, configurable)
  • Real-time progress tracking with upload speed calculation
  • Automatic cleanup on errors (AbortMultipartUpload)
  • Support for resumable uploads via ListParts
  • Comprehensive input validation (part size, part numbers, etc.)
  • Integration tests with MinIO (500+ LOC)
  • File integrity verification across all upload modes

Why: Critical for large files (>100MB). Enables resumable uploads, parallel transfers, and better performance.

v0.13.0 - Server-Side Encryption

Scope: Security basics Size: Small (~150 LOC)

  • SSE-S3 support (AES256)
  • SSE-KMS support (key ARN)
  • Encryption headers in Put/Upload/Copy

Why: Security is table stakes. Simple to add to existing operations.

v0.14.0 - Object Tagging ✅ COMPLETED

Scope: Metadata management Size: Small (4 APIs, ~250 LOC) Released: Nov 27, 2025

Core APIs:

  • PutObjectTagging
  • GetObjectTagging
  • DeleteObjectTagging

Tagging Support in Operations:

  • FilePut (PUT upload)
  • FileUpload (POST upload) - AWS S3 only, MinIO limited
  • CopyObject - AWS S3 supported, MinIO/R2 handled automatically

Features Implemented:

  • Put/get/delete tags on existing objects (up to 10 tags per object)
  • Set tags during object upload operations
  • Tags field in UploadInput and CopyObjectInput
  • URL-encoded tag header formatting
  • Input validation (max 10 tags, required fields)
  • Integration tests with MinIO (and workaround for signature issues)
  • Comprehensive documentation in README

Why: Common requirement for organization/billing. Simple XML operations.

Note: Some MinIO limitations exist for tagging via POST uploads. CopyObject tagging is handled via a 2-step process (copy then tag) to support both AWS and MinIO/R2.

v0.15.0 - Object Versioning ✅ COMPLETED

Scope: Version control Size: Medium (5 APIs, ~300 LOC) Released: Nov 27, 2025

  • PutBucketVersioning
  • GetBucketVersioning
  • ListObjectVersions
  • GetObjectVersion (via FileDownload)
  • DeleteObjectVersion (via FileDelete)

Why: Important for data safety. More complex due to version handling.

v0.16.0 - ACLs & Lifecycle ✅ COMPLETED

Scope: Advanced management Size: Medium (9 APIs, ~500 LOC) Released: Nov 27, 2025

ACLs:

  • PutObjectAcl
  • GetObjectAcl
  • PutBucketAcl
  • GetBucketAcl

Lifecycle:

  • PutBucketLifecycle
  • GetBucketLifecycle
  • DeleteBucketLifecycle

Why: Grouped advanced features. Completes library API surface.

v0.17.0 - CLI Complete

Scope: Full-featured CLI with all library features Size: Large (~2000 LOC)

CLI (cmd/simples3/):

  • Project structure (cobra or custom parser)
  • S3 URI parsing (s3://bucket/key)
  • Config (env vars: AWS_*, flags: --region/--profile)
  • AWS credentials file support (~/.aws/credentials)
  • Basic commands:
    • ls (list buckets/objects)
    • cp (upload/download, with multipart for large files)
    • rm (delete single/batch)
    • mb (make bucket)
    • rb (remove bucket)
    • mv (move/rename using copy+delete)
    • presign (generate URL)
    • sync (local ↔ S3 with diff algorithm)
  • Flags:
    • --recursive (cp/rm)
    • --sse/--sse-kms-key-id (encryption)
    • --tags (tagging)
    • --version-id (versioning)
    • --acl (canned ACLs)
    • --delete (sync)
    • --exclude/--include (patterns)
    • --json (output mode)
    • --dry-run
  • Features:
    • Progress indicators
    • Parallel transfers
    • Better error messages
  • Subcommands:
    • acl get/set
    • lifecycle get/set/delete
    • tags get/set/delete

Why: Library complete. Build full-featured CLI supporting all operations.

v1.0.0 - Stable Release

Focus: Production-ready stability Size: Documentation, tests, polish

Library:

  • All tests passing with 80%+ coverage
  • API documentation complete
  • Performance benchmarks
  • Security audit of signing/crypto code

CLI:

  • CLI tests with real/mocked S3
  • CLI documentation (man pages, --help)
  • Installation guides (brew, apt, etc.)
  • Shell completions (bash, zsh, fish)

Release:

  • CHANGELOG.md complete
  • Migration guide from v0.x
  • Examples for all features
  • API stability guarantee

Why: v1.0 = stability commitment. Library + CLI production-ready.

Reference in New Issue View Git Blame Copy Permalink