S3 Backend for Afero

January 1, 2022 ยท View on GitHub

Build codecov Go Report Card GoDoc

About

It provides an afero filesystem implementation of an S3 backend.

This was created to provide a backend to the ftpserver but can definitely be used in any other code.

I'm very opened to any improvement through issues or pull-request that might lead to a better implementation or even better testing.

Key points

  • Download & upload file streaming
  • 75% coverage (all APIs are tested, but not all errors are reproduced)
  • Very carefully linted

Known limitations

  • File appending / seeking for write is not supported because S3 doesn't support it, it could be simulated by rewriting entire files.
  • Chtimes is not supported because S3 doesn't support it, it could be simulated through metadata.
  • Chmod support is very limited

How to use

Note: Errors handling is skipped for brevity but you definitely have to handle it.


import(
	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/credentials"
	"github.com/aws/aws-sdk-go/aws/session"
  
	s3 "github.com/fclairamb/afero-s3"
)

func main() {
  // You create a session
  sess, _ := session.NewSession(&aws.Config{
    Region:      aws.String(region),
    Credentials: credentials.NewStaticCredentials(keyID, secretAccessKey, ""),
  })

  // Initialize the file system
  s3Fs := s3.NewFs(bucket, sess)

  // And do your thing
  file, _ := fs.OpenFile("file.txt", os.O_WRONLY, 0777)
  file.WriteString("Hello world !")
  file.Close()
}

Thanks

The initial code (which was massively rewritten) comes from: