IamTugy / pyminio

Pyminio is a python client wrapped like the os module to control minio server

Pyminio

Pyminio is a python client wrapped like the os module to control minio server.

I have developed pyminio while trying to work with the minio's original python client with a lot of struggles. I had to read and understand minio's implementations to preform the most simple tasks.

Pyminio is a wrapper to minio, that is more indecative for the user. It works like os module, so you don't need to understand minio's concepts, and just using regular paths.

Content

1. Installation
2. Setting up Pyminio
3. Usage
4. Contribute

Installation

Use the package manager pip to install pyminio.

pip install pyminio

or

git clone https://github.com/mmm1513/pyminio.git
cd pyminio
python setup.py install  # also `pip install ."
# `pip install -e '.[dev]'` for development

Setting up Pyminio

Firstly you need to set up your Minio Docker, and acquire an ENDPOINT (URL), ACCESS_KEY, and a SECRET_KEY.

• If you want to add your own minio object you can pass it in the constructor like so:

  Install python's Minio module.

  from minio import Minio
  from pyminio import Pyminio
  
  minio_obj = Minio(
      endpoint='<your-minio-endpoint>',  # e.g. "localhost:9000/"
      access_key='<your-minio-access-key>',
      secret_key='<your-minio-secret-key>'
  )
  pyminio_client = Pyminio(minio_obj=minio_obj)

• If you don't want to handle with minio, you can do this instead:

  from pyminio import Pyminio
  
  pyminio_client = Pyminio.from_credentials(
      endpoint='<your-minio-endpoint>',  # e.g. "localhost:9000/"
      access_key='<your-minio-access-key>',
      secret_key='<your-minio-secret-key>'
  )

Usage

• mkdirs
• listdir
• exists
• isdir
• truncate
• rmdir
• rm
• cp
• mv
• get
• get_last_object
• put_data
• put_file

mkdirs(self, path: str)

Pyminio.mkdirs will create the given full path if not exists like linux's mkdir -p.

This method must get a directory path or it will raise a ValueError.

>>> pyminio_client.mkdirs('/foo/bar/baz/')
>>> pyminio_client.mkdirs('/foo/bar/baz')
ValueError /foo/bar/baz is not a valid directory path. must be absolute and end with /

listdir(self, path: str, files_only: bool = False, dirs_only: bool = False) -> Tuple[str]

Pyminio.listdir will return the directory's content as a tuple of directories and file names. Works like os's listdir.

This method must get a directory path or it will raise a ValueError.

>>> pyminio_client.listdir('/foo/bar/baz/')
('file_name_1', 'file_name_2', 'directory_name/')

There is an option to use the files_only flag to get only files and dirs_only to get only directories from listdir.

>>> pyminio_client.listdir('/foo/bar/baz/', files_only=True)
('file_name_1', 'file_name_2')
>>> pyminio_client.listdir('/foo/bar/baz/', dirs_only=True)
('directory_name/', )

exists(self, path: str) -> bool

Pyminio.exists will return a boolean that confirm rather this path exists or not in the server. Works like os's path.exists.

/
├── foo
│   └── bar
│       └── baz
│           ├── file_name_1
│           └── file_name_2

>>> pyminio_client.exists('/foo/bar/baz/file_name_1')
True
>>> pyminio_client.exists('/foo/bar/baz/file_name_3')
False
>>> pyminio_client.exists('/all/path/wrong/')  # not existing path
False

isdir(self, path: str)

Pyminio.isdir will return True only if the given path exists and is a directory. Works like os.path.isdir.

>>> pyminio_client.isdir('/foo/bar/baz/file_name_1')  # existed file
False
>>> pyminio_client.isdir('/foo/bar/baz/directory_name/')  # existed directory
True
>>> pyminio_client.isdir('/all/path/wrong/but/directory/')  # not existed directory
False

truncate(self) -> Pyminio

Pyminio.truncate will delete all minio's content from the root directory.

>>> pyminio_client.truncate()

rmdir(self, path: str, recursive: bool = False) -> Pyminio

Pyminio.rmdir will delete the specified directory. Works like linux's rmdir / rm (-r).

It will raise a DirectoryNotEmptyError if given directory is not empty, except if the recursive flag is set and then it will delete given directory's path recursively.

This method must get a directory path or it will raise a ValueError.

>>> pyminio_client.rmdir('/foo/bar/baz/directory_name/')  # empty directory
>>> pyminio_client.rmdir('/foo/bar/')  # non-empty directory
DirectoryNotEmptyError: can not recursively delete non-empty directory
>>> pyminio_client.rmdir('/foo/bar/', recursive=True)

rm(self, path: str, recursive: bool = False) -> Pyminio

Pyminio.rm works like rmdir only that it can delete files too. Works like linux's rm (-r).

>>> pyminio_client.rm('/foo/bar/baz/file_name')

cp(self, from_path: str, to_path: str, recursive: bool = False) -> Pyminio

Pyminio.cp will copy one file or directory to given destination. Works like linux's cp (-r).

This method can only copy recursively when the recursive flag is True. If not, it will raise a ValueError.

How will the copy accure? (all directories are copied recursively in this examples)

src path	dst path	dst exists	new dst	Explain
/foo/bar	/foo/baz	---	/foo/baz	The file's name will be copied from bar to baz as well.
/foo1/bar	/foo2/	True	/foo/bar	The file will be copied to '/foo2/bar'
/foo/bar/	/foo/	True	/foo/	The content of '/foo/bar/' will be copied to '/foo/'
/foo1/bar/	/foo2/	False	/foo2/bar/	'/foo1/bar/' will be copied recursively to '/foo2/bar/'
/foo1/bar/	/foo2/baz	---	---	ValueError will be raised in attempting to copy directory in to a file

>>> pyminio_client.cp('/foo/bar', '/foo/baz')
>>> pyminio_client.cp('/foo1/bar', '/foo2/')
>>> pyminio_client.cp('/foo/bar/', '/foo/', recursive=True)
>>> pyminio_client.cp('/foo1/bar/', '/foo2/', recursive=True)
>>> pyminio_client.cp('/foo1/bar/', '/foo2/baz', recursive=True)
ValueError: can not activate this method from directory to a file.

mv(self, from_path: str, to_path: str, recursive: bool = False) -> Pyminio

Pyminio.mv works like cp only that it removes the source after the transfer has been completed. Works like linux's mv.

This method can only move recursively when the recursive flag is True. If not, it will raise a ValueError.

>>> pyminio_client.mv('/foo/bar/', '/foo/baz/')

get(self, path: str) -> ObjectData

Pyminio.get return an object from given path. This object will be returned as a pyminio.File object or an pyminio.Folder object, that both inherit from pyminio.ObjectData.

This objects will contain metadata, their path and name.

>>> pyminio_client.get('/foo/bar/baz')
File(name='baz', 
     full_path='/foo/bar/baz', 
     metadata=AttrDict({
         'is_dir': False, 
         'last_modified': time.struct_time(...), 
         'size': ..., 
         'content-type': ...
     }), 
     data=...)

get_last_object(self, path: str) -> File

Pyminio.get_last_object will return the last modified object inside a given directory.

This method must get a directory path or it will raise a ValueError.

>>> pyminio_client.get_last_object('/foo/bar/')
File(name='baz', 
     full_path='/foo/bar/baz', 
     metadata=AttrDict({
         'is_dir': False, 
         'last_modified': time.struct_time(...), 
         'size': ..., 
         'content-type': ...
     }), 
     data=...)

put_data(self, path: str, data: bytes, metadata: Dict = None)

Pyminio.put_data gets a path, data in bytes, and some metadata, and create an object inside the given path.

>>> data = b'test'
>>> metadata = {'Pyminio-is': 'Awesome'}
>>> pyminio_client.put_data(path='/foo/bar/baz', data=data, metadata=metadata)

put_file(self, file_path: str, to_path: str, metadata: Dict = None)

Pyminio.put_file works like put_data only that instead of data it gets a path to a file in you computer. Then it will copy this file to the given location.

>>> metadata = {'Pyminio-is': 'Awesome'}
>>> pyminio_client.put_file(to_path='/foo/bar/baz', file_path='/mnt/some_file', metadata=metadata)

Contribute

All contributions are welcome:

• Read the issues, Fork the project and create a new Pull Request.
• Request a new feature creating a New issue with the enhancement tag.
• Find any kind of errors in the code and create a New issue with the details, or fork the project and do a Pull Request.
• Suggest a better or more pythonic way for existing examples.

Work environment

After forking the project and installing the dependencies, (like specified in the installations in part 2) download the minio docker and start an instance in your computer for development and testing.

to run the tests run:

pytest

Don't forget to write tests, and to run all the tests before making a pull request.