athenianco / medvedi

Memory Efficient Deconstructed Vectorized Dataframe Interface

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

MEDVeDI Build status codecov Latest Version Python Versions License

logo

Memory Efficient Deconstructed Vectorized Dataframe Interface.

Design goals:

  • Favor performance over nice syntax features. Sacrifice fool-proof for efficient zero-copy operations.
  • Ensure ideal micro-performance and optimize for moderate data sizes (megabytes).
  • The use-case is API server code that you write once and execute many times.
  • Try to stay compatible with the Pandas interface. There is no Series, however.
  • Rely on numpy.
  • Friends with Arrow.
  • Frequently release GIL and depend on native extensions doing unsafe things.
  • Test only CPython and Linux.
  • Support only x86-64 CPUs with AVX2.
  • Support only Python 3.10+.
  • 100% test coverage.

Otherwise, you should be way better with regular Pandas.

Medvedi is currently heavily used in production of Athenian.

About

Memory Efficient Deconstructed Vectorized Dataframe Interface

License:Apache License 2.0


Languages

Language:Python 52.8%Language:C++ 23.4%Language:Cython 23.0%Language:Makefile 0.4%Language:C 0.4%