flexflow / FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving

https://flexflow.readthedocs.io

flexflow/FlexFlow Issues

cuIpcGetMemHandle triggered CUDA out of memory when I use flexflow on one gpu
Updated 9 days ago4
Add kernels/local-execution support for new `BatchNormAttrs` fields
Updated 10 days ago
Fix profiling wrapper to avoid resource handle issues
Updated 10 days ago
Add `hash` and `fmt` for `TaskBinding` in local execution
Updated 10 days ago
Update files in `kernels` so that the `CMakeLists.txt` src pattern can be changed to `src/cuda/*.cu`
Updated 14 days ago
Update rewriting search in unity algorithm
Updated 15 days ago
Add weight handling for SP decomposition of PCGs
Updated 15 days ago
Add intermediate interface between `ComputationGraphBuilder` and the raw graph interface for testing
Updated 15 days ago
`CHECK_VALID_OP_ATTR` is not that necessary anymore and should be removed
Updated 15 days ago
Add `num_inputs` check to `get_output_shapes(PCGOperatorAttrs, std::vector<ParallelTensorShape>)`
Updated 17 days ago
Improve unit tests for `ParallelComputationGraphBuilder` and `ComputationGraphBuilder`
Updated 18 days ago
Error when I use larger batch size for spec-infer
Updated 22 days ago
multinode python: Legion error 67 alongside NCCL errors.
Updated 23 days ago5
Remove inheritance structure from graph objects
Updated a month ago
Replace/simplify DimOrdered
Updated a month ago
Add ability to get `vector` of all enum values to `proj dtgen`
Updated a month ago
Implement `is_valid_substitution`
Updated a month ago
Implement `as_dot` for graphs in `utils/graph` in a less hacky way
Updated a month ago
Add ability to document dtgen structs using doxygen
Updated a month ago
Add a `SubstitutionBuilder` to make creating `Substitution`s less verbose and error-prone
Updated a month ago
Add function to divide list of PCG inputs into inputs and weights in `op-attrs`
Updated a month ago
Move `TensorSlotsBackingWithoutAddresses` over to dtgen
Updated a month ago
Move `OpTaskSignature` to dtgen, away from visitable
Updated a month ago1
Questions about the measurement of the latency
Updated a month ago2
it is possible to print the simulated running time and simualtion time cost？
Updated 2 months ago
Fix handling for device specific and binding arbitrary device specific types
Updated 2 months ago
Inference with TP>1 and dissagregated qkv projection & attention operator
Closed 2 months ago1
Issue with C++ inference for model meta-llama/Llama-2-70b-hf
Updated 2 months ago
Add support for Tiktoken tokenizer in Request Manager
Closed 2 months ago
Issue with debugging using cuda-gdb
Updated 2 months ago
example inference test in cpp_inference_tests.sh does not terminate
Closed 2 months ago
Sorry, it was a typo
Closed 2 months ago
Issue with FlexFlow LLM Compilation and Generation
Updated 2 months ago1
Fix embedding kernel refactor
Updated 2 months ago
Check whether an `OpTaskInvocation` is valid against an `OpTaskSignature`
Updated 2 months ago
Add tests for managed_ff_stream and handle
Updated 3 months ago
How to enable reduction parallel in substitutions?
Closed 3 months ago
How to enable reduction parallel in substitutions?
Updated 3 months ago
Request for Graph Pruning Algorithm Code Location in FlexLLM
Updated 3 months ago
Fix input, weight, noop in local execution
Updated 3 months ago
Fix `cudnnSetTensorDescriptorFromArrayShape`
Updated 3 months ago
Convert allocator to arena
Updated 3 months ago
Rename `real_type` to `real_type_t`
Updated 3 months ago
Fix calls for softmax_kernels init_kernel
Updated 3 months ago
Change `Allocator` ownership model
Updated 3 months ago
Local Backing: Gradient Tensor Allocation
Updated 3 months ago
Analytical function for estimating communication cost
Updated 3 months ago
Refactor `element_unary_kernels.cpp`
Updated 4 months ago
Implement `is_valid` (or remove it) in parallel shape inference
Updated 4 months ago
Add local logging
Updated 4 months ago