jackfsuia's starred repositories
Awesome-LLM-Inference
📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
RLHF-Label-Tool
用于大模型 RLHF 进行人工数据标注排序的工具。A tool for manual response data annotation sorting in RLHF stage.
persona-hub
Official repo for the paper "Scaling Synthetic Data Creation with 1,000,000,000 Personas"
arxiv-sanity-preserver
Web interface for browsing, search and filtering recent arxiv submissions
Case_or_Rule
exploring whether LLMs perform case-based or rule-based reasoning
crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
codellm-data-preprocess-pipeline
代码大模型 预训练&微调&DPO 数据处理 业界处理pipeline sota
cachecloud
搜狐视频(sohu tv)Redis私有云平台 :支持Redis多种架构(Standalone、Sentinel、Cluster)高效管理、有效降低大规模redis运维成本,提升资源管控能力和利用率。平台提供快速搭建/迁移,运维管理,弹性伸缩,统计监控,客户端整合接入等功能。(CacheCloud is a Redis cloud management platform. It supports Standalone, Sentinel, and Cluster architectures for Redis, effectively reducing large-scale Redis operation and maintenance costs, and improving resource management and utilization. The platform provides rapid construction/migration, operation and maintenance management, elastic scaling, statistical monitoring, client integration and access and other functions)
trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
playwright-python
Python version of the Playwright testing and automation library.
wave-share
Serverless, peer-to-peer, local file sharing through sound
label-studio
Label Studio is a multi-type data labeling and annotation tool with standardized output format
EasySpider
A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。
MediaCrawler
小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 百度贴吧评论回复爬虫
search_with_lepton
Building a quick conversation-based search demo with Lepton AI.
Perplexica
Perplexica is an AI-powered search engine. It is an Open source alternative to Perplexity AI
AutoWebGLM
An LLM-based Web Navigating Agent (KDD'24)
document.ai
基于向量数据库与GPT3.5的通用本地知识库方案(A universal local knowledge base solution based on vector database and GPT3.5)