scaling-laws

There are 1 repository under scaling-laws topic.

huggingface / datablations
Scaling Data-Constrained Language Models
gpt high-performance-computing language-models large-language-models llms scaling-laws
Language:Jupyter Notebook 297
LAION-AI / scaling-laws-openclip
Reproducible scaling laws for contrastive language-image learning (https://arxiv.org/abs/2212.07143)
clip deep-learning few-shot-learning fine-tuning laion openclip pre-training pytorch scaling-laws transfer-learning zero-shot-classification zero-shot-retrieval
Language:Jupyter Notebook 135
kyo-takano / chinchilla
A toolkit for scaling law research ⚖
deep-learning flops pretraining reinforcement-learning scaling-laws llm
Language:Python 34
xiaoyuxie-vico / PyDimension
Dimensionless learning codes for our paper called "Data-driven discovery of dimensionless numbers and governing laws from scarce measurements".
scaling-laws manufacturing deep-learning scientific-machine-learning machine-learning physics-informed partial-differential-equations system-identification
Language:Jupyter Notebook 25
SLAMPAI / large-scale-pretraining-transfer
Code for reproducing the experiments on large-scale pre-training and transfer learning for the paper "Effect of large-scale pre-training on full and few-shot transfer learning for natural and medical images" (https://arxiv.org/abs/2106.00116)
transfer-learning deep-learning medical-imaging imagenet pytorch fine-tuning distributed-training large-scale-learning supercomputing scaling-laws few-shot-learning chest-xray-images chexpert-dataset mimic-cxr padchest-dataset chest-x-ray14 covidx-dataset big-transfer pre-training pre-trained-model
Language:Jupyter Notebook 18
releaunifreiburg / DPL
[NeurIPS 2023] Multi-fidelity hyperparameter optimization with deep power laws that achieves state-of-the-art results across diverse benchmarks.
benchmark computer-vision deep-learning hpo hyperparameter-optimization hyperparameter-tuning large-language-models llm machine-learning mlp natural-language-processing power-laws scaling-laws tabular-data transformer neurips neurips-2023
Language:Python 12
VITA-Group / Data-Efficient-Scaling
[ICML 2023] "Data Efficient Neural Scaling Law via Model Reusing" by Peihao Wang, Rameswar Panda, Zhangyang Wang
data-efficient large-language-models scaling-laws model-reusing
Language:Python 12
christinakim / scaling-laws-for-language-transfer
code for Scaling Laws for Language Transfer Learning
fine-tuning huggingface-transformers language-model openai pre-trained-model pytorch pytorch-lightning scaling-laws transfer-learning transformers
Language:Python 9
supersimple33 / Scaling-Laws
A method for calculating scaling laws for LLMs from publicly available models
large-language-models scaling-laws
Language:Python 3
upunaprosk / small-language-models
Code for CoNLL BabyLM workshop Mini Minds: Exploring Bebeshka and Zlata Baby Models
gpt-2 graphcore ipu language-model pretraining roberta scaling-laws
Language:Jupyter Notebook 1