YingtianDt

brain-score

A framework for evaluating models on their alignment to brain and behavioral measurements (50+ benchmarks)

pycortex

Pycortex is a python-based toolkit for surface visualization of fMRI data

neuroparc

brainio

Data management for quantitative comparison of brains and brain-inspired systems

scenic

Scenic: A Jax Library for Computer Vision Research and Beyond

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

mamba.py

A simple and efficient Mamba implementation in PyTorch and MLX.

cav-mae

Code and Pretrained Models for ICLR 2023 Paper "Contrastive Audio-Visual Masked Autoencoder".

Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

ijepa

Official codebase for I-JEPA, the Image-based Joint-Embedding Predictive Architecture. First outlined in the CVPR paper, "Self-supervised learning from images with a joint-embedding predictive architecture."

S3D_HowTo100M

S3D Text-Video model trained on HowTo100M using MIL-NCE

result_caching

Store results of function calls with respect to the call parameters

VideoMAE

[NeurIPS 2022 Spotlight] VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

VideoMAEv2

[CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking

selavi

This repo covers the implementation for Labelling unlabelled videos from scratch with multi-modal self-supervision, which learns clusters from multi-modal data in a self-supervised way.

OpenSTL

OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

GDT

We present a framework for training multi-modal deep learning models on unlabelled video data by forcing the network to learn invariances to transformations applied to both the audio and video streams.

AVID-CMA

Audio Visual Instance Discrimination with Cross-Modal Agreement

mae_st

Official Open Source code for "Masked Autoencoders As Spatiotemporal Learners"

mmaction2

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

YingtianDt.github.io

model-tools

Helper functions to extract model activations and translate from Machine Learning to Neuroscience

debug_tools

exp_tools

pdbpp

pdb++, a drop-in replacement for pdb (the Python debugger)

individual_event_seg

This repository contains code and experimental materials for the publication "Individual variability in neural event segmentation reflects stimulus content and interpretation""

afd

[ECCV 2022] Is Appearance Free Action Recognition Possible? (SUBMODULE)

contrastive2021

Implementation for paper "Towards the Generalization of Contrastive Self-Supervised Learning" (https://arxiv.org/abs/2111.00743)

SimCLR-cifar10-master

modelzoo_continual

Model Zoos for Continual Learning (ICLR 22)