PKU-Alignment

safe-rlhf

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

omnisafe

OmniSafe is an infrastructural framework for accelerating SafeRL research.

safety-gymnasium

NeurIPS 2023: Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark

Safe-Policy-Optimization

NeurIPS 2023: Safe Policy Optimization: A benchmark repository for safe reinforcement learning algorithms

AlignmentSurvey

AI Alignment: A Comprehensive Survey

beavertails

BeaverTails is a collection of datasets designed to facilitate research on safety alignment in large language models (LLMs).

align-anything

Align Anything: Training Any Modality Model with Feedback

ProAgent

ProAgent: Building Proactive Cooperative Agents with Large Language Models

SafeDreamer

ICLR 2024: SafeDreamer: Safe Reinforcement Learning with World Models

safe-sora

SafeSora is a human preference dataset designed to support safety alignment research in the text-to-video generation field, aiming to enhance the helpfulness and harmlessness of Large Vision Models (LVMs).

ReDMan

ReDMan is an open-source simulation platform that provides a standardized implementation of safe RL algorithms for Reliable Dexterous Manipulation.

ProgressGym

Alignment with a millennium of moral progress.

llms-resist-alignment

Repo for paper "Language Models Resist Alignment"

.github