NRL

Overview

NRL is a lightweight reinforcement learning framework built directly on PyTorch primitives. It focuses on being modular, easy to extend, and suitable for research and engineering workflows without depending on large external infra.

Main features

• Native PyTorch implementation (no heavy external infra).
• Common algorithms implemented (PPO, DPO, GRPO, ...).
• Distributed support using PyTorch primitives and DTensor (data/model/tensor parallelism).
• Support for running training and inference within the same process for easy online evaluation.
• Modular design — easy to replace or extend algorithms, policies and backends.

Repository layout

• nrl/ — core package (entry, algorithms, training, distributed utilities)
• examples/ — runnable examples and configs (e.g., examples/ardf)
• scripts/, tests/ — helper scripts and tests

Installation

Prerequisites: Python and a matching PyTorch build (install a CUDA-enabled PyTorch for GPU usage).

# create a virtualenv
pip install -r requirements.txt

Quick start — run the ardf example

Run the repository entry with the example config:

python3 nrl/entry.py examples/ardf/config.py

Distributed example

python -m torch.distributed.run --nproc_per_node=<N> nrl/entry.py examples/ardf/config.py

Contributing

• Open an Issue to discuss large changes before implementation.
• Provide or update examples and tests when adding features.
• Include benchmarks for performance-sensitive changes.

Running tests

Use the helper script in the repository root to run tests (it sets PYTHONPATH so the local package is loaded):

bash scripts/run_tests.sh

See CONTRIBUTING.md for more details.

License & Contact

See the LICENSE file in the repository root. Use Issues for questions or contact the maintainers.

中文 — 概览

NRL — 基于原生 PyTorch 的轻量级强化学习框架。

主要特性

• 原生 PyTorch 实现（不依赖大型外部 infra）。
• 常见算法实现：DPO、PPO、GRPO 等。
• 基于 PyTorch DTensor 的分布式支持（数据/模型/张量并行、切分等）。
• 支持在同一进程中进行训练与推理，便于在线评估。
• 模块化设计：便于替换或扩展算法、策略与后端。

项目结构（高层）

• nrl/ — 核心包（入口、算法、训练、分布式工具）
• examples/ — 可运行示例与配置（例如 examples/ardf）
• scripts/, tests/ — 实用脚本与测试

安装

# 使用虚拟环境
pip install -r requirements.txt

快速开始 — 运行 `ardf` 示例

python3 nrl/entry.py examples/ardf/config.py

分布式运行（示例）

python -m torch.distributed.run --nproc_per_node=<N> nrl/entry.py examples/ardf/config.py

贡献

• 在提交大改动前建议先通过 Issue 讨论设计。
• 增加功能时请提供或更新示例和测试用例。
• 对性能相关改动建议附带基准或对比数据。

运行测试

请使用仓库根目录下的脚本来运行测试（脚本会把仓库根加入 PYTHONPATH，以便加载本地包）：

bash scripts/run_tests.sh

更多说明见 CONTRIBUTING.md。

许可与联系方式

详见仓库根目录下的 LICENSE 文件。如有问题，请通过 Issue 联系维护者。