From e8bfcfb40b45ee0c5c6aedf1aef44e27c743aa32 Mon Sep 17 00:00:00 2001
From: Alan <alan.d@gfp.com.au>
Date: Thu, 26 Mar 2026 20:50:38 +1100
Subject: [PATCH] add readme

---
 README.md | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 README.md

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..aeba966
--- /dev/null
+++ b/README.md
@@ -0,0 +1,16 @@
+# LoRA Training on AMD
+
+## Overview
+
+### `shell.nix`
+Provides a development environment compatible within Nix. It handles:
+- Installing PyTorch optimized for AMD ROCm (`rocm7.2`).
+- Installing `unsloth` and `unsloth-zoo` for efficient fine-tuning.
+- Installing `marimo` and `ipython` as QOL.
+
+### `train.py`
+A `marimo` script that executes the fine-tuning process:
+- Loads the `unsloth/Qwen3.5-0.8B` model in 16-bit.
+- Prepares a sample dataset (`laion/OIG/unified_chip2.jsonl`).
+- Configures Parameter-Efficient Fine-Tuning (PEFT) using LoRA (Rank 16).
+- Sets up an `SFTTrainer` (using 8-bit AdamW) and trains the model for 100 steps, saving results to `outputs_qwen35`.