# Duplicate File Finder (Java - MD5 Hashtable)

Finds duplicate files by content using MD5 hashing. Uses a `Hashtable<String, List<String>>` where **MD5 hash -> list of filenames**. Groups with 2+ files are reported as duplicates.

The core is a **command-line program** that can be used standalone, wrapped by a **Swing GUI** with tree view and selective deletion.

`Last updated: 2026-08-29 - NPE fix for tree renderer + README refresh`

## Files

- `DuplicateFinder.java` - CLI core + hashtable logic. Can run standalone. (`Hashtable<String, List<String>>` at `DuplicateFinder.java:114`, `filterDuplicates` at `DuplicateFinder.java:145`)
- `DuplicateFinderGUI.java` - Swing GUI wrapper (tree view + delete) that reuses `DuplicateFinder`'s hashtable logic. (`CheckBoxTreeCellRenderer` at `DuplicateFinderGUI.java:347`, `CheckedNode` at `DuplicateFinderGUI.java:56`, `startScan` at `DuplicateFinderGUI.java:474`)
- `build.bat` / `run.bat` - Windows helpers
- `build.sh` / `run.sh` - bash helpers (Linux/macOS/WSL/Git Bash)
- `README.md` - this file

## Requirements

- Java 8+ (tested on Temurin 1.8.0_504). Compatible with Java 8, 11, 17, 21. No external deps.

## Build

```bat
javac DuplicateFinder.java DuplicateFinderGUI.java
```
Or:
```bat
build.bat        :: Windows
./build.sh       :: bash (Linux/macOS/WSL/Git Bash) - chmod +x build.sh first
```

Verify:
```bat
java DuplicateFinder --help
java DuplicateFinderGUI
:: bash:
./run.sh --help
./run.sh         :: launches GUI
```

## CLI Usage

```bat
java DuplicateFinder [options] <directory>
java DuplicateFinder --gui          # launch GUI
java DuplicateFinder --help

java DuplicateFinderGUI             # launch GUI directly
```

Options:
- `-r, --recursive`     recurse into subdirectories (default)
- `--no-recursive`      only top-level directory
- `-h, --help`          show help
- `--gui`               launch Swing GUI

Examples:
```bat
java DuplicateFinder C:\Users\Me\Documents
java DuplicateFinder --no-recursive C:\Temp
java DuplicateFinder --gui
```

Output example:
```
Scanning: C:\Users\Me\Documents
Recursive: true

=== Duplicate Group 1 ===
MD5: 3579c8da7f1e0ad94656e76c886e5125  (3 files)
  C:\...\a.txt  (13 bytes)
  C:\...\b.txt  (13 bytes)
  C:\...\sub\d.txt  (13 bytes)

Summary: 1 duplicate group(s), 3 files involved.
Scanned in 43 ms.
```

Exit codes: 0 success, 1 error (bad args / missing path).

## GUI Usage

```bat
java DuplicateFinderGUI
:: or
java DuplicateFinder --gui
java DuplicateFinderGUI "C:\path\to\start\dir"
```

### GUI Features

- Directory text field + Browse button (JFileChooser - directories only)
- Recursive checkbox
- Find Duplicates / Cancel / Clear Results buttons
- Progress bar + status label (SwingWorker background thread, doesn't freeze UI)
- **Tree view** (`JTree` with checkboxes, `DefaultTreeModel` + `CheckedNode`):
  - Root = scan summary (invisible, `rootVisible=false`), children = duplicate groups, leaves = files
  - Group node: `Group 1 - 3 files - 13 bytes each - MD5: abc...` (bold, directory icon)
  - File node: `filename [parent folder] (size bytes)` with checkbox (file icon, red when checked)
  - Click checkbox or press `Space` on selected node to mark for deletion; group checkbox toggles all children (`setNodeChecked` at `DuplicateFinderGUI.java:423`)
  - Custom `CheckBoxTreeCellRenderer` shows `JCheckBox + JLabel` per row (`DuplicateFinderGUI.java:347`)
- **Tree actions bar**:
  - `Auto-Select (keep 1 per group)` - checks all but first file in each group (safe default, `DuplicateFinderGUI.java:538`)
  - `Clear Checks` - unchecks all (`DuplicateFinderGUI.java:561`)
  - `Delete Checked Files` - deletes with confirmation (shows count + bytes, warns if all in group checked, `DuplicateFinderGUI.java:651`)
  - `Open Location` - opens containing folder via `Desktop.getDesktop().open()` / `explorer`
  - `Expand All` / `Collapse All` (`DuplicateFinderGUI.java:571`)
  - Label: `N file(s) checked, X bytes to free` (`updateCheckedInfo` at `DuplicateFinderGUI.java:636`)
- Context popup (right-click): Auto-Select / Clear / Delete / Open Location / Copy Path
- Menu bar: File/Browse/Exit, Edit/Auto-Select/Delete, View/Expand/Collapse, Help/About
- Selection listener shows full path + size in status bar

### Deletion Flow

1. Scan -> tree populated, groups expanded (`displayResultsTree` at `DuplicateFinderGUI.java:862`, `expandAll`).
2. Use `Auto-Select` or manually check unwanted duplicates (mouse click on row or `Space`).
3. Review `Delete Checked Files` confirmation dialog:
   - Lists files to delete (first 15 + count), total bytes, warnings if entire group selected.
   - Requires YES confirmation (permanent `File.delete()` with `setWritable(true)` fallback).
4. On success: tree nodes removed via `treeModel.removeNodeFromParent()`, empty/single groups pruned, summary updated, status shows `Deleted N file(s)`.
5. On failure (locked/read-only/not found): dialog lists failures, partial successes kept.

> Safety: `Auto-Select` keeps 1 copy per group. Manual delete warns if you check all files in a group.

The GUI calls `DuplicateFinder.buildHashTableWithProgress()` with a `ProgressCallback` to update the progress bar without freezing the EDT.

## How it works

1. `collectFiles(root, recursive)` (`DuplicateFinder.java:78`) walks the file tree (recursive DFS).
2. For each file, `computeMD5(File)` (`DuplicateFinder.java:42`) uses `MessageDigest.getInstance("MD5")` + 8KB buffered `FileInputStream` to compute hex hash.
3. Hashtable population (`DuplicateFinder.java:114`):
   ```java
   Hashtable<String, List<String>> table = new Hashtable<>();
   // md5 -> list
   List<String> list = table.get(md5);
   if (list == null) { list = new ArrayList<>(); table.put(md5, list); }
   list.add(file.getAbsolutePath());
   ```
4. `filterDuplicates()` (`DuplicateFinder.java:145`) keeps only entries where `list.size() > 1`.
5. CLI prints sorted groups; GUI builds `DefaultTreeModel` with `CheckedNode` user objects and `CheckBoxTreeCellRenderer` (`DuplicateFinderGUI.java:347`). Groups sorted by hash, files sorted by path.

Empty files (MD5 `d41d8cd98f00b204e9800998ecf8427e`) are considered duplicates if multiple empties exist. Unreadable files are skipped with warning.

## Testing

Create a `test_data` example:

```bat
mkdir test_data\sub
echo Hello World > test_data\a.txt
copy test_data\a.txt test_data\b.txt
copy test_data\a.txt test_data\sub\d.txt
echo Different > test_data\c.txt
copy /b nul test_data\empty1.txt
copy /b nul test_data\empty2.txt
```

- `a.txt`, `b.txt`, `sub\d.txt` all "Hello World" -> one group of 3
- `empty1.txt`, `empty2.txt` -> one group of 2 (if empties present)
- `bin1.dat`, `sub\bin2.dat` (bytes 0-255) -> one group of 2 if created

Run:
```bat
java DuplicateFinder test_data           # expects 3 groups (or 1 if minimal set above)
java DuplicateFinder --no-recursive test_data  # expects fewer groups (sub excluded)
# GUI: scan test_data, Auto-Select, Delete -> leaves 1 per group, re-scan shows 1 group then 0
```

Manual NPE regression test (fixed 2026-08-29):
```bat
javac -d . DuplicateFinder.java DuplicateFinderGUI.java
java DuplicateFinderGUI
# Click Find Duplicates immediately -> before fix threw NPE at DuplicateFinderGUI.java:391
# After fix: shows "Scanning: ..." placeholder without crash, then tree populates
```

## Changelog

- **2026-08-29 (fix):** `DuplicateFinderGUI.java:391` NPE in `CheckBoxTreeCellRenderer.getTreeCellRendererComponent()` when `data.md5==null` for placeholder root nodes ("Scanning...", "No duplicates...", "Cancelled"). Added null guard for `md5` (length check + empty). Also guarded `DuplicateFinderGUI.java:670` warning `shortMd5` substring. Verified via `RendererTest` + `ScanTest` harness; CLI unaffected.
- **2026-08-29:** Added tree view with checkboxes, selective delete, auto-select keep-one, open location, expand/collapse, context menu.
- **2026-08-28:** Initial CLI + basic GUI (`JTextArea` results).

## Troubleshooting

- `NullPointerException at CheckBoxTreeCellRenderer.getTreeCellRendererComponent(DuplicateFinderGUI.java:391)`: fixed in latest version. Update `DuplicateFinderGUI.java` or re-clone. Recompile with `javac`.
- `javac not found`: install JDK 8+ (Temurin 8 used for testing at `C:\Program Files\Eclipse Adoptium\jdk-8.0.504.1-hotspot\bin\javac.exe`) and add to `PATH` or use `build.bat` which auto-adds that path.
- `HeadlessException` when running GUI on headless server: GUI requires display. Use CLI `java DuplicateFinder <dir>` instead or run with `xvfb`/`X forwarding`.
- Files not deleted: checked for `File.delete()` returning false (locked/read-only). Close programs using the file, then retry. GUI retries with `setWritable(true)`.

## Notes

- Uses `Hashtable` specifically (as required), not `HashMap`.
- Large files are streamed, not loaded fully into memory.
- CLI is read-only (listing only); GUI deletion is opt-in with confirmation and prunes non-duplicate groups.