Reorganize: Python Package Structure + Comprehensive Docs with Mermaid Diagrams

[itsXactlY]

What This PR Does

Major reorganization of JackrabbitDLM into a standard Python package layout with comprehensive documentation, Mermaid diagrams, and technical corrections discovered during code analysis.

---

Structural Changes

New Python Package: jackrabbitdlm/

jackrabbitdlm/
├── __init__.py              # Package root (version, exports Locker + encoding)
├── common/
│   └── encoding.py          # dlmEncode, dlmDecode, ShuffleJSON (shared, importable)
├── client/
│   └── locker.py            # Locker class (refactored, full docstrings)
├── server/
│   └── daemon.py            # Server daemon (cleaned, action handlers extracted)
├── tools/
│   ├── identity/            # Auth config generator
│   └── chart/               # Plotly log visualizer
└── benchmarks/
    └── lock_fighter.py      # Stress tester

New Directories

• scripts/ — Shell helpers (install.sh, launch_dlm.sh, lock_wars.sh)
• config/ — Configuration templates (defaults.yaml)
• tests/ — Test suite (encoding roundtrip + ShuffleJSON)
• docs/ — Extended documentation

---

Code Improvements

Encoding Module (jackrabbitdlm/common/encoding.py)

• Extracted shared dlmEncode/dlmDecode/ShuffleJSON from inline duplication
• Pre-computed lookup tables (ENCODE_TABLE, DECODE_TABLE) for O(1) performance
• Full docstrings explaining the encoding is obfuscation, not encryption

Client Library (jackrabbitdlm/client/locker.py)

• Cleaner API: talk(), _retry(), _retry_data() as internal methods
• Docstrings for all public methods (Lock, Unlock, Put, Get, Erase, Version, IsDLM)
• Imports encoding from common instead of duplicating code
• Full backward compatibility with original DLMLocker.py

Server Daemon (jackrabbitdlm/server/daemon.py)

• Extracted action handlers: _handle_lock(), _handle_unlock(), _handle_get(), _handle_put(), _handle_erase()
• Configurable base dir via JACKRABBIT_BASE environment variable
• Full docstrings with Wikipedia references for each concept
• Identical protocol behavior — drop-in replacement

Shell Scripts

• scripts/install.sh — Proper setup with venv creation and dependency install
• scripts/launch_dlm.sh — Auto-restart wrapper with configurable paths
• scripts/lock_wars.sh — Parallel stress test launcher with argument passthrough

---

README.md Overhaul

ASCII Art

• slant font header (via pyfiglet) — clean, terminal-friendly

5 Mermaid Diagrams

1. Architecture Overview (graph TB) — Clients → TCP → Server internals
2. Request Flow (sequenceDiagram) — Lock/Put/Get/Unlock lifecycle with correct Lock vs DataStore separation
3. Lock State Machine (stateDiagram-v2) — Lock states with note on entity separation
4. Data Flow (flowchart TD) — Put with dynamic disk spillover thresholds
5. Class Diagram (classDiagram) — All classes and their relationships

Sections

• TL;DR with quick start code
• Protocol Reference — Complete field descriptions, auth rules
• Security Model — Ownership enforcement, blind vault, TTL, payload limits
• Performance & Monitoring — All statistics explained
• Comparison Table — vs Redis, ZooKeeper, Etcd, PostgreSQL
• Wikipedia Links — Every deeper concept linked

---

Critical Technical Corrections

During code analysis, we discovered and documented several important details that were previously unclear:

1. Locks and DataStores Are Separate Entities

Locks and DataStores use separate IDs and should use different FileName values. A lock on "my_job" and data on "my_job:state" have independent lifecycles, ownership, and expiration.

# CORRECT: separate resources, separate IDs
lock = Locker("my_job", ID="lock-abc")
data = Locker("my_job:state", ID="data-xyz")

lock.Lock(expire=30)        # Acquires lock
data.Put(data="...", expire=60)  # Stores data
lock.Unlock()               # Lock released, data untouched
data.Erase()                # Data erased, lock unaffected

Unlock does NOT erase data. Erase does NOT unlock.

2. At 33% RAM, Both Lock AND Put Are Disabled

The MemoryOverloadCheck() is called in both _handle_lock and _handle_put. When process RSS exceeds 33% of total system RAM, both operations return {"Status": "NO"}. Only Get, Unlock, and Erase continue working.

3. Disk Spillover Threshold Is Dynamic

The ForceDisk logic has three conditions:

force_disk = (
    (ds_len > 16KB and expire > 10) or                        # Long TTL
    (ds_len > 16KB and ram_pct > 16.5%) or                    # Memory pressure
    (ds_len > 8KB and ram_pct > 24.75%)                       # High pressure
)

RAM Usage	Lock	Put	Disk Threshold
< 16.5%	✓	✓	16 KB
16.5% - 24.75%	✓	✓	16 KB
24.75% - 33%	✓	✓	8 KB
≥ 33%	NO	NO	n/a

---

Preserved

• All original files kept alongside new structure for reference
• LGPL-2.1 license unchanged
• Zero additional dependencies beyond psutil (and plotly for charts)
• Full backward compatibility with existing JSON-over-TCP protocol
• Any language implementing the protocol will continue to work

---

Testing

cd JackrabbitDLM
python3 tests/test_encoding.py
# PASS: encode/decode roundtrip
# PASS: ShuffleJSON
# All tests passed!

---

Reorganized by itsXactlY with Hermes Agent. All technical details verified against source code.