Overview / نگاه کلی

System Architecture / ساختار سیستم

The HAS-RL control pipeline bridges clinician inputs, reinforcement learning decision-making, and robotic compliance control.

Figure 1: Control loop and architecture of HAS-RL. The policy takes the state, anticipatory look-ahead error, and the operator's runtime safety limit κ to generate velocity actions under low-pass impedance contact dynamics.

Core Innovations / نوآوری‌های کلیدی

1. Operator-Conditioned Zero-Shot Safety / تنظیم آنلاین فشار زیروشات

Instead of baking a fixed $F_{\max}$ into policy weights, HAS-RL treats the clinician’s runtime safety setting as an observation variable: \(s_t = \left[x_t, y_t, F_t, e_t, \kappa_t\right]\)

The clinician can freely dial the force limit during an active scan. The policy reacts in real time, modulating contact force without a single constraint breach.

Figure 2: Mid-scan dial adjustment tracking. The policy tracks abrupt shifts in κ with 0 violations.

Figure 3: 7-point sweep across $[3, 12]\,\text{N}$. Contact force scales proportionally with zero breaches.

2. Smoothed Bandit-Lagrangian (SBL) / سازوکار لاگرانژ باندیتی هموارشده

To combine the bounded penalty guarantees of multi-armed bandit multiplier selection (EXP3) with online policy-gradient learning, SBL maintains an exponential moving average over expected multiplier weights: \(\bar{\lambda}_k = (1 - \tau)\bar{\lambda}_{k-1} + \tau \mathbb{E}_{a \sim p_k}[\lambda_a]\)

This resolves the gradient variance and policy collapse caused by raw bandit arm sampling while maintaining zero constraint violations.

3. Two-Sided Acoustic-Window Safety / پنجره ایمنی آکوستیک دوطرفه

Ultrasound imaging requires minimum contact to establish acoustic coupling, while excessive force injures soft tissue. HAS-RL enforces: \(\kappa_{\text{low}} \le F_{\text{contact}} \le \kappa_{\text{high}}\)

This eliminates the “headroom gap” of ceiling-only safety, reaching 10.2 N in $[9, 12]\,\text{N}$ windows versus 7.97 N under naive ceiling constraints.

Quantitative Evaluation / نتایج تجربی

Safety Performance Comparison ($\kappa = 5\,\text{N}$)

Multiplier-Mode Stability Ablation (3 Random Seeds)

Two-Sided Operator Window Performance

Ultrasound B-Mode Image Reconstruction

HAS-RL Reconstruction: High acoustic fidelity maintained with strictly controlled force.

Baseline Reconstruction: Unconstrained force causing continuous safety breaches.

Amirkabir University of Technology Defense Details / جزئیات دفاع

Citation

@thesis{niknam2026hasrl,
  title        = {HAS-RL: Human-Adaptive Safe Reinforcement Learning for Clinician-Configured Force Safety in Robotic Ultrasound Spinal Scanning},
  author       = {Niknam, Mohammad Hadi},
  school       = {Amirkabir University of Technology (Tehran Polytechnic)},
  year         = {2026},
  type         = {Bachelor's Thesis},
  address      = {Tehran, Iran},
  url          = {https://mhadiniknam.github.io/blog/2026/has-rl-bachelor-thesis/}
}