Projects & Code

Working systems, open code.

Research software, teaching packages and applied AI projects. Every card says whether the work runs on a physical robot, in simulation, or as software only.

github.com/sh1755

Research software

Robot systems

Physical robot

Body-Pose and Hand-Gesture Control of a Franka Panda

sh1755/Franka_MediaPipe_BodyPose_HandGesture_Control

Problem. Teleoperating a collaborative arm usually needs a pendant or a joystick, and naive camera-to-joint mapping is unsafe.

Approach. MediaPipe tracks the operator's right wrist relative to the shoulder and streams small Cartesian velocity commands to MoveIt Servo, while a thumb–index pinch opens and closes the Franka Hand.

Calibration of a neutral pose before any motion is allowed
Left-hand dead-man gesture: lowering the hand commands zero velocity immediately
Speed capped at 0.03 m/s by default, with a deadband against tracking noise
MoveIt Servo handles collision checking, joint limits and singularities
Dry-run mode that sends no robot commands

ROS 2 HumbleMoveIt ServoMediaPipeOpenCVFranka Hand

Status: Experimental layer on a lab-validated ROS 2 / MoveIt hardware stack

Code

Simulated Franka Panda reaching for a red cube in PyBullet

Simulation + hardware tooling

Physics-Grounded Action Grammars

sh1755/PGAG-Robotic-Manipulation

Problem. Long-horizon manipulation fails part-way, and full replanning throws away work that is still valid.

Approach. Skills are typed atoms with preconditions and effects; an executor uses the dependency graph to repair only what a failure invalidated. Includes an evaluation harness, calibration tools and simulation scenes.

PythonPyBulletROS 2Isaac LabMuJoCopytest

Status: Active research prototype

Code Case study

Physical robot

LLM Human–Robot Interaction Experiments

sh1755/LLM_HRI

Problem. An LLM must never send a robot an action it cannot perform.

Approach. Experiment code behind the ICARM 2025 study: a GPT-4 prompt returns only whitelisted commands, which trigger pre-taught Franka tasks for coffee and tea preparation (cup, milk, sugar, stir).

PythonGPT-4Franka Desklibfranka

Status: Research code accompanying a published study

Code Case study

Research-led teaching

Teaching packages

Self-contained ROS 2 exercises written for students. Both run in simulation only and say so in their documentation.

Franka Panda over a green table in RViz during a MoveIt 2 pick-and-place exercise

Simulation only

ROS 2 Panda Pick-and-Place with MoveIt 2

sh1755/ROS2-Panda-Pick-and-Place

Problem. Students need a complete, readable example of motion planning, planning scenes and gripper control.

Approach. A teaching package that adds a table and cube to the MoveIt planning scene, plans pose-constrained motions, commands the gripper and attaches the object through a full pick-and-place sequence in RViz.

ROS 2 HumbleMoveIt 2RViz 2Pythonament_python

Status: Teaching resource

Code

Simulation only

ROS 2 Forward & Inverse Kinematics for a 7-DoF Arm

sh1755/ROS2-Forward-Inverse-Kinematics-repository

Problem. FK and IK are usually taught on paper; students rarely see a redundant arm respond to their own maths.

Approach. FK by chained transforms and damped-least-squares Jacobian IK, published as joint states and animated in RViz 2 through approach, grasp, lift, transfer and place stages.

ROS 2 HumblePythonNumPyRViz 2TF

Status: Teaching resource

Code

Applied AI

Retrieval and language systems

NLP and retrieval-augmented generation projects outside robotics, built with local models and vector search.

Software / AI

ConstructRAG AI

sh1755/constructrag-ai

Problem. Comparing construction products across suppliers means searching unstructured catalogues and data sheets.

Approach. A retrieval-augmented generation pipeline — HuggingFace embeddings, a FAISS index and a local Ollama LLM — behind a Streamlit interface for semantic search, supplier comparison and recommendations.

LangChainFAISSHuggingFaceOllamaStreamlit

Status: Applied AI project

Code

Software / AI

ConstructAgent NLP

sh1755/constructagent-nlp

Problem. Natural-language product queries mix intent, constraints and entities that keyword search misses.

Approach. An NLP pipeline for intent detection and entity extraction feeds sentence-transformer retrieval over FAISS and an agentic step that ranks products with explanations.

Sentence TransformersFAISSNLPAgentic AIStreamlit

Status: Applied AI project

Code

All repositories

Snapshot of public repositories, October 2026. Empty or placeholder repositories are not listed.

Repository	Type	Main tools	Link
Body-Pose and Hand-Gesture Control of a Franka Panda Franka_MediaPipe_BodyPose_HandGesture_Control	Physical robot	ROS 2 Humble, MoveIt Servo, MediaPipe	GitHub repository Franka_MediaPipe_BodyPose_HandGesture_Control (opens in a new tab)
Physics-Grounded Action Grammars PGAG-Robotic-Manipulation	Simulation + hardware tooling	Python, PyBullet, ROS 2	GitHub repository PGAG-Robotic-Manipulation (opens in a new tab)
LLM Human–Robot Interaction Experiments LLM_HRI	Physical robot	Python, GPT-4, Franka Desk	GitHub repository LLM_HRI (opens in a new tab)
ROS 2 Panda Pick-and-Place with MoveIt 2 ROS2-Panda-Pick-and-Place	Simulation only	ROS 2 Humble, MoveIt 2, RViz 2	GitHub repository ROS2-Panda-Pick-and-Place (opens in a new tab)
ROS 2 Forward & Inverse Kinematics for a 7-DoF Arm ROS2-Forward-Inverse-Kinematics-repository	Simulation only	ROS 2 Humble, Python, NumPy	GitHub repository ROS2-Forward-Inverse-Kinematics-repository (opens in a new tab)
ConstructRAG AI constructrag-ai	Software / AI	LangChain, FAISS, HuggingFace	GitHub repository constructrag-ai (opens in a new tab)
ConstructAgent NLP constructagent-nlp	Software / AI	Sentence Transformers, FAISS, NLP	GitHub repository constructagent-nlp (opens in a new tab)