PPO won't learn to grasp with the Franka arm

[amsoufi]

Hi,

I am trying to train PPO to do a simple pick and place task with Franka Emika Panda. I have tried multiple setups with discrete actions for the gripper close/open (e.g., 2 cm per each step), increasing the controller's kp to 350, action repeat, etc.

Also, I noticed that in examples/manipulation/grasp_eval.py the grasp and lift is done by the hard coded function env.grasp_and_lift_demo(), so the agent only learns to get close to the object.

I was wondering if I'm doing something wrong or is this supposed to be harder compared to mujoco_playground envs for example? Any suggestions?

Thank you.