Can Simumatik be used as a simulator for reinforcement learning for industrial robots?

I discovered Simumatik for the first time today. I’m looking for a simulator for reinforcement learning for industrial robots. I think Simumatik is the best simulator for that right now. It would be nice if Simumatik used MuJoCo as the physics engine. Specifically, I would like to realize cable routing by multi-axis robots through reinforcement learning, and apply the motions obtained from this to multi-axis robots in the real world. Simumatik can use Python, so I guess reinforcement learning with Pytorch is possible, is that possible?

Hi! And welcome to Simumatik! :slight_smile:

Interesting application!

In this case, I recommend creating a Python program that controls and reads values from your Simumatik Model and then incorporates the Pytorch functionalities. In the academy course “Using Python to program a production line” an example of a Python script controlling a Simumatik can be found.

As of today, Simumatik uses the Bullet physics engine. But there are plans to support multiple physics engines.

Please let us know how it goes!

I would like to be able to load multiple identical units (as many as the number of CPU cores) into a single workspace and have them work in parallel. By using these units, I hope to perform reinforcement learning more efficiently than using a single unit.

Attached below is the reference video.
" raining a CUDA TDS Ant using C++ ARS Linear policy."

By the way, it seems that it is very difficult to perform parallel processing of physics calculations with CUDA. CUDA programs seem to be extremely complicated.


There is a possibility to load multiple components to a system and run in the workspace. Just keep track of the monitoring values physics load and behavior load so they don’t reach above 100%

The workspace will not utilize more CPU cores depending on size of the system, a single system takes as many cores as a big one, but the utilization of the cores differ. So for the sake of Simumatik you don’t have to match the number of cores in CPU with running components.