Random simulator crash during training

[sillycornvalley]

Has anyone completed the train for DRL following cables in the LandscapeMountains environment successfully? I'm currently trying to run it and I've been facing some issues like random simulator crashes etc. The reward generated after like 10000 steps still seems to be very low and I'm not sure if the training is actually happening.
Is there someone who could help me with this?

[msb336]

This is a race condition bug in UpdateableObject::update() with low repro. There is a PR currently being reviewed that solves this problem:
#1970

If you use this branch, you should be able to run long training sessions without crashing for the same reason.

[sillycornvalley]

Okay, thanks. I'll try to use this branch.
By the way, do you know what the maximum generated reward after training would be like? Just a rough approximation, to make sure my algorithm is working as intended. Because even after 10000 steps, the generated reward seems to be very low.

[msb336]

@sillycornvalley
That is very dependent on your environment and reward function. If you are providing sparse rewards (e.g. only rewarding at the end of an episode), especially if your episodes are very long and often do not result in the desired behavior, it can take a very long time to converge on something useful.

Here is an intro level article on how to properly shape reward functions in an RL training simulation