Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

decreasing the cycle amplitude after each period in cyclic lr #783

Closed
Tracked by #1151 ...
minouei-kl opened this issue Jan 11, 2021 · 1 comment
Closed
Tracked by #1151 ...

decreasing the cycle amplitude after each period in cyclic lr #783

minouei-kl opened this issue Jan 11, 2021 · 1 comment
Assignees
Labels

Comments

@minouei-kl
Copy link

currently, the implementation of cyclic lr does not support decreasing policies which could be really useful.
I wanted to have one cycle per epoch and decrease the cycle amplitude gradually so here is what I did:
set cyclic_times equal to my number of epochs that is 12
define a gamma parameter in CyclicLrUpdaterHook and add these lines in the get_lr method:

                if start_iter==0:
                    end_ratio=end_ratio * self.gamma**runner.epoch
                else:
                    start_ratio=start_ratio * self.gamma**runner.epoch

the logic here is since there are two phases that one start from zero to max and the other start from the max to the min, I had to decrease the end_ratio in phase one and start_ratio in phase two.
this is working now and after each epoch, the cycle amplitude drops depends on the gamma.
I'm sure that this is not the best way to implement this so I hope this feature gets implemented in the right way.

@minouei-kl minouei-kl changed the title decreases the cycle amplitude after each period in cyclic lr decreasing the cycle amplitude after each period in cyclic lr Jan 11, 2021
@zhouzaida
Copy link
Collaborator

resolved by #1655

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants