-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reboot pattern "...Rebooting with /sbin/kexec -e..." is missing from syslog after fast-reboot #11993
Comments
@vaibhavhd , what is your plan to fix it ? |
I missed this issue somehow. Do you have any failure logs that you can share? |
It happens occasionally, we saw that the reset happens before the kexec LOG message is written to LOG. If we put sleep of 2 seconds between the LOG message write and the reset, it always get into the LOG but it will impact the scheduling so we will not do it. (it was done only for testing) |
@vaibhavhd - Any update here ? |
@vaibhavhd - Can you please update regarding this issue ? |
Based on my analysis: The issue is due to some services not being killed gracefully, and gets to a hung state when database container go down. Identified services are
When I add explicit service stop for these services, no more error logs appear, and KEXEC string appears in the logs. |
We don't need a new fix in 202205 for this issue. The real reason is that we missed to cherrypick a change to 202205: sonic-net/sonic-utilities#2133 and I have added labels for cherrypicking the changes to 202205. That should fix this issue. With these changes the shutdown part is handled by systemd and happens in terms of systemd service dependencies, which takes care of unexpected errors or hung services in shutdown path.
|
Description
Reboot pattern message is not printed to syslog in fast-reboot - this is not consistent. Running multiple times and checking reboot cause history will show few fast-reboot were done but in the syslog(s) it is not printed each time.
Steps to reproduce the issue:
Describe the results you received:
As mentioned it is missing from syslog in some cases.
Describe the results you expected:
This message should be printed anytime fast-reboot is performed.
Output of
show version
:Output of
show techsupport
:Additional information you deem important (e.g. issue happens only occasionally):
The text was updated successfully, but these errors were encountered: