New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
High load in touch thread [CORE5401] #5674
Comments
Modified by: @AlexPeshkoffreporter: Alexander Peshkov [ alexpeshkoff ] => David Hollings [ dholling ] assignee: Alexander Peshkov [ alexpeshkoff ] |
Modified by: @AlexPeshkoffstatus: Open [ 1 ] => Resolved [ 5 ] resolution: Fixed [ 1 ] Fix Version: 2.5.7 [ 10770 ] |
Commented by: Sean Leyne (seanleyne) Edited to provide original reporter name and for readability. |
Modified by: Sean Leyne (seanleyne)description: A few days ago I posted on Firebird Support "Firebird Recurring Hourly I've look into this a little bit more and notice that the issue seems to lie Each thread calls sem_timedwait [while (!semaphore->tryEnter(delay))] with if (!m_base->touch_time || m_base->touch_time < now) Since touch_time is an integer value, current seconds will be equal to it To resolve shouldn't the comparison check for equality: if (!m_base->touch_time || m_base->touch_time <= now) I'm also curious as to why each thread is required to wake at the exact same => David Hollings posted on Firebird Support "Firebird Recurring Hourly Load". The gist of the post is that he noticed that every hour all Firebird processes would wake up for a second or two causing high load, context switches etc - This using Firebird 2.5.x on Centos 6.x/7.x. I've look into this a little bit more and notice that the issue seems to lie in the trace touch thread - specifically TouchThreadFunc in jrd/trace/TraceConfigStorage.cpp. Each thread calls sem_timedwait [while (!semaphore->tryEnter(delay))] with the next touch time in seconds. The thread then blocks till seconds after epoch + current milliseconds. When it returns it compares to the touch time: Since touch_time is an integer value, current seconds will be equal to it for up to a full second. This results in a delay of zero being calculated and sem_trywait being called repeatedly for up to a second per thread until current seconds > touch_time. To resolve shouldn't the comparison check for equality: I'm also curious as to why each thread is required to wake at the exact same time - would it not reduce contention if each thread blocked for a full hour? |
Modified by: @pavel-zotovstatus: Resolved [ 5 ] => Resolved [ 5 ] QA Status: No test => Cannot be tested |
Modified by: @pavel-zotovstatus: Resolved [ 5 ] => Closed [ 6 ] |
Submitted by: David Hollings (dholling)
David Hollings posted on Firebird Support "Firebird Recurring Hourly Load".
The gist of the post is that he noticed that every hour all Firebird processes would wake up for a second or two causing high load, context switches etc - This using Firebird 2.5.x on Centos 6.x/7.x.
I've look into this a little bit more and notice that the issue seems to lie in the trace touch thread - specifically TouchThreadFunc in jrd/trace/TraceConfigStorage.cpp.
(Please note that I'm not a c++ developer, so there's a good chance that I'm missing something)
Each thread calls sem_timedwait [while (!semaphore->tryEnter(delay))] with the next touch time in seconds. The thread then blocks till seconds after epoch + current milliseconds.
When it returns it compares to the touch time:
if (!m_base->touch_time || m_base->touch_time < now)
Since touch_time is an integer value, current seconds will be equal to it for up to a full second. This results in a delay of zero being calculated and sem_trywait being called repeatedly for up to a second per thread until current seconds > touch_time.
To resolve shouldn't the comparison check for equality:
if (!m_base->touch_time || m_base->touch_time <= now)
I'm also curious as to why each thread is required to wake at the exact same time - would it not reduce contention if each thread blocked for a full hour?
Finally, for my own curiosity, what is the reason for touching the trace file hourly?
Commits: 59cf92b
The text was updated successfully, but these errors were encountered: