New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gbak restore with large number of small blobs very slow using Linux Classic [CORE5653] #5919
Comments
Modified by: @romansimakovassignee: Roman Simakov [ roman-simakov ] |
Modified by: @romansimakovstatus: Open [ 1 ] => Resolved [ 5 ] resolution: Fixed [ 1 ] Fix Version: 3.0.3 [ 10810 ] Fix Version: 4.0 Beta 1 [ 10750 ] |
Commented by: Sean Leyne (seanleyne) Roman, Did you meant to describe the issue more as "gbak restore with large number of small blobs very slow using Classic"? Also, what blob size qualifies as "small"? and how many items in a "huge"? ;-) Finally, would the lack of TCP_NODELAY setting have any impact (performance or otherwise) on non-gbak connections/operations? |
Commented by: @romansimakov I did not investigage statistics of blobs. Its enough for me to see offCPU waits in profiler tools and a lot of blob functions calls. The key point is that there are a lot of TCP packats walking in the network. > Finally, would the lack of TCP_NODELAY setting have any impact (performance or otherwise) on non-gbak connections/operations? It's not a lack. Its respect of config option which is now missed. |
Commented by: Sean Leyne (seanleyne) Roman, Do you have a problem with me changing the issue summary/description? |
Commented by: @romansimakov You want to change summary? If so feel free. |
Modified by: Sean Leyne (seanleyne)description: The problem is that time or restore more then 100x via classic comparing to super. The problem appears if I restore an backup made by gbak via connection to classic server with its own listner. If I run super, superclassic or direct connection without network layer all work good. There is no load neither disk not CPU. I found that TCP_NODELAY socket option is not set for classic listen socket. It set only in Super modes. However it must be set since severl socket options are interited from listen socket. It's described for example here: Moving setting this option out of a condition solves the problem. => The problem is that time or restore more than 100x via classic compared to super. The problem appears if I restore an backup made by gbak via connection to classic server with its own listner. If I run super, superclassic or direct connection without network layer all work good. There is no load neither disk not CPU. I found that TCP_NODELAY socket option is not set for classic listen socket. It set only in Super modes. However it must be set since severl socket options are interited from listen socket. It's described for example here: Moving setting this option out of a condition solves the problem. summary: Very slow restore from gbak containing a huge small blobs via classic => gbak restore with large number of small blobs very slow using Linux Classic |
Commented by: @romansimakov Small gstat -r resultes for the biggest tables JOURNALS (155) JOURNAL_DETAILS (154) |
Modified by: @pavel-zotovstatus: Resolved [ 5 ] => Resolved [ 5 ] QA Status: No test => Cannot be tested Test Details: One can not compare Classic and Super during the same test run in current fbtest implementation. Test Specifics: [Architecture (SS/CS) specific, Platform (Windows/Linux) specific] |
Modified by: @pavel-zotovstatus: Resolved [ 5 ] => Closed [ 6 ] |
Submitted by: @romansimakov
The problem is that time or restore more than 100x via classic compared to super. The problem appears if I restore an backup made by gbak via connection to classic server with its own listner. If I run super, superclassic or direct connection without network layer all work good. There is no load neither disk not CPU.
I found that TCP_NODELAY socket option is not set for classic listen socket. It set only in Super modes. However it must be set since severl socket options are interited from listen socket. It's described for example here:
https://notes.shichao.io/unp/ch7/#checking-if-an-option-is-supported-and-obtaining-the-default
Moving setting this option out of a condition solves the problem.
On Windows it works fine without changing. I'll prepare a patch only for Linux right now.
Commits: 9fdbd21 b31e373
====== Test Details ======
One can not compare Classic and Super during the same test run in current fbtest implementation.
The text was updated successfully, but these errors were encountered: