Issue Details (XML | Word | Printable)

Key: CORE-1476
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Alexander Peshkov
Reporter: Alexander Peshkov
Votes: 1
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Forced writes do not work on Linux

Created: 24/Sep/07 08:25 AM   Updated: 22/Sep/11 12:38 AM
Component/s: Engine
Affects Version/s: 1.0.3, 1.5.2, 1.5.3, 2.0.0, 1.5.4, 2.0.1, 2.1 Alpha 1, 2.1 Beta 1, 2.0.2, 2.0.3
Fix Version/s: 2.1 Beta 2, 2.0.4

Time Tracking:
Not Specified

Environment: Linux, possibly other POSIX platforms too


 Description  « Hide
To make disk writes return only after data is actually placed on disk (not only to OS cache), fcntl() system call is used. It was supposed to set flag O_SYNC, making cache write-transparent. But, according to Linux documentation:
"On Linux this command (i.e. fcntl(fd, F_SETFL, flags)) can only change the O_APPEND, O_ASYNC, O_DIRECT, O_NOATIME, and O_NONBLOCK flags."
Review of kernel source code (2.6.19) and direct speed tests with various open() / fcntl() flags showed that flag O_SYNC is really not affected on Linux. Attempt to set it does not make error to be returned by fcntl(). Moreover, attempt to check current flag's state shows that O_SYNC is set, but actually write() does not send data to disk to the moment of system call return.

For Firebird this all leads to ability to have database broken even with FW=ON. Which is actually known to happen on Linux.

 All   Comments   Work Log   Change History   Version Control   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Dmitry Yemanov added a comment - 19/Mar/07 01:57 PM
Is it possible to backport the fix into 2.0.4?

Alexander Peshkov added a comment - 19/Mar/07 02:51 PM
It's much desired, but we have one problem.
Currently it makes sequential insert of records into the table approx 8 times slower (tested with fb 2.1 with ODS 11.0). PIO_init_data() makes file growth much faster. Now I need to do some more measurements with 2.0.Release and after it decide, how to backport.

kdv added a comment - 27/Sep/07 05:38 AM
Excuse me, but description of this issue does not specify is this related only to kernel 2.6, or to 2.4 and other kernel versions also.
Do you have any information about this?

Alexander Peshkov added a comment - 27/Sep/07 06:07 AM
I have reviewed kernel sources only as a last chance. I've hoped that what I see in documentation and what I get with firebird is a kind of mistake, my mistake and doc's writer mistake. See yourself - system call, NOT doing what one asks it to do (i.e. set flag O_SYNC), returns success instead of error! And this awful behavior is documented! Yes, I know that 'documented bug becomes feature', but...

I'm pretty sure taht all older linux versions do have same behavior - or what we have now is serious regression in 2.6, which could not stay unnoticed for years. Moreover, man is independent from kernel, installing newer kernel does not change man pages regarding system calls. And what we have is documented in it.

Alexander Peshkov added a comment - 27/Sep/07 06:11 AM - edited
There is interesting article in linux kernel tracker:
http://bugzilla.kernel.org/show_bug.cgi?id=5994

The test provided in that article shows that a lot of unixes (including FreeBSD, Solaris and HPUX) are OK from this POV. Certainly, it will be useful to test that operating systems better.

Alexander Peshkov added a comment - 26/Feb/08 08:46 AM
Backported without serious performance optimization.
No idea how to do it safely without ODS 11.1 from firebird 2.1.