New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build process does not produce consistent results [CORE5548] #5816
Comments
Commented by: Bernhard M. Wiedemann (bmwiedemann) fix for first issue is #92 |
Commented by: @AlexPeshkoff I hardly understand how can second issue be fixed. Having creation datetime in the database header is an old useful feature and there is no idea how can we avoid this irreproducible bytes in the build |
Commented by: Bernhard M. Wiedemann (bmwiedemann) There is one nicely designed way to do this, using the SOURCE_DATE_EPOCH environment variable as the 'current' time int, when we have it set during package build. Some programs like TeX that were afraid, that this might accidentally trigger during normal operation, added some extra variable (e.g. SOURCE_DATE_EPOCH_TEX_PRIMITIVES there) to be sure. Can you give me a hint on where in the code that creation datetime is set? 00000000 01 00 00 00 12 00 00 00 00 00 00 00 00 00 00 00 so the change is spread out over 8 bytes |
Commented by: @mkubecek I believe it's this line in PAG_format_header():
See NoThrowTimeStamp::getCurrentTimeStamp() to get idea how to convert time values. |
Commented by: Bernhard M. Wiedemann (bmwiedemann) wrote #93 yet, there are still diffs left in .fdb files. Usually just a single bit e.g. for help.fdb I got 000bbfd0 1a 00 10 00 00 00 00 00 01 00 00 00 1a 00 00 00 maybe something that depends on timing of individual operations? |
Commented by: @hvlad 1. Why this ticket classified as a bug ? This is feature request, imo 2. Why "reproducible" database files is required ? Why "reproducible" binaries is not enough ? |
Commented by: Bernhard M. Wiedemann (bmwiedemann) 1) the separating line between bugs and features can be vague. E.g. when working on reproducible builds I found https://savannah.gnu.org/support/index.php?109234 and https://bugs.launchpad.net/intltool/+bug/1687644 which are pretty definitely bugs. Feel free to change the classification. 2) for some reason, the help.fdb and sample .fdb files get built and shipped as part of our openSUSE packages and we try to make all our packages build reproducibly. |
Commented by: @hvlad 1. Race conditions you found in that projects during build could be real bugs, yes, but i see no relation with current ticket, sorry. 2. The only database file required to run Firebird v3 is security3.fdb. |
Modified by: @hvladissuetype: Bug [ 1 ] => Improvement [ 4 ] |
Commented by: @hvlad As for --------------------------------------------- e.g. for help.fdb I got 000bbfd0 1a 00 10 00 00 00 00 00 01 00 00 00 1a 00 00 00 maybe something that depends on timing of individual operations? the difference is in pag::pag_generation field, it changes every time page is written to disk. |
Commented by: @mkubecek I must admit I'm not really sure what is help.fdb used for. As for employee.fdb, while firebird can run without it, it's often used for examples (e.g. in documentation) so I believe completely omitting it would be a mistake (currently it's in firebird-examples subpackage). If it really came to the worst, it might by probably generated from post-install script but I would prefer to avoid such dirty tricks. While I can see value in reproducible build results, I see it rather as "nice to have" than "must have" kind of feature. |
Commented by: @AlexPeshkoff Just FYI - help.fdb is help file for QLI. If you type in QLI 'help <command>' help.fdb is used. |
Commented by: @asfernandes I would not suprise if a next person asks "what is QLI?" :) |
Commented by: Sean Leyne (seanleyne) Edited Summary for better context |
Modified by: Sean Leyne (seanleyne)summary: firebird does not build reproducibly => Build process does not produce consistent results |
Commented by: @hvlad Sean, build process does produced consistent results. Original summary was better and more clear, imo. |
Commented by: Bernhard M. Wiedemann (bmwiedemann) I just got an idea for a nicer way to be able to build firebird twice with identical results: in the end of the build we can run something like and that fdb-strip program can then streamline hdr_creation_date and pag::pag_generation field values. IMHO, it would make sense to maintain such a tool as part of firebird, because the data structure definitions and functions are available and guaranteed to be current. |
Commented by: @hvlad Bernhard, if you absolutely need this feature, i see no problem with additional build-time utility. Do you need any help to implement it ? |
Commented by: Bernhard M. Wiedemann (bmwiedemann) I'm busy fixing other packages and do not know the firebird codebase, so I would not get to it soon... |
Commented by: @hvlad Bernhard, i'll look at it a bit later |
Commented by: Damyan Ivanov (dam) My (failed) attempt at stripping random stuff from .fdb files is at https://anonscm.debian.org/cgit/pkg-firebird/3.0.git/tree/debian/fdb-r15y-prune.cpp It does two things: It seems to work (used in the Debian package builds) and the resulting employee.fdb seems correct (isql -x does not complain, tested during the package build, on all Debian architectures). However, there are still varying bits - perhaps index trees or something like that. Just FYI, to help you not reinvent the wheel, maybe. |
Commented by: @hvlad Damyan, thanks for your efforts. Could you show few samples of that varying bits ? To check database validity use gfix -validate -full |
Commented by: Damyan Ivanov (dam) Hello, Vlad, sorry about the delay in responding. All the data about reproducibility testing of firebird3.0 is at https://tests.reproducible-builds.org/debian/rb-pkg/firebird3.0.html . Just pick architecture/version from the left and follow the "differences" link. Here are a few samples of differences, as detected by the reproducible-builds effort (long URLs). Comparisons are made after running the fdb-r15y-prune tool. Validation using gfix -validate -full passes. Sometimes there are differences in the security database too: |
Commented by: Damyan Ivanov (dam) Re-reading fdb-r15y-prune.cpp, it also resets pag_generation to '1' and pag_reserved to '0', so the remaining differences are elsewhere. |
Commented by: @hvlad Damyan, i see that some pages differs in a swept state. I think it is worth to add a sweep step before pruning. |
Commented by: Damyan Ivanov (dam) Here's a diff after adding a sweep before pruning: --- employee-1.fdb |
Commented by: Damyan Ivanov (dam) I tried to find out what actual data makes the differences. Looking at the first difference: --- employee-1.fdb This is a data page, containing record data. The record is spread from 0x24d9e8 to 0x24da80. It is flagged as stream BLOB (flags 0x30), but appears to contain ARRAY data. The array has 5 (6?) elements of type VARCHAR(15) - "English\n", "French\n", "Spanish\n" and "\n". The storage of the values takes 17 bytes - 2 for the length of the VARCHAR and 15 for the actual data. The actual data is padded so that it takes 15 bytes, and this padding is where the difference is. I'll see if I can make fdb-r15y-prune to wipe these unused padding bytes, but that would require some extra intelligence like linking record data to the table structure. Currently it is relatively simple examining pages one by one and wiping unused ranges based only on the page header. |
Commented by: Damyan Ivanov (dam) The more I delve in the details, the more I think that making the engine produce byte-for-byte equal results when given equal input is not the way to go. There are too many moving parts in the engine, and there is more than one way to describe the same data in the database file. Instead, I'd rather deferring the creation of the security database and all others (employee, qli's, help) to installation time. This is easily done for security3.fdb, which is created using simple SQL script. employee.fdb is mostly populated with SQL, but array and blob data is added via empbuild.e. Firebird 3 already supports feeding text blobs via SQL, but arrays are still a problem (See CORE710). My solution for the Debian package will be to 1) remove employee.fdb; and 2) create security3.fdb upon package installation. This way no .fdb files are part of the packages, making them deterministic. (qli is not part of the Debian packages since 2011) I am not happy to drop employee.fdb from the packages since it is a nice sample database using many features. I think that the ideal fix would be to rework employee.fdb a bit to avoid usage of arrays, and create it entirely out of plain SQL. |
Commented by: @hvlad You could try to use ServerMode = Classic while build databases. |
Commented by: @dyemanov You may package employee.fbk instead of .fdb, its layout should be more consistent. Those who need examples will restore this backup themselves. |
Commented by: @AlexPeshkoff Or appropriate gbak command to restore database may be added to the package install script. What about security.db. Presence of creating it SQL script is just a legacy way to install legacy plugin. SRP plugin creates all required data structures itself on first run, and this is recommended way to go for new plugins. I.e. if you do not use legacy auth - security.db may be created empty at install time. |
Commented by: Damyan Ivanov (dam) Thanks for the replies, gbak may be the way to go (backup during build, restore during install), but still the backups are different every time: $ gbak -user sysdba -b emp.fdb emp.fbk1 $ diffoscope emp.fbk1 emp.fbk2 Would you accept a patch replacing arrays in eployee.fdb with detail-tables (making it re-buildable via SQL)? I'd rather not spend more time on binary formats. |
Commented by: @AlexPeshkoff I dislike such patch. Arrays are not actively supported feature but they are not even deprecated. What about the difference you see - if it's the only difference it's not too big problem. gbak stores current datetime and certainly it changes. May be adding a switch to it making that time to be a day when 'official' set of packages is built will be a good fix? |
Commented by: Damyan Ivanov (dam) Alex, Yes, I would try such a switch if it existed. But maybe it is not necessary after all. I discovered the 'faketime' utility which can fool programs (via a library loaded using LD_PRELOAD) about the current time. I'll report back if it works with gbak. |
Commented by: @AlexPeshkoff Damyan, re > I would try such a switch if it existed I wanted to say that if the only problem will stay missing switch it's not too hard to add it. |
Commented by: Damyan Ivanov (dam) It seems the date is not the only source of differences. See https://tests.reproducible-builds.org/debian/dbdtxt/unstable/amd64/firebird3.0_3.0.2.32703.ds4-8.diffoscope.txt The major difference here is the contents of the backup file. This is probably because the data in the database is not guaranteed to have any particular order. I doubt this is solvable or even worth solving. To me it seems that the only solution to this is to use plain SQL in the package. Today this means not filling the array columns, but the other "solution" is to not have employees example database at all. |
Submitted by: Bernhard M. Wiedemann (bmwiedemann)
Votes: 2
When working on reproducible builds for openSUSE,
I found that the firebird package produced different results on every build.
The differences come from two sources:
1.
g++ orders functions in http://libEngine12.so and fbintl
depending on random order of files in the build system's filesystem.
2.
Additionally, help.fdb and other sample .fdb files contain some bytes that differ between builds.
See https://reproducible-builds.org/ for why this matters.
Commits: 3278b68
The text was updated successfully, but these errors were encountered: