Issue Details (XML | Word | Printable)

Key: CORE-4384
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Dmitry Yemanov
Reporter: Kovalenko Dmitry
Votes: 0
Watchers: 3
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Problems when a table grows beyond 65535 pointer pages

Created: 04/Apr/14 04:17 PM   Updated: 23/Sep/15 11:21 AM
Component/s: Engine
Affects Version/s: 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.5.0, 2.1.4, 2.5.1, 2.1.5, 2.5.2, 2.1.5 Update 1, 2.5.2 Update 1, 3.0 Alpha 1, 3.0 Alpha 2
Fix Version/s: 2.5.4, 3.0 Beta 2

File Attachments: 1. Zip Archive inserts-batch-and-sql.zip (1 kB)
2. Zip Archive miscelan-logs.zip (112 kB)


QA Status: Done successfully
Test Details: There is no .fbt file for this ticket, but I've done such test and result was: table CAN be of such size that more than 64K PP will be related to it.


 Description  « Hide
The USHORT pp_sequence number wraps and it may lead to unexpected issues like an infinite loop inside extend_relation() or locate_space(). This issue manifests itself when a table (including garbage) gets bigger than 256GB (for 4KB page) ... 4TB (for 16KB page) and remains even after a garbage collection pass.

 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Pavel Zotov added a comment - 30/Jan/15 03:32 PM - edited
http://svn.code.sf.net/p/firebird/code/firebird/trunk/ChangeLog:
> 2014-12-29 17:40 dimitr <...>
> This commit is expected to fix CORE-4384: Problems when a table grows beyond 65535 pointer pages. Testing would be appreciated.

I've done somewhat like test about ability to create and fill table with more than 65K pointer pages.
Short conclusion: we CAN create such table, at least on database with page_size = 4K.

Environment: OS = Windows XP, FB = WI-T6.3.0.31543, SuperServer; database = page_4k, fw_OFF

firebird.conf:
==========

DefaultDbCachePages = 64K
LockHashSlots = 22111
TempCacheLimit = 2147483647

RemoteServicePort = 3333

MaxUnflushedWrites = -1
MaxUnflushedWriteTime = -1

ExternalFileAccess = Restrict C:\1INSTALL\FBTEST
MaxUserTraceLogSize = 99999

FileSystemCacheThreshold = 65536K
AuthServer = Legacy_Auth,Srp
AuthClient = Legacy_Auth,Srp,Win_Sspi
WireCrypt = Disabled
UserManager = Legacy_UserManager
BugcheckAbort = 1
======

Two tables are in this database:

-- main:
recreate table t(id bigint, s01 varchar(8), s02 varchar(8));

-- log with info for later analyzing of inserts speed:
recreate table ins_log(
    dts_beg timestamp, dts_end timestamp,
    elapsed_ms int computed by(datediff(millisecond from dts_beg to dts_end)),
   gen_after_ins bigint, pp_after_ins int
);
commit;

recreate sequence g;
commit;

Test was run during several days, final database size = ~305Gb.

Result:
======

Database "E:\DATA\HUGE_CORE_4384.FDB"
Database header page information:
Flags 0
Generation 14032
System Change Number 0
Page size 4096
ODS version 12.0
Oldest transaction 1149
Oldest active 11496
Oldest snapshot 11496
Next transaction 11497
Sequence number 0
Next attachment ID 5741
Implementation HW=Intel/i386 little-endian OS=Windows CC=MSVC
Shadow count 0
Page buffers 0
Next header page 0
Database dialect 3
Creation date Jan 18, 2015 19:02:14
Attributes

    Variable header data:
*END*


Database file sequence:
File E:\DATA\HUGE_CORE_4384.FDB is the only file

Analyzing database pages ...
T (128)
    Primary pointer page: 211, Index root page: 212
    Pointer pages: 127855, data page slots: 103306800
    Data pages: 103306800, average fill: 67%
    Primary pages: 103306800, secondary pages: 0, swept pages: 0
    Empty pages: 5, full pages: 103306794
    Fill distribution:
0 - 19% = 5
20 - 39% = 0
40 - 59% = 1
60 - 79% = 103306794
80 - 99% = 0



I decided to check trivial COUNT operation on this table and create INDEX on it.

Results are in attach "miscelan-logs.zip".