New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allocate data pages in group of sequential ordered pages (extents) [CORE4556] #4874
Comments
Modified by: @hvladassignee: Vlad Khorsun [ hvlad ] |
Commented by: Sean Leyne (seanleyne) Vlad, Isn't it a reality that other engines allocate an "extent" as the minimum table size? Why would we add extra overhead for checking if a page has already been assigned/used, why not use an 'extent' each time? |
Commented by: @hvlad Sean, > Isn't it a reality that other engines allocate an "extent" as the minimum table size? > Why would we add extra overhead for checking if a page has already been assigned/used, why not use an 'extent' each time? |
Commented by: Sean Leyne (seanleyne) "If we have a lot of small tables we wont waste space allocating whole extent for them" But the "minimal space is good' rational overlooks: It would seem to be simpler to have the engine always allocate in "extents" and allow for the size of the extents to be defined as a database parameter [default = 8] to manage the size of an empty database (this would also address size issues for ultra small deployment -- ie. mobile), rather than having to 'count' the number of pages to decide if allocation will be in pages or extents. Q: What would happen if a data had "master" and "details" tables where the data ratio between the tables was 1 to 4? Based on the description, after sometime had passed, the resulting disk page usage for the tables could be: Am I correct? (if so, then to describe the extents as aligned on 8 page boundary is incorrect) Q: If pages 21-28 were released from the Details table, would those tables be released for use a single pages or only full extents? |
Commented by: Sean Leyne (seanleyne) Small changes for readability |
Modified by: Sean Leyne (seanleyne)description: DIscussed in fb-devel (see "RFC: Data page allocation algorithm" at 26 Dec 2013)
Such group of pages is often called "extent". I offer to change page allocation algorithm for tables as following:
OS-level prefetch to work more efficient (it will read not just a bunch of pages of random objects but => DIscussed in fb-devel (see "RFC: Data page allocation algorithm" at 26 Dec 2013) I offer to allocate data pages not one-by-one (as currently) but in group of sequential ordered pages. Such group of pages is often called "extent". I offer to change page allocation algorithm for tables as follows: This new algorithm will: |
Commented by: @hvlad > But the "minimal space is good' rational overlooks: 1. Consider following: 2. Yes, there will be fragmentation of first 8 pages. We can live with it. > It would seem to be simpler to have the engine always allocate in "extents" and allow for the size of the extents to be defined as a database parameter [default = 8] to manage the size of an empty database (this would also address size issues for ultra small deployment -- ie. mobile), It *would seem to be simpler* only until you start to implement it. Also, we trying to avoid new configuration parameters when not needed. > rather than having to 'count' the number of pages to decide if allocation will be in pages or extents. > Q: What would happen if a data had "master" and "details" tables where the data ratio between the tables was 1 to 4? Based on the description, You are wrong when calculate extent boundary as 12-19, 21-28 and 30-37 :) It could be 8-15, 16-24, etc. > Q: If pages 21-28 were released from the Details table, would those tables be released for use a single pages or only full extents? PS why do you prefer to discuss all this stuff here (at tracker) and 10 months after my initial post in fb-devel ? |
Modified by: @pcisarstatus: Resolved [ 5 ] => Closed [ 6 ] |
Submitted by: @hvlad
Votes: 1
DIscussed in fb-devel (see "RFC: Data page allocation algorithm" at 26 Dec 2013)
I offer to allocate data pages not one-by-one (as currently) but in group of sequential ordered pages. Such group of pages is often called "extent".
I offer to change page allocation algorithm for tables as follows:
- if table is empty or small (have no full extent allocated) then data pages is allocated one-by-one (as currently)
- if table already have at least one full extent allocated, next request for new page will allocate the whole extent of pages
- size of extent is 8 pages
- every such extent is aligned at 8-pages boundary
This new algorithm will:
- reduce page-level fragmentation (all pages in extent are adjacent),
- allow OS-level prefetch to work more efficient (it will read not just a bunch of pages of random objects but pages related to the same table), and
- allow in the future to read and write in a large chunks making IO more efficient.
Commits: 8ba9d33 FirebirdSQL/fbt-repository@6e6dbbe
The text was updated successfully, but these errors were encountered: