Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sweep doesn't increase OIT [CORE6188] #6433

Closed
firebird-automations opened this issue Nov 15, 2019 · 7 comments
Closed

Sweep doesn't increase OIT [CORE6188] #6433

firebird-automations opened this issue Nov 15, 2019 · 7 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: Lucas Schatz (arvanus)

Hello.

I'm running Sweep manually and for some reason Sweep doesn't change OIT
Details:
It's running in a offline server (backup server)
Firebird process were stopped, no active user.
It's running sweep really fast (<2 minutes), apparently does nothing and returns no message, returning code from process is zero.
I'm guessing that it's running fast because Sweep is doing it's job, but isn't marking the pages as successfully sweeped, is it right?
Is it related to CORE5869? In this case, Next is lesser than 2^27
It's happening at 6 of 18 databases running in remotely separate servers
gfix -v returns 0 warnings
gfix -v -full returns 3 warnings
gfix -mend returns 3 warnings (0 fixes)
A older backup sweep works fine, but gfix -v -full returns 1 warning

I'd say that it is a bug, because not only it doesn't increase OIT, but also gives no other error or warning, if someone could give me a tip of what to do, or what could make this happens I'll be glad
I'd like to know if simply gbak and restore isn't the only solution.

/opt/firebird/firebird.log:
backup Thu Nov 14 12:15:53 2019
Sweep is started by SYSDBA
Database "/db/db_no_sweep.fdb"
OIT 54057265, OAT 104895344, OST 104895344, Next 104895358

backup Thu Nov 14 12:17:15 2019
Sweep is finished
Database "/db/db_no_sweep.fdb"
OIT 54057265, OAT 104895344, OST 104895344, Next 104895359

backup Thu Nov 14 13:09:50 2019
Sweep is started by SYSDBA
Database "/db/db_no_sweep.fdb"
OIT 54057265, OAT 104895344, OST 104895344, Next 104895359

backup Thu Nov 14 13:09:52 2019
Sweep is finished
Database "/db/db_no_sweep.fdb"
OIT 54057265, OAT 104895344, OST 104895344, Next 104895360

[root@backup db]#⁠ /opt/firebird/bin/gfix -user sysdba -password masterkey -v db_no_sweep.fdb (no output)
/opt/firebird/firebird.log:

backup Thu Nov 14 14:00:38 2019
Database: /db/db_no_sweep.fdb
Validation started

backup Thu Nov 14 14:21:43 2019
Database: /db/db_no_sweep.fdb
Validation finished: 0 errors, 0 warnings, 0 fixed

backup Thu Nov 14 14:24:47 2019
Sweep is started by SYSDBA
Database "/db/db_no_sweep.fdb"
OIT 54057265, OAT 104895361, OST 104895361, Next 104895361

backup Thu Nov 14 14:24:56 2019
Sweep is finished
Database "/db/db_no_sweep.fdb"
OIT 54057265, OAT 104895361, OST 104895361, Next 104895362

[root@backup db]#⁠ /opt/firebird/bin/gfix -user sysdba -password masterkey -v -full db_no_sweep.fdb

Summary of validation errors
Number of record level warnings : 3

/opt/firebird/firebird.log:
backup Thu Nov 14 14:28:56 2019
Database: /db/db_no_sweep.fdb
Validation started

backup Thu Nov 14 14:35:24 2019
Database: /db/db_no_sweep.fdb
Warning: Relation has 2 orphan backversions {0 in use} in table CONTAS_A_RECEBER (247)

backup Thu Nov 14 14:35:37 2019
Database: /db/db_no_sweep.fdb
Warning: Relation has 3 orphan backversions {0 in use} in table NFE_LOTE_DADOS (253)

backup Thu Nov 14 14:46:07 2019
Database: /db/db_no_sweep.fdb
Warning: Relation has 8 orphan backversions {0 in use} in table NF_SAIDA_CAPA (575)

backup Thu Nov 14 14:53:53 2019
Database: /db/db_no_sweep.fdb
Validation finished: 0 errors, 3 warnings, 0 fixed

[root@backup db]#⁠ /opt/firebird/bin/gfix -user sysdba -password masterkey -mend db_no_sweep.fdb
/opt/firebird/firebird.log:
Summary of validation errors
Number of record level warnings : 3

backup Thu Nov 14 18:27:53 2019
Database: /db/db_no_sweep.fdb
Validation started

backup Thu Nov 14 18:34:54 2019
Database: /db/db_no_sweep.fdb
Warning: Relation has 2 orphan backversions {0 in use} in table CONTAS_A_RECEBER (247)

backup Thu Nov 14 18:35:07 2019
Database: /db/db_no_sweep.fdb
Warning: Relation has 3 orphan backversions {0 in use} in table NFE_LOTE_DADOS (253)

backup Thu Nov 14 18:46:01 2019
Database: /db/db_no_sweep.fdb
Warning: Relation has 8 orphan backversions {0 in use} in table NF_SAIDA_CAPA (575)

backup Thu Nov 14 18:53:44 2019
Database: /db/db_no_sweep.fdb
Validation finished: 0 errors, 3 warnings, 0 fixed

/opt/firebird/bin/gfix -user sysdba -password masterkey -sweep db_no_sweep.fdb

/opt/firebird/firebird.log:
backup Thu Nov 14 17:32:36 2019
Sweep is started by SYSDBA
Database "/db/db_no_sweep.fdb"
OIT 54057265, OAT 104895366, OST 104895366, Next 104895366

backup Thu Nov 14 17:32:38 2019
Sweep is finished
Database "/db/db_no_sweep.fdb"
OIT 54057265, OAT 104895366, OST 104895366, Next 104895367

[root@backup db]#⁠ /opt/firebird/bin/gstat -h /db/db_no_sweep.fdb

Database "/db/db_no_sweep.fdb"
Gstat execution time Thu Nov 14 17:42:28 2019

Database header page information:
Flags 0
Generation 104179175
System Change Number 0
Page size 16384
ODS version 12.0
Oldest transaction 54057265
Oldest active 104895367
Oldest snapshot 104895367
Next transaction 104895367
Sequence number 0
Next attachment ID 245555
Implementation HW=AMD/Intel/x64 little-endian OS=Linux CC=gcc
Shadow count 0
Page buffers 0
Next header page 0
Database dialect 3
Creation date Jul 30, 2019 21:01:32
Attributes

Variable header data:
    Sweep interval:         0
    \*END\*

Gstat completion time Thu Nov 14 17:42:28 2019

Note:
Running the same command at an old backup, it worked correctly, so probably something happened between one backup and the other:

backup Thu Nov 14 08:37:02 2019
Sweep is started by SYSDBA
Database "/db/db.fdb"
OIT 39619669, OAT 39644979, OST 39644979, Next 41051917

backup Thu Nov 14 08:38:08 2019
Sweep is finished
Database "/db/db.fdb"
OIT 41051917, OAT 39644979, OST 39644979, Next 41051918

backup Thu Nov 14 08:45:16 2019
Sweep is started by SYSDBA
Database "/db/db.fdb"
OIT 41051917, OAT 41051918, OST 41051918, Next 41051918

backup Thu Nov 14 08:45:16 2019
Sweep is finished
Database "/db/db.fdb"
OIT 41051918, OAT 41051918, OST 41051918, Next 41051919

[root@backup db]#⁠ /opt/firebird/bin/gfix -user sysdba -password masterkey -v -full db.fdb
Summary of validation errors
Number of record level warnings : 1

backup Thu Nov 14 17:36:55 2019
Database: /db/db.fdb
Validation started

backup Thu Nov 14 17:42:38 2019
Database: /db/db.fdb
Warning: Relation has 2 orphan backversions {0 in use} in table CONTAS_A_RECEBER (247)

backup Thu Nov 14 17:55:41 2019
Database: /db/db.fdb
Validation finished: 0 errors, 1 warnings, 0 fixed

[root@backup db]#⁠ /opt/firebird/bin/gstat -h /db/db.fdb

Database "/db/db.fdb"
Gstat execution time Thu Nov 14 17:42:32 2019

Database header page information:
Flags 0
Generation 40861176
System Change Number 0
Page size 16384
ODS version 12.0
Oldest transaction 41051918
Oldest active 41051919
Oldest snapshot 41051919
Next transaction 41051919
Sequence number 0
Next attachment ID 105975
Implementation HW=AMD/Intel/x64 little-endian OS=Linux CC=gcc
Shadow count 0
Page buffers 0
Next header page 0
Database dialect 3
Creation date Jul 30, 2019 21:01:32
Attributes

Variable header data:
    Sweep interval:         0
    \*END\*

Gstat completion time Thu Nov 14 17:42:32 2019

Any idea is welcome :)
Thanks

@firebird-automations
Copy link
Collaborator Author

Modified by: Lucas Schatz (arvanus)

summary: Sweep doesn't increase IT => Sweep doesn't increase OIT

@firebird-automations
Copy link
Collaborator Author

Modified by: Lucas Schatz (arvanus)

security: Developers [ 10012 ] =>

@firebird-automations
Copy link
Collaborator Author

Commented by: Lucas Schatz (arvanus)

Note: running gstat -data, I've got some tables with different number of swept pages from primary pages, example:

TDR$SYNC_NF_ENTRADA (604)
Primary pointer page: 1297, Index root page: 1298
Pointer pages: 5, data page slots: 13776
Data pages: 13776, average fill: 83%
Primary pages: 13776, secondary pages: 0, swept pages: 12609
Empty pages: 7, full pages: 13645
Fill distribution:
0 - 19% = 7
20 - 39% = 70
40 - 59% = 21
60 - 79% = 2107
80 - 99% = 11571

@firebird-automations
Copy link
Collaborator Author

Commented by: Lucas Schatz (arvanus)

"Problem" solved: it was an two-phase transaction in limbo
Found using:
>/opt/firebird/bin/gfix /db/db_no_sweep.fdb -l -pr
rollbacked the transaction, and sweep is working now!

Note: is it possible to make it more verbose to find out what is working?

Thanks

@firebird-automations
Copy link
Collaborator Author

Commented by: Lucas Schatz (arvanus)

Could some admin close this issue?
Apparently I can't do it .
Thanks

@firebird-automations
Copy link
Collaborator Author

Modified by: @hvlad

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Won't Fix [ 2 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant