Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Engine may hang due to races when starting crypt thread and simultaneous shutdown [CORE6360] #6601

Closed
firebird-automations opened this issue Jul 9, 2020 · 5 comments
Closed

Engine may hang due to races when starting crypt thread and simultaneous shutdown [CORE6360] #6601

firebird-automations opened this issue Jul 9, 2020 · 5 comments
Assignees
@AlexPeshkoff
Labels
affect-version: 3.0.6 affect-version: 4.0 Beta 2 component: engine fix-version: 3.0.7 fix-version: 4.0 RC 1 priority: major type: bug

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @artyom-smirnov

Relate to CORE6338

When CM thread starting it assigns its internal attachment flag "ATT_crypt_thread", but on short connection this attachment and shutdown racing and flag may be never assigned therefore attachment and CM thread can not properly stopped causing server hang.

Reproducing:

#⁠!/bin/bash
set -e

echo Auto = y > plugins/DbCrypt_example.conf

rm tmp.fdb
echo "create database 'tmp.fdb';" | bin/isql -u sysdba -p masterkey

while true; do
cp tmp.fdb repro.fdb; echo 'alter database encrypt with "DbCrypt_example";'| bin/isql -u sysdba -p masterkey repro.fdb
done

Commits: bcd6ca8 ebddc8f
@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

reporter: Alexander Peshkov [ alexpeshkoff ] => Artyom Smirnov [ artyom.smirnov ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

Link: This issue relate to CORE6338 [ CORE6338 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 4.0 RC 1 [ 10930 ]

Fix Version: 3.0.7 [ 10940 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @artyom-smirnov

Another issue related to CryptoManager. It less disruptive issue and hardly to achieve. Looks like sometime CryptoManager::cryptThread calling PAG_attachment_id(tempDbb) but it never released. Can be reproduced same script as above. It causing assertion in debug mode.

BackupManager::localStateLock::lockCounter > 0

Assertion (lockCounter.value() == 0) failure: /workspace/rdb/firebird_3_0/src/jrd/../jrd/../jrd/../common/classes/rwlock.h 233
./test.sh: line 14: 331765 Done echo 'alter database encrypt with "DbCrypt_example";'
331766 Aborted (core dumped) | bin/isql -u sysdba -p masterkey repro.fdb

Backtrace:

#⁠0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#⁠1 0x00007fd54b093859 in __GI_abort () at abort.c:79
#⁠2 0x00007fd548fd508e in fb_assert_impl (msg=0x7fd5494fd91d "lockCounter.value() == 0",
file=0x7fd5494fd8d0 "/workspace/rdb/firebird_3_0/src/jrd/../jrd/../jrd/../common/classes/rwlock.h", line=233, do_abort=true)
at /workspace/rdb/firebird_3_0/src/include/../common/gdsassert.h:48
#⁠3 0x00007fd548feac5d in Firebird::RWLock::~RWLock (this=0x7fd548e8fd70, __in_chrg=<optimized out>)
at /workspace/rdb/firebird_3_0/src/jrd/../jrd/../jrd/../common/classes/rwlock.h:233
#⁠4 0x00007fd54916b78d in Jrd::BackupManager::~BackupManager (this=0x7fd548e8fcd0, __in_chrg=<optimized out>)
at /workspace/rdb/firebird_3_0/src/jrd/nbak.cpp:951
#⁠5 0x00007fd548fe8e05 in Jrd::Database::~Database (this=0x7fd548e8d850, __in_chrg=<optimized out>)
at /workspace/rdb/firebird_3_0/src/jrd/Database.cpp:97
#⁠6 0x00007fd5490d9940 in Jrd::Database::destroy (toDelete=0x7fd548e8d850) at /workspace/rdb/firebird_3_0/src/jrd/../jrd/../jrd/Database.h:361
#⁠7 0x00007fd5490d1edb in JRD_shutdown_database (dbb=0x7fd548e8d850, flags=7) at /workspace/rdb/firebird_3_0/src/jrd/jrd.cpp:6753
#⁠8 0x00007fd5490d3007 in purge_attachment (tdbb=0x7ffd463b1cb0, sAtt=0x7fd548c96cd0, flags=2) at /workspace/rdb/firebird_3_0/src/jrd/jrd.cpp:7123
#⁠9 0x00007fd5490c4d02 in Jrd::JAttachment::freeEngineData (this=0x7fd548c96ea0, user_status=0x7ffd463b1e40, forceFree=false)
at /workspace/rdb/firebird_3_0/src/jrd/jrd.cpp:2931
#⁠10 0x00007fd5490c4b70 in Jrd::JAttachment::detach (this=0x7fd548c96ea0, user_status=0x7ffd463b1e40)
at /workspace/rdb/firebird_3_0/src/jrd/jrd.cpp:2891
#⁠11 0x00007fd5490ef355 in Firebird::IAttachmentBaseImpl<Jrd::JAttachment, Firebird::CheckStatusWrapper, Firebird::IReferenceCountedImpl<Jrd::JAttachment, Firebird::CheckStatusWrapper, Firebird::Inherit<Firebird::IVersionedImpl<Jrd::JAttachment, Firebird::CheckStatusWrapper, Firebird::InheritFirebird::IAttachment\ > > > >::cloopdetachDispatcher (self=0x7fd548c96ea8, status=0x7ffd463b1f48)
at /workspace/rdb/firebird_3_0/src/include/firebird/IdlFbInterfaces.h:9051
#⁠12 0x00007fd54b6b2735 in Firebird::IAttachment::detachFirebird::CheckStatusWrapper\ (this=0x7fd548c96ea8, status=0x7ffd463b1f40)
at /workspace/rdb/firebird_3_0/src/include/firebird/IdlFbInterfaces.h:1964
#⁠13 0x00007fd54b69f4e5 in Why::YAttachment::detach (this=0x7fd54b60d2d0, status=0x7ffd463b1f40) at /workspace/rdb/firebird_3_0/src/yvalve/why.cpp:5468
#⁠14 0x00007fd54b6d6e38 in Firebird::IAttachmentBaseImpl<Why::YAttachment, Firebird::CheckStatusWrapper, Firebird::IReferenceCountedImpl<Why::YAttachment, Firebird::CheckStatusWrapper, Firebird::Inherit<Firebird::IVersionedImpl<Why::YAttachment, Firebird::CheckStatusWrapper, Firebird::InheritFirebird::IAttachment\ > > > >::cloopdetachDispatcher (self=0x7fd54b60d2d8, status=0x560f3c14b0e8 <fbStatusObj+8>)
at /workspace/rdb/firebird_3_0/src/include/firebird/IdlFbInterfaces.h:9051
--Type <RET> for more, q to quit, c to continue without paging--
#⁠15 0x0000560f3c0413e5 in Firebird::IAttachment::detachFirebird::CheckStatusWrapper\ (this=0x7fd54b60d2d8, status=0x560f3c14b0e0 <fbStatusObj>)
at /workspace/rdb/firebird_3_0/src/include/firebird/IdlFbInterfaces.h:1964
#⁠16 0x0000560f3c04c7ab in do_isql () at /workspace/rdb/firebird_3_0/temp/Debug/isql/isql.cpp:5690
#⁠17 0x0000560f3c04239a in ISQL_main (argc=6, argv=0x7ffd463b2818) at /workspace/rdb/firebird_3_0/temp/Debug/isql/isql.cpp:1753
#⁠18 0x0000560f3c04206a in main (argc=6, argv=0x7ffd463b2818) at /workspace/rdb/firebird_3_0/temp/Debug/isql/isql.cpp:1659

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AlexPeshkoff AlexPeshkoff

Labels
affect-version: 3.0.6 affect-version: 4.0 Beta 2 component: engine fix-version: 3.0.7 fix-version: 4.0 RC 1 priority: major type: bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@AlexPeshkoff @firebird-automations