Issue Details (XML | Word | Printable)

Key: CORE-3032
Type: Bug Bug
Status: Open Open
Priority: Critical Critical
Assignee: Unassigned
Reporter: Gili Buzaglo
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Firebird Core

fbserver terminates and creates a core dump

Created: 06/Jun/10 06:52 PM   Updated: 08/Jun/10 07:47 AM
Component/s: None
Affects Version/s: 2.1.1
Fix Version/s: None

File Attachments: 1. GZip Archive fbCore1.gz (8.28 MB)
2. GZip Archive fbCore2.gz (5.39 MB)
3. GZip Archive fbCore3.gz (4.94 MB)
4. GZip Archive fbCore4.gz (4.81 MB)

Environment: solaris x86 superserver


 Description  « Hide
our firebird server crashes at customer site. I have a few core dumps and here are the 'where' outputs from dbx:
1)
(dbx) where
=>[1] jrd8_cancel_operation(0x80a9dcf, 0xfdcfebfc, 0xfe6c05e8, 0x1, 0x1000000, 0x0), at 0x818c427
  [2] gds__cancel_operation(0xfdcfebfc, 0xfe6c173c, 0x1, 0x0, 0xfdcfec04, 0xfdcfebfc), at 0x80a9dcf
  [3] rem_port::disconnect(0xfe8a14cc, 0xfdeaa778, 0xfdeaaa2c, 0xfe6c0bf4, 0x0, 0x0), at 0x808b7e5
  [4] process_packet(0xfe8a14cc, 0xfdeaa778, 0xfdeaaa2c, 0xfdcfef98, 0x83fe110, 0x83dded9), at 0x808dbb5
  [5] loopThread(0x2, 0xfe6c11dc, 0xfdcfefdc, 0xfe9c69b0, 0xfea42ec0, 0x0), at 0x808e53c
  [6] _GLOBAL__N_global_mutex::threadStart(0xfe6c11dc), at 0x80a2688
  [7] 0xfe9c71c0(0xfe863a40), at 0xfe9c71c0
  [8] 0xfe9c74b0(), at 0xfe9c74b0


2)
(dbx) where
current thread: t@7
=>[1] jrd8_rollback_transaction(0xfd99ebfc, 0xfe84dca8, 0xfd4, 0x1000000, 0x0, 0x0), at 0x8195f3e
  [2] isc_rollback_transaction(0xfd99ebfc, 0xfe84e674, 0x0, 0x0, 0xfd99ec04, 0x0), at 0x80ab6f3
  [3] rem_port::end_transaction(0xfe8a5934, 0x1f, 0xfe6c1078, 0xfe6c0ca0, 0x0, 0x0), at 0x8087fee
  [4] process_packet(0xfe8a5934, 0xfe6c0ca0, 0xfe6c0f54, 0xfd99ef98, 0x83fe110, 0x83dded9), at 0x808dcdd
  [5] loopThread(0x2, 0xfe84d1c0, 0xfd99efdc, 0xfe9c69b0, 0xfea42ec0, 0x0), at 0x808e53c
  [6] _GLOBAL__N_global_mutex::threadStart(0xfe84d1c0), at 0x80a2688
  [7] _thr_setup(0xfe861240), at 0xfe9c71c0
  [8] _lwp_start(0x1, 0xfffffff4, 0x1, 0xfffffffc, 0x1, 0xfd99e8d0), at 0xfe9c74b0

3)

>[1] jrd8_release_request(0xfdc9ea80, 0xfe29d170, 0xb10, 0x1000000, 0x0, 0x0), at 0x81963ee
  [2] isc_release_request(0xfdc9ea80, 0xfc97df7c, 0xfe9bed28, 0x8467dc0, 0x1000000, 0xfc97df24), at 0x80abb54
  [3] release_request(0xfc970014, 0xfe863a40, 0xfdc9eb18, 0x80a032c, 0x4c, 0x8467e80), at 0x8237e23
  [4] GDS_DSQL_FREE_CPP(0xfdc9ebfc, 0xfe29d204, 0x2, 0x1000000, 0x0, 0x0), at 0x82389cc
  [5] isc_dsql_free_statement(0xfdc9ebfc, 0xfe29d1b4, 0x2, 0x0, 0xfdc9ec04, 0x0), at 0x80adee9
  [6] rem_port::end_statement(0xfe8a5934, 0xfdb92008, 0xfdb91aec, 0xfc9b2b44, 0x0, 0x0), at 0x808b480
  [7] process_packet(0xfe8a5934, 0xfdb91aec, 0xfdb91da0, 0xfdc9ef98, 0x83fe110, 0x83dded9), at 0x808d7c0
  [8] loopThread(0x2, 0xfe292f2c, 0xfdc9efdc, 0xfe9c69b0, 0xfea42ec0, 0x0), at 0x808e53c
  [9] _GLOBAL__N_global_mutex::threadStart(0xfe292f2c), at 0x80a2688
  [10] _thr_setup(0xfe863a40), at 0xfe9c71c0
  [11] _lwp_start(0xfc9b2b44, 0xfdc9ebd8, 0x83dde83, 0x8, 0xfdb3da90, 0xfdc9e750), at 0xfe9c74b0

4)
>[1] free_request(0x2000d000, 0x803f5e8, 0xfe9bef24, 0xfea32a40, 0x84633c0, 0xfea3d000), at 0x8085ec2
  [2] link_request(0x83dded9, 0x803f6a5, 0x2000, 0x804770a, 0x8047704, 0x0), at 0x808681b
  [3] SRVR_multi_thread(0xfe8aa198, 0x2, 0x3, 0x0, 0x0, 0x8335a14), at 0x808bbbb
  [4] server_main(0x1, 0x8047c64, 0x83fe110, 0x808e844), at 0x80858e4
  [5] main(0x8085160, 0x1, 0x8047c64), at 0x808e85f



I also have the firebird.log file, and at the relevant time it prints:
cms1a (Client) Wed May 12 20:28:00 2010
        /usr/local/firebird/bin/fbguard: /usr/local/firebird/bin/fbserver terminated abnormally (-1)



cms1a (Client) Wed May 12 20:28:00 2010
        /usr/local/firebird/bin/fbguard: guardian starting bin/fbserver



cms1a (Server) Wed May 12 20:28:00 2010
        64 bit i/o support is on.


cms1a (Server) Wed May 12 20:28:00 2010
        Open file limit increased from 256 to 65536


cms1a (Client) Wed May 12 20:28:17 2010
        /usr/local/firebird/bin/fbguard: /usr/local/firebird/bin/fbserver terminated abnormally (-1)



cms1a (Client) Wed May 12 20:28:17 2010
        /usr/local/firebird/bin/fbguard: guardian starting bin/fbserver



cms1a (Server) Wed May 12 20:28:17 2010
        64 bit i/o support is on.


cms1a (Server) Wed May 12 20:28:17 2010
        Open file limit increased from 256 to 65536


cms1a (Client) Wed May 12 20:28:21 2010
        /usr/local/firebird/bin/fbguard: /usr/local/firebird/bin/fbserver te

 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Gili Buzaglo added a comment - 06/Jun/10 07:05 PM
Those are core dumps of fbserver

Gili Buzaglo added a comment - 06/Jun/10 07:08 PM
core dump

Gili Buzaglo added a comment - 06/Jun/10 07:10 PM
core dump

Gili Buzaglo added a comment - 06/Jun/10 07:13 PM
At the java client application I get those exceptions:
org.firebirdsql.jdbc.FBSQLException: GDS Exception. Error reading data from the connection.
        at org.firebirdsql.jdbc.FBConnection.checkEndTransaction(FBConnection.java:1008)
        at org.firebirdsql.jdbc.FBPreparedStatement.<init>(FBPreparedStatement.java:83)
        at org.firebirdsql.jdbc.FBConnection.prepareStatement(FBConnection.java:223)
        at cloverleaf.manager.database.db.CashConnection.prepareCashStatement(CashConnection.java:102)
        at cloverleaf.manager.database.db.QueryHelper1.getAllControllers(QueryHelper1.java:125)
        at cloverleaf.manager.mbe.mfeserver.MfeServerImpl.getAllControllers(MfeServerImpl.java:995)
        at cloverleaf.manager.mbe.snmp.agent.cloverleafmib.CllfStorageTable.updateNow(CllfStorageTable.java:41)
        at cloverleaf.manager.mbe.snmp.agent.AgentMib.checkUpdate(AgentMib.java:1037)
        at cloverleaf.manager.mbe.snmp.agent.AgentMib.getNumStorages(AgentMib.java:1048)
        at cloverleaf.manager.mbe.snmp.agent.cloverleafmib.CllfPoolGroup.getCllfNoStorages(CllfPoolGroup.java:44)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:324)
        at com.sun.management.jmx.MBeanServerImpl.invoke(MBeanServerImpl.java:1628)
        at com.sun.management.jmx.MBeanServerImpl.invoke(MBeanServerImpl.java:1523)
        at com.ireasoning.protocol.snmp.SnmpBaseAgent.a(SnmpBaseAgent.java:2321)
        at com.ireasoning.protocol.snmp.SnmpBaseAgent.a(SnmpBaseAgent.java:2483)
        at com.ireasoning.protocol.snmp.SnmpBaseAgent.a(SnmpBaseAgent.java:1906)
        at com.ireasoning.protocol.snmp.l.run(l.java:238)
        at a.a.a.a.a.a.ch.run(ch.java:728)
        at java.lang.Thread.run(Thread.java:534)
at org.firebirdsql.gds.GDSException: Error reading data from the connection.
        at org.firebirdsql.jgds.GDS_Impl.isc_commit_transaction(GDS_Impl.java:544)
        at org.firebirdsql.jca.FBManagedConnectionFactory.commit(FBManagedConnectionFactory.java:684)
        at org.firebirdsql.jca.FBManagedConnection.internalCommit(FBManagedConnection.java:410)
        at org.firebirdsql.jca.FBLocalTransaction.internalCommit(FBLocalTransaction.java:156)
        at org.firebirdsql.jdbc.FBConnection.checkEndTransaction(FBConnection.java:1004)
        at org.firebirdsql.jdbc.FBPreparedStatement.<init>(FBPreparedStatement.java:83)
        at org.firebirdsql.jdbc.FBConnection.prepareStatement(FBConnection.java:223)
        at cloverleaf.manager.database.db.CashConnection.prepareCashStatement(CashConnection.java:102)
        at cloverleaf.manager.database.db.QueryHelper1.getAllControllers(QueryHelper1.java:125)
        at cloverleaf.manager.mbe.mfeserver.MfeServerImpl.getAllControllers(MfeServerImpl.java:995)

Gili Buzaglo added a comment - 08/Jun/10 07:47 AM
hi all
I think the first 3 core dumps are a result of a security tool attack.
But I dont know about the last one.