Issue Details (XML | Word | Printable)

Key: CORE-2270
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Alexander Peshkov
Reporter: Jaroslaw Swierczynski
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Firebird Core

isql consumes all memory and crashes when run in zlogin console

Created: 09/Jan/09 11:14 AM   Updated: 08/Nov/09 09:13 PM
Component/s: ISQL
Affects Version/s: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.0.4, 2.5 Alpha 1, 2.1.1, 2.0.5, 2.1.2
Fix Version/s: 2.5 Beta 1, 2.1.3, 2.0.6

Time Tracking:
Not Specified

Environment: Solaris 10, all architectures

Planning Status: Unspecified


 Description  « Hide
When you try to run isql inside a non-global zone that you entered using "zlogin -C", it will start to consume all available memory and eventually crash.

The reason for that is a bug in Solaris that makes the terminal not have its size set when the terminal is started with "zlogin -C". The bug is described here:

http://docs.sun.com/app/docs/doc/820-0428/6nc5u3kom?a=view#gejte

This reveals two bugs in isql or actualy in the editline library that is included in the Firebird sources:

1. Consuming all available memory. The libary tries to read the terminal's size. It gets some random values - negative or an incredibly huge numbers. The negative numbers are corrected in extern/editline/src/term.c:term_change_size(), however the huge positive numbers are not which leads to an attempt of allocating a huge amount of memory in term_alloc_display(). This way isql hangs and often makes system unresponsive (swap allocation).

2. Segmentation fault. When isql finally eats up all the memory and it cannot get more, in two places return codes are not validated. The library will not detect that the memory was not allocated and will try to dereference a null pointer.

The second problem is easy to fix, however the first one needs to be fixed in Solaris. For Firebird I can think only of a workaround which will detect insane terminal sizes and correct them just as it corrects negative sizes.

Patch:

diff -ru firebird-B2_1_Release-20081218.orig/extern/editline/src/readline.c firebird-B2_1_Release-20081218/extern/editline/src/readline.c
--- firebird-B2_1_Release-20081218.orig/extern/editline/src/readline.c Mon Apr 9 14:57:41 2007
+++ firebird-B2_1_Release-20081218/extern/editline/src/readline.c Fri Jan 9 13:36:16 2009
@@ -351,7 +351,8 @@
        static int used_event_hook;

        if (e == NULL || h == NULL)
- rl_initialize();
+ if (rl_initialize() == -1)
+ return NULL;

        rl_done = 0;

diff -ru firebird-B2_1_Release-20081218.orig/extern/editline/src/term.c firebird-B2_1_Release-20081218/extern/editline/src/term.c
--- firebird-B2_1_Release-20081218.orig/extern/editline/src/term.c Mon Apr 9 14:57:41 2007
+++ firebird-B2_1_Release-20081218/extern/editline/src/term.c Fri Jan 9 13:37:16 2009
@@ -347,7 +347,8 @@
                return (-1);
        (void) memset(el->el_term.t_val, 0, T_val * sizeof(int));
        term_outfile = el->el_outfile;
- (void) term_set(el, NULL);
+ if (term_set(el, NULL) == -1)
+ return (-1);
        term_init_arrow(el);
        return (0);
 }
@@ -1025,8 +1026,8 @@
        /*
          * Just in case
          */
- Val(T_co) = (cols < 2) ? 80 : cols;
- Val(T_li) = (lins < 1) ? 24 : lins;
+ Val(T_co) = (cols < 2 || cols > 10000) ? 80 : cols;
+ Val(T_li) = (lins < 1 || lins > 10000) ? 24 : lins;

        /* re-make display buffers */
        if (term_rebuffer_display(el) == -1)

 All   Comments   Work Log   Change History   Version Control   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Alexander Peshkov added a comment - 09/Jan/09 11:47 AM
I'm far not sure it's worth fixing host-OS bugs if we can easily avoid the, specially when this is client utility, not server. The simplest way is to build with switch --without-editiline for this OS. Or add a notice in readme that isql should not be used in such environment. Or use SUN's suggested workaround - "use the stty command to set the correct number of rows and columns once you login with zlogin".

I suggest to close this issue as won't fix.

Jaroslaw Swierczynski added a comment - 09/Jan/09 06:24 PM
You've made a valid point, Alex. However that doesn't change the fact that editline has a bug - it allows an abnormal memory consumption that could kill a system. In this case it's a Solaris bug, revealed under certain conditions. But it could happen everywhere, googling I could find many reports of invalid terminal sizes returned by ioctl TIOCSWINSZ. The values are already checked against negatives, so why not check them also against large numbers? The effort required is minimal, changes are simple and there is no chance of breaking anything. My opinion is that OS bugs are no excuse for misbehaving software if you can easily make it work better.

Dmitry Yemanov added a comment - 10/Jan/09 02:59 AM
I tend to agree with Jaroslaw on his points.

Alexander Peshkov added a comment - 11/Jan/09 01:21 PM
Patch applied

Alexander Peshkov added a comment - 11/Jan/09 01:23 PM
No idea why were marked all unreleased versions....

Jaroslaw Swierczynski added a comment - 11/Jan/09 04:31 PM
Sorry, I just marked all versions >= 2.0 which were available.