When you try to run isql inside a non-global zone that you entered using "zlogin -C", it will start to consume all available memory and eventually crash.

The reason for that is a bug in Solaris that makes the terminal not have its size set when the terminal is started with "zlogin -C". The bug is described here:

http://docs.sun.com/app/docs/doc/820-0428/6nc5u3kom?a=view#gejte

This reveals two bugs in isql or actualy in the editline library that is included in the Firebird sources:

1. Consuming all available memory. The libary tries to read the terminal's size. It gets some random values - negative or an incredibly huge numbers. The negative numbers are corrected in extern/editline/src/term.c:term_change_size(), however the huge positive numbers are not which leads to an attempt of allocating a huge amount of memory in term_alloc_display(). This way isql hangs and often makes system unresponsive (swap allocation).

2. Segmentation fault. When isql finally eats up all the memory and it cannot get more, in two places return codes are not validated. The library will not detect that the memory was not allocated and will try to dereference a null pointer.

The second problem is easy to fix, however the first one needs to be fixed in Solaris. For Firebird I can think only of a workaround which will detect insane terminal sizes and correct them just as it corrects negative sizes.

Patch:

diff -ru firebird-B2_1_Release-20081218.orig/extern/editline/src/readline.c firebird-B2_1_Release-20081218/extern/editline/src/readline.c
--- firebird-B2_1_Release-20081218.orig/extern/editline/src/readline.c Mon Apr 9 14:57:41 2007
+++ firebird-B2_1_Release-20081218/extern/editline/src/readline.c Fri Jan 9 13:36:16 2009
@@ -351,7 +351,8 @@
        static int used_event_hook;

        if (e == NULL || h == NULL)
- rl_initialize();
+ if (rl_initialize() == -1)
+ return NULL;

        rl_done = 0;

diff -ru firebird-B2_1_Release-20081218.orig/extern/editline/src/term.c firebird-B2_1_Release-20081218/extern/editline/src/term.c
--- firebird-B2_1_Release-20081218.orig/extern/editline/src/term.c Mon Apr 9 14:57:41 2007
+++ firebird-B2_1_Release-20081218/extern/editline/src/term.c Fri Jan 9 13:37:16 2009
@@ -347,7 +347,8 @@
                return (-1);
        (void) memset(el->el_term.t_val, 0, T_val * sizeof(int));
        term_outfile = el->el_outfile;
- (void) term_set(el, NULL);
+ if (term_set(el, NULL) == -1)
+ return (-1);
        term_init_arrow(el);
        return (0);
 }
@@ -1025,8 +1026,8 @@
        /*
          * Just in case
          */
- Val(T_co) = (cols < 2) ? 80 : cols;
- Val(T_li) = (lins < 1) ? 24 : lins;
+ Val(T_co) = (cols < 2 || cols > 10000) ? 80 : cols;
+ Val(T_li) = (lins < 1 || lins > 10000) ? 24 : lins;

        /* re-make display buffers */
        if (term_rebuffer_display(el) == -1)