Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

isql consumes all memory and crashes when run in zlogin console [CORE2270] #2696

Closed
firebird-automations opened this issue Jan 9, 2009 · 12 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: Jaroslaw Swierczynski (swiergot)

When you try to run isql inside a non-global zone that you entered using "zlogin -C", it will start to consume all available memory and eventually crash.

The reason for that is a bug in Solaris that makes the terminal not have its size set when the terminal is started with "zlogin -C". The bug is described here:

http://docs.sun.com/app/docs/doc/820-0428/6nc5u3kom?a=view#gejte

This reveals two bugs in isql or actualy in the editline library that is included in the Firebird sources:

1. Consuming all available memory. The libary tries to read the terminal's size. It gets some random values - negative or an incredibly huge numbers. The negative numbers are corrected in extern/editline/src/term.c:term_change_size(), however the huge positive numbers are not which leads to an attempt of allocating a huge amount of memory in term_alloc_display(). This way isql hangs and often makes system unresponsive (swap allocation).

2. Segmentation fault. When isql finally eats up all the memory and it cannot get more, in two places return codes are not validated. The library will not detect that the memory was not allocated and will try to dereference a null pointer.

The second problem is easy to fix, however the first one needs to be fixed in Solaris. For Firebird I can think only of a workaround which will detect insane terminal sizes and correct them just as it corrects negative sizes.

Patch:

diff -ru firebird-B2_1_Release-20081218.orig/extern/editline/src/readline.c firebird-B2_1_Release-20081218/extern/editline/src/readline.c
--- firebird-B2_1_Release-20081218.orig/extern/editline/src/readline.c Mon Apr 9 14:57:41 2007
+++ firebird-B2_1_Release-20081218/extern/editline/src/readline.c Fri Jan 9 13:36:16 2009
@@ -351,7 +351,8 @@
static int used_event_hook;

    if \(e == NULL \|\| h == NULL\)

- rl_initialize();
+ if (rl_initialize() == -1)
+ return NULL;

    rl\_done = 0;

diff -ru firebird-B2_1_Release-20081218.orig/extern/editline/src/term.c firebird-B2_1_Release-20081218/extern/editline/src/term.c
--- firebird-B2_1_Release-20081218.orig/extern/editline/src/term.c Mon Apr 9 14:57:41 2007
+++ firebird-B2_1_Release-20081218/extern/editline/src/term.c Fri Jan 9 13:37:16 2009
@@ -347,7 +347,8 @@
return (-1);
(void) memset(el->el_term.t_val, 0, T_val * sizeof(int));
term_outfile = el->el_outfile;
- (void) term_set(el, NULL);
+ if (term_set(el, NULL) == -1)
+ return (-1);
term_init_arrow(el);
return (0);
}
@@ -1025,8 +1026,8 @@
/*
* Just in case
*/
- Val(T_co) = (cols < 2) ? 80 : cols;
- Val(T_li) = (lins < 1) ? 24 : lins;
+ Val(T_co) = (cols < 2 || cols > 10000) ? 80 : cols;
+ Val(T_li) = (lins < 1 || lins > 10000) ? 24 : lins;

    /\* re\-make display buffers \*/
    if \(term\_rebuffer\_display\(el\) == \-1\)

Commits: 462c442 bf6c81b 004be85

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

I'm far not sure it's worth fixing host-OS bugs if we can easily avoid the, specially when this is client utility, not server. The simplest way is to build with switch --without-editiline for this OS. Or add a notice in readme that isql should not be used in such environment. Or use SUN's suggested workaround - "use the stty command to set the correct number of rows and columns once you login with zlogin".

I suggest to close this issue as won't fix.

@firebird-automations
Copy link
Collaborator Author

Commented by: Jaroslaw Swierczynski (swiergot)

You've made a valid point, Alex. However that doesn't change the fact that editline has a bug - it allows an abnormal memory consumption that could kill a system. In this case it's a Solaris bug, revealed under certain conditions. But it could happen everywhere, googling I could find many reports of invalid terminal sizes returned by ioctl TIOCSWINSZ. The values are already checked against negatives, so why not check them also against large numbers? The effort required is minimal, changes are simple and there is no chance of breaking anything. My opinion is that OS bugs are no excuse for misbehaving software if you can easily make it work better.

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

I tend to agree with Jaroslaw on his points.

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

Patch applied

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 2.5 Beta 1 [ 10251 ]

Fix Version: 2.1.3 [ 10302 ]

Fix Version: 2.0.6 [ 10303 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

No idea why were marked all unreleased versions....

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

Version: 2.5 Beta 1 [ 10251 ] =>

Version: 3.0 Initial [ 10301 ] =>

Version: 2.5 RC1 [ 10300 ] =>

Version: 2.5.0 [ 10221 ] =>

Version: 3.0 Alpha 1 [ 10331 ] =>

Version: 2.1.3 [ 10302 ] =>

Version: 2.0.6 [ 10303 ] =>

Version: 2.5.1 [ 10333 ] =>

Version: 3.0 Beta 1 [ 10332 ] =>

Version: 3.0.0 [ 10048 ] =>

@firebird-automations
Copy link
Collaborator Author

Commented by: Jaroslaw Swierczynski (swiergot)

Sorry, I just marked all versions >= 2.0 which were available.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

QA Status: No test

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Closed [ 6 ] => Closed [ 6 ]

QA Status: No test => Cannot be tested

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment