Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A number of errors when database name is longer than 255 symbols [CORE6248] #6492

Closed
firebird-automations opened this issue Feb 16, 2020 · 19 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @pavel-zotov

Is related to CORE6419

Following script illustrates athe problem (of course, most probably it can occur with too long PATH that contains from dozen of nested folders rather than single file name):

set list on;

shell del C:\temp\123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123.fdb 2>nul;
shell del C:\temp\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234.fdb 2>nul;
shell del C:\temp\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678.fdb 2>nul;

--set echo on;

-- Length of full path + file name = 255 bytes is maximal limit for DB referencing using mon$database_name and/or rdb$get_context('SYSTEM', 'DB_NAME'):
create database 'C:\temp\123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123.fdb';

show database; -- passed OK

select mon$database_name, rdb$get_context('SYSTEM', 'DB_NAME') from mon$database; -- passed OK

commit;
drop database;

------------------------------------------------------------------------ this needs to be fixed: -------------------------------------------------------------

-- When length of full path+file name is 256 then DB is created but can not be referred using RDB$GET_CONTEXT() or MON$DATABASE:
create database 'localhost:C:\temp\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234.fdb';

show database; -- passed OK. WHY ??

select mon$database_name from mon$database; -- Statement failed, SQLSTATE = 22001 / ... -string right truncation / -expected length 255, actual 256

select rdb$get_context('SYSTEM', 'DB_NAME') from mon$database; -- Statement failed, SQLSTATE = 22001 / ... -string right truncation / -expected length 255, actual 256

commit;

drop database;

PS.

-- When length of full path+file name is 260 then "the system cannot find the path specified" raises.
-- Statement failed, SQLSTATE = 08001 / I/O error during "CreateFile (create)" operation for file ... / -Error while trying to create file
create database 'localhost:C:\temp\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678.fdb';
drop database;
quit;

Commits: 74f7b30 684e17d

====== Test Details ======

STDOUT-logs of backup, restore and gstat currently (09-mar-2020) have only truncated name of DB with length about 230...240 chars.
Test verifies presense of such truncated strings rather then expected full path + name of DB.

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

NBACKUP can work with .FDB file which name is less or equal 253 (ascii ?) character (at least on Winows).

After this following error raises:
[
PROBLEM ON "begin backup: commit".
I/O error during "CreateFile (create)" operation for file "<...>.FDB.delta"
-Error while trying to create file
-<the system cannot find the path specified> // localized message here
SQLCODE:-902
]

Adding of suffix "..delta" to DB name can not be neither avoided nor replaced with some other (shorter) text.

So, for practice usage max length of path + DB filename must be limited with 253 bytes rather than 255 as this ticket title states.
Even if some other OS allows to create files with bigger length because such file will not be restored on Windows.

@firebird-automations
Copy link
Collaborator Author

Commented by: @mrotteveel

I'd suggest that instead, the columns of these tables should be changed to allow for longer filenames. And one Windows, checks should be made for API usage, because recent versions of Windows do support longer file paths that 255 characters, though I'm not exactly sure if that requires a different API or specific settings.

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

> instead, the columns of these tables should be changed to allow for longer filenames.

Sorry, this i did not understand. Do you mean to increase size of RDB$ / MON$ tables text fields ?

> recent versions of Windows do support longer file paths that 255 characters

Such API can provoke to create database with path+file name length beyond this limit and, consequently, this makes such DB not usable on OS which still has limit about 255 chars.
I doubt that this is good opportunity...

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

One more limitation was encountered in record-level backup (i.e. gbak or fbsvcmgr action_backup): maximum length of path + name of DB can be 245 characters.

This is the only action that has such limit; all other have greater than this (nbackup = 253, others = 255).
Tested utilities / modes:

isql \-i <script with "create database <longname\_db\>" statement\>
isql \-x <localhost:longname\_db\>; 
gstat \-r <localhost:longname\_db\>; 
gfix \-w async \| \-sweep \| \-shut full \-force 0 \| \-online <localhost:longname\_db\>; 
nbackup \-L \| \-N ; 
fb\_lock\_print \-c \-d <longname\_db\>; 
fbvsvcmgr action\_trace \| action\_validate  \| action\_repair \| action\_nbak \| action\_nrest \| action\_backup \| action\_restore

====

For path+name length = 246 characters action backup / restore FAILED with following result:

1) gbak -b -v localhost:drive:\\path\\to\\name\.fdb drive:\\path\\to\\name\.fbk
-- shows:

gbak:readied database <CUTED-OFF name on 261th character, ellipsis instead of tailing text>
gbak:creating file <CUTED-OFF name on 261th character, ellipsis instead of tailing text>
gbak:starting transaction
gbak:database <CUTED-OFF name on 261th character, ellipsis instead of tailing text>
gbak:writing domains
gbak: writing domain RDB$1
gbak:writing shadow files
gbak:writing character sets
. . .
gbak:writing SQL roles
gbak:writing names mapping
gbak:closing file, committing, and finishing. 18432 bytes written
gbak:text for attribute 1 is too large in put_asciz(), truncating to 255 bytes

So, .fbk file *WILL* be created and it seems that we can restore from it.
But restore will fail:

2) gbak -rep -v drive:\\path\\to\\name\.fbk localhost:drive:\\path\\to\\name\.fdb
-- shows:

gbak:opened file <CUTED-OFF name on 261th character, ellipsis instead of tailing text>
gbak:transportable backup -- data in XDR format
gbak: backup file is compressed
gbak:backup version is 11
gbak:do not recognize backup attribute 108 -- continuing
gbak:do not recognize backup attribute 53 -- continuing
gbak:do not recognize backup attribute 49 -- continuing
gbak:do not recognize backup attribute 51 -- continuing
gbak:do not recognize backup attribute 52 -- continuing
gbak:do not recognize backup attribute 73 -- continuing
gbak:do not recognize backup attribute 49 -- continuing
gbak:do not recognize backup attribute 51 -- continuing
gbak:do not recognize backup attribute 55 -- continuing
gbak:do not recognize backup attribute 51 -- continuing
gbak: ERROR:Expected database description record
gbak:Exiting before completion due to errors

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

It have no sence as you can't force user to not move\rename existing database file to make full path longer than 255 chars.

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

What exactly 'have no sence' ?
Path consisted of dozen other sub-dirs (e.g. in UUID-form) can easy be near this limit.
IMO, limitation must be:
1) the same for every utility / mode (gbak / nbackup / isql: create database statement etc)
2) less than 250 characters for any OS; currently one may create database with len = 259 chars (at least on Windows 8.1 x64), but such DB can not be used normally.

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

> as you can't force user to not move\rename

Sorry, could not understand this properly when first read.
If user does move such database from <path+file> less then 250 to some 'deepest' place - this will be his own problem. I think that after getting errors he will change his mind and return DB to normal place.
What I suppose is: FB must prevent *creation* of database using such long path+names.

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Alex,
I hope you not going to implement it as is ?

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

No, I plan to truncate database name in found by Pavel places

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

summary: Ability to create database with length more than 255 characters should be prohibited => A number of errors when database name is longer than 255 symbols

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

Fixed code of gbak, monitoring tables and system function rdb$get_context(). Behavior is consistent with "good old" database info call used in 'show db' in isql, i.e. database name is truncated to 255 chars. Appropriate warnings are issued.

.delta suffix added in nbackup is not an issue at all - ALTER DATABASE ADD DIFFERENCE FILE should be used for databases with long name.

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 4.0 Beta 2 [ 10888 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

> Fixed code of gbak ... database name is truncated to 255 chars

This:

gbak:readied database <CUTED-OFF name on 261th character, ellipsis instead of tailing text>
gbak:creating file <CUTED-OFF name on 261th character, ellipsis instead of tailing text>
gbak:starting transaction
gbak:database <CUTED-OFF name on 261th character, ellipsis instead of tailing text>

- still exists (at least on WI-T4.0.0.1796 ), and visible name of DB file is ** 243 ** chars rather 250.

(example:
gbak:database c:\temp\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345...
)

Why can't gbak print FULL name of database file, without truncation and adding ellipsis ?

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

PS. Further, if gbak switches contain '-st tdrw' then name of DB file becomes yet shorter: 235, 243 & 238:

gbak: time delta reads writes
gbak: 0.274 0.274 82 1 readied database c:\temp\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567...
gbak: 0.274 0.000 0 0 creating file c:\temp\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890...
gbak: 0.274 0.000 0 0 starting transaction
gbak: 0.276 0.001 10 1 database c:\temp\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345...
gbak: 0.276 0.000 0 0 writing domains
. . .

Am i right in guess that some limitation in gbak utility exists for "decoration" purposes to make text width no more than 260 chars (when we run it without '-st tdrw') or 290 chars (when we use '-st tdrw') ?

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

PPS.

Command "gstat -h <dbname_with_length_255_characters>" truncates visible part of DB file name to 241 crahacter.

For example, if name of database is:

C:\FBTESTING\QA\FBT-REPO\TMP\ABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255.FDB
(length = 255 chars)

-- then "gstat -h" for this DB will show in the 1st line:

Database "C:\FBTESTING\QA\FBT-REPO\TMP\ABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC25...

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Done with caveats

Test Details: STDOUT-logs of backup, restore and gstat currently (09-mar-2020) have only truncated name of DB with length about 230...240 chars.
Test verifies presense of such truncated strings rather then expected full path + name of DB.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @asfernandes

Link: This issue is related to CORE6419 [ CORE6419 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment