Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade from 3.0.3 to 3.0.4 or 3.0.5-SNAPSHOT is broken. Results in "Encryption key did not meet algorithm requirements of Symmetric/Arc4" [JDBC543] #576

Closed
firebird-automations opened this issue Aug 22, 2018 · 21 comments

Comments

@firebird-automations
Copy link

Submitted by: VENKATESH DODDATHIMMAIAH (venkatesh)

Is related to JDBC544
Is related to JDBC545
Relate to JDBC546

Attachments:
AppDb.java
HelloJaybird.java
pom.xml
Encryption key did not meet algorithm requirements of SymmetricArc4.txt
mobileclient.jar
Build_Log_SonaType.txt
Crypto_Permission_Check_Failed.txt
Java_1.8_161 - Jaybird JDK18 - 3.0.5-SNAPSHOT.txt

Application
-------------------------------------------------------------------
A Java application for Firebird 3.0.3 via Jaybird 3.0.3 (being upgraded to 3.0.5-SNAPSHOT to utilize Fix from JDBC542).

History
-------------------------------------------------------------------
1. Jaybird 3.0.3 had issue - JDBC542
2. It is fixed in 3.0.5-SNAPSHOT as per the ticket
3. While upgrading from 3.0.3 to 3.0.5-SNAPSHOT, maven build throws exceptions as in attached log file "Encryption key did not meet algorithm requirements of SymmetricArc4.txt"
4. The upgrade was attempted by just changing the version from 3.0.3 to 3.0.5-SNAPSHOT

Other analysis
1. With jaybird-jdk17 and 3.0.4 also, the exception is thrown
2. With jaybird-jdk18 and 3.0.4 also, the exception is thrown
3. With jaybird-jdk17 and 3.0.5-SNAPSHOT also, the exception is thrown. However, the issue of JDBC542 seems to have been fixed.

Firebird configuration
WireCrypt = Enabled. Have also changed this to Required, but still results in exception.

Simulation
-------------------------------------------------------------------
Attached is a sample application (with two Java files and associated pom) which can be used to directly simulate the issue.

Prerequisites
-------------------------------------------------------------------
1. Sample & valid FDB database file in a specific file system location
2. Firebird database server

Steps
-------------------------------------------------------------------
1. Change the source file "AppDb.java" to reflect above database file path & associated credentials
2. Build the application from the source as a maven project to get "mobileclient.jar"
3. Ensure Firebird server is up and running
4. Run the jar from a command prompt using "java -jar mobileclient.jar" and watch for the informational logs

Observation
-------------------------------------------------------------------
1. The exception logs in the attached file are visible

In contrary if project is built with 3.0.3, exceptions are not thrown

@firebird-automations
Copy link
Author

Modified by: VENKATESH DODDATHIMMAIAH (venkatesh)

Attachment: AppDb.java [ 13289 ]

Attachment: HelloJaybird.java [ 13290 ]

Attachment: pom.xml [ 13291 ]

@firebird-automations
Copy link
Author

Modified by: VENKATESH DODDATHIMMAIAH (venkatesh)

Attachment: Encryption key did not meet algorithm requirements of SymmetricArc4.txt [ 13292 ]

@firebird-automations
Copy link
Author

Commented by: @mrotteveel

I can't reproduce the problem. Could you send me a copy of the mobileclient.jar built on your machine so that I can check what the difference is?

How did you install the snapshot into your local repository and what is your Maven version? I do run into the "No suitable driver found" issue if I install using
mvn install:install-file -Dfile=jaybird-jdk17-3.0.5-SNAPSHOT.jar -DgroupId=org.firebirdsql.jdbc -DartifactId=jaybird-jdk17 -Dversion=3.0.5-SNAPSHOT -Dpackaging=jar

This will generate a POM without dependencies, which means that the required connector-api dependencies is missing.

Instead, extract the META-INF/maven/org.firebirdsql.jdbc/jaybird-jdk17.jar/pom.xml from the archive and use

mvn install:install-file -Dfile=jaybird-jdk17-3.0.5-SNAPSHOT.jar -DpomFile=pom.xml

Alternatively, add an explicit dependency on connector-api:

    <dependency\>
        <groupId\>javax\.resource</groupId\>
        <artifactId\>connector\-api</artifactId\>
        <version\>1\.5</version\>
    </dependency\>

@firebird-automations
Copy link
Author

Commented by: VENKATESH DODDATHIMMAIAH (venkatesh)

First I had tried to have the jar deployed in a local maven repository. This approach resulted in the exception. Below is the command used
Next, I cleaned all references to local maven versions and just updated the version in the original project pom file to pull from maven central directly. Even this approach is resulting in same exception.

I have attached the sample "mobileclient.jar" which throws exceptions. It points to Firebird example Employee.fdb with SYSDBA/masterkey.

Command used
------------------------------------------
mvn deploy:deploy-file -Dfile="<PATH_TO>\Jaybird-3.0.5-SNAPSHOT-JDK_1.7\jaybird-3.0.5-SNAPSHOT.jar" -DgroupId=org.firebirdsql.jdbc -DartifactId=jaybird-jdk17 -Dversion=3.0.5-SNAPSHOT -Dpackaging=jar -Durl=file:./maven-repository

Will try out both the above approaches and update.

@firebird-automations
Copy link
Author

Modified by: VENKATESH DODDATHIMMAIAH (venkatesh)

Attachment: mobileclient.jar [ 13294 ]

@firebird-automations
Copy link
Author

Commented by: @mrotteveel

The mobileclient.jar you attached was built with Jaybird 3.0.4, which for me exhibits the exact same behavior as I can reproduce with 3.0.3 (that is, the problem you reported in JDBC542). Which is what I expect with that version.

The only problem I observe when using 3.0.5-SNAPSHOT is when installing Jaybird into the local repository like you did, because then it fails because of the missing dependency information in the pom that maven generates when it installs like that.

You will need to install the dependency with an explicit POM, as detailed in my previous comment, or otherwise try removing the 3.0.5-SNAPSHOT from your local repository, and instead add https://oss.sonatype.org/content/repositories/snapshots as a snapshot repository in your maven config. I have uploaded the 3.0.5-SNAPSHOT versions there.

@firebird-automations
Copy link
Author

Commented by: VENKATESH DODDATHIMMAIAH (venkatesh)

I removed all firebird jdbc driver references from local maven and then configured maven to pull only from sonatype.

I have attached the build log which captures downloading logs of 3.0.5-SNAPSHOT from sonatype repository.

When we run the final built jar, results in same exception.

@firebird-automations
Copy link
Author

Modified by: VENKATESH DODDATHIMMAIAH (venkatesh)

Attachment: Build_Log_SonaType.txt [ 13295 ]

@firebird-automations
Copy link
Author

Commented by: @mrotteveel

Can you upload the mobileclient.jar you built that way? And the exception you see now is still that "Encryption key did not meet algorithm requirements of Symmetric/Arc4"?

The only other option I can think of that you are using a security policy that disallows RC4 (which is used by the Firebird 3 wire encryption which was introduced in Jaybird 3.04). Could you show the output of :

java -Djava.security.debug=jca -jar mobileclient.jar

@firebird-automations
Copy link
Author

Commented by: @mrotteveel

With some experimentation, I managed to reproduce the problem. Your Java version is probably applying the limited Cryptographic Jurisdiction Policy (this is the default for Java 8 Update 152 and earlier), and given the implementation we use RC4 with a keysize of 160, while the limited policy only allows key sizes of 128 or smaller.

For some reason I had assumed that this policy did not influence RC4, and my Java installs have been using unlimited for years.

You have the following options:

* Change your policy to unlimited, see https://stackoverflow.com/a/3864276/466862
* Upgrade to Java 8 update 161 or higher (those versions default to the unlimited policy)
* Disable WireCrypt in Firebird (not advisable)
* Disable wireCrypt in Jaybird (see https://www.firebirdsql.org/file/documentation/drivers_documentation/java/3.0.x/release_notes.html#wire-encryption-support), unfortunately this is not possible for FBEventManager yet (I just noticed I didn't add support for this property there).

@firebird-automations
Copy link
Author

Modified by: @mrotteveel

Link: This issue is related to JDBC544 [ JDBC544 ]

@firebird-automations
Copy link
Author

Modified by: @mrotteveel

Link: This issue is related to JDBC545 [ JDBC545 ]

@firebird-automations
Copy link
Author

Commented by: @mrotteveel

I just noticed you are already using Java 8 update 161 which should default to the unlimited policy. In that case it would still be helpful to have the output of

java -Djava.security.debug=jca -jar mobileclient.jar

Or is it possible that your Java install has explicitly set the policy to limited?

@firebird-automations
Copy link
Author

Modified by: @mrotteveel

Link: This issue relate to JDBC546 [ JDBC546 ]

@firebird-automations
Copy link
Author

Commented by: VENKATESH DODDATHIMMAIAH (venkatesh)

Awesome! I think you nailed it.

I could see below in the logs
----------------------------------------------------
:
Cipher: Crypto Permission check failed
Cipher: granted: (CryptoPermission * 128)
Cipher: requesting: (CryptoPermission ARCFOUR 160)
:
----------------------------------------------------

Full log attached as "Crypto_Permission_Check_Failed.txt"

Will follow the link - "https://stackoverflow.com/questions/3862800/invalidkeyexception-illegal-key-size/3864276#3864276" to increase the limit. Hopefully it should resolve.

@firebird-automations
Copy link
Author

Modified by: VENKATESH DODDATHIMMAIAH (venkatesh)

Attachment: Crypto_Permission_Check_Failed.txt [ 13296 ]

@firebird-automations
Copy link
Author

Commented by: @mrotteveel

Yes, that debug logging is the same as what I saw when I changed my configuration to the limited policy. Switching to unlimited should then solve it.

I have created a number of follow up tickets to address this further: JDBC544 (allow wireCrypt to be disabled for FBEventManagr), JDBC545 (document this problem and solutions) and JDBC546 (let Jaybird default to wireCrypt DISABLED if the cipher + key size is disallowed by the policy).

Thanks for taking the time of reporting this and helping me in finding the underlying problem.

@firebird-automations
Copy link
Author

Commented by: @mrotteveel

To be clear, even with the limited policy the connection (and event manager) will work, it will just log this as warning and the connection will not be encrypted, unless the wireCrypt is set to REQUIRED, in which case this will be a connection error.

As part of JDBC546, I will see if I can make the logging in this case less intrusive (eg maybe log the full stacktrace only once on warn and for the rest on debug) if checking the permission is too involved.

@firebird-automations
Copy link
Author

Commented by: VENKATESH DODDATHIMMAIAH (venkatesh)

Options to manage crypto policy limitations

Option 1:
To download the packages as in - http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
Reference
- https://stackoverflow.com/questions/6481627/java-security-illegal-key-size-or-default-parameters?noredirect=1&lq=1
- https://golb.hplar.ch/2017/10/JCE-policy-changes-in-Java-SE-8u151-and-8u152.html

Option 2:
In u151 and u152 version of Java 8, we can directly handle in code as
"Security.setProperty("crypto.policy", "unlimited");"

Option 3:
Set the unlimited policy in the <jre_home>/lib/security/java.security file by uncommenting line #⁠crypto.policy=unlimited
We didn't have an entry itself in the file. So we couldn't try on Option 3. Perhaps this is for even older versions.

Our production servers are at Java "1.8 161" and our dev at "1.8 144", but lot has changed between these multiple versions.
For our dev - we had to go with Option 1, which worked. Also, we prefer & intend to keep it in sync with prod java version i.e. 1.8 161.
For our prod - we didn't have to do anything and it worked. But if any prod server java versions falling between 151 and 152, might need Option 2 as a better approach over Option 1. Even this approach worked.

The sample was verified for both issues, this and especially the original JDBC542
1. No Encryption related exceptions observed
2. When Firebird was killed, only one entry of IOException was logged and CPU remained stable

Log attached "Java_1.8_161 - Jaybird JDK18 - 3.0.5-SNAPSHOT.txt"

I greatly appreciate all the time and support in resolving these issues and clearing the way for production roll outs. Thank you once again.

@firebird-automations
Copy link
Author

Modified by: VENKATESH DODDATHIMMAIAH (venkatesh)

Attachment: Java_1.8_161 - Jaybird JDK18 - 3.0.5-SNAPSHOT.txt [ 13297 ]

@firebird-automations
Copy link
Author

Modified by: @mrotteveel

status: Open [ 1 ] => Closed [ 6 ]

resolution: Fixed [ 1 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants