According to the SQL standard, floating point values that aren't regular values should be considered errors. The engine shouldn't produce NaN or Infinity:
SQL> select asin(2), cot(0) from rdb$database;
SQL> select acos(2) - acos(2) from rdb$database;
We have two options: detect the problem once it has occurred (and give a generic message) or check the parameters. Since we know the functions we offer, the latter seems feasible.