You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Allow some tests to be specified as non-critical at the definition level (.fbt file), individually for different FB versions. In the case of actual vs expected mismatch they should be reported as warnings instead of failures and if there are no other critical failures the overall test run should be treated as "passed".
This would allow tests to be written early, as soon as some bug is registered. This is especially important for some well-known (including legacy) issues that remain alive for many years. They are not regressions, so they should not invalidate test runs. But automagically reminding developers about them would be a good thing.
I believe this is a much better approach than remembering what failures are actually OK for the given version.
The text was updated successfully, but these errors were encountered:
Submitted by: @dyemanov
Allow some tests to be specified as non-critical at the definition level (.fbt file), individually for different FB versions. In the case of actual vs expected mismatch they should be reported as warnings instead of failures and if there are no other critical failures the overall test run should be treated as "passed".
This would allow tests to be written early, as soon as some bug is registered. This is especially important for some well-known (including legacy) issues that remain alive for many years. They are not regressions, so they should not invalidate test runs. But automagically reminding developers about them would be a good thing.
I believe this is a much better approach than remembering what failures are actually OK for the given version.
The text was updated successfully, but these errors were encountered: