Add a macro for testing assertion in both debug and production builds

If we have an assert then in a debug build we want an abort() to occur.
In a production build we want the function to return an error.

This introduces a new macro to assist with that. The idea is to replace
existing use of OPENSSL_assert() with this new macro. The problem with
OPENSSL_assert() is that it aborts() on an assertion failure in both debug
and production builds. It should never be a library's decision to abort a
process (we don't get to decide when to kill the life support machine or
the nuclear reactor control system). Additionally if an attacker can
cause a reachable assert to be hit then this can be a source of DoS attacks
e.g. see CVE-2017-3733, CVE-2015-0293, CVE-2011-4577 and CVE-2002-1568.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3496)

1 files changed, 18 insertions, 0 deletions

diff --git a/e_os.h b/e_os.h
index 914a2bf4ac..49a0c3ed18 100644
--- a/e_os.h
+++ b/e_os.h
@@ -13,6 +13,7 @@
 # include <openssl/opensslconf.h>
 
 # include <openssl/e_os2.h>
+# include <openssl/crypto.h>
 /*
  * <openssl/e_os2.h> contains what we can justify to make visible to the
  * outside; this file e_os.h is not part of the exported interface.
@@ -544,6 +545,23 @@ struct servent *getservbyname(const char *name, const char *proto);
 # define CRYPTO_memcmp memcmp
 #endif
 
+#ifdef NDEBUG
+# define ossl_assert(x) (int)(x)
+#else
+__owur static ossl_inline int ossl_assert_int(int expr, const char *exprstr,
+                                              const char *file, int line)
+{
+    if (!expr)
+        OPENSSL_die(exprstr, file, line);
+
+    return expr;
+}
+
+# define ossl_assert(x) ossl_assert_int((int)(x), "Assertion failed: "#x, \
+                                         __FILE__, __LINE__)
+
+#endif
+
 #ifdef  __cplusplus
 }
 #endif