iobuf: Increase the size of the buffer. Add iobuf_set_buffer_size.

* common/iobuf.c (IOBUF_BUFFER_SIZE): Rename to
DEFAULT_IOBUF_BUFFER_SIZE and increase to 64k.
(iobuf_buffer_size): New var.  Always use this instead of the macro.
(iobuf_set_buffer_size): New.
(struct file_filter_ctx_t): Add field delayed_rc.
(file_filter) [!W32]: Try to fill the supplied buffer.
--

I did some test to see whether this has an effect.  A test program
piped 100 million random bytes to gpg to symmetric encryption only w/0
compression.  Single read means the old behaviour, multi read the new
behaviour which fills up the buffer when the read(2) returned only 4k
in once call.

8k buffer single read
        User time (seconds): 0.09
        System time (seconds): 0.04
        Percent of CPU this job got: 6%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.04

8k buffer multi read
       User time (seconds): 0.08
       System time (seconds): 0.05
       Percent of CPU this job got: 6%
       Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.04

64k buffer single read
        User time (seconds): 0.09
        System time (seconds): 0.06
        Percent of CPU this job got: 6%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.20

64k buffer multi read
        User time (seconds): 0.11
        System time (seconds): 0.06
        Percent of CPU this job got: 8%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.01

128k buffer single read
        User time (seconds): 0.09
        System time (seconds): 0.05
        Percent of CPU this job got: 7%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.05

128k buffer multi read
        User time (seconds): 0.11
        System time (seconds): 0.05
        Percent of CPU this job got: 8%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.01

512k buffer single read:
        User time (seconds): 0.08
        System time (seconds): 0.08
        Percent of CPU this job got: 7%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.21

512k buffer multi read:
        User time (seconds): 0.10
        System time (seconds): 0.06
        Percent of CPU this job got: 7%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.05

Does not make much of a difference :-(.  Maybe it changes depending on
the type of used filters.

Signed-off-by: Werner Koch <wk@gnupg.org>

1 files changed, 6 insertions, 0 deletions

diff --git a/common/iobuf.h b/common/iobuf.h
index 22e02daad..16156383c 100644
--- a/common/iobuf.h
+++ b/common/iobuf.h
@@ -259,6 +259,12 @@ struct iobuf_struct
 EXTERN_UNLESS_MAIN_MODULE int iobuf_debug_mode;
 
 
+/* Change the default size for all IOBUFs to KILOBYTE.  This needs to
+ * be called before any iobufs are used and can only be used once.
+ * Returns the current value.  Using 0 has no effect except for
+ * returning the current value.  */
+unsigned int iobuf_set_buffer_size (unsigned int kilobyte);
+
 /* Returns whether the specified filename corresponds to a pipe.  In
    particular, this function checks if FNAME is "-" and, if special
    filenames are enabled (see check_special_filename), whether