doc: document loadable module support

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>

16 files changed, 260 insertions, 6 deletions

diff --git a/doc/basic.texi b/doc/basic.texi
index cea33eaa8..05d72bc80 100644
--- a/doc/basic.texi
+++ b/doc/basic.texi
@@ -18,6 +18,7 @@ daemons.
 * Config Commands::             Commands used in config files
 * Terminal Mode Commands::      Common commands used in a VTY
 * Common Invocation Options::   Starting the daemons
+* Loadable Module Support::     Using extension modules
 * Virtual Terminal Interfaces:: Interacting with the daemons
 @end menu
 
@@ -372,6 +373,51 @@ Print program version.
 @end table
 
 
+@node Loadable Module Support
+@section Loadable Module Support
+
+FRR supports loading extension modules at startup.  Loading, reloading or
+unloading modules at runtime is not supported (yet).  To load a module, use
+the following command line option at daemon startup:
+
+@table @samp
+@item -M @var{module:options}
+@itemx --module @var{module:options}
+
+Load the specified module, optionally passing options to it.  If the module
+name contains a slash (/), it is assumed to be a full pathname to a file to
+be loaded.  If it does not contain a slash, the
+@code{@value{INSTALL_PREFIX_MODULES}} directory is searched for a module of
+the given name; first with the daemon name prepended (e.g. @code{zebra_mod}
+for @code{mod}), then without the daemon name prepended.
+
+This option is available on all daemons, though some daemons may not have
+any modules available to be loaded.
+@end table
+
+
+@subsection The SNMP Module
+
+If SNMP is enabled during compile-time and installed as part of the package,
+the @code{snmp} module can be loaded for the @command{zebra},
+@command{bgpd}, @command{ospfd}, @command{ospf6d} and @command{ripd} daemons.
+
+The module ignores any options passed to it.  Refer to @ref{SNMP Support}
+for information on its usage.
+
+
+@subsection The FPM Module
+
+If FPM is enabled during compile-time and installed as part of the package,
+the @code{fpm} module can be loaded for the @command{zebra} daemon.  This
+provides the Forwarding Plane Manager ("FPM") API.
+
+The module expects its argument to be either @code{netlink} or
+@code{protobuf}, specifying the encapsulation to use.  @code{netlink} is the
+default, and @code{protobuf} may not be available if the module was built
+without protobuf support.  Refer to @ref{zebra FIB push interface} for more
+information.
+
 
 @node Virtual Terminal Interfaces
 @section Virtual Terminal Interfaces
diff --git a/doc/bgpd.8.in b/doc/bgpd.8.in
index 9026f2cde..0df1b1dce 100644
--- a/doc/bgpd.8.in
+++ b/doc/bgpd.8.in
@@ -27,6 +27,9 @@ bgpd \- a BGPv4, BGPv4\+, BGPv4\- routing engine for use with @PACKAGE_FULLNAME@
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B bgpd 
@@ -76,6 +79,11 @@ When the program terminates, retain routes added by \fBbgpd\fR.
 \fB\-S\fR, \fB\-\-skip_runas\fR
 Skip setting the process effective user and group.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+The \fBsnmp\fR module may be available for
+\fBbgpd\fR, if the package was built with SNMP support.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES
diff --git a/doc/defines.texi.in b/doc/defines.texi.in
index 43d744293..0fadba964 100644
--- a/doc/defines.texi.in
+++ b/doc/defines.texi.in
@@ -13,6 +13,7 @@
 @set INSTALL_PREFIX_ETC @CFG_SYSCONF@
 @set INSTALL_PREFIX_SBIN @CFG_SBIN@
 @set INSTALL_PREFIX_STATE @CFG_STATE@
+@set INSTALL_PREFIX_MODULES @CFG_MODULE@
 @set INSTALL_USER @enable_user@
 @set INSTALL_GROUP @enable_group@
 @set INSTALL_VTY_GROUP @enable_vty_group@
diff --git a/doc/dev-modules.md b/doc/dev-modules.md
new file mode 100644
index 000000000..87bc96318
--- /dev/null
+++ b/doc/dev-modules.md
@@ -0,0 +1,119 @@
+# Module and Hook support (developer docs)
+
+## What it does
+
+It uses `dlopen()` to load DSOs at startup.
+
+
+## Limitations
+
+* can't load, unload, or reload during runtime.  This just needs some work
+  and can probably be done in the future.
+* doesn't fix any of the "things need to be changed in the code in the library"
+  issues.  Most prominently, you can't add a CLI node because CLI nodes are
+  listed in the library...
+* if your module crashes, the daemon crashes.  Should be obvious.
+* **does not provide a stable API or ABI**.  Your module must match a version
+  of FRR and you may have to update it frequently to match changes.
+* **does not create a license boundary**.  Your module will need to link
+  libzebra and include header files from the daemons, meaning it will be
+  GPL-encumbered.
+
+
+## Installation
+
+Look for `moduledir` in `configure.ac`, default is normally
+`/usr/lib64/frr/modules` but depends on `--libdir` / `--prefix`.
+
+The daemon's name is prepended when looking for a module, e.g. "snmp" tries
+to find "zebra_snmp" first when used in zebra.  This is just to make it nicer
+for the user, with the snmp module having the same name everywhere.
+
+Modules can be packaged separately from FRR.  The SNMP and FPM modules are
+good candidates for this because they have dependencies (net-snmp / protobuf)
+that are not FRR dependencies.  However, any distro packages should have an
+"exact-match" dependency onto the FRR package.  Using a module from a
+different FRR version will probably blow up nicely.
+
+For snapcraft (and during development), modules can be loaded with full path
+(e.g. -M `$SNAP/lib/frr/modules/zebra_snmp.so`).  Note that libtool puts output
+files in the .libs directory, so during development you have to use
+`./zebra -M .libs/zebra_snmp.so`.
+
+
+## Creating a module
+
+... best to look at the existing SNMP or FPM modules.
+
+Basic boilerplate:
+
+```
+#include "hook.h"
+#include "module.h"
+
+static int
+module_init (void)
+{
+  hook_register(frr_late_init, module_late_init);
+  return 0;
+}
+
+FRR_MODULE_SETUP(
+	.name = "my module",
+	.version = "0.0",
+	.description = "my module",
+	.init = module_init,
+)
+```
+
+The `frr_late_init` hook will be called after the daemon has finished its
+other startup and is about to enter the main event loop;  this is the best
+place for most initialisation.
+
+
+## Compiler & Linker magic
+
+There's a `THIS_MODULE` (like in the Linux kernel), which uses `visibility`
+attributes to restrict it to the current module.  If you get a linker error
+with `_frrmod_this_module`, there is some linker SNAFU.  This shouldn't be
+possible, though one way to get it would be to not include libzebra (which
+provides a fallback definition for the symbol).
+
+libzebra and the daemons each have their own `THIS_MODULE`, as do all loadable
+modules.  In any other libraries (e.g. `libfrrsnmp`), `THIS_MODULE` will use
+the definition in libzebra;  same applies if the main executable doesn't use
+`FRR_DAEMON_INFO` (e.g. all testcases).
+
+The deciding factor here is "what dynamic linker unit are you using the symbol
+from."  If you're in a library function and want to know who called you, you
+can't use `THIS_MODULE` (because that'll just tell you you're in the library).
+Put a macro around your function that adds `THIS_MODULE` in the *caller's
+code calling your function*.
+
+The idea is to use this in the future for module unloading.  Hooks already
+remember which module they were installed by, as groundwork for a function
+that removes all of a module's installed hooks.
+
+There's also the `frr_module` symbol in modules, pretty much a standard entry
+point for loadable modules.
+
+
+## Hooks
+
+Hooks are just points in the code where you can register your callback to
+be called.  The parameter list is specific to the hook point.  Since there is
+no stable API, the hook code has some extra type safety checks making sure
+you get a compiler warning when the hook parameter list doesn't match your
+callback.  Don't ignore these warnings.
+
+
+## Relation to MTYPE macros
+
+The MTYPE macros, while primarily designed to decouple MTYPEs from the library
+and beautify the code, also work very nicely with loadable modules -- both
+constructors and destructors are executed when loading/unloading modules.
+
+This means there is absolutely no change required to MTYPEs, you can just use
+them in a module and they will even clean up themselves when we implement
+module unloading and an unload happens.  In fact, it's impossible to create
+a bug where unloading fails to de-register a MTYPE.
diff --git a/doc/isisd.8.in b/doc/isisd.8.in
index 9ffcbc618..542c28993 100644
--- a/doc/isisd.8.in
+++ b/doc/isisd.8.in
@@ -23,6 +23,9 @@ isisd \- an IS-IS routing engine for use with @PACKAGE_FULLNAME@.
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B isisd
@@ -63,6 +66,11 @@ interfaces.
 \fB\-u\fR, \fB\-\-user \fR\fIuser\fR
 Specify the user to run as. Default is \fI@enable_user@\fR.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+There are currently no such modules for
+\fBisisd\fR in the base package.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES
diff --git a/doc/ldpd.8.in b/doc/ldpd.8.in
index 1683de46c..2d68a31a5 100644
--- a/doc/ldpd.8.in
+++ b/doc/ldpd.8.in
@@ -23,6 +23,9 @@ ldpd \- an LDP engine for use with @PACKAGE_FULLNAME@.
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B ldpd
@@ -63,6 +66,11 @@ interfaces.
 \fB\-u\fR, \fB\-\-user \fR\fIuser\fR
 Specify the user to run as. Default is \fI@enable_user@\fR.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+There are currently no such modules for
+\fBldpd\fR in the base package.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES
diff --git a/doc/main.texi b/doc/main.texi
index dfe02e1b5..706baa25f 100644
--- a/doc/main.texi
+++ b/doc/main.texi
@@ -385,7 +385,8 @@ ip protocol rip route-map RM1
 
 Zebra supports a 'FIB push' interface that allows an external
 component to learn the forwarding information computed by the Frr
-routing suite.
+routing suite.  This is a loadable module that needs to be enabled
+at startup as described in @ref{Loadable Module Support}.
 
 In Frr, the Routing Information Base (RIB) resides inside
 zebra. Routing protocols communicate their best routes to zebra, and
@@ -440,9 +441,9 @@ independently.
 @end itemize
 
 As mentioned before, zebra encodes routes sent to the FPM in netlink
-format by default. The format can be controlled via the
-@code{--fpm_format} command-line option to zebra, which currently
-takes the values @code{netlink} and @code{protobuf}.
+format by default. The format can be controlled via the FPM module's
+load-time option to zebra, which currently takes the values @code{netlink}
+and @code{protobuf}.
 
 The zebra FPM interface uses replace semantics. That is, if a 'route
 add' message for a prefix is followed by another 'route add' message,
diff --git a/doc/nhrpd.8.in b/doc/nhrpd.8.in
index c5e4f7e32..09b662ae7 100644
--- a/doc/nhrpd.8.in
+++ b/doc/nhrpd.8.in
@@ -23,6 +23,9 @@ nhrpd \- a Next Hop Routing Protocol routing engine for use with @PACKAGE_FULLNA
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B nhrpd
@@ -63,6 +66,11 @@ interfaces.
 \fB\-u\fR, \fB\-\-user \fR\fIuser\fR
 Specify the user to run as. Default is \fI@enable_user@\fR.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+There are currently no such modules for
+\fBnhrpd\fR in the base package.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES
diff --git a/doc/ospf6d.8.in b/doc/ospf6d.8.in
index 7f94782be..02d9d8083 100644
--- a/doc/ospf6d.8.in
+++ b/doc/ospf6d.8.in
@@ -23,6 +23,9 @@ ospf6d \- an OSPFv3 routing engine for use with @PACKAGE_FULLNAME@.
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B ospf6d
@@ -64,6 +67,11 @@ interfaces.
 \fB\-u\fR, \fB\-\-user \fR\fIuser\fR
 Specify the user to run as. Default is \fI@enable_user@\fR.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+The \fBsnmp\fR module may be available for
+\fBospf6d\fR, if the package was built with SNMP support.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES
diff --git a/doc/ospfd.8.in b/doc/ospfd.8.in
index 1b86551ca..6bad77771 100644
--- a/doc/ospfd.8.in
+++ b/doc/ospfd.8.in
@@ -23,6 +23,9 @@ ospfd \- an OSPFv2 routing engine for use with @PACKAGE_FULLNAME@.
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B ospfd
@@ -66,6 +69,11 @@ Specify the user to run as. Default is \fI@enable_user@\fR.
 \fB\-a\fR, \fB\-\-apiserver \fR
 Enable OSPF apiserver. Default is disabled.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+The \fBsnmp\fR module may be available for
+\fBospfd\fR, if the package was built with SNMP support.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES
diff --git a/doc/pimd.8.in b/doc/pimd.8.in
index 60b844b1e..3fb060e56 100644
--- a/doc/pimd.8.in
+++ b/doc/pimd.8.in
@@ -26,6 +26,9 @@ pimd \- a PIM routing for use with @PACKAGE_FULLNAME@.
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B pimd
@@ -70,6 +73,11 @@ interfaces.
 \fB\-u\fR, \fB\-\-user \fR\fIuser\fR
 Specify the user to run as. Default is \fI@enable_user@\fR.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+There are currently no such modules for
+\fBpimd\fR in the base package.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .TP
diff --git a/doc/ripd.8.in b/doc/ripd.8.in
index 6db5ac364..a84668e6d 100644
--- a/doc/ripd.8.in
+++ b/doc/ripd.8.in
@@ -23,6 +23,9 @@ ripd \- a RIP routing engine for use with @PACKAGE_FULLNAME@.
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B ripd
@@ -67,6 +70,11 @@ Specify the user to run as. Default is \fI@enable_user@\fR.
 \fB\-r\fR, \fB\-\-retain\fR 
 When the program terminates, retain routes added by \fBripd\fR.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+The \fBsnmp\fR module may be available for
+\fBripd\fR, if the package was built with SNMP support.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES
diff --git a/doc/ripngd.8.in b/doc/ripngd.8.in
index 4c5f2bb11..98039219a 100644
--- a/doc/ripngd.8.in
+++ b/doc/ripngd.8.in
@@ -23,6 +23,9 @@ ripngd \- a RIPNG routing engine for use with @PACKAGE_FULLNAME@.
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B ripngd
@@ -67,6 +70,11 @@ Specify the user to run as. Default is \fI@enable_user@\fR.
 \fB\-r\fR, \fB\-\-retain\fR 
 When the program terminates, retain routes added by \fBripd\fR.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+There are currently no such modules for
+\fBripngd\fR in the base package.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES
diff --git a/doc/snmp.texi b/doc/snmp.texi
index c2c889de7..d9656941d 100644
--- a/doc/snmp.texi
+++ b/doc/snmp.texi
@@ -8,6 +8,10 @@ but is able to connect to a SNMP agent using the SMUX protocol
 (@cite{RFC1227}) or the AgentX protocol (@cite{RFC2741}) and make the
 routing protocol MIBs available through it.
 
+Note that SNMP Support needs to be enabled at compile-time and loaded as
+module on daemon startup.  Refer to @ref{Loadable Module Support} on
+the latter.
+
 @menu
 * Getting and installing an SNMP agent::
 * AgentX configuration::
diff --git a/doc/watchfrr.8.in b/doc/watchfrr.8.in
index 813f87abd..82098e1b0 100644
--- a/doc/watchfrr.8.in
+++ b/doc/watchfrr.8.in
@@ -108,13 +108,13 @@ Set the logging
 (LOG_DEBUG), but higher number can be supplied if extra debugging messages
 are required.
 .TP
-.BI \-m " number" "\fR, \fB\-\-min\-restart\-interval " number
+.BI \-\-min\-restart\-interval " number
 Set the minimum
 .I number
 of seconds to wait between invocations of the daemon restart commands (the
 default value is "60").
 .TP
-.BI \-M " number" "\fR, \fB\-\-max\-restart\-interval " number
+.BI \-\-max\-restart\-interval " number
 Set the maximum
 .I number
 of seconds to wait between invocations of the daemon restart commands (the
diff --git a/doc/zebra.8.in b/doc/zebra.8.in
index 4599a8563..f5b8bd4d8 100644
--- a/doc/zebra.8.in
+++ b/doc/zebra.8.in
@@ -23,6 +23,9 @@ zebra \- a routing manager for use with associated @PACKAGE_FULLNAME@ components
 ] [
 .B \-g
 .I group
+] [
+.B \-M
+.I module:options
 ]
 .SH DESCRIPTION
 .B zebra 
@@ -86,6 +89,14 @@ maximum before starting zebra.
 
 Note that this affects Linux only.
 .TP
+\fB\-M\fR, \fB\-\-module \fR\fImodule:options\fR
+Load a module at startup.  May be specified more than once.
+The \fBsnmp\fR and \fBfpm\fR modules may be
+available for \fBzebra\fR, if the package was built with SNMP and FPM support
+respectively.  The \fBfpm\fR module takes an additional colon-separated
+argument specifying the encapsulation, either \fBnetlink\fR or \fBprotobuf\fR.
+It should thus be loaded with \fB-M fpm:netlink\fR or \fB-M fpm:protobuf\fR.
+.TP
 \fB\-v\fR, \fB\-\-version\fR
 Print the version and exit.
 .SH FILES