1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
|
// Copyright (C) 2018-2021 Internet Systems Consortium, Inc. ("ISC")
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
/**
@page congestionHandling Congestion Handling in Kea DHCP Servers
@section background What is Congestion?
Congestion occurs when servers are subjected to client queries
faster than they can be fulfilled. Subsequently, the servers begin
accumulating a backlog of pending queries. The longer the high rate of
traffic continues the farther behind the servers fall. Depending on the
client implementations, those that fail to get leases either give up or simply
continue to retry forever. In the former case, the server may eventually
recover. The latter case is vicious cycle from which the server is unable
to escape.
In a well-planned deployment, the number and capacity of servers is matched
to the maximum client loads expected. As long as capacity is matched to
load, congestion does not occur. If the load is routinely too heavy, then
the deployment needs to be re-evaluated. Congestion typically occurs when
there is a network event that causes overly large numbers of clients to
simultaneously need leases such as recovery after a network outage.
@section introduction Congestion Handling Overview
Kea 1.5.0 introduces a new feature referred to as Congestion Handling. The
goal of Congestion Handling is to help the servers mitigate the peak
in traffic by fulfilling as many of the most relevant requests as possible
until it subsides.
Prior to Kea 1.5.0, Kea DHCP servers read inbound packets directly
from the interface sockets in the main application thread. This meant that
packets waiting to be processed were held in socket buffers themselves. Once
these buffers fill any new packets are discarded. Under swamped conditions
the servers can end up processing client packets that may no longer be
relevant, or worse are redundant. In other words, the packets waiting in
the FIFO socket buffers become increasingly stale.
Congestion Handling offers the ability to configure the server to use a
separate thread to read packets from the interface socket buffers. As the
thread reads packets from the buffers they are added to an internal "packet
queue". The server's main application thread processes packets from this queue
rather than the socket buffers. By structuring it this way, we've introduced
a configurable layer which can make decisions on which packets to process,
how to store them, and the order in which they are processed by the server.
The default packet queue implementation for both Kea DHCPv4 and DHCPv6 servers
is a simple ring buffer. Once it reaches capacity, new packets get added to
the back of queue by discarding packets from the front of queue. Rather than
always discarding the newest packets, we now always discard the oldest
packets. The capacity of the buffer (i.e. the maximum number of packets the
buffer can contain) is configurable.
@section custom-implementations Custom Packet Queues
It is possible to replace the default packet queue implementation with a
custom implementation by registering it with your Kea server via a hook
library. The steps for doing this are listed below:
-# Develop a derivation of the interface isc::dhcp::PacketQueue
-# Registering and un-registering your implementation via Hook library
-# Configure your Kea server to use your derivation
(If you are not familiar with writing Kea hook libraries, you may wish to
read @ref hooksdgDevelopersGuide before continuing).
@subsection packet-queue-derivation Developing isc::dhcp::PacketQueue Derivations
@subsection packet-queue-derivation-basics The Basics
Your custom packet queue must derive from the class template,
isc::dhcp::PacketQueue. The class is almost entirely abstract and
deliberately brief to provide developers wide latitude in the internals
of their solutions.
The template argument, @c PacketTypePtr, is expected to be either
isc::dhcp::Pkt4Ptr or isc::dhcp::Pkt6Ptr, depending upon which
protocol the implementation will handle. Please note that the
while following text and examples largely focus on DHCPv4 out
of convenience as the concepts are identical for DHCPv6. For
completeness there are code snippets at the end of this
chapter for DHCPv6.
The two primary functions of interest are:
-# isc::dhcp::PacketQueue::enqueuePacket() - This function is invoked by
the receiver thread each time a packet has been read from an interface
socket buffer and should be added to the queue. It is passed a pointer to
the unpacked client packet (isc::dhcp::Pkt4Ptr or isc::dhcp::Pkt6Ptr), and
a reference to the isc::dhcp::SocketInfo describing the interface socket
from which the packet was read. Your derivation is free to use whatever
logic you deem appropriate to decide if a given packet should be added
to the queue or dropped. The socket information is passed along to be used
(or not) in your decision making. The simplest derivation would add every
packet, every time.
-# isc::dhcp::PacketQueue::dequeuePacket() - This function is invoked by the
server's main thread whenever the receiver thread indicates that packets are
ready. Which packet is dequeued and returned is entirely up to your
derivation.
The remaining functions that you'll need to implement are self-explanatory.
How your actual "queue" is implemented is entirely up to you. Kea's default
implementation using a ring buffer based on Boost's boost::circular_buffer
(please refer to isc::dhcp::PacketQueueRing, isc::dhcp::PacketQueueRing4 and
isc::dhcp::PacketQueueRing6). The most critical aspects to remember when
developing your implementation are:
-# It MUST be thread safe since queuing and dequeuing packets are done by
separate threads. (You might considering using std::mutex and std::lock_guard).
-# Its efficiency (or lack thereof) will have a direct impact on server
performance. You will have to consider the dynamics of your deployment
to determine where the trade-off lies between the volume of packets responded
to and preferring to respond to some subset of those packets.
@subsection packet-queue-derivation-factory Defining a Factory
isc::dhcp::IfaceMgr using two derivations of isc::dhcp::PacketQueueMgr (one for
DHCPv4 and one for DHCPv6), to register queue implementations and instantiate
the appropriate queue type based the current configuration. In order to
register your queue implementation your hook library must provide a factory
function that will be used to create packet queues. This function will be
invoked by the server during the configuration process to instantiate the
appropriate queue type. For DHCPv4, the factory should be as follows:
@code
PackQueue4Ptr factory(isc::data::ConstElementPtr parameters)
@endcode
and for DHCPv6:
@code
PackQueue6Ptr factory(isc::data::ConstElementPtr parameters)
@endcode
The factory's only argument is an isc::data::ConstElementPtr. This is will be
an isc::data::MapElement instance containing the contents of the configuration
element "dhcp-queue-control" from the Kea server's configuration. It will
always have the following two values:
-# "enable-queue" - used by isc::dhcp::IfaceMgr to know whether
congestion handling is enabled. Your implementation need not do anything
with this value.
-# "queue-type" - name of the registered queue implementation to use.
It is used by isc::dhcp::IfaceMgr to invoke the appropriate queue factory.
Your implementation must pass this value through to the isc::dhcp::PacketQueue
constructor.
Beyond that you may add whatever additional values you may require. In
other words, the content is arbitrary so long as it is valid JSON. It is
up to your factory implementation to examine the contents and use them
to construct a queue instance.
@subsection packet-queue-derivation-example An Example
Let's suppose you wish to develop a queue for DHCPv4 and your implementation
requires two configurable parameters: capacity and threshold. Your class
declaration might look something like this:
@code
class YourPacketQueue4 : public isc::dhcp::PacketQueue<isc::dhcp::Pkt4Ptr> {
public:
// Logical name you will register your factory under.
static const std::string QUEUE_TYPE;
// Factory for instantiating queue instances.
static isc::dhcp::PacketQueue4Ptr factory(isc::data::ConstElementPtr params);
// Constructor
YourPacketQueue4(const std::string& queue_type, size_t capacity, size_t threshold)
: isc::dhcp::PacketQueue<isc::dhcp::Pkt4Ptr>(queue_type) {
// your constructor steps here
}
// Adds a packet to your queue using your secret formula based on threshold.
virtual void enqueuePacket(isc::dhcp::Pkt4Ptr packet, const dhcp::SocketInfo& source);
// Fetches the next packet to process from your queue using your other secret formula.
virtual isc::dhcp::Pkt4Ptr dequeuePacket();
: // Imagine you prototyped the rest of the functions
};
@endcode
Your factory implementation would then look something like this:
@code
const std::string QUEUE_TYPE = "Your-Q4";
isc::dhcp::PacketQueue4Ptr
YourPacketQueue4::factory(isc::data::ConstElementPtr parameters) {
// You need queue-type to pass into the base class.
// It's guaranteed to be here.
std::string queue_type = isc::data::SimpleParser::getString(parameters, "queue-type");
// Now you need to fetch your required parameters.
size_t capacity;
try {
capacity = isc::data::SimpleParser::getInteger(parameters, "capacity");
} catch (const std::exception& ex) {
isc_throw(isc::dhcp::InvalidQueueParameter, "YourPacketQueue4:factory:"
" 'capacity' parameter is missing/invalid: " << ex.what());
}
size_t threshold;
try {
threshold = isc::data::SimpleParser::getInteger(parameters, "threshold");
} catch (const std::exception& ex) {
isc_throw(isc::dhcp::InvalidQueueParameter, "YourPacketQueue4:factory:"
" 'threshold' parameter is missing/invalid: " << ex.what());
}
// You should be all set to create your queue instance!
isc::dhcp::PacketQueue4Ptr queue(new YourPacketQueue4(queue_type, capacity, threshold));
return (queue);
}
@endcode
Kea's configuration parser cannot know your parameter requirements and thus
can only flag JSON syntax errors. Thus it is important for your factory to
validate your parameters according to your requirements and throw meaningful
exceptions when they are not met. This allows users to know what to correct.
@subsection packet-queue-registration Registering Your Implementation
All hook libraries must provide a load() and unload() function. Your hook
library should register you queue factory during load() and un-register it
during unload(). Picking up with the our example, those functions might
look something like this:
@code
// This function is called when the library is loaded.
//
// param - handle library handle (we aren't using it)
// return - 0 when initialization is successful, 1 otherwise
int load(LibraryHandle& /* handle */) {
try {
// Here you register your DHCPv4 queue factory
isc::dhcp::IfaceMgr::instance().getPacketQueueMgr4()->
registerPacketQueueFactory(YourPacketQueue4::QUEUE_TYPE,
YourPacketQueue::factory);
} catch (const std::exception& ex) {
LOG_ERROR(your_logger, YOUR_LOAD_FAILED)
.arg(ex.what());
return (1);
}
LOG_INFO(your_logger, YOUR_LOAD_OK);
return (0);
}
// This function is called when the library is unloaded.
//
// return - 0 if deregistration was successful, 1 otherwise
int unload() {
// You need to remove your queue factory. This must be done to make sure
// your queue instance is destroyed before your library is unloaded.
isc::dhcp::IfaceMgr::instance().getPacketQueueMgr4()->
unregisterPacketQueueFactory(YourPacketQueue4::QUEUE_TYPE);
LOG_INFO(your_logger, YOUR_UNLOAD_OK);
return (0);
}
@endcode
@subsection packet-queue-factory Configuring Kea to use YourPacketQueue4
You're almost there. You developed your implementation, you've unit tested it
(You did unit test it right?). Now you just have to tell Kea to load it and
use it. Continuing with the example, your kea-dhcp4 configuration would need
to look something like this:
@code
{
"Dhcp4":
{
...
"hooks-libraries": [
{
# Loading your hook library!
"library": "/somepath/lib/libyour_packet_queue.so"
}
# any other hook libs
],
...
"dhcp-queue-control": {
"enable-queue": true,
"queue-type": "Your-Q4",
"capacity" : 100,
"threshold" : 75
},
...
}
@endcode
@subsection packet-queue-example-dhcpv6 DHCPv6 Example Snippets
For completeness, this section includes the example from above
implemented for DHCPv6.
DHCPv6 Class declaration:
@code
class YourPacketQueue6 : public isc::dhcp::PacketQueue<isc::dhcp::Pkt6Ptr> {
public:
// Logical name you will register your factory under.
static const std::string QUEUE_TYPE;
// Factory for instantiating queue instances.
static isc::dhcp::PacketQueue6Ptr factory(isc::data::ConstElementPtr params);
// Constructor
YourPacketQueue6(const std::string& queue_type, size_t capacity, size_t threshold)
: isc::dhcp::PacketQueue<isc::dhcp::Pkt6Ptr>(queue_type) {
// your constructor steps here
}
// Adds a packet to your queue using your secret formula based on threshold.
virtual void enqueuePacket(isc::dhcp::Pkt6Ptr packet, const dhcp::SocketInfo& source);
// Fetches the next packet to process from your queue using your other secret formula.
virtual isc::dhcp::Pkt6Ptr dequeuePacket();
: // Imagine you prototyped the rest of the functions
};
@endcode
DHCPv6 Factory implementation:
@code
const std::string QUEUE_TYPE = "Your-Q6";
isc::dhcp::PacketQueue6Ptr
YourPacketQueue6::factory(isc::data::ConstElementPtr parameters) {
// You need queue-type to pass into the base class.
// It's guaranteed to be here.
std::string queue_type = isc::data::SimpleParser::getString(parameters, "queue-type");
// Now you need to fetch your required parameters.
size_t capacity;
try {
capacity = isc::data::SimpleParser::getInteger(parameters, "capacity");
} catch (const std::exception& ex) {
isc_throw(isc::dhcp::InvalidQueueParameter, "YourPacketQueue6:factory:"
" 'capacity' parameter is missing/invalid: " << ex.what());
}
size_t threshold;
try {
threshold = isc::data::SimpleParser::getInteger(parameters, "threshold");
} catch (const std::exception& ex) {
isc_throw(isc::dhcp::InvalidQueueParameter, "YourPacketQueue6:factory:"
" 'threshold' parameter is missing/invalid: " << ex.what());
}
// You should be all set to create your queue instance!
isc::dhcp::PacketQueue6Ptr queue(new YourPacketQueue6(queue_type, capacity, threshold));
return (queue);
}
@endcode
DHCPv6 Hook load/unload functions
@code
// This function is called when the library is loaded.
//
// param - handle library handle (we aren't using it)
// return - 0 when initialization is successful, 1 otherwise
int load(LibraryHandle& /* handle */) {
try {
// Here you register your DHCPv6 queue factory
isc::dhcp::IfaceMgr::instance().getPacketQueueMgr6()->
registerPacketQueueFactory(YourPacketQueue6::QUEUE_TYPE,
YourPacketQueue::factory);
} catch (const std::exception& ex) {
LOG_ERROR(your_logger, YOUR_LOAD_FAILED)
.arg(ex.what());
return (1);
}
LOG_INFO(your_logger, YOUR_LOAD_OK);
return (0);
}
// This function is called when the library is unloaded.
//
// return - 0 if deregistration was successful, 1 otherwise
int unload() {
// You need to remove your queue factory. This must be done to make sure
// your queue instance is destroyed before your library is unloaded.
isc::dhcp::IfaceMgr::instance().getPacketQueueMgr6()->
unregisterPacketQueueFactory(YourPacketQueue6::QUEUE_TYPE);
LOG_INFO(your_logger, YOUR_UNLOAD_OK);
return (0);
}
@endcode
Server configuration for kea-dhcp6:
@code
{
"Dhcp6":
{
...
"hooks-libraries": [
{
# Loading your hook library!
"library": "/somepath/lib/libyour_packet_queue.so"
}
# any other hook libs
],
...
"dhcp-queue-control": {
"enable-queue": true,
"queue-type": "Your-Q6",
"capacity" : 100,
"threshold" : 75
},
...
}
@endcode
*/
|