0
0
mirror of https://gitlab.nic.cz/labs/bird.git synced 2024-12-23 18:21:54 +00:00
Commit Graph

420 Commits

Author SHA1 Message Date
Maria Matejka
b50224a003 Merge branch 'master' into HEAD 2021-12-01 13:04:52 +01:00
Maria Matejka
bb63e99d78 Page allocator moved from pools to IO loops.
The resource pool system is highly hierarchical and keeping spare pages
in pools leads to unnecessarily complex memory management.

Loops have a flat hiearchy, at least for now, and it is therefore much
easier to keep care of pages, especially in cases of excessive virtual memory
fragmentation.
2021-12-01 13:00:54 +01:00
Maria Matejka
385b3ea395 For safer memory allocations, resources are bound to loops.
Also all loops have their basic resource pool for allocations which are
auto-freed when the loop is stopping.
2021-11-30 21:38:25 +01:00
Maria Matejka
f772afc525 Memory statistics split into Effective and Overhead
This feature is intended mostly for checking that BIRD's allocation
strategies don't consume much memory space. There are some cases where
withdrawing routes in a specific order lead to memory fragmentation and
this output should give the user at least a notion of how much memory is
actually used for data storage and how much memory is "just allocated"
or used for overhead.

Also raising the "system allocator overhead estimation" from 8 to 16
bytes; it is probably even more. I've found 16 as a local minimum in
best scenarios among reachable machines. I couldn't find any reasonable
method to estimate this value when BIRD starts up.

This commit also fixes the inaccurate computation of memory overhead for
slabs where the "system allocater overhead estimation" was improperly
added to the size of mmap-ed memory.
2021-11-27 22:54:15 +01:00
Maria Matejka
644e9ca94e Directly mapped pages are kept for future use if temporarily not needed 2021-11-24 19:42:52 +00:00
Maria Matejka
1b39473993 Introducing basic RCU primitives for lock-less shared data structures 2021-11-22 19:05:44 +01:00
Maria Matejka
794a4eefa1 Keeping un-unmmappable pages until they can be reused
On Linux, munmap() may fail with ENOMEM when virtual memory is too
fragmented. Working this around by just keeping such blocks for future
use.
2021-11-22 19:05:44 +01:00
Maria Matejka
f0507f05ce Route sources have an explicit owner
This commit prevents use-after-free of routes belonging to protocols
which have been already destroyed, delaying also all the protocols'
shutdown until all of their routes have been finally propagated through
all the pipes down to the appropriate exports.

The use-after-free was somehow hypothetic yet theoretically possible in
rare conditions, when one BGP protocol authors a lot of routes and the
user deletes that protocol by reconfiguring in the same time as next hop
update is requested, causing rte_better() to be called on a
not-yet-pruned network prefix while the owner protocol has been already
freed.

In parallel execution environments, this would happen an inter-thread
use-after-free, causing possible heisenbugs or other nasty problems.
2021-11-22 19:05:44 +01:00
Maria Matejka
6e841b3153 Adding a generic cork mechanism for events 2021-11-22 19:05:43 +01:00
Maria Matejka
94eb0858c2 Converting the former BFD loop to a universal IO loop and protocol loop.
There is a simple universal IO loop, taking care of events, timers and
sockets. Primarily, one instance of a protocol should use exactly one IO
loop to do all its work, as is now done in BFD.

Contrary to previous versions, the loop is now launched and cleaned by
the nest/proto.c code, allowing for a protocol to just request its own
loop by setting the loop's lock order in config higher than the_bird.

It is not supported nor checked if any protocol changed the requested
lock order in reconfigure. No protocol should do it at all.
2021-11-22 19:05:43 +01:00
Maria Matejka
a4451535c6 Unified time for whole BIRD
In previous versions, every thread used its own time structures,
effectively leading to different time in every thread and strange
logging messages.

The time processing code now uses global atomic variables to keep
current time available for fast concurrent reading and safe updates.
2021-11-22 19:05:43 +01:00
Maria Matejka
df3264f51f Lock position checking allows for safe lock unions 2021-11-22 19:05:43 +01:00
Maria Matejka
1289c1c5ee Coroutines: A simple and lightweight parallel execution framework. 2021-11-22 19:05:43 +01:00
Maria Matejka
1db83a507a Locking subsystem: Just a global BIRD lock to begin with. 2021-11-22 19:05:43 +01:00
Maria Matejka
feb17ced23 Dropping the POSIX thread-local variables in favor of much easier-to-use C11 thread-local variables 2021-11-22 19:05:43 +01:00
Maria Matejka
b5061659d3 POSIX threads and thread-local storage is needed for concurrent execution 2021-11-22 19:05:43 +01:00
Maria Matejka
8d706aedba Fixing expensive list checks. Debug only commit. 2021-11-22 19:05:43 +01:00
Maria Matejka
2d7e42cc59 Type checking in SKIP_BACK macro 2021-11-22 19:05:43 +01:00
Maria Matejka
44f26c49f9 Special table hooks rectified.
* internal tables are now more standalone, having their own import and
  export hooks
* route refresh/reload uses stale counter instead of stale flag,
  allowing to drop walking the table at the beginning
* route modify (by BGP LLGR) is now done by a special refeed hook,
  reimporting the modified routes directly without filters
2021-11-22 19:05:43 +01:00
Maria Matejka
eb937358c0 Preference moved to RTA and set explicitly in protocols 2021-10-13 19:09:04 +02:00
Maria Matejka
6cd3771378 Multipage allocation
We can also quite simply allocate bigger blocks. Anyway, we need these
blocks to be aligned to their size which needs one mmap() two times
bigger and then two munmap()s returning the unaligned parts.

The user can specify -B <N> on startup when <N> is the exponent of 2,
setting the block size to 2^N. On most systems, N is 12, anyway if you
know that your configuration is going to eat gigabytes of RAM, you are
almost forced to raise your block size as you may easily get into memory
fragmentation issues or you have to raise your maximum mapping count,
e.g. "sysctl vm.max_map_count=(number)".
2021-10-13 19:01:22 +02:00
Maria Matejka
e5a8eec6d7 Linpools may use pages instead of xmalloc 2021-10-13 19:00:36 +02:00
Maria Matejka
bea582cbb5 fixup! Bound allocated pages to resource pools with page caches to avoid unnecessary syscalls 2021-10-13 18:59:45 +02:00
Maria Matejka
7f0e598208 Bound allocated pages to resource pools with page caches to avoid unnecessary syscalls 2021-09-10 18:13:50 +02:00
Maria Matejka
923a6644b2 Fixed memory poisoning in slab 2021-09-10 17:38:22 +02:00
Ondrej Zajicek (work)
ee9516dbe8 Lib: Fix static assert macro 2021-06-06 17:23:45 +02:00
Toke Høiland-Jørgensen
589f7d1e4f Nest: Allow MAC algorithms to specify min/max key length
Add min/max key length fields to the MAC algorithm description and
validate configured keys before they are used.
2021-06-06 16:28:18 +02:00
Toke Høiland-Jørgensen
35f88b305a Nest: Allow specifying security keys as hex bytes as well as strings
Add support for specifying a password in hexadecimal format, The result
is the same whether a password is specified as a quoted string or a
hex-encoded byte string, this just makes it more convenient to input
high-entropy byte strings as MAC keys.
2021-06-06 16:28:18 +02:00
Toke Høiland-Jørgensen
f1a824190c Lib: Add tests for blake2s and blake2b
Import the blake2-kat.h header with test vector output from the blake
reference implementation, and add tests to mac_test.c to compare the
output of the Bird MAC algorithm implementations with that reference
output.

Since the reference implementation only has test vectors for the full
output size, there are no tests for the smaller-sized output variants.
2021-06-06 16:28:09 +02:00
Toke Høiland-Jørgensen
725d9af94a Lib: Add Blake2s and Blake2b hash functions
The Babel MAC authentication RFC recommends implementing Blake2s as one of
the supported algorithms. In order to achieve do this, add the blake2b and
blake2s hash functions for MAC authentication. The hashing function
implementations are the reference implementations from blake2.net.

The Blake2 algorithms allow specifying an arbitrary output size, and the
Babel MAC spec says to implement Blake2s with 128-bit output. To satisfy
this, we add two different variants of each of the algorithms, one using
the default size (256 bits for Blake2s, 512 bits for Blake2b), and one
using half the default output size.

Update to BIRD coding style done by committer.
2021-06-06 16:26:58 +02:00
Ondrej Zajicek (work)
e5724f71d2 sysdep: Add wrapper to get random bytes - update
Simplify the code and fix an issue with getentropy() return value.
2021-06-06 16:26:06 +02:00
Toke Høiland-Jørgensen
c48ebde5ce sysdep: Add wrapper to get random bytes
Add a wrapper function in sysdep to get random bytes, and required checks
in configure.ac to select how to do it. The configure script tries, in
order, getrandom(), getentropy() and reading from /dev/urandom.
2021-06-06 16:26:06 +02:00
Maria Matejka
ebd5751cde Babel: Seqno requests are properly decoupled from neighbors when the underlying interface disappears
When an interface disappears, all the neighbors are freed as well. Seqno
requests were anyway not decoupled from them, leading to strange
segfaults. This fix adds a proper seqno request list inside neighbors to
make sure that no pointer to neighbor is kept after free.
2021-05-30 13:29:21 +02:00
Ondrej Zajicek (work)
c3c691e95c Flowspec: Documentation update 2021-05-18 20:41:01 +02:00
Ondrej Zajicek (work)
abc9ccc5cb Flowspec: Label field should use numeric operator and not bitmask operator 2021-05-18 20:23:08 +02:00
Ondrej Zajicek (work)
dd8481cc1c Flowspec: Do not use comma for bitmask operators
For numeric operators, comma is used for disjunction in expressions like
"10, 20, 30..40". But for bitmask operators, comma is used for
conjunction in a way that does not really make much sense. Use always
explicit logical operators (&& and ||) to connect bitmask operators.

Thanks to Matt Corallo for the bugreport.
2021-05-18 19:54:18 +02:00
Trisha Biswas
e5468d1685 Filter: Add MPLS label route attribute
Add support to set or read outgoing MPLS labels using filters. Currently
this supports the addition of one label per route for the first next hop.

Minor changes by committer.
2021-05-17 17:50:04 +02:00
Ondrej Zajicek (work)
d114959e3a Flowspec: Fix values for true/false operators
RFC 8955 is pretty clear that 000 is false and 111 is true.
2021-05-14 18:44:52 +02:00
Ondrej Zajicek (work)
69a33c92ff Flowspec: Add code for conversion of flowspec parts to interval lists
Implement function flow_explicate_part() to convert flowspec numeric
expressions to a simple list of (disjoint, sorted) intervals. That could
be used in filters to build f_tree-based int-sets from them.
2021-05-14 18:33:15 +02:00
Ondrej Zajicek (work)
9c41e1ca3e Lib: Fix handling of buffers in timestamp formatting
The code in tm_format_real_time() mixed up two buffers and their
sizes, which may cause crash in MRT dumping code.

Thanks to Piotr Wydrych for the bugreport.
2021-04-12 17:01:31 +02:00
Maria Matejka
a9938b1792 Resources: added mb_move() to complement rmove() for memory blocks 2021-03-30 21:56:08 +02:00
Maria Matejka
886dd92eee Slab: head now uses bitmask for used/free nodes info instead of lists
From now, there are no auxiliary pointers stored in the free slab nodes.
This led to strange debugging problems if use-after-free happened in
slab-allocated structures, especially if the structure's first member is
a next pointer.

This also reduces the memory needed by 1 pointer per allocated object.
OTOH, we now rely on pages being aligned to their size's multiple, which
is quite common anyway.
2021-03-25 16:47:48 +01:00
Ondrej Zajicek (work)
7be3af7fa6 Rate-limit scheduling of work-events
In general, events are code handling some some condition, which is
scheduled when such condition happened and executed independently from
I/O loop. Work-events are a subgroup of events that are scheduled
repeatedly until some (often significant) work is done (e.g. feeding
routes to protocol). All scheduled events are executed during each
I/O loop iteration.

Separate work-events from regular events to a separate queue and
rate limit their execution to a fixed number per I/O loop iteration.
That should prevent excess latency when many work-events are
scheduled at one time (e.g. simultaneous reload of many BGP sessions).
2021-03-12 15:35:56 +01:00
Ondrej Zajicek (work)
d06a875b04 Filter: Recursive filter iteration code
Add macros for recursive filter iteration that allows to examine
all instructions reachable from a filter.
2021-02-07 19:21:42 +01:00
Toke Høiland-Jørgensen
db2d29073a lib/slab: introduce sl_allocz() function and use it in Babel
The babel protocol code was initialising objects returned from the slab
allocator by assigning to each of the struct members individually, but
wasn't touching the NODE member while doing so. This leads to warnings on
debug builds since commit:

baac700906 ("List expensive check.")

To fix this, introduce an sl_allocz() variant of the slab allocator which
will zero out the memory before returning it, and switch all the babel call
sites to use this version. The overhead for doing this should be negligible
for small objects, and in the case of babel, the largest object being
allocated was being zeroed anyway, so we can drop the memset in
babel_read_tlv().
2020-11-24 02:36:31 +01:00
Maria Matejka
59238768b3 Slab: Init node in slab head to NULLs. 2020-05-01 15:19:12 +02:00
Maria Matejka
ea259d6201 Timer: Adding missing initializer. 2020-05-01 15:19:12 +02:00
Maria Matejka
9ac13d7af2 Lists: Replaced replace_node() by update_node() which is the only use of that function. 2020-05-01 15:19:12 +02:00
Maria Matejka
e26a5195dd Lists: fix a stupid sanitizer bug 2020-05-01 15:19:12 +02:00
Maria Matejka
3bb10b4d31 Uninitialized list nodes fixes 2020-05-01 15:19:12 +02:00